satmo.wrappers module¶

satmo.wrappers.bin_map_batcher(begin, end, sensor_codes, south, north, west, east, data_root, binning_resolution=1, mapping_resolution=1000, day_vars=None, night_vars=None, flags=None, proj=None, overwrite=True, n_threads=1)[source]¶: Batch processing of L3m data from L2 for several dates, sensors and variables

satmo.wrappers.bin_map_wrapper(date, sensor_codes, south, north, west, east, data_root, binning_resolution=1, mapping_resolution=1000, day_vars=None, night_vars=None, flags=None, proj=None, overwrite=True)[source]¶

Wrapper to run l2bin and l3mapgen for a list of variables and a given date

The function automatically handles the production of the right intermediary L3b files.

Parameters:

date (str or datetime) – Date of the data to process
sensor_codes (list) – List to sensor codes to include in the processing
south (int or float) – south latitude of mapped file extent
north (int or float) – north latitude of mapped file extent
west (int or float) – west longitude of mapped file extent
east (int or float) – east longitude of mapped file extent
data_root (str) – Root of the data archive.
binning_resolution (int or str) – See resolve argument in l2bin doc
mapping_resolution (int) – Resolution of the output L3m file in meters
day_vars (list) – List of day variables to map (will result in one L3m file being produced for each var and each sensor)
night_vars (list) – List of night variables to map (will result in one L3m file being produced for each var and each sensor)
flags (list) – A list of flags to mask invalid data (e.g. [‘CLDICE’, ‘LAND’, ‘HIGLINT’]) If None (default), a default list of flag for the L3 suite is fetched from the global variable FLAGS. When processing several variables belonging to different suites, it is not recommended to set a fixed list of flags, since they normally differ between suites.
proj (str) – Optional proj4 string. If None (default), a lambert Azimutal Equal Area projection (laea), centered on the provided extent is used.
overwrite (bool) – Overwrite existing final (L3m) and intermediary (L3b) files

Returns:

This function is used for its side effects of binning and then mapping data, producing as a result various L3b and L3m files

satmo.wrappers.l2_append_batcher(begin, end, sensor_codes, var, suite, data_root, n_threads=1)[source]¶: Compute a new variable and append it to existing L2 files in batch mode

satmo.wrappers.l2_append_wrapper(date, sensor_codes, var, suite, data_root)[source]¶

Wrapper to run l2_append on all matching files of a given date

Parameters:	date (datetime or str) – Date of L2 data to process sensor_codes (list) – List of strings corresponding to sensor codes to process var (str) – Short name of the variable to process. Must exist in the global variable BAND_MATH_FUNCTIONS. suite (str) – Suite name of the L2 files containing the required input bands. data_root (str) – Root of the data archive
Returns:	The list of files to which a variable was appended. However, the function is mostly used for its side effect of appending a variable to each file of a list of netcdf files.
Return type:	list

satmo.wrappers.l2gen_batcher(begin, end, sensor_codes, var_list, suite, data_root, night=False, get_anc=True, n_threads=1)[source]¶: Batch L2 processing with parallel support; to be ran from cli

satmo.wrappers.l2gen_wrapper(date, sensor_codes, var_list, suite, data_root, night=False, get_anc=True)[source]¶

Runs l2gen on all the files of a given date.

All possible errors are caught

Parameters:

date (datetime or str) – Date of L2 data to process
sensor_codes (list) – List of strings corresponding to sensor codes to process
var_list (list) – List of strings representing the variables to process
suite (str) – Output L2 suite name
data_root (str) – Root of the data archive
night (bool) – Is night data? Default to False
get_anc (bool) – Download ancillary data for improved atmospheric correction. Defaults to True.

Examples

>>> import satmo
>>> l2gen_wrapper(date='2004-11-21', sensor_codes=['A', 'T', 'V'],
>>>               var_list=['chlor_a'], data_root='/export/isilon/datos2/satmo2_data',
>>>               suite='OC3', night=False, get_anc=True)

Returns:	List of processed L2 files
Return type:	list

satmo.wrappers.l2mapgen_batcher(begin, end, sensor_codes, var, south, north, west, east, data_root, night=False, flags=None, width=5000, outmode='tiff', overwrite=False, threshold=0, n_threads=1)[source]¶: Batch L2m processing with parallel support; to be ran from cli

satmo.wrappers.l2mapgen_wrapper(date, sensor_codes, var, south, north, west, east, data_root, night=False, flags=None, width=5000, outmode='tiff', overwrite=False, threshold=0)[source]¶

Runs l2mapgen for a given variable on all the files of a given date.

All possible errors are caugth

Parameters:

date (datetime or str) – Date of L2 data to process
sensor_codes (list) – List of strings corresponding to sensor codes to process
var (str) – Variable to process
south (float) – Southern border of output extent (in DD)
north (float) – Northern border of output extent (in DD)
west (float) – Western border of output extent (in DD)
east (float) – Eastern border of output extent (in DD)
data_root (str) – Root of the data archive
night (bool) – Is night data? Default to False
flags (list) – List of flags to apply. Defaults to None, in which case default flags for the product suite are retrieved from the FLAGS global variable and used
width (int) – Width in pixels of the output image
outmode (str) – See seadas l2mapgen doc
overwrite (bool) – Overwrite existing files? Return ValueError if file exists and overwrite is set to False (default)
threshold (float) – Minimum percentage of the filled pixels

satmo.wrappers.l3bin_map_batcher(begin, end, delta, sensor_codes, var_list, south, north, west, east, composite, data_root, mapping_resolution=1000, night=False, proj=None, overwrite=False, n_threads=1)[source]¶

Takes a begin date, an end date and a compositing period to batch process temporal composites

Uses the l3bin and l3mapgen seadas utilities and support parallel processing

Parameters:	begin (datetime.datetime or str) – Begin date of the first composite end (datetime.datetime or str) – Begin date of the last composite delta (int or str) – composite length in days if int, ‘month’ if str.

satmo.wrappers.l3bin_map_wrapper(date_list, sensor_codes, var_list, south, north, west, east, composite, data_root, mapping_resolution=1000, night=False, proj=None, overwrite=False)[source]¶

Run l3bin and l3mapgen for a list of dates, sensor_codes, and variables

automatically retrieve suites that need to be processed from variables

Parameters:

sensor_codes (list) – List of sensor codes to include (e.g.: [‘A’, ‘T’, ‘V’]
date_list (list) – List of dates
var_list (list) – List of variables to process
south (float) – Southern border of output extent (in DD)
north (float) – Northern border of output extent (in DD)
west (float) – Western border of output extent (in DD)
east (float) – Eastern border of output extent (in DD)
data_root (str) – Root of the data archive
composite (str) – Composite type (8DAY, MON)
resolution (int) – MApping resolution in meters. Defaults to 1000
night (bool) – Is it night processing?
overwrite (bool) – Overwrite existing files?

satmo.wrappers.l3bin_wrapper(sensor_codes, date_list, suite_list, south, north, west, east, composite, data_root, overwrite=False)[source]¶

Wrapper to run l3bin without having to name explicitly input or output files

Parameters:	sensor_codes (list) – List of sensor codes to include (e.g.: [‘A’, ‘T’, ‘V’] date_list (list) – List of dates suite_list (list) – List of L3 suites to include south (float) – Southern border of output extent (in DD) north (float) – Northern border of output extent (in DD) west (float) – Western border of output extent (in DD) east (float) – Eastern border of output extent (in DD) data_root (str) – Root of the data archive composite (str) – Composite type (8DAY, MON) overwrite (bool) – Overwrite existing files?
Returns:	List of L3b files generated. Mostly used for its side effect of generating temporally composited L3b files from daily L3b files.
Return type:	list

satmo.wrappers.make_daily_composite(date, variable, suite, data_root, resolution, sensor_codes='all', fun='mean', filename=None, preview=True, overwrite=False)[source]¶

Wrapper for making daily composites (from multiple sensors)

Parameters:	date (str or datetime) – Date of the data to be composited variable (str) – L3m variable (e.g. ‘chlor_a’) suite (str) – L3m suite (e.g. ‘CHL’) data_root (str) – Root of the data archive resolution (str) – e.g. ‘1km’ sensor_codes (list) – Sensors to include in the composite. Defaults to ‘all’, which is automatically transformed to [‘A’, ‘T’, ‘V’] for aqua, terra and viirs. fun (str) – Compositing function. Can be ‘mean’, ‘median’, ‘min’, ‘max’. Defaults to ‘mean’ filename (str) – Optional output filename. Defaults to None, in which case the filename is automatically generated. preview (bool) – Generate a png preview. Defaults to True overwrite (bool) – Overwrite existing L3m file. Defaults to False
Returns:	The filename of the created composite.
Return type:	str

Examples

>>> import satmo

>>> satmo.make_daily_composite(date='2016-01-01', variable='chlor_a', suite='CHL',
                               resolution='2km',
                               data_root='/home/ldutrieux/sandbox/satmo2_data',
                               overwrite=True, fun='mean', sensor_codes=['A', 'V'])

satmo.wrappers.make_daily_composite_error_catcher(date, variable, suite, data_root, resolution, sensor_codes='all', fun='mean', filename=None, preview=True, overwrite=False)[source]¶

satmo.wrappers.nrt_wrapper(day_or_night, pp_type, var_list, north, south, west, east, data_root, binning_resolution=1, mapping_resolution=1000, eight_day=False, month=False, flags=None, proj=None)[source]¶

Main wrapper to be called from CLI for NRT operation of the system

Parameters:

day_or_night (str) – ‘day’ or ‘night’
pp_type (str) – ‘nrt’ (for near real time) or ‘refined’ (for refined processing
var_list (list) – list of variables to process after data download (e.g. [‘chlor_a’, ‘sst’])
north (float) – north latitude of bounding box in DD
south (float) – south latitude of bounding box in DD
west (float) – west longitude of bounding box in DD
east (float) – east longitude of bounding box in DD
data_root (str) – Root of the data archive
binning_resolution (int or str) – See resolve argument in l2bin doc
mapping_resolution (int) – Resolution of the output L3m file in meters
flags (list) – A list of flags to mask invalid data (e.g. [‘CLDICE’, ‘LAND’, ‘HIGLINT’]) If None (default), a default list of flag for the L3 suite is fetched from the global variable FLAGS. When processing several variables belonging to different suites, it is not recommended to set a fixed list of flags, since they normally differ between suites.
proj (str) – Optional proj4 string. If None (default), a longlat is used
eight_day (bool) – Generate 8 days temporal composites (defaults to False)
month (bool) – Generate monthly temporal composites (defaults to False)

Returns:

The function is used for it’s side effects of downloading data, and processing: associated variables

Examples

>>> # The idea is to call this wrapper several time within a scheduler
>>> import satmo

>>> satmo.nrt_wrapper(day_or_night='day', pp_type='nrt', var_list=['chlor_a', 'sst', 'nflh', 'Kd_490'],
                      north=33, south=3, west=-122, east=-72,
                      data_root='/export/isilon/datos2/satmo2_data/',
                      eight_day=True)

satmo.wrappers.nrt_wrapper_l1(north, south, west, east, var_list, data_root, n_threads=1)[source]¶

Wrapper to be called from nrt command line once a day

Handles subscription based download of L1A files, L2 processing, computation of additional indices, and mapping of each individual file, separately to longlat.

Parameters:

north (float) – north latitude of bounding box in DD
south (float) – south latitude of bounding box in DD
west (float) – west longitude of bounding box in DD
east (float) – east longitude of bounding box in DD
var_list (list) – List of additional variables to generate using l2_append
data_root (str) – Root of the data archive
n_threads (int) – Number of threads to use for running l2gen in parallel

Returns:

The function is used for its side effect of running the L1A-download to L2m processing chain

satmo.wrappers.refined_processing_wrapper_l1(north, south, west, east, var_list, data_root, delay=30)[source]¶

Wrapper to be called from nrt command line once a day

Reprocesses L2 OC2 suite a month after initial processing for improved atmospheric correction

Parameters:

north (float) – north latitude of bounding box in DD
south (float) – south latitude of bounding box in DD
west (float) – west longitude of bounding box in DD
east (float) – east longitude of bounding box in DD
data_root (str) – Root of the data archive
delay (int) – Delay in number of days between data acquisition and refined processing. Defaults to 30.

Returns:

The function is used for its side effect of running l2gen on 1 month old L1A data

satmo.wrappers.subscriptions_download(sub_list, data_root, refined=False)[source]¶

Update a local archive using a list of data subscription numbers

Parameters:	sub_list (list of int) – List of subscription numbers data_root (str) – Root of the data archive refined (bool) – Do the subscriptions refer to refined processing data (defaults to True), in which case the function will compare file size between the local and remote archives before deciding or not to download the file.
Returns:	A list of file paths corresponding to the local paths of downloaded files
Return type:	list

Examples

>>> import satmo
>>> satmo.subscriptions_download([1821, 1823], data_root='/export/isilon/datos2/satmo2_data',
                                 refined=False)

>>> from satmo.global_variables import SUBSCRIPTIONS
>>> satmo.subscriptions_download(SUBSCRIPTIONS['L1A']['day'],
                                 data_root='/export/isilon/datos2/satmo2_data',
                                 refined=False)

satmo.wrappers.timerange_daily_composite(begin, end, variable, suite, data_root, resolution, sensor_codes='all', fun='mean', preview=True, overwrite=False, n_threads=1)[source]¶

Produce daily composite for individual dates in a time-range in batch

Parameters:	begin (datetime or str) – Beginning of time range. ‘yyyy-mm-dd’ if str end (datetime or str) – End of time range. ‘yyyy-mm-dd’ if str n_threads (int) – Number of threads to use for running the make_daily_composite function in parallel. others – See help of make_daily_composite for the other parameters.

Examples

>>> import satmo

>>> satmo.timerange_daily_composite(begin='2015-01-01', end='2016-06-30',
                                    variable='chlor_a', suite='CHL',
                                    data_root='/home/ldutrieux/sandbox/satmo2_data',
                                    resolution='2km')

satmo.wrappers.timerange_download(sensors, begin, end, write_dir, north, south, west, east, day=True, night=True, product='L1A', overwrite=False, check_integrity=False)[source]¶

Queries and download data for a given timerange

Wrapper around downdload and query

Parameters:	sensors – (list) list of strings, Valid entries are ‘am’ (aqua), ‘tm’ (terra), ‘sw’ (seawifs), ‘v0’ (viirs) begin (datetime or str) – Beginning of time range. ‘yyyy-mm-dd’ if str end (datetime or str) – End of time range. ‘yyyy-mm-dd’ if str write_dir (str) – Host directory to which the data will be written north (float) – north latitude of bounding box in DD south (float) – south latitude of bounding box in DD west (float) – west longitude of bounding box in DD east (float) – east longitude of bounding box in DD day (bool) – Order day data ? night (bool) – Order night data ? product (str) – Product to order, defaults to ‘L1A’. Other possible values include ‘CHL’ for L2_OC, ‘SST’ for L2_SST, and ‘SST4’ for L2_SST4 overwrite (bool) – Should existing files on the host be overwritten check_integrity (bool) – Only makes sense if overwrite is set to False (when updating the archive)
Returns:	list of boolean
Return type:	list