satmo.download module

satmo.download.download_file(url, write_dir, overwrite=False, check_integrity=False, timeout=15)[source]

Generic file download function

Downloads a file from a URL, and write it to a user defined location. The function will raise a ConnectionError (via requests) if a connection cannot be established with the server (e.g.: network is down) and an HttpResourceNotAvailable if the file appear not to exist on the server

Parameters:
  • url (str) – download url
  • write_dir (str) – Host directory to which the data will be written
  • overwrite (bool) – Should the file be overwritten if already existing on local host
  • check_integrity (bool) – Only makes sense if overwrite is set to False (when updating the archive)
  • timeout (float) – How long to wait (in seconds) for the server to send data before giving up and raising a requests.ConnectionError
Returns:

The filename of the downloaded data
Return type:

str
satmo.download.download_robust(url, base_dir, n_retries=5, pause_retries=10, overwrite=False, check_integrity=False)[source]

Robust download of a list of urls

Handles connection errors, optionally check for existing files and integrity Suitable for real time use and for updating an archive without overwriting existing data

Parameters:
  • url (str) – Download url
  • base_dir (str) – root of the archive tree on the host
  • n_retries (int) – Number of retries when the download fails because of a ConnectionError
  • pause_retries (int) – Duration (in seconds) between retries
  • overwrite (bool) – Should existing files on the host be overwritten
  • check_integrity (bool) – Only makes sense if overwrite is set to False (when updating the archive)
Returns:

Boolean, True when successful, False otherwise
satmo.download.download_to_tree(url, base_dir, overwrite=False, check_integrity=False)[source]

Download an ocean product file and automatically write it to the right location

The idea is that the function immediately knows where to write the file by parsing the filename (present in url) Directories where data are downloaded do not need to exist before running the function Similarly to download_file, the function will raise a ConnectionError (via requests) if a connection cannot be established with the server (e.g.: network is down) and an HttpResourceNotAvailable if the file appear not to exist on the server. It might be worth re-trying when catching requests.ConnectionError, going to the next file is probably the best way if catching a HttpResourceNotAvailable

Parameters:
  • url (str) – download url
  • base_dir (str) – root of the archive tree on the host
  • overwrite (bool) – Should the file be overwritten if already existing on local host
  • check_integrity (bool) – Only makes sense if overwrite is set to False (when updating the archive)
Returns:

The filename of the downloaded file
Return type:

str