fetch_file

Documentation for eth_defi.vault_report.data.fetch_file function.

fetch_file(url, path, params=None, max_age=datetime.timedelta(seconds=21600), timeout=120.0)

Download a file with a progress bar, reusing a fresh cached copy.

The cache freshness is the file’s modification time, see get_cache_age(), so no separate metadata file is needed and deleting the file forces a download.

The file is downloaded to a temporary name first and renamed into place, so an interrupted download never leaves a truncated cache file behind. The body is streamed in 1 MiB chunks, so the large price Parquet never sits in memory, with a tqdm_loggable progress bar that stays visible in non-interactive logs.

Error messages include url but never params, so an API key passed in params does not end up in logs or tracebacks.

Also used by eth_defi.vault_report.benchmarks for the FRED CSV.

Parameters

  • url (str) – URL to download, without secrets.

  • path (pathlib.Path) – Local cache path.

  • params (Optional[dict]) – Query parameters, e.g. an API key.

  • max_age (datetime.timedelta) – Reuse the cached file if it is younger than this.

  • timeout (float) – HTTP connect/read timeout in seconds.

Returns

Path to the downloaded file.

Return type

pathlib.Path