vault.data_file_export

Documentation for eth_defi.vault.data_file_export Python module.

Export vault database files to Cloudflare R2.

Uploads price databases, metadata pickles, and Core3 risk intelligence DuckDB so the backtester can download them without running the full scan pipeline.

Functions

get_data_file_paths(base_path[, core3_db_path])

Build the data file list uploaded to R2.

main()

Run the data file export script.

upload_files_to_r2(file_paths, bucket_name, ...)

Upload a list of files to R2 bucket, excluding tmp* files.
get_data_file_paths(base_path, core3_db_path=None)

Build the data file list uploaded to R2.

Parameters
Returns

Files to upload, including optional files that may be skipped later if they do not exist.

Return type

list[pathlib.Path]

main()

Run the data file export script.

Reads R2 configuration from environment variables, uploads vault data files to configured buckets, and creates daily backups in the alternative bucket when enabled.

Return type

None

upload_files_to_r2(file_paths, bucket_name, endpoint_url, access_key_id, secret_access_key, key_prefix='', public_url='')

Upload a list of files to R2 bucket, excluding tmp* files.

Parameters

  • file_paths (list[pathlib.Path]) – List of file paths to upload.

  • bucket_name (str) – R2 bucket name.

  • endpoint_url (str) – R2 API endpoint URL.

  • access_key_id (str) – R2 access key ID.

  • secret_access_key (str) – R2 secret access key.

  • key_prefix (str) – Prefix for S3 keys, e.g. test- for test uploads.

  • public_url (str) – Public base URL for logging final download URLs.

Returns

Number of files uploaded.

Return type

int