vault.data_file_export

Documentation for eth_defi.vault.data_file_export Python module.

Export vault database files to Cloudflare R2.

Uploads price databases, metadata pickles, and Core3 risk intelligence and exchange-rate DuckDB files so the backtester can download them without running the full scan pipeline.

Functions

`get_data_file_paths`(base_path[, ...])	Build the data file list uploaded to R2.
`main`()	Run the data file export script.
`resolve_exchange_rate_database_path`([base_path])	Resolve the exchange-rate DuckDB database path.
`upload_files_to_r2`(file_paths, bucket_name, ...)	Upload a list of files to R2 bucket, excluding tmp* files.

resolve_exchange_rate_database_path(base_path=None)

Resolve the exchange-rate DuckDB database path.

The exchange-rate bundle is produced by the currency API scanner. Use the same CURRENCY_API_DB_PATH / CURRENCY_API_DATABASE_PATH override contract as the all-chain scanner so exports upload the database that the scanner writes. Without an override, data exports use exchange-rates.duckdb under the active pipeline data directory.

Parameters: base_path (Optional[pathlib.Path]) – Pipeline data directory. None falls back to the standalone currency API default path.
Returns: Path to the exchange-rate DuckDB database.
Return type: pathlib.Path

get_data_file_paths(base_path, core3_db_path=None, exchange_rate_db_path=None)

Build the data file list uploaded to R2.

Parameters

base_path (pathlib.Path) – Pipeline data directory.
core3_db_path (Optional[pathlib.Path]) – Optional Core3 DuckDB path override.
exchange_rate_db_path (Optional[pathlib.Path]) – Optional exchange-rate DuckDB path override.

Returns

Files to upload, including optional files that may be skipped later if they do not exist.

Return type

list[pathlib.Path]

upload_files_to_r2(file_paths, bucket_name, endpoint_url, access_key_id, secret_access_key, key_prefix='', public_url='')

Upload a list of files to R2 bucket, excluding tmp* files.

Parameters

file_paths (list[pathlib.Path]) – List of file paths to upload.
bucket_name (str) – R2 bucket name.
endpoint_url (str) – R2 API endpoint URL.
access_key_id (str) – R2 access key ID.
secret_access_key (str) – R2 secret access key.
key_prefix (str) – Prefix for S3 keys, e.g. test- for test uploads.
public_url (str) – Public base URL for logging final download URLs.

Returns

Number of files uploaded.

Return type

int

main()

Run the data file export script.

Reads R2 configuration from environment variables, uploads vault data files to configured buckets, and creates daily backups in the alternative bucket when enabled.

Return type: None