currency_api.database

Documentation for eth_defi.currency_api.database Python module.

DuckDB persistence for historical exchange rates.

Stores one row per (date, base_currency, quote_currency, source) plus a quote-level gap table (unavailable_rates) so the scanner can resume on completeness rather than on MAX(date) and never re-fetch genuinely missing cells forever.

See eth_defi/currency_api/README-currency-api.md for the schema overview.

Classes

CurrencyRateDatabase

DuckDB database for storing historical exchange rates.
class CurrencyRateDatabase

Bases: object

DuckDB database for storing historical exchange rates.

Three tables:

  • exchange_rates — the data, uniquely maintained by (date, base_currency, quote_currency, source). rate is the raw API value (units of quote per 1 unit of base).

  • unavailable_rates — quote-level gap tracking for cells confirmed to have no data (whole-date 404s, individually missing quotes and given-up persistent errors), so they are not re-fetched on every run.

  • fetch_attempts — internal bookkeeping of the consecutive transient-failure count per (date, base_currency, source), used to give up on a stuck date after a bounded number of attempts.

Example:

from pathlib import Path
from eth_defi.currency_api.database import CurrencyRateDatabase

db = CurrencyRateDatabase(Path("/tmp/rates.duckdb"))
print(db.get_rates_dataframe())
db.close()

Open (or create) the database and ensure the schema exists.

Parameters

path – Path to the DuckDB file. Parent directories are created if needed.

__init__(path)

Open (or create) the database and ensure the schema exists.

Parameters

path (pathlib.Path) – Path to the DuckDB file. Parent directories are created if needed.

Return type

None

save()

Force a checkpoint so data is flushed to disk.

Return type

None

close()

Close the database connection.

Return type

None

upsert_rates(date_rates)

Idempotently upsert the rates for one date.

Re-running with the same key overwrites the value and written_at via transactional delete-then-insert, so a repeated scan produces no duplicates without relying on DuckDB primary-key indexes.

Parameters

date_rates (eth_defi.currency_api.client.DateRates) – Parsed rates to store.

Return type

None

record_unavailable(date, base_currency, quote_currency, source, reason, http_status=None)

Record a single rate cell as permanently unavailable.

Parameters

  • date (datetime.date) – Date of the missing cell.

  • base_currency (str) – Base currency code.

  • quote_currency (str) – Quote currency code that has no data.

  • source (str) – Provider identifier.

  • reason (str) – date_404 (whole date 404 on both hosts), quote_missing (quote absent from an otherwise-200 body), or persistent_error (given up after exceeding the transient-failure budget).

  • http_status (Optional[int]) – HTTP status that confirmed the gap, if applicable.

Return type

None

No-op for a cell that already has data in exchange_rates — the two tables are kept mutually exclusive. This mirrors upsert_rates(), which deletes the gap row when data arrives; together they ensure a cell is never recorded as both present and unavailable (e.g. when a date that already has stored data later 404s on a tail refetch, or hits give-up).

get_transient_attempts(base_currency, source)

Return the consecutive transient-failure count per date.

Parameters

  • base_currency (str) – Base currency to filter on.

  • source (str) – Provider identifier to filter on.

Returns

Mapping of date to the number of consecutive transient failures recorded so far. Dates with no failures are absent.

Return type

dict[datetime.date, int]

set_transient_attempts(date, base_currency, source, attempts)

Persist the consecutive transient-failure count for a date.

Parameters

  • date (datetime.date) – Date that failed.

  • base_currency (str) – Base currency code.

  • source (str) – Provider identifier.

  • attempts (int) – New cumulative count of consecutive transient failures.

Return type

None

clear_transient_attempts(date, base_currency, source)

Reset the transient-failure counter for a date once it resolves.

Parameters

  • date (datetime.date) – Date that succeeded or was confirmed unavailable.

  • base_currency (str) – Base currency code.

  • source (str) – Provider identifier.

Return type

None

get_present_pairs(base_currency, source)

Return (date, quote_currency) pairs already stored in exchange_rates.

Parameters

  • base_currency (str) –

  • source (str) –

Return type

set[tuple[datetime.date, str]]

get_unavailable_pairs(base_currency, source)

Return (date, quote_currency) pairs recorded in unavailable_rates.

Parameters

  • base_currency (str) –

  • source (str) –

Return type

set[tuple[datetime.date, str]]

row_count()

Return the total number of stored exchange rate rows.

Return type

int

get_min_date(base_currency, source)

Return the earliest stored date for a base/source, or None if empty.

Parameters

  • base_currency (str) –

  • source (str) –

Return type

Optional[datetime.date]

get_max_date(base_currency, source)

Return the latest stored date for a base/source, or None if empty.

Parameters

  • base_currency (str) –

  • source (str) –

Return type

Optional[datetime.date]

get_rates_dataframe(base_currency=None, source=None)

Return stored rates as a DataFrame ordered by date then quote.

Parameters

  • base_currency (Optional[str]) – Optional base currency filter.

  • source (Optional[str]) – Optional source filter.

Returns

DataFrame with columns date, base_currency, quote_currency, rate, source, written_at.

Return type

pandas.DataFrame