vault.duckdb_backup

Documentation for eth_defi.vault.duckdb_backup Python module.

Private, consistent DuckDB backups with a durable per-database 48-hour gate.

A conditional R2 state write claims each attempt before touching database contents. Immutable attempt receipts survive local-state loss and failed runs. Only stopped owners can be snapshotted: an exclusive DuckDB connection must succeed before checkpoint and copy. R2 conditional writes are documented at https://developers.cloudflare.com/r2/api/s3/api/.

Functions

backup_databases(client, bucket, registry[, ...])	Back up due, exclusively opened databases once per two days at most.
backup_key_root()	Resolve a stable namespace without resetting cadence on prefix changes.
create_private_backup_client()	Create the private data client from the scanner's environment settings.
database_registry(base_path, existing_paths)	Resolve the scanner database allowlist and configured path overrides.
fetch_legacy_backup_anchor(client, bucket, ...)	Bootstrap the new cadence from existing flat and daily legacy transfers.
reserve_backup_window(client, bucket, ...[, ...])	Reserve a shared daily/HF maintenance window without scheduler starvation.

Classes

DuckDBBackup

Explicitly registered database with a stable logical identity.

Exceptions

DuckDBBackupError

Report an incomplete backup batch without hiding successful members.

class DuckDBBackup

Bases: object

Explicitly registered database with a stable logical identity.

Nested caches are included only by explicitly naming their owner and path.

name: str

Stable identity used for remote cadence and immutable snapshots.

path: pathlib.Path

Operator-owned stopped database file.

__init__(name, path)

Parameters

• name (str) –

• path (pathlib.Path) –

Return type

None

exception DuckDBBackupError

Bases: RuntimeError

Report an incomplete backup batch without hiding successful members.

__init__(statuses)

Parameters

statuses (dict[str, str]) –

Return type

None

__new__(**kwargs)

add_note(note, /)

Add a note to the exception

with_traceback(tb, /)

Set self.__traceback__ to tb and return self.

create_private_backup_client()

Create the private data client from the scanner’s environment settings.

Prefer R2_DATA_* credentials, with the existing R2_VAULT_METADATA_* variables as fallbacks. Bucket selection remains explicit at each caller. See https://developers.cloudflare.com/r2/api/s3/api/.

Returns

Authenticated S3-compatible client; credentials are never logged.

Return type

Any

backup_key_root()

Resolve a stable namespace without resetting cadence on prefix changes.

Prefixed deployments must explicitly select a persistent logical namespace, preventing staging from sharing production gates or compatibility objects.

Returns

Remote scheduling and immutable snapshot prefix.

Return type

str

database_registry(base_path, existing_paths)

Resolve the scanner database allowlist and configured path overrides.

Existing export paths supply canonical risk, settlement and currency paths. Unknown files are logged for an operator to register, never bulk uploaded.

Parameters

• base_path (pathlib.Path) – Active pipeline state directory.

• existing_paths (list[pathlib.Path]) – Already resolved export paths, possibly overridden.

Returns

Explicit database registry with unique names and paths.

Return type

list[eth_defi.vault.duckdb_backup.DuckDBBackup]

fetch_legacy_backup_anchor(client, bucket, filename, now)

Bootstrap the new cadence from existing flat and daily legacy transfers.

The first scheduled backup must also respect uploads/copies made by the old exporter during the preceding 48 hours. Only remote HEAD metadata is read.

Parameters

• client (Any) – Private R2 client.

• bucket (str) – Private bucket.

• filename (str) – Existing compatibility object basename.

• now (datetime.datetime) – Naive UTC scheduler clock.

Returns

Latest legacy upload/copy timestamp, or None when absent.

Return type

Optional[datetime.datetime]

backup_databases(client, bucket, registry, now=None, maintenance_id=None)

Back up due, exclusively opened databases once per two days at most.

The gate precedes hashing, snapshotting, upload and compatibility copy. Snapshot keys are immutable. Unchanged due databases update the check receipt without transferring bytes. Errors are observable and throttled too; other registered databases are attempted independently. No local file is deleted.

Parameters

• client (Any) – Authenticated R2 client with conditional write support.

• bucket (str) – Private data bucket only.

• registry (list[eth_defi.vault.duckdb_backup.DuckDBBackup]) – Explicit logical database registry.

• now (Optional[datetime.datetime]) – Optional injected naive UTC time for focused tests.

• maintenance_id (Optional[str]) – Named reservation owner; routine exporters pass None.

Returns

Per-database missing, deferred, unchanged, uploaded or failed.

Return type

dict[str, str]

reserve_backup_window(client, bucket, reservation_id, names, *, action='reserve', now=None)

Reserve a shared daily/HF maintenance window without scheduler starvation.

A bucket-wide conditional receipt has one active named reservation. Routine exporters defer its databases while it is active; the named maintenance exporter still observes every database’s original 48-hour gate. Heartbeats renew a 72-hour lease; cancel releases it without resetting backup clocks.

Parameters

• client (Any) – R2 client.

• bucket (str) – Private bucket.

• reservation_id (str) – Operator-supplied maintenance name.

• names (list[str]) – Logical database identities reserved together.

• action (str) – reserve, heartbeat or cancel.

• now (Optional[datetime.datetime]) – Optional naive UTC test clock.

Returns

Reservation receipt with the earliest common backup time.

Return type

dict