tokenised_fund.superstate.backfill

Documentation for eth_defi.tokenised_fund.superstate.backfill Python module.

Backfill the reviewed Superstate USTB lead and its NAV history.

This migration is intentionally address-scoped. It upserts only the Ethereum USTB lead and metadata row, removes only that reader state, and passes only USTB to the historical scanner. It never resets a chain cursor, re-discovers all Ethereum vaults, or deletes unrelated Parquet rows.

Run with:

source .local-test.env && PROTOCOLS=superstate poetry run python scripts/backfill-tokenised-funds.py

Environment variables:

DRY_RUN

Show the migration plan without writing files. Default: false.

SCAN_PRICES

Rebuild USTB’s daily NAV/share history. Default: true.

FREQUENCY

1d (default) or 1h historical sampling interval.

START_BLOCK / END_BLOCK

Optional inclusive bounds for a controlled repair.

VAULT_DB_PATH, UNCLEANED_PRICE_DATABASE, CLEANED_PRICE_DATABASE and READER_STATE_DATABASE

Optional state-path overrides for an isolated or production run.

Functions

create_detection()

Create the reviewed USTB scanner detection.

create_lead()

Create the USTB lead without scanning unrelated Ethereum events.

main()

Run the USTB-only lead and NAV-history migration.

parse_bool_env(name, *, default)

Read a conventional Boolean environment setting.

parse_path_env(name, default)

Read a local state-path override.

read_reader_states(path)

Load trusted state while retaining every unrelated vault entry.

resolve_frequency()

Validate the requested historical sample interval.

upsert_ustb_metadata(vault_db, row, end_block)

Upsert only USTB metadata without changing shared discovery cursors.

write_reader_states(path, reader_states)

Atomically persist the complete retained reader-state map.
parse_bool_env(name, *, default)

Read a conventional Boolean environment setting.

Parameters

  • name (str) – Environment variable name.

  • default (bool) – Value used when it is absent.

Returns

Parsed Boolean.

Return type

bool

parse_path_env(name, default)

Read a local state-path override.

Parameters

  • name (str) – Environment variable name.

  • default (pathlib.Path) – Normal production location.

Returns

Expanded configured path.

Return type

pathlib.Path

resolve_frequency()

Validate the requested historical sample interval.

Returns

Scanner frequency.

Raises

ValueError – If the requested frequency is unsupported.

Return type

Literal[‘1h’, ‘1d’]

create_detection()

Create the reviewed USTB scanner detection.

Returns

Hardcoded, non-event-derived USTB detection.

Return type

eth_defi.erc_4626.core.ERC4262VaultDetection

create_lead()

Create the USTB lead without scanning unrelated Ethereum events.

Returns

Reviewed USTB lead.

Return type

eth_defi.erc_4626.discovery_base.PotentialVaultMatch

read_reader_states(path)

Load trusted state while retaining every unrelated vault entry.

Parameters

path (pathlib.Path) – Reader-state pickle location.

Returns

Current state map or an empty map.

Return type

dict[eth_defi.vault.base.VaultSpec, dict]

write_reader_states(path, reader_states)

Atomically persist the complete retained reader-state map.

Parameters
Returns

None.

Return type

None

upsert_ustb_metadata(vault_db, row, end_block)

Upsert only USTB metadata without changing shared discovery cursors.

VaultDatabase.update_leads_and_rows() is the shared merge path, but it also advances the chain-wide discovery cursor. A targeted migration must not make unrelated Ethereum discovery skip blocks, so retain the exact pre-existing cursor value, including the case where no cursor was present, around this single-address upsert.

Parameters

  • vault_db (eth_defi.vault.vaultdb.VaultDatabase) – Existing shared vault database.

  • row (dict) – Fresh USTB scanner metadata row.

  • end_block (int) – Metadata read block supplied to the underlying merge operation.

Returns

None.

Return type

None

main()

Run the USTB-only lead and NAV-history migration.

Returns

None.

Return type

None