tokenised_fund.wisdomtree.backfill

Documentation for eth_defi.tokenised_fund.wisdomtree.backfill Python module.

Backfill only WisdomTree WTGXX metadata and price history.

This migration never removes an Ethereum-wide scan range. It upserts only the WTGXX lead and metadata row, removes only its reader state before scanning, and passes WTGXX as vault_addresses to the shared Parquet scanner. Cleaned history replacement is likewise restricted to its one VaultSpec id.

Run with:

source .local-test.env && WISDOMTREE_DATASPAN_API_KEY=... PROTOCOLS=wisdomtree poetry run python scripts/backfill-tokenised-funds.py

Functions

create_detection(deployment_block, first_seen_at)	Create WTGXX routing data without an Ethereum-wide event replay.
create_lead(deployment_block, first_seen_at)	Create the one reviewed WTGXX lead.
main()	Read configuration and execute the scoped migration.
parse_bool_env(name, *[, default])	Parse a conventional boolean environment value.
parse_path_env(name, default)	Read an optional output-path override.
read_reader_states(path)	Read trusted local reader states.
read_vault_database(path)	Read existing metadata without discarding unrelated rows.
remove_selected_reader_states(states)	Remove WTGXX state while preserving every unrelated reader state.
require_price_scan_key()	Fail closed before history writes if issuer NAV cannot be authenticated.
resolve_start_block()	Choose the earliest WTGXX history start block.
run_backfill(*, dry_run, scan_prices, ...)	Run the complete address-scoped WTGXX migration.
selected_vault_addresses()	Return the only addresses this migration may alter.
selected_vault_spec_ids()	Return the only cleaned-history ids this migration may replace.
upsert_selected_metadata(vault_db, *, ...)	Upsert WTGXX metadata without changing the Ethereum discovery watermark.
write_reader_states(path, states)	Atomically persist the complete preserved state mapping.

parse_bool_env(name, *, default=False)

Parse a conventional boolean environment value.

Parameters

• name (str) – Environment variable name.

• default (bool) – Value when the variable is absent.

Returns

Parsed boolean value.

Return type

bool

parse_path_env(name, default)

Read an optional output-path override.

Parameters

• name (str) – Environment variable name.

• default (pathlib.Path) – Production default path.

Returns

Resolved configured path.

Return type

pathlib.Path

selected_vault_addresses()

Return the only addresses this migration may alter.

Returns

Singleton lower-case WTGXX address set.

Return type

set[str]

selected_vault_spec_ids()

Return the only cleaned-history ids this migration may replace.

Returns

Singleton WTGXX vault spec id set.

Return type

set[str]

require_price_scan_key()

Fail closed before history writes if issuer NAV cannot be authenticated.

Raises

RuntimeError – If the documented DataSpan API key is missing.

Return type

None

read_vault_database(path)

Read existing metadata without discarding unrelated rows.

Parameters

path (pathlib.Path) – Metadata pickle path.

Returns

Existing or empty database.

Return type

eth_defi.vault.vaultdb.VaultDatabase

read_reader_states(path)

Read trusted local reader states.

Parameters

path (pathlib.Path) – Reader-state pickle path.

Returns

Complete state mapping, or empty mapping.

Return type

dict[eth_defi.vault.base.VaultSpec, dict]

remove_selected_reader_states(states)

Remove WTGXX state while preserving every unrelated reader state.

Parameters

states (dict[eth_defi.vault.base.VaultSpec, dict]) – Existing complete reader-state mapping.

Returns

Mapping without the selected WTGXX state.

Return type

dict[eth_defi.vault.base.VaultSpec, dict]

write_reader_states(path, states)

Atomically persist the complete preserved state mapping.

Parameters

• path (pathlib.Path) – Reader-state pickle path.

• states (dict[eth_defi.vault.base.VaultSpec, dict]) – Complete state mapping.

Return type

None

create_detection(deployment_block, first_seen_at)

Create WTGXX routing data without an Ethereum-wide event replay.

Parameters

• deployment_block (int) – First block with WTGXX proxy code.

• first_seen_at (datetime.datetime) – Corresponding naive UTC timestamp.

Returns

Scanner-compatible detection.

Return type

eth_defi.erc_4626.core.ERC4262VaultDetection

create_lead(deployment_block, first_seen_at)

Create the one reviewed WTGXX lead.

Parameters

• deployment_block (int) – First block with WTGXX proxy code.

• first_seen_at (datetime.datetime) – Corresponding naive UTC timestamp.

Returns

WTGXX lead.

Return type

eth_defi.erc_4626.discovery_base.PotentialVaultMatch

upsert_selected_metadata(vault_db, *, end_block, row)

Upsert WTGXX metadata without changing the Ethereum discovery watermark.

VaultDatabase.update_leads_and_rows() also maintains the ordinary chain-wide scan cursor. A one-address migration must leave that cursor untouched because it says nothing about unrelated Ethereum discovery.

Parameters

• vault_db (eth_defi.vault.vaultdb.VaultDatabase) – Existing metadata database to update.

• end_block (int) – Block used to build WTGXX metadata.

• row (dict) – WTGXX scan row.

Return type

None

resolve_start_block()

Choose the earliest WTGXX history start block.

HyperSync fills any missing timestamp cache entries, so the scan starts at deployment unless the operator explicitly supplies START_BLOCK.

Returns

Explicit override or the WTGXX deployment block.

Return type

int

run_backfill(*, dry_run, scan_prices, clean_prices, frequency, vault_db_path, raw_price_path, cleaned_price_path, reader_state_path)

Run the complete address-scoped WTGXX migration.

Parameters

• dry_run (bool) – Do not write metadata, states or Parquet rows.

• scan_prices (bool) – Rebuild WTGXX raw history.

• clean_prices (bool) – Replace only WTGXX cleaned history after raw scan.

• frequency (Literal['1h', '1d']) – Shared scanner sample frequency.

• vault_db_path (pathlib.Path) – Metadata database path.

• raw_price_path (pathlib.Path) – Raw price Parquet path.

• cleaned_price_path (pathlib.Path) – Cleaned price Parquet path.

• reader_state_path (pathlib.Path) – Reader-state pickle path.

Return type

None

main()

Read configuration and execute the scoped migration.

Return type

None