tokenised_fund.asseto.backfill

Documentation for eth_defi.tokenised_fund.asseto.backfill Python module.

Backfill historical Asseto vault data into the shared vault pipeline.

This is a targeted production tool for the EVM products currently published by Asseto’s public product registry. It does not rediscover whole chains and updates only the selected Asseto vault identifiers. Historical rows combine on-chain ERC-20 supply with Asseto’s verified Pricer NAV/share where available, or its public daily display-NAV history for other products.

Usage:

source .local-test.env
export JSON_RPC_ETHEREUM="https://your-archive-ethereum-rpc"
PROTOCOLS=asseto poetry run python scripts/backfill-tokenised-funds.py

Useful environment variables:

Variable	Description
`DRY_RUN`	If `true`, print and validate the planned work without database writes.
`NETWORKS`	Optional comma-separated chain ids or known names, e.g. `177,hashkey`.
`JSON_RPC_<CHAIN>`	Archive-capable RPC URL for each selected supported chain, e.g. `JSON_RPC_ETHEREUM`. Products are skipped when this is not set.
`HYPERSYNC_API_KEY`	Required for cached HyperSync block-timestamp reads during price backfill.
`PRODUCTS`	Optional comma-separated Asseto symbols, e.g. `AoABT`.
`ASSETO_SCAN_PRICES`	If `false`, update only metadata. Default: `true`.
`ASSETO_CLEAN_PRICES`	If `true`, replace only selected histories in cleaned prices. Default: `true`.
`MAX_WORKERS`	Historical multicall worker count. Default: `8`.
`FREQUENCY`	Historical price frequency. Asseto supports daily `1d` samples only.
`START_BLOCK` / `END_BLOCK`	Optional global scan range overrides.
`VAULT_DB_PATH` / `UNCLEANED_PRICE_DATABASE`	Optional production database path overrides.
`CLEANED_PRICE_DATABASE` / `READER_STATE_DATABASE`	Optional cleaned price and reader-state path overrides.

The backfill removes stale reader state only for selected Asseto vaults. This is required because the targeted scanner replaces those rows from its explicit start block onwards; retained later state would otherwise skip the rewrite. Only chains registered by the shared project registry and supported by HyperSync are included. Asseto products on unsupported chains, such as HashKey or Pharos until they are added to the project, are reported and skipped.

Functions

`backfill_chain`(chain_id, products, *, ...)	Backfill one Asseto EVM chain.
`build_vaults`(web3, products, token_cache)	Build Asseto vault adapters and attach deployment block hints.
`create_asseto_detection`(product)	Create a synthetic shared scanner detection for an Asseto product.
`create_asseto_lead`(product)	Create a synthetic discovery lead for an Asseto product.
`create_runtime_product`(product, ...)	Convert one public registry entry to a temporary scanner product.
`fetch_contract_deployment_block`(web3, ...)	Find the first block containing runtime code for an Asseto token.
`get_asseto_rpc_env`(chain_id)	Return the normal RPC environment variable for a supported chain.
`get_chain_selector_names`(chain_id)	Return accepted `NETWORKS` selector names for a chain.
`is_supported_asseto_chain`(chain_id)	Check whether a chain is eligible for the shared Asseto backfill.
`iter_selected_products`()	Iterate current eligible Asseto registry products filtered by environment.
`main`()	Run the targeted Asseto historical backfill.
`parse_bool_env`(name, *[, default])	Parse a boolean environment variable.
`parse_csv_env`(name)	Parse a comma-separated environment variable.
`parse_optional_int_env`(name)	Parse an optional integer environment variable.
`parse_path_env`(name, default)	Parse an optional filesystem path environment variable.
`read_asseto_json_rpc_url`(chain_id)	Read the script-local Asseto JSON-RPC URL from its environment variable.
`read_reader_states`(path)	Read trusted local historical reader states.
`read_vault_database`(path)	Read or initialise the vault metadata database.
`resolve_frequency`()	Resolve the daily historical sampling frequency for Asseto.
`resolve_price_scan_start_block`(products)	Resolve the earliest history start for selected Asseto products.
`select_cleanable_vault_ids`(products, rows)	Select Asseto histories supported by the stablecoin-only cleaner.
`write_reader_states`(path, states)	Atomically write historical reader state.

parse_bool_env(name, *, default=False)

Parse a boolean environment variable.

Parameters

name (str) – Environment variable name.
default (bool) – Value returned when the variable is unset.

Returns

Parsed boolean value.

Return type

bool

parse_csv_env(name)

Parse a comma-separated environment variable.

Parameters: name (str) – Environment variable name.
Returns: Lowercase values, or None when unset.
Return type: Optional[set[str]]

parse_optional_int_env(name)

Parse an optional integer environment variable.

Parameters: name (str) – Environment variable name.
Returns: Integer value, or None when unset.
Return type: Optional[int]

parse_path_env(name, default)

Parse an optional filesystem path environment variable.

Parameters

name (str) – Environment variable name.
default (pathlib.Path) – Default production path.

Returns

Expanded selected path.

Return type

pathlib.Path

resolve_frequency()

Resolve the daily historical sampling frequency for Asseto.

Asseto’s public NAV history contains daily observations. The scanner must therefore not generate artificial hourly rows by carrying a daily value forward between price publications.

Returns: The required daily sampling interval.
Raises: ValueError – If an hourly or other unsupported frequency is requested.
Return type: Literal[‘1h’, ‘1d’]

get_chain_selector_names(chain_id)

Return accepted NETWORKS selector names for a chain.

Parameters: chain_id (int) – EVM chain id.
Returns: Numeric and configured textual selector values.
Return type: set[str]

is_supported_asseto_chain(chain_id)

Check whether a chain is eligible for the shared Asseto backfill.

Parameters: chain_id (int) – Asseto product chain id.
Returns: True when the project registers the chain and HyperSync supports it.
Return type: bool

get_asseto_rpc_env(chain_id)

Return the normal RPC environment variable for a supported chain.

Parameters: chain_id (int) – Asseto product chain id.
Returns: The project-standard JSON_RPC_<CHAIN> environment variable.
Return type: str

read_asseto_json_rpc_url(chain_id)

Read the script-local Asseto JSON-RPC URL from its environment variable.

Parameters: chain_id (int) – Asseto product chain id.
Returns: Configured archive-capable RPC URL for a supported chain.
Raises: ValueError – If the standard RPC variable is unset.
Return type: str

iter_selected_products()

Iterate current eligible Asseto registry products filtered by environment.

Returns: Unique registry products on supported chains with configured RPC URLs.
Return type: collections.abc.Iterable[eth_defi.tokenised_fund.asseto.offchain_api.AssetoOffchainProduct]

resolve_price_scan_start_block(products)

Resolve the earliest history start for selected Asseto products.

The normal scanner’s incremental reader state must not decide the start of a targeted rewrite. Begin at the earliest Asseto deployment unless the operator supplied START_BLOCK. HyperSync populates any missing block timestamps, so an existing local timestamp cache must never truncate the requested history.

Parameters: products (list[eth_defi.tokenised_fund.asseto.constants.AssetoProduct]) – Selected products on one EVM chain.
Returns: Explicit override or the earliest deployment block.
Return type: int

create_asseto_detection(product)

Create a synthetic shared scanner detection for an Asseto product.

Parameters: product (eth_defi.tokenised_fund.asseto.constants.AssetoProduct) – Asseto product metadata.
Returns: Detection compatible with metadata scan record generation.
Return type: eth_defi.erc_4626.core.ERC4262VaultDetection

create_asseto_lead(product)

Create a synthetic discovery lead for an Asseto product.

Parameters: product (eth_defi.tokenised_fund.asseto.constants.AssetoProduct) – Asseto product metadata.
Returns: Lead compatible with the vault metadata database.
Return type: eth_defi.erc_4626.discovery_base.PotentialVaultMatch

read_vault_database(path)

Read or initialise the vault metadata database.

Parameters: path (pathlib.Path) – Vault metadata database path.
Returns: Existing or empty database.
Return type: eth_defi.vault.vaultdb.VaultDatabase

read_reader_states(path)

Read trusted local historical reader states.

Parameters: path (pathlib.Path) – Reader-state pickle file path.
Returns: Existing states, or an empty mapping when the file does not exist.
Return type: dict[eth_defi.vault.base.VaultSpec, dict]

write_reader_states(path, states)

Atomically write historical reader state.

Parameters

path (pathlib.Path) – Reader-state pickle file path.
states (dict[eth_defi.vault.base.VaultSpec, dict]) – Reader state mapping returned by the historical scanner.

Return type

None

build_vaults(web3, products, token_cache)

Build Asseto vault adapters and attach deployment block hints.

Parameters

web3 (web3.main.Web3) – Connected chain Web3 instance.
products (list[eth_defi.tokenised_fund.asseto.constants.AssetoProduct]) – Selected products on this chain.
token_cache (eth_defi.token.TokenDiskCache) – Shared token metadata cache.

Returns

Configured Asseto vault adapters.

Return type

list[eth_defi.vault.base.VaultBase]

select_cleanable_vault_ids(products, rows)

Select Asseto histories supported by the stablecoin-only cleaner.

Asseto publishes products denominated in currencies such as HKD. Their raw NAV histories are valid, but the shared cleaned-price pipeline deliberately filters out denominations not recognised by is_stablecoin_like(). Passing those identifiers to the strict replacement helper would make it interpret the expected filtering as accidental data loss.

Parameters

products (collections.abc.Iterable[eth_defi.tokenised_fund.asseto.constants.AssetoProduct]) – Asseto products whose raw histories were scanned.
rows (dict[eth_defi.vault.base.VaultSpec, dict]) – Fresh vault metadata rows keyed by vault specification.

Returns

Canonical vault identifiers eligible for cleaned-history replacement.

Return type

set[str]

fetch_contract_deployment_block(web3, address, end_block)

Find the first block containing runtime code for an Asseto token.

Parameters

web3 (web3.main.Web3) – Archive-capable connection for the product chain.
address (eth_typing.evm.HexAddress) – Asseto ERC-20 token address.
end_block (int) – Highest block that may be checked.

Returns

First block containing contract code.

Raises

ValueError – If the public registry address has no contract code on this chain.

Return type

int

create_runtime_product(product, deployment_block, first_seen_at)

Convert one public registry entry to a temporary scanner product.

Public registry products do not all expose Asseto’s request/claim manager and Pricer contracts. The generic adapter therefore uses their published daily NAV history, while preserving the exact token identity and deployment information read from the configured archive RPC.

Parameters

product (eth_defi.tokenised_fund.asseto.offchain_api.AssetoOffchainProduct) – Asseto public EVM product entry.
deployment_block (int) – First token-code block found on-chain.
first_seen_at (datetime.datetime) – Naive UTC deployment timestamp.

Returns

Runtime Asseto adapter product metadata.

Return type

eth_defi.tokenised_fund.asseto.constants.AssetoProduct

backfill_chain(chain_id, products, *, dry_run, scan_prices, clean_prices, frequency, vault_db, vault_db_path, price_database_path, cleaned_price_database_path, reader_state_database_path, token_cache)

Backfill one Asseto EVM chain.

Metadata and historical writes use the same shared database and scanner paths as the normal vault pipeline, but both are limited to selected Asseto identifiers.

Parameters

chain_id (int) – EVM chain id.
products (list[eth_defi.tokenised_fund.asseto.offchain_api.AssetoOffchainProduct]) – Selected Asseto products on this chain.
dry_run (bool) – Whether filesystem/database writes are disabled.
scan_prices (bool) – Whether to scan supply, NAV, and TVL history.
clean_prices (bool) – Whether to replace selected cleaned price histories.
frequency (Literal['1h', '1d']) – Historical sampling interval.
vault_db (eth_defi.vault.vaultdb.VaultDatabase) – In-memory metadata database.
vault_db_path (pathlib.Path) – Metadata database output path.
price_database_path (pathlib.Path) – Uncleaned historical price parquet path.
cleaned_price_database_path (pathlib.Path) – Cleaned historical price parquet path.
reader_state_database_path (pathlib.Path) – Reader-state pickle path.
token_cache (eth_defi.token.TokenDiskCache) – Shared token metadata cache.

Returns

Summary row for operator output.

Return type

dict[str, object]

main()

Run the targeted Asseto historical backfill.

Return type: None