vault.rpc_scan_state

Documentation for eth_defi.vault.rpc_scan_state Python module.

Rollback-compatible sidecars for scanner retry and probe scheduling.

Critical reader, metadata and cycle-state schemas remain unchanged. These versioned JSON files can be ignored by an older release without losing history.

The all-chain scheduler and lead scanner own these sidecars under the pipeline writer lock. Retry/probe files are scheduling hints; verified qualification comes from reader history. The publication journal has a stronger role: it bridges the two separate price-file and reader-pickle publications so a process restart cannot replay already committed prices from an obsolete reader cursor.

Module Attributes

fetch_remaining_state_budget(boundary, ...)

Compatibility alias; this calculation uses persisted observations and makes no provider request.

Functions

calculate_remaining_state_budget(boundary, ...)	Estimate time before unread state is evicted from the measured window.
classify_rpc_scan_failure(error)	Separate retryable transport failures from deterministic code errors.
fetch_reader_publication_progress(...)	Recover committed local progress when legacy state persistence failed.
fetch_remaining_state_budget(boundary, ...)	Compatibility alias; this calculation uses persisted observations and makes no provider request.
is_contract_read_failure(error)	Recognise expected contract failures without hiding programmer errors.
is_metadata_due(entry, features, ...[, force])	Refresh metadata when protocol identity changes or a deadline arrives.
is_probe_due(entry, now, qualified)	Decide admission probing before constructing an adapter.
load_rpc_scan_state(path)	Load a versioned sidecar, never silently discard a damaged file.
record_chain_backoff(path, chain, category)	Persist next-attempt timing independently of successful scan timestamps.
record_metadata_failure(pending, ...)	Bound candidate retries and move persistent negatives out of the queue.
record_probe_result(entries, address, now, ...)	Store a bounded probe schedule and raw observation outcome.
save_reader_publication_journal(...)	Durably prepare reader progress before replacing the price file.
save_rpc_scan_state(path, entries)	Atomically publish sidecar progress under the caller's pipeline lock.

load_rpc_scan_state(path)

Load a versioned sidecar, never silently discard a damaged file.

A missing sidecar is normal on initial deployment or rollback.

Parameters

path (pathlib.Path) – Sidecar path under the mounted pipeline directory.

Returns

Mutable entry mapping; absent files start with an empty mapping.

Return type

dict

save_rpc_scan_state(path, entries)

Atomically publish sidecar progress under the caller’s pipeline lock.

Parent directories are created but historical state is never replaced.

Parameters

• path (pathlib.Path) – Destination sidecar path.

• entries (dict) – JSON-compatible scheduling/capability mapping.

Returns

None.

Return type

None

classify_rpc_scan_failure(error)

Separate retryable transport failures from deterministic code errors.

Contract reverts do not become whole-chain retry storms. Adapter version failures are handled per candidate before they reach this boundary.

Used by phase boundaries, admission batches and the scheduler’s retry gate. Production fallback providers wrap JSON-RPC failures in ExtraValueError; native Web3 error handling alone would miss those transport outcomes. Keep recognition narrow so unrelated ValueError/RuntimeError defects remain visible instead of being deferred indefinitely as provider outages.

Parameters

error (BaseException) – Original phase exception.

Returns

transient or internal.

Return type

str

record_chain_backoff(path, chain, category, now=None, retention_seconds=None)

Persist next-attempt timing independently of successful scan timestamps.

Monad’s observed state-retention margin can cap additional retry delay. A successful chain removes its backoff. Lock deferrals never call this.

Parameters

• path (pathlib.Path) – New backoff sidecar, never the legacy cycle-state JSON.

• chain (str) – Scheduler chain name.

• category (Optional[str]) – Failure category, or None after success.

• now (Optional[datetime.datetime]) – Naive UTC attempt time.

• retention_seconds (Optional[float]) – Observed remaining state window, when available.

Returns

None.

Return type

None

is_probe_due(entry, now, qualified)

Decide admission probing before constructing an adapter.

Previously meaningful vaults continue their ordinary historical scanning. A probe cache never overrides authoritative reader qualification.

Parameters

• entry (Optional[dict]) – Prior probe outcome or None for a new candidate.

• now (datetime.datetime) – Naive UTC scheduling time.

• qualified (bool) – Eligibility from authoritative verified reader history.

Returns

Whether a current admission probe is needed.

Return type

bool

record_probe_result(entries, address, now, first_seen_at, tvl_usd, error)

Store a bounded probe schedule and raw observation outcome.

No qualification flag is persisted. Errors have a finite retry deadline; previously tiny mature vaults are checked at least weekly.

Called only for admission candidates. Young vaults can receive their first deposit soon after deployment, so daily probes trade a few reads for timely inclusion. A current meaningful TVL accelerates the next probe, but cannot alone establish the reader-history qualification used by price scanning.

Parameters

• entries (dict) – Mutable sidecar entries keyed by vault address.

• address (eth_typing.evm.HexAddress) – Lower-case vault address.

• now (datetime.datetime) – Naive UTC observation/attempt time.

• first_seen_at (datetime.datetime) – Deployment/first activity time from detection data.

• tvl_usd (Optional[str]) – Verified USD observation as decimal text, or None.

• error (Optional[str]) – Explicit unavailable outcome or None after success.

Returns

None.

Return type

None

save_reader_publication_journal(journal_path, temporary_prices, output_prices, states)

Durably prepare reader progress before replacing the price file.

Rename preserves the temporary file’s inode, size and nanosecond mtime. These identify the exact local publication without hashing a large Parquet. An interrupted pre-publication receipt cannot match the previous file.

The Parquet writer calls this after validating/syncing its temporary file and before os.replace(). The scheduler recovers only a matching published inode, merges reader cursors monotonically, then writes the legacy pickle before consuming the receipt. This is local crash recovery, not a portable restore receipt: copied or subsequently replaced files need not match.

Parameters

• journal_path (pathlib.Path) – New sidecar beside the critical reader-state pickle.

• temporary_prices (pathlib.Path) – Verified and fsynced Parquet about to be renamed.

• output_prices (pathlib.Path) – Final Parquet path.

• states (dict) – Reader state already serialised using the legacy schema.

Returns

None.

Return type

None

fetch_reader_publication_progress(journal_path, output_prices)

Recover committed local progress when legacy state persistence failed.

The journal is usable only if the published file matches its exact local identity. A receipt from an unpublished or replaced output yields no state. Invalid receipts fail loudly rather than discard critical history.

Parameters

• journal_path (pathlib.Path) – Optional publication sidecar from the upgraded writer.

• output_prices (pathlib.Path) – Current price Parquet.

Returns

Legacy-shaped reader state ready to merge monotonically.

Return type

dict

is_metadata_due(entry, features, classifier_version, now, force=False)

Refresh metadata when protocol identity changes or a deadline arrives.

Missing provenance is refreshed once. Explicit operator refreshes bypass the sidecar without altering the event discovery cursor.

Parameters

• entry (Optional[dict]) – Last metadata observation or negative result.

• features (list[str]) – Current sorted classification feature names.

• classifier_version (str) – Current classification implementation hash.

• now (datetime.datetime) – Naive UTC scheduling time.

• force (bool) – Explicit operator refresh.

Returns

Whether adapter metadata must be read.

Return type

bool

record_metadata_failure(pending, observations, address, features, classifier_version, now, category)

Bound candidate retries and move persistent negatives out of the queue.

Unsupported versions and deterministic contract failures are checked weekly. Transient failures receive up to three daily attempts before weekly checks. New classification or explicit force still makes negatives immediately due.

Initial discovery and queue recovery share this transition. The bounded queue avoids repeatedly constructing a known unsupported adapter on every chain tick, while a negative observation with an expiry permits recovery after a contract upgrade or provider repair without replaying event history.

Parameters

• pending (dict) – Mutable resumable candidate queue.

• observations (dict) – Mutable metadata status catalogue.

• address (eth_typing.evm.HexAddress) – Lower-case vault address.

• features (list[str]) – Sorted current protocol features.

• classifier_version (str) – Current feature implementation hash.

• now (datetime.datetime) – Naive UTC failure time.

• category (str) – Redacted failure category, never provider error text.

Returns

None.

Return type

None

is_contract_read_failure(error)

Recognise expected contract failures without hiding programmer errors.

Providers vary between Solidity revert errors and plain JSON-RPC VM errors. Only known contract-execution markers qualify for per-candidate deferral.

Parameters

error (BaseException) – Original constructor/reader exception.

Returns

Whether the error describes unavailable contract execution/data.

Return type

bool

calculate_remaining_state_budget(boundary, committed_block, block_seconds, now)

Estimate time before unread state is evicted from the measured window.

A fresh capability observation does not reset the age of unscanned history. The estimate subtracts both head-to-committed-block lag and observation age.

Parameters

• boundary (dict) – Actual bounded provider-state observation.

• committed_block (int) – Last durably committed chain reader block.

• block_seconds (float) – Configured approximate block interval.

• now (datetime.datetime) – Naive UTC scheduling time.

Returns

Remaining seconds, possibly negative to expose an existing gap.

Return type

float

fetch_remaining_state_budget(boundary, committed_block, block_seconds, now)

Compatibility alias; this calculation uses persisted observations and makes no provider request. Scheduler callers use the explicit calculate_ name.

Parameters

• boundary (dict) –

• committed_block (int) –

• block_seconds (float) –

• now (datetime.datetime) –

Return type

float