hypersync.hypersync_timestamp

Documentation for eth_defi.hypersync.hypersync_timestamp Python module.

Block timestamp and hash bulk loading using Hypersync API.

Replace slow and expensive eth_getBlockByNumber calls with Hypersync API.

Example:

blocks = get_block_timestamps_using_hypersync(
    hypersync_client,
    chain_id=1,
    start_block=10_000_000,
    end_block=10_000_100,
)

# Includes blocks that do not contain transactions
# E.g https://etherscan.io/block/10000007
assert len(blocks) == 101

block = blocks[10_000_100]

assert block.block_number == 10_000_100
assert block.block_hash == "0x427b4ae39316c0df7ba6cd61a96bf668eff6e3ec01213b0fbc74f9b7a0726e7b"
assert block.timestamp_as_datetime == datetime.datetime(2020, 5, 4, 13, 45, 31)

Functions

fetch_block_timestamps_using_hypersync_cached(...)	Sync wrapper with retry and exponential backoff.
fetch_block_timestamps_using_hypersync_cached_async(...)	Quickly get block timestamps using Hypersync API and a local cache file.
fetch_exact_block_timestamps_using_hypersync_cached(*, ...)	Synchronously fetch irregular exact timestamps with retry and backoff.
fetch_exact_block_timestamps_using_hypersync_cached_async(*, ...)	Fetch selected block timestamps through Hypersync and the shared cache.
fetch_sparse_block_timestamps_using_hypersync_cached(...)	Synchronously fetch sparse sampled timestamps with retry/backoff.
fetch_sparse_block_timestamps_using_hypersync_cached_async(...)	Fetch a regular sparse timestamp sample through the shared cache.
fetch_timestamp_cache_headers_async(client, ...)	Fetch timestamp-only pages with a rate-limit slot for every request.
get_block_timestamps_using_hypersync(client, ...)	Quickly get block timestamps using Hypersync API.
get_block_timestamps_using_hypersync_async(...)	Read block timestamps using Hypersync API.
get_hypersync_block_height(client)	Get the latest block known to Hypersync.
get_hypersync_block_height_with_retries(client)	Get latest Hypersync block height with retry/backoff.
is_hypersync_next_block_range_error(e)	Check if a Hypersync stream failed on an internal pagination boundary.
is_hypersync_rate_limit_error(e)	Check if a Hypersync RuntimeError is a rate limit error.
is_hypersync_retryable_runtime_error(e)	Check if a Hypersync RuntimeError should be handled by retry logic.
raise_if_recoverable_hypersync_flaky(e, context)	Re-raise a recoverable Hypersync RuntimeError as HypersyncFlaky.

Classes

CachedBlockTimestamp

Block number and timestamp persisted in the timestamp cache.

Exceptions

HypersyncFlaky

Hypersync stream flaky error, e.g.

class CachedBlockTimestamp

Bases: object

Block number and timestamp persisted in the timestamp cache.

block_number: int

EVM block number.

timestamp: int

Unix timestamp in seconds.

__init__(block_number, timestamp)

Parameters

• block_number (int) –

• timestamp (int) –

Return type

None

exception HypersyncFlaky

Bases: Exception

Hypersync stream flaky error, e.g. timeout or rate limit.

__init__(*args, **kwargs)

__new__(**kwargs)

add_note(note, /)

Add a note to the exception

with_traceback(tb, /)

Set self.__traceback__ to tb and return self.

is_hypersync_rate_limit_error(e)

Check if a Hypersync RuntimeError is a rate limit error.

The Rust client raises RuntimeError after exhausting its internal retries. The rate limit can surface in two different textual forms depending on where in the client it is detected:

• As an HTTP status, e.g. ... 429 Too Many Requests ....

• As a server-side budget message wrapped inside a stream inner receiver error, e.g.:

  inner receiver
  Caused by:
    0: get initial data
    1: rate limited by server (remaining=0/100 reqs, resets_in=15s).
       To increase your rate limits, upgrade your plan at https://envio.dev/app/api-tokens
  

  This second form contains no 429 token, so we also match the rate limited by server wording. See the resets_in note in the retry callers: we deliberately do not parse it and instead rely on the caller’s fixed backoff (typically longer than the reset window).

Parameters

e (Exception) –

Return type

bool

is_hypersync_next_block_range_error(e)

Check if a Hypersync stream failed on an internal pagination boundary.

Some Hypersync backends can fail near the indexed chain head with an inner receiver error where next_block is at the lower boundary of the server-side subrange. Treat this as a flaky stream error so the caller can retry or wait for the backend to index more blocks.

Parameters

e (Exception) –

Return type

bool

is_hypersync_retryable_runtime_error(e)

Check if a Hypersync RuntimeError should be handled by retry logic.

Parameters

e (Exception) –

Return type

bool

raise_if_recoverable_hypersync_flaky(e, context)

Re-raise a recoverable Hypersync RuntimeError as HypersyncFlaky.

The Rust Hypersync client raises a bare RuntimeError for both server-side rate limiting and near-head pagination glitches. Wrapping them as HypersyncFlaky lets the caller’s retry/backoff loop recover instead of crashing the whole scan. Non-recoverable errors are left untouched so the caller can re-raise them with a bare raise.

Parameters

• context (str) – Where the error happened, included in the wrapped message, e.g. "stream setup [vault-prices]".

• e (RuntimeError) –

Return type

None

async get_block_timestamps_using_hypersync_async(client, chain_id, start_block, end_block, timeout=120.0, display_progress=True, progress_throttle=10000, validate_chain_id=True, reason=None)

Read block timestamps using Hypersync API.

Instead of hammering eth_getBlockByNumber JSON-RPC endpoint, we can get block timestamps using Hypersync API 1000x faster.

Parameters

• chain_id (int) – Expected chain ID. Validated against the client unless validate_chain_id is False.

• start_block (int) – Start block, inclusive

• end_block (int) – End block, inclusive

• client (hypersync.HypersyncClient) – Hypersync client to use

• validate_chain_id (bool) – When True (default), verify the client is connected to the expected chain before streaming. Set to False when the caller has already validated (e.g. the cached path).

• reason (Optional[str]) – Human-readable label for this request, included in log and error messages to help track which caller is consuming API quota.

• timeout (float) –

• display_progress (bool) –

Return type

collections.abc.AsyncIterable[eth_defi.event_reader.block_header.BlockHeader]

get_block_timestamps_using_hypersync(client, chain_id, start_block, end_block, display_progress=True)

Quickly get block timestamps using Hypersync API.

Wraps get_block_timestamps_using_hypersync_async().

You want to use fetch_block_timestamps_using_hypersync_cached() cached version.

Returns

Block number -> header mapping

Parameters

• client (hypersync.HypersyncClient) –

• chain_id (int) –

• start_block (int) –

• end_block (int) –

• display_progress (bool) –

Return type

dict[eth_typing.evm.BlockNumber, eth_defi.event_reader.block_header.BlockHeader]

async fetch_timestamp_cache_headers_async(client, chain_id, start_block, end_block, *, display_progress=True, validate_chain_id=True, reason=None)

Fetch timestamp-only pages with a rate-limit slot for every request.

Cached timestamps do not need block hashes. Request only numbers and timestamps, and advance through explicit pages so the Rust stream cannot make unthrottled internal requests. include_all_blocks also returns empty blocks. See the HyperSync query reference.

Parameters

• client (Union[hypersync.HypersyncClient, eth_defi.hypersync.session.ThrottledHypersyncClient]) – Configured HyperSync client for this chain.

• chain_id (int) – Expected EVM chain identifier.

• start_block (int) – Inclusive first block.

• end_block (int) – Inclusive last block.

• display_progress (bool) – Display page progress when true.

• validate_chain_id (bool) – Verify the chain before reading when true.

• reason (Optional[str]) – Context for provider errors.

Returns

Actual block numbers and Unix timestamps returned by HyperSync.

Return type

collections.abc.AsyncIterable[eth_defi.hypersync.hypersync_timestamp.CachedBlockTimestamp]

get_hypersync_block_height(client)

Get the latest block known to Hypersync.

Wrapped around the async function.

Parameters

client (hypersync.HypersyncClient) –

Return type

int

get_hypersync_block_height_with_retries(client, attempts=3, retry_sleep=30, reason='block-height-check')

Get latest Hypersync block height with retry/backoff.

Hypersync height checks are one-shot API calls and can hit the same 429 rate limits as streams. Use this helper when a caller needs a height check before opening a stream.

Parameters

• client (hypersync.HypersyncClient) – Hypersync client.

• attempts (int) – Maximum number of attempts before raising the last HypersyncFlaky.

• retry_sleep (int) – Sleep time between attempts, in seconds.

• reason (str) – Human-readable operation label for logs.

Returns

Latest block number known to Hypersync.

Return type

int

async fetch_block_timestamps_using_hypersync_cached_async(client, chain_id, start_block, end_block, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, chunk_size=100000)

Quickly get block timestamps using Hypersync API and a local cache file.

• Uses timestamp-only Hypersync pages and a persistent DuckDB cache.

• Large ranges are split into chunks of chunk_size blocks. Each HTTP page acquires its own rate-limit slot, including truncated responses. Progress is saved after each chunk so a provider failure retains earlier completed chunks. Nearby interior holes are fetched together.

Parameters

• chunk_size (int) – Maximum number of blocks in one durable cache chunk. Defaults to 100 000 (~2 days on Polygon, ~3 days on Binance).

• client (hypersync.HypersyncClient) –

• chain_id (int) –

• start_block (int) –

• end_block (int) –

• display_progress (bool) –

Returns

Block number -> datetime mapping

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer

fetch_block_timestamps_using_hypersync_cached(client, chain_id, start_block, end_block, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, attempts=5, chunk_size=100000)

Sync wrapper with retry and exponential backoff.

See fetch_block_timestamps_using_hypersync_cached_async() for documentation.

Parameters

• attempts – Work around Hypersync timeout issues

• chunk_size (int) – Maximum block range committed as one cache chunk. Each page within the chunk is fetched through the configured request limiter.

• client (hypersync.HypersyncClient) –

• chain_id (int) –

• start_block (int) –

• end_block (int) –

• display_progress (bool) –

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer

async fetch_exact_block_timestamps_using_hypersync_cached_async(*, client, chain_id, block_numbers, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, checkpoint_frequency=25, max_concurrency=5)

Fetch selected block timestamps through Hypersync and the shared cache.

The caller may supply an irregular set of event blocks. The function anti-joins those exact blocks against the persistent DuckDB cache, fetches only misses, and checkpoints them incrementally without filling the gaps between distant events. See Envio’s stream configuration documentation.

Parameters

• client (Union[hypersync.HypersyncClient, eth_defi.hypersync.session.ThrottledHypersyncClient]) – Hypersync client for the requested chain.

• chain_id (int) – Expected EVM chain id.

• block_numbers (collections.abc.Iterable[int]) – Exact non-negative EVM blocks whose timestamps are required.

• cache_path (pathlib.Path) – Shared per-chain timestamp cache directory.

• display_progress (bool) – Whether to display sampled timestamp progress.

• checkpoint_frequency (int) – Maximum number of new timestamps held before a durable cache write.

• max_concurrency (int) – Maximum one-block streams awaited together. The shared client limiter still governs total request starts across the batch.

Returns

Cache-backed timestamp slicer containing every indexed requested block.

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer

async fetch_sparse_block_timestamps_using_hypersync_cached_async(client, chain_id, start_block, end_block, step, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, checkpoint_frequency=25, max_concurrency=5)

Fetch a regular sparse timestamp sample through the shared cache.

Historical state readers may sample one block per hour or day from chains that produce millions of blocks per month. The exact-block implementation prevents the cache warm-up from reading intervening headers.

Parameters

• client (hypersync.HypersyncClient) – Hypersync client for the requested chain.

• chain_id (int) – Expected EVM chain identifier.

• start_block (int) – First sampled block, inclusive.

• end_block (int) – Historical reader end block, exclusive.

• step (int) – Number of blocks between historical samples.

• cache_path – Shared per-chain timestamp-cache directory.

• display_progress (bool) – Whether to display sampled timestamp progress.

• checkpoint_frequency (int) – Maximum new timestamps held before a durable cache write.

• max_concurrency (int) – Maximum exact-block streams awaited together.

Returns

Cache-backed timestamp slicer containing every indexed sample.

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer

fetch_exact_block_timestamps_using_hypersync_cached(*, client, chain_id, block_numbers, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, attempts=5)

Synchronously fetch irregular exact timestamps with retry and backoff.

Completed cache checkpoints survive retries, so only unresolved event blocks are requested after a transient Hypersync failure.

Parameters

• client (Union[hypersync.HypersyncClient, eth_defi.hypersync.session.ThrottledHypersyncClient]) – Hypersync client for the requested chain.

• chain_id (int) – Expected EVM chain identifier.

• block_numbers (collections.abc.Iterable[int]) – Exact non-negative EVM blocks whose timestamps are required.

• cache_path (pathlib.Path) – Shared per-chain timestamp-cache directory.

• display_progress (bool) – Whether to display exact-block progress.

• attempts (int) – Maximum Hypersync attempts.

Returns

Cache-backed timestamp slicer containing every indexed requested block.

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer

fetch_sparse_block_timestamps_using_hypersync_cached(client, chain_id, start_block, end_block, step, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, attempts=5)

Synchronously fetch sparse sampled timestamps with retry/backoff.

The regular range is delegated to the exact-block cache collector so intervening headers are never fetched.

Parameters

• attempts (int) – Maximum Hypersync attempts; completed checkpoints survive retries.

• client (hypersync.HypersyncClient) –

• chain_id (int) –

• start_block (int) –

• end_block (int) –

• step (int) –

• display_progress (bool) –

Returns

Cache-backed timestamp slicer containing every requested sample.

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer