hypersync.hypersync_timestamp

Documentation for eth_defi.hypersync.hypersync_timestamp Python module.

Block timestamp and hash bulk loading using Hypersync API.

Replace slow and expensive eth_getBlockByNumber calls with Hypersync API.

Example:

blocks = get_block_timestamps_using_hypersync(
    hypersync_client,
    chain_id=1,
    start_block=10_000_000,
    end_block=10_000_100,
)

# Blocks missing if they do not contain transactions
# E.g https://etherscan.io/block/10000007
assert len(blocks) == 101

block = blocks[10_000_100]

assert block.block_number == 10_000_100
assert block.block_hash == "0x427b4ae39316c0df7ba6cd61a96bf668eff6e3ec01213b0fbc74f9b7a0726e7b"
assert block.timestamp_as_datetime == datetime.datetime(2020, 5, 4, 13, 45, 31)

Functions

fetch_block_timestamps_using_hypersync_cached(...)

Sync wrapper with retry and exponential backoff.

fetch_block_timestamps_using_hypersync_cached_async(...)

Quickly get block timestamps using Hypersync API and a local cache file.

fetch_sparse_block_timestamps_using_hypersync_cached(...)

Synchronously fetch sparse sampled timestamps with retry/backoff.

fetch_sparse_block_timestamps_using_hypersync_cached_async(...)

Fetch only exact sampled timestamps through Hypersync and the shared cache.

get_block_timestamps_using_hypersync(client, ...)

Quickly get block timestamps using Hypersync API.

get_block_timestamps_using_hypersync_async(...)

Read block timestamps using Hypersync API.

get_hypersync_block_height(client)

Get the latest block known to Hypersync.

get_hypersync_block_height_with_retries(client)

Get latest Hypersync block height with retry/backoff.

is_hypersync_next_block_range_error(e)

Check if a Hypersync stream failed on an internal pagination boundary.

is_hypersync_rate_limit_error(e)

Check if a Hypersync RuntimeError is a rate limit error.

is_hypersync_retryable_runtime_error(e)

Check if a Hypersync RuntimeError should be handled by retry logic.

raise_if_recoverable_hypersync_flaky(e, context)

Re-raise a recoverable Hypersync RuntimeError as HypersyncFlaky.

Exceptions

HypersyncFlaky

Hypersync stream flaky error, e.g.
exception HypersyncFlaky

Bases: Exception

Hypersync stream flaky error, e.g. timeout or rate limit.

__init__(*args, **kwargs)
__new__(**kwargs)
add_note(note, /)

Add a note to the exception

with_traceback(tb, /)

Set self.__traceback__ to tb and return self.

is_hypersync_rate_limit_error(e)

Check if a Hypersync RuntimeError is a rate limit error.

The Rust client raises RuntimeError after exhausting its internal retries. The rate limit can surface in two different textual forms depending on where in the client it is detected:

  • As an HTTP status, e.g. ... 429 Too Many Requests ....

  • As a server-side budget message wrapped inside a stream inner receiver error, e.g.:

    inner receiver
Caused by:
  0: get initial data
  1: rate limited by server (remaining=0/100 reqs, resets_in=15s).
     To increase your rate limits, upgrade your plan at https://envio.dev/app/api-tokens

    This second form contains no 429 token, so we also match the rate limited by server wording. See the resets_in note in the retry callers: we deliberately do not parse it and instead rely on the caller’s fixed backoff (typically longer than the reset window).

Parameters

e (Exception) –

Return type

bool

is_hypersync_next_block_range_error(e)

Check if a Hypersync stream failed on an internal pagination boundary.

Some Hypersync backends can fail near the indexed chain head with an inner receiver error where next_block is at the lower boundary of the server-side subrange. Treat this as a flaky stream error so the caller can retry or wait for the backend to index more blocks.

Parameters

e (Exception) –

Return type

bool

is_hypersync_retryable_runtime_error(e)

Check if a Hypersync RuntimeError should be handled by retry logic.

Parameters

e (Exception) –

Return type

bool

raise_if_recoverable_hypersync_flaky(e, context)

Re-raise a recoverable Hypersync RuntimeError as HypersyncFlaky.

The Rust Hypersync client raises a bare RuntimeError for both server-side rate limiting and near-head pagination glitches. Wrapping them as HypersyncFlaky lets the caller’s retry/backoff loop recover instead of crashing the whole scan. Non-recoverable errors are left untouched so the caller can re-raise them with a bare raise.

Parameters

  • context (str) – Where the error happened, included in the wrapped message, e.g. "stream setup [vault-prices]".

  • e (RuntimeError) –

Return type

None

async get_block_timestamps_using_hypersync_async(client, chain_id, start_block, end_block, timeout=120.0, display_progress=True, progress_throttle=10000, validate_chain_id=True, reason=None)

Read block timestamps using Hypersync API.

Instead of hammering eth_getBlockByNumber JSON-RPC endpoint, we can get block timestamps using Hypersync API 1000x faster.

Parameters

  • chain_id (int) – Expected chain ID. Validated against the client unless validate_chain_id is False.

  • start_block (int) – Start block, inclusive

  • end_block (int) – End block, inclusive

  • client (hypersync.HypersyncClient) – Hypersync client to use

  • validate_chain_id (bool) – When True (default), verify the client is connected to the expected chain before streaming. Set to False when the caller has already validated (e.g. the cached path).

  • reason (Optional[str]) – Human-readable label for this request, included in log and error messages to help track which caller is consuming API quota.

  • timeout (float) –

  • display_progress (bool) –

Return type

AsyncIterable[eth_defi.event_reader.block_header.BlockHeader]

get_block_timestamps_using_hypersync(client, chain_id, start_block, end_block, display_progress=True)

Quickly get block timestamps using Hypersync API.

Wraps get_block_timestamps_using_hypersync_async().

You want to use fetch_block_timestamps_using_hypersync_cached() cached version.

Returns

Block number -> header mapping

Parameters

  • client (hypersync.HypersyncClient) –

  • chain_id (int) –

  • start_block (int) –

  • end_block (int) –

  • display_progress (bool) –

Return type

dict[eth_typing.evm.BlockNumber, eth_defi.event_reader.block_header.BlockHeader]

get_hypersync_block_height(client)

Get the latest block known to Hypersync.

Wrapped around the async function.

Parameters

client (hypersync.HypersyncClient) –

Return type

int

get_hypersync_block_height_with_retries(client, attempts=3, retry_sleep=30, reason='block-height-check')

Get latest Hypersync block height with retry/backoff.

Hypersync height checks are one-shot API calls and can hit the same 429 rate limits as streams. Use this helper when a caller needs a height check before opening a stream.

Parameters

  • client (hypersync.HypersyncClient) – Hypersync client.

  • attempts (int) – Maximum number of attempts before raising the last HypersyncFlaky.

  • retry_sleep (int) – Sleep time between attempts, in seconds.

  • reason (str) – Human-readable operation label for logs.

Returns

Latest block number known to Hypersync.

Return type

int

async fetch_block_timestamps_using_hypersync_cached_async(client, chain_id, start_block, end_block, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, chunk_size=100000)

Quickly get block timestamps using Hypersync API and a local cache file.

  • Ultra fast, used optimised Hypersync streaming and DuckDB local cache.

  • Large ranges are split into chunks of chunk_size blocks so that each chunk opens a separate Hypersync stream() call. This keeps individual requests small, lets the Python-side rate limiter pace them, and — crucially — saves progress after each chunk so that a 429 failure only loses the current chunk, not all prior work.

Parameters

  • chunk_size (int) – Maximum number of blocks per Hypersync streaming request. Defaults to 100 000 (~2 days on Polygon, ~3 days on Binance).

  • client (hypersync.HypersyncClient) –

  • chain_id (int) –

  • start_block (int) –

  • end_block (int) –

  • display_progress (bool) –

Returns

Block number -> datetime mapping

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer

fetch_block_timestamps_using_hypersync_cached(client, chain_id, start_block, end_block, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, attempts=5)

Sync wrapper with retry and exponential backoff.

See fetch_block_timestamps_using_hypersync_cached_async() for documentation.

Parameters

  • attempts – Work around Hypersync timeout issues

  • client (hypersync.HypersyncClient) –

  • chain_id (int) –

  • start_block (int) –

  • end_block (int) –

  • display_progress (bool) –

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer

async fetch_sparse_block_timestamps_using_hypersync_cached_async(client, chain_id, start_block, end_block, step, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, checkpoint_frequency=25, max_concurrency=5)

Fetch only exact sampled timestamps through Hypersync and the shared cache.

Historical state readers may sample one block per hour or day from chains that produce millions of blocks per month. Fetching every intervening header wastes Hypersync quota and can make the first backfill impossible under a rate-limited API key. This path anti-joins the sampled block numbers against the persistent DuckDB cache, fetches only misses, and checkpoints them incrementally.

Parameters

  • client (hypersync.HypersyncClient) – Hypersync client for the requested chain.

  • chain_id (int) – Expected EVM chain id.

  • start_block (int) – First sampled block, inclusive.

  • end_block (int) – Historical reader end block, exclusive.

  • step (int) – Number of blocks between historical samples.

  • cache_path – Shared per-chain timestamp cache directory.

  • display_progress (bool) – Whether to display sampled timestamp progress.

  • checkpoint_frequency (int) – Maximum number of new timestamps held before a durable cache write.

  • max_concurrency (int) – Maximum one-block streams awaited together. The shared client limiter still governs total request starts across the batch.

Returns

Cache-backed timestamp slicer containing every requested sample.

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer

fetch_sparse_block_timestamps_using_hypersync_cached(client, chain_id, start_block, end_block, step, cache_path=PosixPath('/home/runner/.tradingstrategy/block-timestamp'), display_progress=True, attempts=5)

Synchronously fetch sparse sampled timestamps with retry/backoff.

Parameters

  • attempts (int) – Maximum Hypersync attempts; completed checkpoints survive retries.

  • client (hypersync.HypersyncClient) –

  • chain_id (int) –

  • start_block (int) –

  • end_block (int) –

  • step (int) –

  • display_progress (bool) –

Returns

Cache-backed timestamp slicer containing every requested sample.

Return type

eth_defi.event_reader.timestamp_cache.BlockTimestampSlicer