hypersync.session

Documentation for eth_defi.hypersync.session Python module.

Throttle-aware Hypersync client wrapper with stream tuning.

Provides a drop-in replacement for hypersync.HypersyncClient that rate-limits every API call (stream, recv, get_chain_id, get_height) through a shared SQLite-backed token bucket, and allows centralised configuration of Hypersync StreamConfig tuning parameters (concurrency, batch sizes, response byte limits).

This follows the same pyrate_limiter + SQLiteBucket throttling pattern used for Hyperliquid, GRVT, Lighter and Derive sessions (see e.g. eth_defi.hyperliquid.session), adapted for the async Rust FFI client instead of requests.Session.

For stream tuning parameter documentation see Envio HyperSync StreamConfig tuning.

Usage:

from eth_defi.hypersync.session import create_throttled_hypersync_client

# Create a client with custom concurrency for dense workloads
client = create_throttled_hypersync_client(
    hypersync.ClientConfig(url=url, bearer_token=api_key),
    concurrency=20,
    batch_size=5000,
)

# Use exactly like a regular HypersyncClient — throttling is transparent
# and StreamConfig is built automatically from stored tuning params
receiver = await client.stream(query)
res = await receiver.recv()

Module Attributes

`DEFAULT_HYPERSYNC_REQUESTS_PER_MINUTE`	Conservative default: 150 RPM leaves 25% headroom below Hypersync's 200 RPM limit.
`DEFAULT_HYPERSYNC_MAX_NUM_RETRIES`	Disable internal Rust client retries so that 429 errors surface immediately to Python-side retry logic with proper backoff and durable progress saves.

Functions

`create_throttled_hypersync_client`(config[, ...])	Create a Hypersync client with built-in SQLite-backed rate limiting.
`get_hypersync_concurrency_from_env`()	Read `HYPERSYNC_CONCURRENCY` from the environment.
`get_hypersync_rpm_from_env`()	Read `HYPERSYNC_RPM` from the environment.
`is_hypersync_client`(obj)	Check if obj is a Hypersync client (native or throttled).
`open_hypersync_stream`(client, query)	Open a Hypersync stream, dispatching correctly for both client types.

Classes

ThrottledHypersyncClient

Drop-in wrapper for hypersync.HypersyncClient with built-in rate limiting and stream tuning.

HYPERSYNC_RATE_LIMIT_SQLITE_DATABASE = PosixPath('/home/runner/.tradingstrategy/hypersync/rate-limit.sqlite')

Default SQLite database path for Hypersync rate limiting state.

Using SQLite ensures thread/process-safe rate limiting across parallel workers and scanner instances that share the same API key.

DEFAULT_HYPERSYNC_REQUESTS_PER_MINUTE = 150: Conservative default: 150 RPM leaves 25% headroom below Hypersync’s 200 RPM limit.

DEFAULT_HYPERSYNC_MAX_NUM_RETRIES = 0: Disable internal Rust client retries so that 429 errors surface immediately to Python-side retry logic with proper backoff and durable progress saves. The Rust client’s internal retries are invisible to our rate limiter and waste API quota on tight loops.

class ThrottledHypersyncClient

Bases: object

Drop-in wrapper for hypersync.HypersyncClient with built-in rate limiting and stream tuning.

Throttles one-shot API calls (stream, get_chain_id, get_height) through a shared pyrate_limiter.Limiter with an SQLite-backed token bucket.

Stream tuning parameters (concurrency, batch sizes, response byte limits) are stored on the client and used to build a hypersync.StreamConfig automatically when stream() is called without an explicit config.

recv() is intentionally not throttled — adding a Python-side delay on recv() only slows down buffer reads without reducing actual API load. Internal Rust retries are disabled (max_num_retries=0) so 429 errors surface immediately to the Python-side retry logic.

For stream tuning parameter documentation see Envio HyperSync StreamConfig tuning.

Note

This class intentionally does not subclass hypersync.HypersyncClient (a Rust-backed PyO3 class). It relies on duck typing — callers that accept HypersyncClient should also accept this wrapper. Use is_hypersync_client() for isinstance-style checks that accept both types.

Parameters

client – The underlying native hypersync.HypersyncClient.
limiter – Shared pyrate_limiter.Limiter for rate limiting.
concurrency – Number of requests in flight — the main throughput knob. None uses the Hypersync server default (10).
batch_size – Initial block range for the first wave of requests, before density has been measured. None uses the server default (1000).
min_batch_size – Lower limit on projected block count per request. None uses the server default.
max_batch_size – Hard cap on blocks per request. None means no cap.
response_bytes_ceiling – Upper target for response size in bytes (macOS wheel). None uses the server default.
response_bytes_floor – Lower target for response size in bytes (macOS wheel). None uses the server default.
response_bytes_target – Target response size in bytes (Linux wheel). None uses the server default.

Note

The hypersync 1.1.0 Rust wheel exposes different response-byte parameter names on different platforms. Pass whichever your platform supports; unsupported names are silently ignored.

__init__(client, limiter, *, concurrency=None, batch_size=None, min_batch_size=None, max_batch_size=None, response_bytes_ceiling=None, response_bytes_floor=None, response_bytes_target=None, max_buffered_bytes=None)

Parameters

client (hypersync.HypersyncClient) –
limiter (pyrate_limiter.limiter.Limiter) –
concurrency (Optional[int]) –
batch_size (Optional[int]) –
min_batch_size (Optional[int]) –
max_batch_size (Optional[int]) –
response_bytes_ceiling (Optional[int]) –
response_bytes_floor (Optional[int]) –
response_bytes_target (Optional[int]) –
max_buffered_bytes (Optional[int]) –

create_stream_config(**overrides)

Build a hypersync.StreamConfig from stored tuning params.

Any keyword arguments override the stored values for this single call. Only non-None values are passed to the constructor.

Example:

# Use all stored defaults
config = client.create_stream_config()

# Override concurrency for one call
config = client.create_stream_config(concurrency=30)

Return type: hypersync.StreamConfig

async stream(query, config=None)

Open a stream, throttled at the setup level.

When config is None, a hypersync.StreamConfig is built automatically from the stored tuning parameters via create_stream_config(). Pass an explicit config to override completely.

Parameters: config (Optional[hypersync.StreamConfig]) –
Return type: hypersync.QueryResponseStream

async get_chain_id(): Get chain ID, throttled.

async get_height(): Get latest block height, throttled.

is_hypersync_client(obj)

Check if obj is a Hypersync client (native or throttled).

Use instead of isinstance(obj, hypersync.HypersyncClient) so that ThrottledHypersyncClient wrappers are also accepted.

Return type: bool

async open_hypersync_stream(client, query)

Open a Hypersync stream, dispatching correctly for both client types.

For ThrottledHypersyncClient, calls stream(query) which builds a hypersync.StreamConfig from stored tuning parameters and logs them.

For a native hypersync.HypersyncClient, passes a bare StreamConfig() with all server defaults.

Use this instead of calling client.stream(query, config) directly so that tuning parameters are applied transparently.

Parameters

client (Union[hypersync.HypersyncClient, eth_defi.hypersync.session.ThrottledHypersyncClient]) – Either a native HypersyncClient or a ThrottledHypersyncClient.
query (hypersync.Query) – The Hypersync query to stream.

Return type

hypersync.QueryResponseStream

get_hypersync_rpm_from_env()

Read HYPERSYNC_RPM from the environment.

Returns DEFAULT_HYPERSYNC_REQUESTS_PER_MINUTE when the variable is unset or blank. Raises ValueError with a clear message when a non-empty value cannot be parsed as an integer.

Return type: int

get_hypersync_concurrency_from_env()

Read HYPERSYNC_CONCURRENCY from the environment.

Returns None when the variable is unset or blank (meaning use the Hypersync server default of 10). Raises ValueError when the value is not a positive integer.

Return type: Optional[int]

create_throttled_hypersync_client(config, requests_per_minute=150, limiter=None, *, concurrency=None, batch_size=None, min_batch_size=None, max_batch_size=None, response_bytes_ceiling=None, response_bytes_floor=None, response_bytes_target=None, max_buffered_bytes=None)

Create a Hypersync client with built-in SQLite-backed rate limiting.

For stream tuning parameter documentation see Envio HyperSync StreamConfig tuning.

Parameters

config (hypersync.ClientConfig) – Hypersync client configuration (URL, bearer token, etc.).
requests_per_minute (int) – Maximum API requests per minute. Defaults to 150 (75% of the Hypersync free-tier 200 RPM limit). Ignored when limiter is provided.
limiter (Optional[pyrate_limiter.limiter.Limiter]) – Optional pre-existing pyrate_limiter.Limiter to share across multiple clients (e.g. lead discovery + price scanning on the same API key). When None, a new limiter is created.
concurrency (Optional[int]) – Number of requests in flight — the main throughput knob. None uses the Hypersync server default (10).
batch_size (Optional[int]) – Initial block range for the first wave of requests. None uses the server default (1000).
min_batch_size (Optional[int]) – Lower limit on projected block count per request.
max_batch_size (Optional[int]) – Hard cap on blocks per request.
response_bytes_ceiling (Optional[int]) – Upper target for response size in bytes (macOS wheel).
response_bytes_floor (Optional[int]) – Lower target for response size in bytes (macOS wheel).
response_bytes_target (Optional[int]) – Target response size in bytes (Linux wheel).
max_buffered_bytes (Optional[int]) – Cap on bytes of fetched-but-undelivered data (Linux wheel).

Returns

A ThrottledHypersyncClient wrapping a native HypersyncClient.

Return type

eth_defi.hypersync.session.ThrottledHypersyncClient