hyperliquid.daily_metrics

Documentation for eth_defi.hyperliquid.daily_metrics Python module.

Hyperliquid daily vault metrics with DuckDB storage.

This module provides a daily pipeline for scanning Hyperliquid native vault metrics and storing them in a DuckDB database. It computes ERC-4626-style share prices using the time-weighted return approach, reusing the share price calculation from eth_defi.hyperliquid.combined_analysis.

The pipeline:

  1. Bulk-fetches all vaults from the stats-data API

  2. Filters by TVL and open status

  3. Fetches per-vault portfolio history via vaultDetails

  4. Computes share prices from portfolio history

  5. Stores daily prices and metadata in DuckDB

Example:

from eth_defi.hyperliquid.session import create_hyperliquid_session
from eth_defi.hyperliquid.daily_metrics import run_daily_scan, HyperliquidDailyMetricsDatabase

session = create_hyperliquid_session(requests_per_second=2.75)
db = run_daily_scan(session, min_tvl=5_000, max_vaults=500)
print(f"Stored metrics for {db.get_vault_count()} vaults")
db.close()

Functions

fetch_and_store_vault(session, db, summary)

Fetch a single vault's details and store metrics in the database.

portfolio_to_combined_dataframe([...])

Convert vaultDetails portfolio history into a combined DataFrame with share prices.

run_daily_scan(session[, db_path, min_tvl, ...])

Run the daily Hyperliquid vault metrics scan.

Classes

HyperliquidDailyMetricsDatabase

DuckDB database for storing Hyperliquid vault daily metrics.

HyperliquidDailyPriceRow

A single Hyperliquid daily price row ready for DuckDB upsert.
class HyperliquidDailyPriceRow

Bases: object

A single Hyperliquid daily price row ready for DuckDB upsert.

written_at: Optional[datetime.datetime]

When this row was actually written/fetched (naive UTC)

as_db_tuple()

Convert the row to the current 21-column DuckDB layout.

Return type

tuple[object, …]

__init__(vault_address, date, share_price, tvl, cumulative_pnl, cumulative_volume=None, daily_pnl=0.0, daily_return=0.0, follower_count=None, apr=None, is_closed=None, allow_deposits=None, leader_fraction=None, leader_commission=None, daily_deposit_count=None, daily_withdrawal_count=None, daily_deposit_usd=None, daily_withdrawal_usd=None, epoch_reset=None, data_source='api', written_at=None)
Parameters
Return type

None

portfolio_to_combined_dataframe(portfolio_all_time=None, *, portfolio=None)

Convert vaultDetails portfolio history into a combined DataFrame with share prices.

Derives pnl_update and netflow_update from the portfolio history, then feeds them through _calculate_share_price() to compute ERC-4626-style share prices.

When portfolio (the full period dict) is provided, all available periods (day, week, month, allTime) are merged to produce a higher-resolution time series. This yields more data points for recent history compared to using allTime alone.

The derivation:

  • pnl_update[i] = pnl_history[i] - pnl_history[i-1]

  • netflow_update[i] = (account_value[i] - account_value[i-1]) - pnl_update[i]

  • cumulative_account_value[i] = account_value[i]

Parameters
Returns

DataFrame with timestamp index and columns: pnl_update, netflow_update, cumulative_pnl, cumulative_netflow, cumulative_account_value, total_assets, total_supply, share_price, epoch_reset

Return type

pandas.DataFrame

class HyperliquidDailyMetricsDatabase

Bases: eth_defi.hyperliquid.vault_metrics_db.HyperliquidMetricsDatabaseBase

DuckDB database for storing Hyperliquid vault daily metrics.

Inherits shared metadata and lifecycle methods from HyperliquidMetricsDatabaseBase.

Stores daily share price time series and vault metadata. The share prices are computed using time-weighted returns from the Hyperliquid API portfolio history.

Example:

from pathlib import Path
from eth_defi.hyperliquid.daily_metrics import HyperliquidDailyMetricsDatabase

db = HyperliquidDailyMetricsDatabase(Path("/tmp/metrics.duckdb"))

# Query vault data
df = db.get_all_daily_prices()
print(df)

db.close()
__init__(path)
Parameters

path (pathlib.Path) –

upsert_daily_prices(rows, cutoff_date=None)

Bulk upsert daily price rows for a vault.

Parameters

  • rows (list[eth_defi.hyperliquid.daily_metrics.HyperliquidDailyPriceRow]) –

    List of HyperliquidDailyPriceRow items.

    The is_closed, allow_deposits, leader_fraction, and leader_commission fields should be None for historical rows and only set for the latest (today’s) row, so that we track how these values evolve over time.

    The flow columns (daily_deposit_count etc.) should be None for dates outside the backfill window and 0 for dates with no activity within the backfill window. COALESCE preserves existing values when None is passed.

  • cutoff_date (Optional[datetime.date]) – If provided, only store rows up to this date (inclusive). Used for incremental scanning / testing.

get_all_daily_prices()

Get all daily price data across all vaults.

Returns

DataFrame with all daily price records, ordered by vault then date.

Return type

pandas.DataFrame

get_vault_daily_prices(vault_address)

Get daily price data for a specific vault.

Parameters

vault_address (eth_typing.evm.HexAddress) – Vault address to query.

Returns

DataFrame with price records for this vault, ordered by date.

Return type

pandas.DataFrame

update_historical_daily_flows(vault_address, daily_flows, start_date, end_date)

Backfill ledger flows on existing historical price observations.

A normal scan can only attach flow data to portfolio observations that Hyperliquid still returns. Because the API downsamples old portfolio history, an old daily price row may remain in DuckDB even though a new vaultDetails response no longer contains that date. This method updates those existing rows directly from separately fetched userNonFundingLedgerUpdates data. Dates without a ledger event are explicitly stored as zero, which distinguishes a proven zero-flow day from an unknown historical day.

Existing price, NAV, PnL, and metadata values are never changed. The update is intentionally allowed to replace an old zero withdrawal, because older parser versions read Hyperliquid’s zero-valued usdc placeholder instead of netWithdrawnUsd.

Parameters

  • vault_address (eth_typing.evm.HexAddress) – Lowercase or checksummed Hyperliquid vault address.

  • daily_flows (dict[datetime.date, tuple[int, int, float, float]]) – Ledger events aggregated by UTC date as (deposit_count, withdrawal_count, deposit_usd, withdrawal_usd).

  • start_date (datetime.date) – First existing daily observation to backfill, inclusive.

  • end_date (datetime.date) – Last existing daily observation to backfill, inclusive.

Returns

Number of existing daily price rows updated.

Return type

int

get_existing_dates(vault_address)

Get all dates with existing data for a vault.

Parameters

vault_address (eth_typing.evm.HexAddress) – Vault address to query.

Returns

Set of dates that already have data in the database.

Return type

set[datetime.date]

get_vault_daily_price_count(vault_address)

Get the number of daily price records for a vault.

Parameters

vault_address (eth_typing.evm.HexAddress) – Vault address to query.

Returns

Number of daily price records.

Return type

int

get_vault_last_date(vault_address)

Get the last date with price data for a vault.

Parameters

vault_address (eth_typing.evm.HexAddress) – Vault address to query.

Returns

The latest date, or None if no data.

Return type

Optional[datetime.date]

get_leader_fraction_history(vault_address)

Get the recorded leader_fraction snapshots for a vault.

Returns only rows where leader_fraction was recorded (non-NULL). Each row represents a day when the scanner ran and observed the value. Over time, daily scans build up a sparse history of leader capital ownership that can be used to detect threshold crossings.

Parameters

vault_address (str) – Vault address to query (will be lowercased).

Returns

DataFrame with columns date and leader_fraction, ordered by date ascending. Empty if no snapshots recorded.

Return type

pandas.DataFrame

delete_vault_daily_prices(vault_address)

Delete all daily price records for a vault.

Used by the healing script to clear corrupted share prices before re-computing from fresh API data.

Parameters

vault_address (eth_typing.evm.HexAddress) – Vault address to delete.

Returns

Number of rows deleted.

Return type

int

recompute_vault_share_prices(vault_address)

Recompute share prices for a vault from stored tvl/pnl data.

Uses the stored tvl (total_assets), daily_pnl (pnl_update), and cumulative_pnl columns to reconstruct netflow_update, then feeds the data through _calculate_share_price() with the current chain-linked epoch reset logic.

This allows healing of share price data without re-fetching from the Hyperliquid API.

Parameters

vault_address (eth_typing.evm.HexAddress) – Vault address to recompute.

Returns

Dict with before/after statistics: {"rows": int, "old_sp_min": float, "old_sp_max": float, "new_sp_min": float, "new_sp_max": float, "epoch_resets": int, "changed_rows": int}

Return type

dict

recompute_all_share_prices()

Recompute share prices for all vaults in the database.

Iterates over every vault and calls recompute_vault_share_prices() for each.

Returns

Summary dict: {"total_vaults": int, "vaults_with_changes": int, "vaults_with_epoch_resets": int, "total_changed_rows": int, "per_vault": dict[str, dict]}

Return type

dict

detect_broken_vaults()

Detect vaults with anomalous share price data.

Checks for four classes of breakage:

  1. Extreme daily returns (> 10,000%): typically epoch reset artefacts where share price jumped to/from 1.0

  2. Share price at cap (>= 9,999): overflow from zero total_supply

  3. Share price stuck at 1.0: share price identical for 5+ rows, suggesting failed epoch reset logic

  4. Missing epoch_reset column: rows computed with old code that lacked the epoch_reset column (all NULL)

Returns

DataFrame with columns: vault_address, name, issue_type, affected_rows, example_value

Return type

pandas.DataFrame

save()

Force a checkpoint to ensure data is written to disk.

close()

Close the database connection.

get_all_tracked_addresses()

Return all vault addresses that have any price data.

Returns

Set of lowercased vault addresses.

Return type

set[str]

get_all_vault_metadata()

Get metadata for all vaults.

Returns

DataFrame with one row per vault.

Return type

pandas.DataFrame

get_latest_leader_fractions()

Get the latest leader_fraction for each vault.

Queries the most recent row per vault that has a non-NULL leader_fraction value.

Returns

Dict mapping lowercased vault address to leader_fraction.

Return type

dict[str, float]

get_recently_tracked_addresses(within_days=4)

Return vault addresses with price data within the last within_days days.

Uses a pure date cutoff so that whole-day semantics are preserved for the daily table (DATE column) and behave identically for the HF table (TIMESTAMP column — DuckDB casts DATE to midnight).

Parameters

within_days (int) – Number of days to look back from today.

Returns

Set of lowercased vault addresses.

Return type

set[str]

get_vault_count()

Get the number of unique vaults with price data.

Return type

int

mark_vaults_disappeared(known_addresses)

Set TVL to zero for vaults that disappeared from the API.

Also writes tombstone price rows so downstream consumers see a fresh row reflecting removal.

Parameters

known_addresses (set[str]) – Lowercased vault addresses currently present in the bulk API.

tombstone_stale_vaults(known_api_addresses, wind_down_days=4)

Write tombstone rows for vaults whose wind-down window has expired.

A vault is eligible for tombstoning when:

  1. It has existing price data in the database

  2. It is NOT present in the current bulk API listing

  3. Its most recent price row is older than wind_down_days

  4. It does not already have a tombstone row

The tombstone carries forward the last known share_price and cumulative_pnl so return calculations are not distorted.

Parameters

  • known_api_addresses (set[str]) – Vaults still in the API (never tombstoned).

  • wind_down_days (int) – Days after last price row before tombstoning.

Returns

Number of tombstone rows written.

Return type

int

update_vault_tvl_bulk(updates)

Bulk-update TVL, is_closed, and APR for existing vaults.

Only updates rows that already exist in vault_metadata.

Parameters

updates (list[tuple[float, bool, float | None, str]]) – List of tuples (tvl, is_closed, apr, vault_address).

upsert_vault_metadata(vault_address, name, leader, description, is_closed, relationship_type, create_time, commission_rate, follower_count, tvl, apr, allow_deposits=True, flow_data_earliest_date=None)

Insert or update a vault’s metadata.

Parameters
fetch_and_store_vault(session, db, summary, cutoff_date=None, timeout=30.0, flow_backfill_days=7)

Fetch a single vault’s details and store metrics in the database.

Parameters
Returns

True if the vault was successfully processed.

Return type

bool

run_daily_scan(session, db_path=PosixPath('/home/runner/.tradingstrategy/vaults/hyperliquid-vaults.duckdb'), min_tvl=5000, max_vaults=20000, max_workers=16, cutoff_date=None, timeout=30.0, vault_addresses=None, flow_backfill_days=7, full_scan=False)

Run the daily Hyperliquid vault metrics scan.

  1. Bulk-fetches all vaults from stats-data API

  2. Filters by TVL and vault limit (or by explicit address list)

  3. Fetches per-vault details and computes share prices

  4. Fetches deposit/withdrawal events for flow metrics

  5. Stores everything in DuckDB

Parameters

  • session (eth_defi.hyperliquid.session.HyperliquidSession) – HTTP session with rate limiting. Use create_hyperliquid_session().

  • db_path (pathlib.Path) – Path to the DuckDB database file.

  • min_tvl (float) – Minimum TVL in USD to include a vault. Ignored when vault_addresses is provided.

  • max_vaults (int) – Maximum number of vaults to process (sorted by TVL descending). Ignored when vault_addresses is provided.

  • max_workers (int) – Number of parallel workers for fetching vault details.

  • cutoff_date (Optional[datetime.date]) – If provided, only store price data up to this date. Used for incremental scanning / testing.

  • timeout (float) – HTTP request timeout.

  • vault_addresses (Optional[list[str]]) – If provided, only scan these specific vault addresses. Overrides min_tvl and max_vaults filters.

  • flow_backfill_days (int) – Number of complete days to backfill deposit/withdrawal flow data. Only complete days are fetched (up to yesterday). Set to 0 to disable flow fetching. Use a large value (e.g. 365) for initial deep backfill.

  • full_scan (bool) – When True, bypass the min_tvl filter and process all vaults that exist in the database and are still present in the bulk API listing, regardless of their current TVL. This is intended to be run periodically (e.g. once a week) to refresh data for vaults that have dropped below the daily TVL threshold but still have historical data worth keeping current. The max_vaults limit still applies to cap total work. Ignored when vault_addresses is provided.

Returns

The metrics database instance.

Return type

eth_defi.hyperliquid.daily_metrics.HyperliquidDailyMetricsDatabase