lighter.vault_data_export

Documentation for eth_defi.lighter.vault_data_export Python module.

Export Lighter pool data into the ERC-4626 pipeline format.

This module bridges the Lighter-specific DuckDB data into the formats consumed by the existing ERC-4626 vault metrics pipeline:

• Synthetic VaultRow entries for the VaultDatabase pickle

• Raw price DataFrames matching the uncleaned Parquet schema, so that Lighter data goes through the same cleaning pipeline as EVM vaults

• Merge functions to append Lighter data into existing files

Example:

from pathlib import Path
from eth_defi.lighter.daily_metrics import LighterDailyMetricsDatabase
from eth_defi.lighter.vault_data_export import merge_into_vault_database, merge_into_uncleaned_parquet

db = LighterDailyMetricsDatabase(Path("daily-metrics.duckdb"))

merge_into_vault_database(db, vault_db_path)
merge_into_uncleaned_parquet(db, uncleaned_parquet_path)

db.close()

Functions

build_raw_prices_dataframe(db)	Build a raw prices DataFrame from the Lighter DuckDB.
create_lighter_pool_row(account_index, name, ...)	Create a synthetic VaultRow for a Lighter pool.
get_lighter_price_deployments(prices_df)	Resolve deployments represented by a Lighter raw-price frame.
merge_into_uncleaned_parquet(db, parquet_path)	Merge Lighter daily prices into the uncleaned Parquet file.
merge_into_vault_database(db, vault_db_path)	Merge Lighter pool metadata into an existing VaultDatabase pickle.

get_lighter_price_deployments(prices_df)

Resolve deployments represented by a Lighter raw-price frame.

The frame must contain synthetic Lighter addresses in its address column. Unknown addresses abort the merge because treating an incomplete export as a whole-chain replacement could discard the other deployment’s history now that Ethereum and Robinhood share synthetic chain 9998.

Parameters

prices_df (pandas.DataFrame) – Lighter raw-price DataFrame with string address values.

Returns

Deployment configurations represented by at least one row.

Return type

set[eth_defi.lighter.constants.LighterAPIConfig]

create_lighter_pool_row(account_index, name, description, tvl, created_at, operator_fee=0.0, is_llp=False, status=0, deployment=LighterAPIConfig(slug='ethereum', name='Lighter Ethereum', chain_id=9998, deployment_chain_id=1, api_url='https://mainnet.zklighter.elliot.ai', app_url='https://app.lighter.xyz', address_prefix='lighter-pool', denomination='USDC', lockup=datetime.timedelta(seconds=300), llp_account_index_override=None))

Create a synthetic VaultRow for a Lighter pool.

Builds a VaultRow that matches what calculate_vault_record() expects, using the Lighter synthetic chain ID.

Lighter pool operator fees are already reflected in the share price (internalised skimming model), so the pipeline treats the share price as net of fees.

Parameters

• account_index (int) – Pool account index on the Lighter platform.

• name (str) – Pool display name.

• description (Optional[str]) – Pool description text.

• tvl (float) – Current TVL in the deployment’s collateral currency.

• created_at (Optional[datetime.datetime]) – Pool creation timestamp.

• operator_fee (float) – Operator fee percentage (e.g. 10.0 = 10%).

• is_llp (bool) – Whether this is the LLP protocol pool.

• status (int) – Pool status code from the API (0 = active).

• deployment (eth_defi.lighter.constants.LighterAPIConfig) – Lighter deployment configuration. Defaults to Ethereum for backwards compatibility.

Returns

Tuple of (VaultSpec, VaultRow).

Return type

tuple[eth_defi.vault.base.VaultSpec, eth_defi.vault.vaultdb.VaultRow]

build_raw_prices_dataframe(db)

Build a raw prices DataFrame from the Lighter DuckDB.

Produces rows matching the schema of the EVM vault scanner (export()), so Lighter data can go through the same cleaning pipeline (process_raw_vault_scan_data()) as ERC-4626 vaults.

The output has timestamp as a column (not index), matching the raw uncleaned Parquet format.

Parameters

db (eth_defi.lighter.daily_metrics.LighterDailyMetricsDatabase) – The Lighter daily metrics database.

Returns

DataFrame with columns matching the uncleaned Parquet schema.

Return type

pandas.DataFrame

merge_into_vault_database(db, vault_db_path)

Merge Lighter pool metadata into an existing VaultDatabase pickle.

Reads the existing pickle, upserts Lighter VaultRow entries (keyed by VaultSpec), and writes back. Idempotent: running twice produces the same result.

If the pickle file does not exist, creates a new VaultDatabase.

Parameters

• db (eth_defi.lighter.daily_metrics.LighterDailyMetricsDatabase) – The Lighter daily metrics database.

• vault_db_path (pathlib.Path) – Path to the VaultDatabase pickle file.

Returns

The updated VaultDatabase.

Return type

eth_defi.vault.vaultdb.VaultDatabase

merge_into_uncleaned_parquet(db, parquet_path)

Merge Lighter daily prices into the uncleaned Parquet file.

Writes Lighter raw data in the same format as the EVM vault scanner, so the standard cleaning pipeline (process_raw_vault_scan_data()) can process all vaults together.

Reads the existing Parquet, removes prior rows only for deployments present in the fresh export, appends fresh Lighter daily price rows, and writes back. Idempotent: running twice produces the same result. This deployment-scoped replacement preserves Robinhood history during an Ethereum-only scan, and vice versa.

If the Parquet file does not exist, creates a new one.

Parameters

• db (eth_defi.lighter.daily_metrics.LighterDailyMetricsDatabase) – The Lighter daily metrics database.

• parquet_path (pathlib.Path) – Path to the uncleaned Parquet file (typically vault-prices-1h.parquet).

Returns

The combined DataFrame.

Return type

pandas.DataFrame