lighter.vault_data_export

Documentation for eth_defi.lighter.vault_data_export Python module.

Export Lighter pool data into the ERC-4626 pipeline format.

This module bridges the Lighter-specific DuckDB data into the formats consumed by the existing ERC-4626 vault metrics pipeline:

  • Synthetic VaultRow entries for the VaultDatabase pickle

  • Raw price DataFrames matching the uncleaned Parquet schema, so that Lighter data goes through the same cleaning pipeline as EVM vaults

  • Merge functions to append Lighter data into existing files

Example:

from pathlib import Path
from eth_defi.lighter.daily_metrics import LighterDailyMetricsDatabase
from eth_defi.lighter.vault_data_export import merge_into_vault_database, merge_into_uncleaned_parquet

db = LighterDailyMetricsDatabase(Path("daily-metrics.duckdb"))

merge_into_vault_database(db, vault_db_path)
merge_into_uncleaned_parquet(db, uncleaned_parquet_path)

db.close()

Functions

build_raw_prices_dataframe(db)

Build a raw prices DataFrame from the Lighter DuckDB.

create_lighter_pool_row(account_index, name, ...)

Create a synthetic VaultRow for a Lighter pool.

merge_into_uncleaned_parquet(db, parquet_path)

Merge Lighter daily prices into the uncleaned Parquet file.

merge_into_vault_database(db, vault_db_path)

Merge Lighter pool metadata into an existing VaultDatabase pickle.
build_raw_prices_dataframe(db)

Build a raw prices DataFrame from the Lighter DuckDB.

Produces rows matching the schema of the EVM vault scanner (export()), so Lighter data can go through the same cleaning pipeline (process_raw_vault_scan_data()) as ERC-4626 vaults.

The output has timestamp as a column (not index), matching the raw uncleaned Parquet format.

Parameters

db (eth_defi.lighter.daily_metrics.LighterDailyMetricsDatabase) – The Lighter daily metrics database.

Returns

DataFrame with columns matching the uncleaned Parquet schema.

Return type

pandas.DataFrame

create_lighter_pool_row(account_index, name, description, tvl, created_at, operator_fee=0.0, is_llp=False, status=0)

Create a synthetic VaultRow for a Lighter pool.

Builds a VaultRow that matches what calculate_vault_record() expects, using the Lighter synthetic chain ID.

Lighter pool operator fees are already reflected in the share price (internalised skimming model), so the pipeline treats the share price as net of fees.

Parameters

  • account_index (int) – Pool account index on the Lighter platform.

  • name (str) – Pool display name.

  • description (Optional[str]) – Pool description text.

  • tvl (float) – Current TVL in USDC.

  • created_at (Optional[datetime.datetime]) – Pool creation timestamp.

  • operator_fee (float) – Operator fee percentage (e.g. 10.0 = 10%).

  • is_llp (bool) – Whether this is the LLP protocol pool.

  • status (int) – Pool status code from the API (0 = active).

Returns

Tuple of (VaultSpec, VaultRow).

Return type

tuple[eth_defi.vault.base.VaultSpec, eth_defi.vault.vaultdb.VaultRow]

merge_into_uncleaned_parquet(db, parquet_path)

Merge Lighter daily prices into the uncleaned Parquet file.

Writes Lighter raw data in the same format as the EVM vault scanner, so the standard cleaning pipeline (process_raw_vault_scan_data()) can process all vaults together.

Reads the existing Parquet, removes any prior Lighter rows (chain == 9998), appends fresh Lighter daily price rows, and writes back. Idempotent: running twice produces the same result.

If the Parquet file does not exist, creates a new one.

Parameters
Returns

The combined DataFrame.

Return type

pandas.DataFrame

merge_into_vault_database(db, vault_db_path)

Merge Lighter pool metadata into an existing VaultDatabase pickle.

Reads the existing pickle, upserts Lighter VaultRow entries (keyed by VaultSpec), and writes back. Idempotent: running twice produces the same result.

If the pickle file does not exist, creates a new VaultDatabase.

Parameters
Returns

The updated VaultDatabase.

Return type

eth_defi.vault.vaultdb.VaultDatabase