vault_report.vault_checks

Documentation for eth_defi.vault_report.vault_checks Python module.

Investability check of the vault report’s top lists.

Some vaults rank high in the report but are not investable in practice: they lend against collateral nobody can sell, their depositors cannot exit, or they are scams. The data we collect cannot show this. An LLM agent, Claude CLI or Codex CLI running the check-top-list-vaults skill, researches the candidates and decides. See .claude/plans/2026-09-26-vault-report-investability-check.md, .claude/skills/check-top-list-vaults/SKILL.md and the Investability check section of eth_defi/vault_report/README-vault-report.md.

The check runs in rounds, see run_vault_checks():

The report’s top lists are collected with a buffer of extra vaults, see eth_defi.vault_report.report.collect_top_lists().
New candidates of in-scope protocols get deterministic facts, see eth_defi.vault_report.vault_probes, and the agent’s decisions. Other candidates are not_in_scope.
Excluded vaults are dropped and the lists are collected again. Vaults that moved up into a list are checked in the next round. If in-scope vaults are still unchecked after the last round, the report stops.
The largest in-scope vaults of the average yield charts are prescreened with the probes only; those raising signals go to the agent in a last round.

Division of labour: code reads everything it can read reliably (positions, redeemable liquidity, DEX liquidity, liquidity history), so runs are cheaper, more repeatable and less exposed to web content; the agent adds the judgement and the outside research (block explorers, forums, X, news).

The agent is not trusted blindly. Its decisions file is validated strictly, bound to the exact candidate list by a SHA-256 digest, and any invalid, stale or missing file aborts the report rather than silently keeping every vault, see read_check_decisions(). It runs without a sandbox, by the product owner’s decision, but the skill restricts it to reading and to writing only the decisions file and eth_defi/vault/flag.py; every flag.py change it makes is printed for human review and never committed by the pipeline.

Version 1 checks Morpho, Euler and 40acres vaults for suspicious collateral and missing exit liquidity; CHECK_SCOPE grows as more protocols are covered. Likely scams are also blacklisted in eth_defi/vault/flag.py by the agent, and undecided vaults get a review_needed entry there.

The outputs are the round files in the report bundle (vault-check-candidates-N.json, vault-check-facts-N.json, vault-check-decisions-N.json and the agent transcript vault-check-agent-N.jsonl), the vault_checks entry of report.json and the dated excluded vaults Markdown record, see render_excluded_vaults_markdown().

Module Attributes

`SCHEMA_VERSION`	Version of the candidate and decision file formats.
`SCOPE_VERSION`	Version of `CHECK_SCOPE`
`RULES_VERSION`	Version of the decision rules in the skill, as the date the rules last changed
`CHECK_SCOPE`	Protocol slug -> checks applied.
`DECISIONS`	Allowed decisions
`CATEGORIES`	Allowed exclusion categories
`CONFIDENCE_LEVELS`	Allowed confidence levels
`CLAUDE_CHECK_MODEL`	Claude CLI model for the check agent, pinned to a full model id so a CLI update moving the `sonnet` alias cannot silently change the cost
`CLAUDE_CHECK_EFFORT`	Claude CLI thinking effort for the check agent: `low`, `medium`, `high`, `xhigh` or `max`

Functions

`apply_check_decisions`(vaults_df, excluded)	Drop excluded vaults from a vault set.
`build_agent_command`(agent, prompt[, model, ...])	Build the unsandboxed, web-enabled CLI command for the check agent.
`build_agent_prompt`(round_)	Create the prompt that points the agent to the skill and the round's files.
`build_check_candidates`(lists)	Turn the report's top lists into deduplicated candidates.
`candidate_depth`(target, buffer_ratio)	Number of candidates collected for a list showing `target` vaults.
`check_blacklist_entries`(decisions, entries)	Check that every blacklisted vault has an entry in `flag.py`.
`excluded_rows`(result)	Excluded vaults as plain records for the CSV and the manifest.
`find_unreviewed_uncertain`(decisions, entries)	Find `uncertain` vaults without an entry in `flag.py`.
`parse_decision`(record)	Parse one decision record written by the agent or an editor.
`parse_naive_utc_timestamp`(value)	Parse an ISO 8601 timestamp written by the agent into a naive UTC datetime.
`read_check_decisions`(path, candidates, ...)	Read and validate the agent's decisions for one round.
`read_flag_entries`(flag_file)	Read the vault flags of `VAULT_FLAGS_AND_NOTES` from the `flag.py` source.
`read_overrides`(path)	Read hand-written decisions that override the agent.
`render_excluded_vaults_markdown`(result, ...)	Render the vaults the investability check left out of a report, as a dated Markdown document.
`run_check_agent`(command, cwd, log_path, ...)	Run the agent CLI, streaming its JSONL output to a log file.
`run_vault_checks`(collect_lists, ...[, ...])	Check the report's top lists in rounds until every listed vault is decided.
`show_flag_diff`(repository_root)	Show the agent's uncommitted `flag.py` changes for review.
`summarise_checks`(result)	Summary of the check for `report.json`.
`write_candidates_file`(path, candidates, ...)	Write the candidates the agent must check.

Classes

`CheckCandidate`	A vault that would appear in one of the report's top lists.
`CheckDecision`	The check's verdict on one candidate.
`CheckResult`	Decisions of all rounds.
`CheckRound`	Files of one check round, in the report bundle.
`VaultCheckSettings`	How to run the investability check.

Exceptions

CheckValidationError

The investability check cannot produce trustworthy decisions.

SCHEMA_VERSION = 1

Version of the candidate and decision file formats.

The three versions are written to the candidates file and must be copied into the decisions file header: bump one whenever the file format, CHECK_SCOPE or the skill’s decision rules change, so saved decisions made under the old rules are no longer reused.

SCOPE_VERSION = 1: Version of CHECK_SCOPE

RULES_VERSION = '2026-09-26': Version of the decision rules in the skill, as the date the rules last changed

CHECK_SCOPE: dict[str, tuple[str, ...]] = {'40acres': ('no_exit_liquidity',), 'euler': ('suspicious_collateral', 'no_exit_liquidity'), 'morpho': ('suspicious_collateral', 'no_exit_liquidity')}

Protocol slug -> checks applied. Vaults of other protocols are not_in_scope.

Adding a protocol needs a row here, a probe in eth_defi.vault_report.vault_probes and a section in the skill, plus a SCOPE_VERSION bump.

DECISIONS = frozenset({'exclude', 'keep', 'not_in_scope', 'uncertain'}): Allowed decisions

CATEGORIES = frozenset({'broken_data', 'no_exit_liquidity', 'other', 'scam', 'suspicious_collateral'}): Allowed exclusion categories

CONFIDENCE_LEVELS = frozenset({'high', 'low', 'medium'}): Allowed confidence levels

FLAG_FILE = PosixPath('eth_defi/vault/flag.py'): Path of the vault flag file the agent edits for blacklist and review_needed entries, relative to the repository root

SKILL_FILE = PosixPath('.claude/skills/check-top-list-vaults/SKILL.md'): Skill the agent follows, relative to the repository root the agent runs in

CLAUDE_CHECK_MODEL = 'claude-sonnet-5-5': Claude CLI model for the check agent, pinned to a full model id so a CLI update moving the sonnet alias cannot silently change the cost

CLAUDE_CHECK_EFFORT = 'medium': Claude CLI thinking effort for the check agent: low, medium, high, xhigh or max

AgentName

Supported agent CLIs

alias of Literal[‘claude’, ‘codex’]

exception CheckValidationError

Bases: ValueError

The investability check cannot produce trustworthy decisions.

Raised when the agent’s decisions file is missing, stale or malformed, when the agent fails or times out, when a blacklisted vault has no flag.py entry, or when top-list vaults remain unchecked. The report pipeline lets it propagate: publishing unchecked rankings is worse than stopping, and the operator can rerun with the saved rounds reused.

__init__(*args, **kwargs)

__new__(**kwargs)

add_note(note, /): Add a note to the exception

with_traceback(tb, /): Set self.__traceback__ to tb and return self.

class CheckCandidate

Bases: object

A vault that would appear in one of the report’s top lists.

Written to the round’s candidates file for the agent, and used to render the excluded vaults Markdown record. Built from a top list row by build_check_candidates().

vault_id: str: {chain_id}-{address}

name: str: Vault name

chain: str: Chain name

address: eth_typing.evm.HexAddress: Vault address

protocol: str: Protocol name

protocol_slug: str: Protocol slug

curator: Optional[str]: Curator name, if any

features: list[str]: ERC-4626 feature names

tvl_usd: float: Current TVL in US dollars

one_month_return: Optional[float]: Annualised one-month and three-month returns

three_months_volatility: Optional[float]: Annualised three-month volatility

link: Optional[str]: Vault page on the website

protocol_link: Optional[str]: Protocol’s own page for the vault

lists: list[str]: Lists the vault would appear in, with its rank, e.g. table:lending#3

property in_scope: bool: Whether version 1 checks this vault.

__init__(vault_id, name, chain, address, protocol, protocol_slug, curator, features, tvl_usd, one_month_return, three_months_return, three_months_volatility, link, protocol_link, lists=<factory>)

Parameters

vault_id (str) –
name (str) –
chain (str) –
address (eth_typing.evm.HexAddress) –
protocol (str) –
protocol_slug (str) –
curator (Optional[str]) –
features (list[str]) –
tvl_usd (float) –
one_month_return (Optional[float]) –
three_months_return (Optional[float]) –
three_months_volatility (Optional[float]) –
link (Optional[str]) –
protocol_link (Optional[str]) –
lists (list[str]) –

Return type

None

class CheckDecision

Bases: object

The check’s verdict on one candidate.

Comes from three sources: the agent’s decisions file, the editor’s overrides file, or the pipeline itself for not_in_scope vaults. All agent and editor records pass through parse_decision().

vault_id: str: {chain_id}-{address}

decision: str: exclude, keep, uncertain or not_in_scope; only exclude removes the vault from the report

category: Optional[str]: Exclusion category, or None

suspicious_item: Optional[str]: Short phrase naming the problem, e.g. RSS elephanToken collateral

reason: Optional[str]: One plain-text sentence for readers

evidence: list[dict]: Evidence: {"source": url or data reference, "observed_at": ISO timestamp}

confidence: Optional[str]: high, medium or low

blacklist: bool: Whether the vault is blacklisted permanently in flag.py

vault_flag: Optional[str]: VaultFlag value of the blacklist entry

__init__(vault_id, decision, category=None, suspicious_item=None, reason=None, evidence=<factory>, confidence=None, blacklist=False, vault_flag=None)

Parameters

vault_id (str) –
decision (str) –
category (Optional[str]) –
suspicious_item (Optional[str]) –
reason (Optional[str]) –
evidence (list[dict]) –
confidence (Optional[str]) –
blacklist (bool) –
vault_flag (Optional[str]) –

Return type

None

class CheckRound

Bases: object

Files of one check round, in the report bundle.

The file names carry only the round number, so a rerun with the same bundle or a VAULT_CHECK_DECISIONS directory finds the earlier round’s decisions by name; the digest then decides whether they still apply.

number: int: Round number, from 1

candidates_path: pathlib.Path: Candidate file, vault-check-candidates-N.json, written by the pipeline

facts_path: pathlib.Path: Deterministic facts file, vault-check-facts-N.json, written by the pipeline; not written when the round’s decisions are reused

decisions_path: pathlib.Path: Agent decisions file, vault-check-decisions-N.json, written by the agent

candidates_digest: str: SHA-256 of the candidates file; the agent copies it into the decisions header

__init__(number, candidates_path, facts_path, decisions_path, candidates_digest)

Parameters

number (int) –
candidates_path (pathlib.Path) –
facts_path (pathlib.Path) –
decisions_path (pathlib.Path) –
candidates_digest (str) –

Return type

None

class VaultCheckSettings

Bases: object

How to run the investability check.

Built from environment variables by from_env() in the operator scripts generate-monthly-vault-report.py and check-top-list-vaults.py.

agent: Optional[Literal['claude', 'codex']]: Agent CLI, or None to only reuse existing decisions (VAULT_CHECK_AGENT=reuse); a round without reusable decisions then fails instead of spending tokens

model: Optional[str]: Model override for the agent CLI; Claude CLI defaults to CLAUDE_CHECK_MODEL

effort: Optional[str]: Thinking effort override for Claude CLI, defaults to CLAUDE_CHECK_EFFORT

reuse_dirs: list[pathlib.Path]: Directories searched for reusable decisions, in addition to the report bundle

overrides_path: Optional[pathlib.Path]: Hand-written decisions that override the agent, e.g. from an editor’s review

timeout: float: Agent timeout per round, in seconds; the whole process group is killed after it

repository_root: pathlib.Path: Repository root the agent runs in; the skill, flag.py and the probe scripts are found from here

max_workers: int: Parallel threads for the deterministic probes

classmethod from_env(default_agent='none')

Read the settings from environment variables.

See the Investability check section of README-vault-report.md for the variables: VAULT_CHECK_AGENT, VAULT_CHECK_MODEL, VAULT_CHECK_EFFORT, VAULT_CHECK_DECISIONS, VAULT_CHECK_OVERRIDES, VAULT_CHECK_TIMEOUT (minutes) and MAX_WORKERS.

reuse runs the check without an agent: it only accepts saved decisions that match the current candidates, which is how an editor reruns the report with overrides without paying for another agent run.

Parameters: default_agent (str) – VAULT_CHECK_AGENT when unset: claude, codex, reuse or none. The report script defaults to none, so a plain run costs no LLM tokens; check-top-list-vaults.py defaults to claude.
Returns: Settings, or None when the check is disabled with none.
Return type: Optional[eth_defi.vault_report.vault_checks.VaultCheckSettings]

__init__(agent=None, model=None, effort=None, reuse_dirs=<factory>, overrides_path=None, timeout=3600.0, repository_root=PosixPath('/home/runner/work/web3-ethereum-defi/web3-ethereum-defi'), max_workers=8)

Parameters

agent (Optional[Literal['claude', 'codex']]) –
model (Optional[str]) –
effort (Optional[str]) –
reuse_dirs (list[pathlib.Path]) –
overrides_path (Optional[pathlib.Path]) –
timeout (float) –
repository_root (pathlib.Path) –
max_workers (int) –

Return type

None

class CheckResult

Bases: object

Decisions of all rounds.

Consumed by the report renderer, which drops excluded from every ranking and chart, and by the excluded vaults record and the report.json summary.

decisions: dict[str, eth_defi.vault_report.vault_checks.CheckDecision]: Vault id -> decision, including not_in_scope and editor overrides

candidates: dict[str, eth_defi.vault_report.vault_checks.CheckCandidate]: Vault id -> candidate, every vault any round or the prescreen saw

rounds: list[eth_defi.vault_report.vault_checks.CheckRound]: Rounds that ran

property excluded: frozenset[str]: Ids of excluded vaults.

property excluded_candidates: list[eth_defi.vault_report.vault_checks.CheckCandidate]: Excluded candidates, in the order of the list and rank they would have appeared at.

property uncertain: list[eth_defi.vault_report.vault_checks.CheckCandidate]: Candidates the check could not decide.

__init__(decisions=<factory>, candidates=<factory>, rounds=<factory>)

Parameters

decisions (dict[str, eth_defi.vault_report.vault_checks.CheckDecision]) –
candidates (dict[str, eth_defi.vault_report.vault_checks.CheckCandidate]) –
rounds (list[eth_defi.vault_report.vault_checks.CheckRound]) –

Return type

None

candidate_depth(target, buffer_ratio)

Number of candidates collected for a list showing target vaults.

The buffer lets the first round also decide the vaults that move up when others are excluded, so most reports need a single agent round.

Parameters

target (int) – Vaults shown.
buffer_ratio (float) – Extra share, e.g. 0.5 for 50% more.

Returns

Candidates to collect.

Return type

int

build_check_candidates(lists)

Turn the report’s top lists into deduplicated candidates.

A vault often appears in several lists, e.g. a lending table and its chart, but is researched once. The lists entries tell the agent how prominent the vault would be and order the excluded vaults record.

Parameters: lists (dict[str, pandas.DataFrame]) – List name -> ranked vaults indexed by vault id, best first, from eth_defi.vault_report.report.collect_top_lists(). Rows are top vaults JSON records: name, chain, address, protocol, protocol_slug, curator_name, features, current_nav, one_month_cagr_best, three_months_cagr_best, three_months_volatility, trading_strategy_link and link, any of which may be missing or NaN.
Returns: Vault id -> candidate, each recording the lists and ranks it appears in, in the order of lists.
Return type: dict[str, eth_defi.vault_report.vault_checks.CheckCandidate]

write_candidates_file(path, candidates, data_end_at)

Write the candidates the agent must check.

The file is the agent’s input, and its digest binds the agent’s output to it. The digest is taken from the bytes on disk, and the JSON is written with sorted keys, so the same candidates, data date, scope and rules give the same digest on a rerun. Any change, such as a refreshed download moving a TVL or a rank, gives a new digest, so decisions made for other candidates are never reused, see read_check_decisions().

Parameters

path (pathlib.Path) – Output JSON file, vault-check-candidates-N.json in the report bundle.
candidates (list[eth_defi.vault_report.vault_checks.CheckCandidate]) – In-scope candidates.
data_end_at (datetime.datetime) – Report data date.

Returns

SHA-256 hex digest of the written file.

Return type

str

parse_decision(record)

Parse one decision record written by the agent or an editor.

Enforces the record rules of the skill. An exclusion removes a vault from a published report, so it must say what is wrong (category and suspicious item), explain it to readers (reason) and be traceable (evidence with a source and an observation time). A blacklist hides the vault on the website and in the data exports permanently, so it needs high confidence and a flag from BAD_FLAGS; that the flag.py entry was actually written is checked separately, see check_blacklist_entries().

Parameters: record (dict) – Decision JSON object, see the output schema in the skill.
Returns: Decision.
Raises: CheckValidationError – On an unknown decision, category or confidence, or an incomplete exclusion or blacklist.
Return type: eth_defi.vault_report.vault_checks.CheckDecision

parse_naive_utc_timestamp(value)

Parse an ISO 8601 timestamp written by the agent into a naive UTC datetime.

The agent may write Z, an explicit UTC offset such as +02:00, or no offset. Offset timestamps are converted to UTC before the offset is dropped; timestamps without an offset are taken to be UTC already.

Parameters: value (str) – ISO 8601 timestamp, e.g. 2026-09-30T12:00:00+02:00.
Returns: Naive UTC datetime.
Return type: datetime.datetime

read_check_decisions(path, candidates, candidates_digest, data_end_at, max_evidence_age=datetime.timedelta(days=7))

Read and validate the agent’s decisions for one round.

Used both for a fresh agent run and to test whether saved decisions can be reused, see _find_reusable(). The checks, in order:

Header binding. The schema, scope and rules versions, the data date and the candidates file digest must all equal the current run. The digest binds the decisions to the exact candidate list the agent was given: decisions saved for another download, another month or a different set of candidates cannot be applied to this one, and an agent that ignored the prompt and wrote a file from memory is caught.
Records. Each record must pass parse_decision(), and a vault may appear only once.
Coverage. Exactly the round’s candidates: no decisions for other vaults, none missing, and no not_in_scope, because every candidate given to the agent is in scope and an unchecked vault must not slip into a ranking.
Freshness. Exit liquidity changes daily, so a no_exit_liquidity exclusion must rest on evidence observed shortly before the data date.

Parameters

path (pathlib.Path) – Decisions JSON file written by the agent, or a saved one from an earlier run.
candidates (list[eth_defi.vault_report.vault_checks.CheckCandidate]) – Candidates of the round.
candidates_digest (str) – SHA-256 of the round’s candidates file, see write_candidates_file().
data_end_at (datetime.datetime) – Report data date.
max_evidence_age (datetime.timedelta) – Liquidity evidence must be observed at most this long before the data date.

Returns

Vault id -> decision.

Raises

CheckValidationError – When the file does not match the round or a decision is invalid. Never falls back to keeping everything: a silent fallback would publish rankings nobody checked.

Return type

dict[str, eth_defi.vault_report.vault_checks.CheckDecision]

read_overrides(path)

Read hand-written decisions that override the agent.

Overrides use the decision record format, without a file header, so an editor can keep them across report runs. Unlike agent decisions they are not digest-bound: an override is a human decision about a vault, not about one candidate list. They are applied when a round’s candidates are triaged, before its candidates file is written, so an overridden vault is never sent to the agent. As a consequence, a new override for a vault changes the candidates file, and its digest, of the round that would have checked it. See Review workflow in README-vault-report.md.

Parameters: path (pathlib.Path) – JSON file with a list of decision records (VAULT_CHECK_OVERRIDES).
Returns: Vault id -> decision.
Return type: dict[str, eth_defi.vault_report.vault_checks.CheckDecision]

check_blacklist_entries(decisions, entries)

Check that every blacklisted vault has an entry in flag.py.

blacklist: true in the decisions file is only the agent’s claim. The blacklist takes effect through the flag.py entry, so the claim is verified against the source the agent edited, and the flag must be the one the decision names. A mismatch is a hard error in _check_round(): the report would say a vault is blacklisted when the website still shows it.

Parameters

decisions (dict[str, eth_defi.vault_report.vault_checks.CheckDecision]) – Decisions of the round.
entries (dict[str, str | None]) – flag.py entries, see read_flag_entries().

Returns

Vault ids missing from VAULT_FLAGS_AND_NOTES or flagged differently.

Return type

list[str]

find_unreviewed_uncertain(decisions, entries)

Find uncertain vaults without an entry in flag.py.

The skill asks the agent to record every uncertain vault with VaultFlag.review_needed, so the doubt survives the run: the bundle is not in source control, but flag.py is, and its public note tells website readers the vault is under review. A vault with any entry, e.g. an older bad flag, counts as recorded. A missing entry only produces a warning, because the decisions themselves are still valid.

Parameters

decisions (dict[str, eth_defi.vault_report.vault_checks.CheckDecision]) – Decisions of the round.
entries (dict[str, str | None]) – flag.py entries, see read_flag_entries().

Returns

Vault ids of uncertain decisions missing from VAULT_FLAGS_AND_NOTES.

Return type

list[str]

read_flag_entries(flag_file)

Read the vault flags of VAULT_FLAGS_AND_NOTES from the flag.py source.

flag.py is parsed from source, because the agent edits it while this process already has the module loaded, so the imported VAULT_FLAGS_AND_NOTES is the pre-run state. Parsing instead of reloading also leaves the loaded module untouched for the rest of the report run and never executes agent-written code.

The dictionary is expected in the form the add-vault-note skill writes:

VAULT_FLAGS_AND_NOTES: dict[str, tuple[VaultFlag | None, str]] = {
    # Vault name
    "0xabc...": (VaultFlag.misleading_valuation, SOME_MESSAGE),
}

Only the flag is read; the message constant is not resolved. Keys that are not string literals, e.g. **OTHER unpacking, are skipped.

Parameters: flag_file (pathlib.Path) – Path to eth_defi/vault/flag.py.
Returns: Lowercased vault address -> VaultFlag member name, or None for a note without a flag.
Raises: CheckValidationError – When the dictionary is not found, e.g. it was renamed or the agent broke its literal form.
Return type: dict[str, str | None]

build_agent_prompt(round_)

Create the prompt that points the agent to the skill and the round’s files.

The prompt stays short and agent-neutral: the rules live in the skill, which Claude CLI and Codex CLI both open because the prompt says so. It hands over the digest the agent must copy into the decisions header, see read_check_decisions(), and repeats the rules whose violation would break the unattended run: asking a question would stall it, committing would bypass the flag.py review, and ending the turn before background sub-agents finish would leave no decisions file. Writing the decisions file last means a file that exists is a finished one.

Parameters

round – Check round.
round_ (eth_defi.vault_report.vault_checks.CheckRound) –

Returns

Prompt text.

Return type

str

build_agent_command(agent, prompt, model=None, effort=None)

Build the unsandboxed, web-enabled CLI command for the check agent.

The agent needs a shell for onchain reads with the repository’s probe script and JSON_RPC_* variables, web search and fetch for explorers, forums, X and news, and write access to the decisions file and flag.py. It runs without a sandbox: the product owner chose to keep it simple over the bwrap or container isolation a review proposed. The skill restricts it to reading and to those two writes, and every flag.py change is reviewed before merge.

Claude CLI, see the Claude Code CLI reference:
- -p print mode runs one unattended turn;
- --model and --effort default to CLAUDE_CHECK_MODEL and CLAUDE_CHECK_EFFORT to limit the token spend;
- --permission-mode dontAsk with an explicit --allowedTools list grants the tools without interactive prompts, which would stall an unattended run; --dangerously-skip-permissions is deliberately not used for an internet-connected agent;
- --output-format stream-json --verbose streams events line by line, so the transcript grows during the run and progress is visible; text mode buffers until the end and looks hung;
- --no-session-persistence keeps the run out of later claude --continue sessions.
Codex CLI, see the Codex CLI documentation:
- --search enables live web search; it is a top-level flag and must come before exec;
- exec --json streams JSONL events for the same reason as above;
- --ephemeral keeps no session;
- --sandbox danger-full-access disables the Codex sandbox, which would block the network and the RPC reads.

Read .claude/docs/agent-tricks-and-troubleshooting.md before changing the flags: it lists the failure modes they avoid, such as buffered output that looks hung, permission prompts and a CLI waiting on stdin.

Parameters

agent (Literal['claude', 'codex']) – claude or codex.
prompt (str) – Prompt text.
model (Optional[str]) – Optional model override; without one, Codex uses its configured default model.
effort (Optional[str]) – Optional Claude CLI thinking effort override; ignored by Codex.

Returns

Command and arguments.

Return type

list[str]

run_check_agent(command, cwd, log_path, decisions_path, timeout, poll_interval=60.0)

Run the agent CLI, streaming its JSONL output to a log file.

Guards against the ways an unattended agent run fails silently:

Stale output. The decisions file is deleted first, so a file from an earlier run can never be mistaken for this run’s result.
Waiting for input. stdin is closed (/dev/null); an open stdin makes codex exec wait for more prompt input forever.
Looking hung. stdout, the JSONL event stream, goes straight to the transcript file and stderr to a .err file next to it. Every poll_interval the elapsed time and the number of events so far are logged, so the operator can tell a working agent from a stuck one during a run that takes tens of minutes.
Runaway runs. The agent starts in a new session, i.e. its own process group. On timeout the whole group is killed, including the shells, Python probes and sub-agents the CLI started; killing only the CLI process would leave them running and spending tokens.
Abandoned sub-agents. In print mode the Claude CLI stops waiting for the agent’s background sub-agents after 600 seconds and exits, before the decisions file is written. CLAUDE_CODE_PRINT_BG_WAIT_CEILING_MS=0 removes that ceiling; timeout still bounds the run. Codex ignores the variable.

Parameters

command (list[str]) – Output of build_agent_command().
cwd (pathlib.Path) – Repository root.
log_path (pathlib.Path) – JSONL transcript of the agent run, vault-check-agent-N.jsonl in the report bundle; kept for review and debugging.
decisions_path (pathlib.Path) – Decisions file the agent must write.
timeout (float) – Seconds before the agent is stopped. Checked at each poll, so the run may overshoot it by up to poll_interval.
poll_interval (float) – Seconds between progress log lines and timeout checks.

Raises

CheckValidationError – When the agent fails, times out or does not write the decisions file. A clean exit without the file is a failure too: the CLI can end its turn early, e.g. when it stops waiting for sub-agents.

Return type

None

show_flag_diff(repository_root)

Show the agent’s uncommitted flag.py changes for review.

The agent never commits; the operator reviews this diff and ships it in a pull request of its own. The diff includes any uncommitted flag.py changes made before the run, which is why _check_round() compares it with the diff taken before the agent started.

Parameters: repository_root (pathlib.Path) – Repository root.
Returns: git diff output; empty when unchanged.
Return type: str

run_vault_checks(collect_lists, comparable_df, data_end_at, prices_path, output_dir, settings, max_rounds=3, aggregate_df=None, prescreen_min_tvl=1000000, max_escalations=20)

Check the report’s top lists in rounds until every listed vault is decided.

Rounds. Each round collects the top lists from the vaults not yet excluded, so when a vault is excluded the next one moves up into the list. Candidates already decided, overridden by the editor or out of scope are triaged without the agent; the rest go to _check_round(). The loop ends when a collection brings no new in-scope vault. The buffer in the lists means this is usually after the first or second round.

Round limit. If the last allowed round still leaves new in-scope vaults in the lists, the check raises instead of publishing them: a vault nobody checked would be shown as a recommendation, which is the failure the check exists to prevent. The original plan only logged a warning, which an operator can miss while the vault is published anyway. The finished rounds’ decisions are saved in the bundle, so a rerun with a higher ReportCriteria.check_max_rounds reuses them.

Prescreen. The average yield charts use hundreds of vaults, too many for the agent. After the top lists, the probes alone read the in-scope vaults of those charts with at least prescreen_min_tvl TVL. The prescreen never excludes a vault itself: vaults with suspicion signals, the largest first, go to the agent in a final round, at most max_escalations of them to bound the token spend. The facts already read are handed to that round, so the vaults are not probed twice.

Parameters

collect_lists (collections.abc.Callable[[pandas.DataFrame], dict[str, pandas.DataFrame]]) – Function (comparable_df) -> dict[list name, DataFrame] returning the top lists with their buffers, see eth_defi.vault_report.report.collect_top_lists().
comparable_df (pandas.DataFrame) – Eligible vaults with an identified protocol.
data_end_at (datetime.datetime) – Report data date.
prices_path (Optional[pathlib.Path]) – Vault price Parquet, for the liquidity history.
output_dir (pathlib.Path) – Report bundle; the round files are written here.
settings (eth_defi.vault_report.vault_checks.VaultCheckSettings) – Agent and reuse settings.
max_rounds (int) – Maximum number of top list rounds.
aggregate_df (Optional[pandas.DataFrame]) – Vaults of the average yield charts, indexed by vault id with protocol_slug and current_nav columns, or None to skip the prescreen.
prescreen_min_tvl (float) – Minimum TVL of a vault to prescreen, in US dollars.
max_escalations (int) – Maximum number of prescreened vaults sent to the agent.

Returns

Decisions of all rounds, with overrides applied.

Raises

CheckValidationError – When a round fails, or in-scope top-list vaults are still unchecked after max_rounds.

Return type

eth_defi.vault_report.vault_checks.CheckResult

apply_check_decisions(vaults_df, excluded)

Drop excluded vaults from a vault set.

The report applies it to the comparable vaults before any ranking, chart or the hero image is selected, and to the inflows and outflows; the TVL totals keep excluded vaults, because they report where money is, not where to invest.

Parameters

vaults_df (pandas.DataFrame) – Vaults indexed by vault id.
excluded (frozenset[str]) – Excluded vault ids, see CheckResult.excluded.

Returns

Vaults without the excluded ones.

Return type

pandas.DataFrame

render_excluded_vaults_markdown(result, data_end_at, report_title)

Render the vaults the investability check left out of a report, as a dated Markdown document.

The blog post does not list excluded vaults. This document is the record of what was left out and why, for the editor, the pull request review and the repository history. The report writes it as {date}-excluded-vaults.md to the bundle and to eth_defi/vault_report/excluded-vaults/; the operator commits it with the report’s pull request and posts it as a PR comment.

Sections: the excluded vaults in list and rank order, the uncertain vaults that stayed in the report under review, and a summary of the run with the decision counts and the decisions file names to look up the evidence. Agent-written text is flattened into plain table cells, see _markdown_cell(); the evidence URLs stay in the decisions files and report.json.

Parameters

result (eth_defi.vault_report.vault_checks.CheckResult) – Check result.
data_end_at (datetime.datetime) – Report data date.
report_title (str) – Title of the report post.

Returns

Markdown document.

Return type

str

excluded_rows(result)

Excluded vaults as plain records for the CSV and the manifest.

Used for tables/excluded.csv in the bundle, the excluded entry of report.json and the operator’s summary table in the report script.

Parameters: result (eth_defi.vault_report.vault_checks.CheckResult) – Check result.
Returns: One record per excluded vault, with the decision and its evidence.
Return type: list[dict]

summarise_checks(result)

Summary of the check for report.json.

Records the versions and each round’s candidates digest, so a later reader can tell which rules and which candidate lists the decisions were made under.

Parameters: result (eth_defi.vault_report.vault_checks.CheckResult) – Check result.
Returns: Counts, round files and the excluded vaults with evidence.
Return type: dict