Zero to gated agent in five minutes.
No API keys. No external accounts. No telemetry daemons. One clean install, one recorded golden trace, and an automated regression gate protecting every PR.
From local execution to automated CI gate.
Install the CLI & Library
Install AgentDiff into your virtual environment via pip or uv. Pure Python, zero telemetry, and zero background daemons.
# Install via pip
pip install agent-trajectory-diff
# Or add to uv project
uv add agent-trajectory-diffInitialize Your Project with agentdiff init
Run the zero-config wizard. It auto-detects your agent framework (LangGraph, CrewAI, OpenAI Agents, OpenTelemetry), writes agentdiff.toml, and generates your GitHub Actions CI gate workflow.
# Auto-detect framework and generate config + gate workflow
agentdiff init --scenario customer_support --runs 3
# Or include the /agentdiff approve bot workflow
agentdiff init --with-approveRecord an N-Run Baseline Envelope
Capture variance bands across multiple runs so non-deterministic agents don't flake the CI gate on harmless timing or token jitter.
# Record a 3-run baseline envelope
agentdiff record my_agent_module:run \
--input '{"query": "Generate Q3 sales analysis"}' \
--runs 3 \
--out baselines/customer_support.envelope.jsonGate CI & Bless with /agentdiff approve
Run the comparator in CI. Hard invariants (loops, error cascades) strictly block merging, while reviewers can bless intended improvements directly from PR comments.
# Diff candidate run against statistical baseline envelope
agentdiff diff baselines/customer_support.envelope.json traces/pr_candidate.json \
--fail-on-regression \
--pr ${{ github.event.number }}
# Reviewers bless accepted improvements right on GitHub:
# Comment: /agentdiff approveTwo ways to integrate in your workflow.
Whether you prefer unit test assertions in pytest or repository-wide threshold governance in agentdiff.toml:
Programmatic Pytest Assertions
Run your golden test baselines directly through standard pytest runners. Automatically flag regressions with expressive assert_no_regressions() assertions.
def test_agent_checkout():
baseline = load_trace("golden.json")
candidate = run_agent(input_data)
assert_no_regressions(
compare(baseline, candidate),
max_divergence=0.25,
allow_loops=False
)Declarative Governance Policy
Commit your regression thresholds once in agentdiff.toml. Audit threshold alterations directly on PRs to prevent silent test weakening.
[scenario.default]
mode = "statistical"
sample_runs = 3
max_cost_increase_pct = 5.0
[scenario.default.hard_invariants]
fail_on_identical_loops = true
max_tool_repeats = 3
[scenario.default.tolerances]
step_count_std_dev = 2.0
divergence_ceiling = 0.35Ready to automate on GitHub?
Install the official GitHub App or drop the Action into your repository for instant diagnostic comments and in-PR approval.