Quickstart: Pipeline Runner

Use the pipeline runner when you want a full staged run instead of only an in-memory model fit.

The runner writes:

• a run manifest
• copied and resolved config files
• fitted model artefacts
• posterior predictive assessment outputs
• decomposition, diagnostics, and response-curve artefacts

Full bundled demo

From the repository root, run the complete demo with:

python3 runme.py --demo timeseries

Other bundled demos are:

• timeseries_controls
• geo_fe
• geo_cre
• geo_panel
• geo_brand_panel

timeseries and timeseries_controls use the released time_series estimator preset. geo_fe and geo_cre use the released one-unit FE and CRE presets. geo_panel and geo_brand_panel use the advanced dimensions.panel surface; they are not named RE, FE, or CRE estimators. The named RE preset remains release-gated. See the demo catalogue for the exact semantics and release status of each recipe.

List them explicitly with:

python3 runme.py --list-demos

runme.py is a convenience wrapper around the structured pipeline. It resolves the demo config under data/demo/<demo_name>/config.yml and runs the pipeline for you.

Run the named panel presets with:

python3 runme.py --demo geo_fe
python3 runme.py --demo geo_cre

Read Choose an Estimator before treating either recipe as the basis for a real model.

The timeseries demo requests four chains with 2,000 tuning and 3,000 retained draws per chain for Stage 20. Its enabled validation stage separately requests four chains with 2,000 tuning and 2,000 retained draws per chain. Explicit validation.sampler settings override main-fit CLI settings for that refit. make smoke_mmm launches this full demo without reducing either budget.

Bounded software smoke

Use this synthetic example to check execution with a small, explicit budget. Run the following from the repository root in the installed Abacus environment. It creates a separate scratch configuration and dataset, leaving the bundled demos unchanged. Repeating the setup overwrites these two scratch inputs; each pipeline execution creates a new run directory.

from pathlib import Path

import numpy as np
import pandas as pd
import yaml

workspace = Path("sandbox/runner-smoke")
workspace.mkdir(parents=True, exist_ok=True)
rng = np.random.default_rng(42)
data = pd.DataFrame({
    "date": pd.date_range("2024-01-01", periods=24, freq="W-MON"),
    "tv": rng.uniform(10, 100, 24),
    "search": rng.uniform(5, 50, 24),
})
data["revenue"] = 100 + 0.4 * data["tv"] + 0.8 * data["search"] + rng.normal(0, 2, 24)
data.to_csv(workspace / "data.csv", index=False)
config = {
    "data": {"dataset_path": "data.csv", "date_column": "date"},
    "target": {"column": "revenue", "type": "revenue"},
    "estimator": {"type": "time_series"},
    "media": {
        "channels": ["tv", "search"],
        "adstock": {"type": "geometric", "l_max": 4},
        "saturation": {"type": "logistic"},
    },
    "original_scale_vars": ["y", "channel_contribution"],
    "fit": {
        "draws": 20, "tune": 20, "chains": 2, "cores": 1,
        "random_seed": 42, "progressbar": False,
    },
    "validation": {"enabled": False},
}
(workspace / "config.yml").write_text(yaml.safe_dump(config), encoding="utf-8")

Then run:

python3 -m abacus.pipeline.runner \
  --config sandbox/runner-smoke/config.yml \
  --output-dir sandbox/runner-smoke/results \
  --run-name execution_check \
  --prior-samples 5 \
  --random-seed 42 \
  --curve-samples 10 \
  --curve-points 10

Stage 20 uses two chains, each with 20 tuning and 20 retained draws, on one core. Stage 35 performs no fit: run_manifest.json must record validation as skipped, and no holdout scoring artefacts are expected. Prior sensitivity, both AI stages and optimisation are also skipped because their configuration blocks are absent. The run should complete through Stage 80’s evidence inventory. The resolved main-fit settings are in 00_run_metadata/config.resolved.yaml.

These small sampling budgets check software execution only. Warnings or retained divergences can occur; do not interpret these fitted estimates. To exercise holdout scoring, prepare a separate config with explicit budgets for both fits and follow Blocked Holdout Validation. A skipped validation stage is not evidence of predictive performance.

Run the pipeline from Python

The direct Python API below runs the bundled geo-panel model. It is separate from the bounded synthetic smoke above. Check the selected YAML’s validation budget before running it:

from pathlib import Path

from abacus.pipeline import PipelineRunConfig, run_pipeline

result = run_pipeline(
    PipelineRunConfig(
        config_path=Path("data/demo/geo_panel/config.yml"),
        output_dir=Path("results"),
        run_name="geo_panel_quickstart",
        prior_samples=10,
        draws=200,
        tune=200,
        chains=2,
        cores=2,
        random_seed=42,
        curve_samples=50,
        curve_points=50,
    )
)

print(result.run_dir)
print(result.manifest_path)

If the YAML config already contains data.dataset_path, you do not need to pass dataset_path again.

Run the thin CLI directly

The pipeline also exposes a thin CLI in abacus.pipeline.runner:

python3 -m abacus.pipeline.runner \
  --config data/demo/geo_panel/config.yml \
  --output-dir results \
  --run-name geo_panel_quickstart \
  --prior-samples 10 \
  --draws 200 \
  --tune 200 \
  --chains 2 \
  --cores 2 \
  --random-seed 42 \
  --curve-samples 50 \
  --curve-points 50

The CLI prints the final run directory and manifest when the pipeline completes. When the run manifest records them, it also prints paths to the estimator summary, diagnostics summary, and interpretation report. These paths are relative to the run directory.

Pipeline completion means that the configured stages finished. It does not mean that the fitted model passed its diagnostic gates or is suitable for interpretation. Review the listed diagnostic and interpretation artefacts before using model outputs.

Override data paths

Use one of these patterns:

Configured relative paths are resolved relative to the YAML config directory.

If you want Stage 50 to use different warn/fail cutoffs, add a runner-only diagnostics.thresholds block to the YAML. See YAML Configuration.

What you get back

run_pipeline(...) returns a PipelineRunResult with:

• run_dir
• manifest_path

The runner creates all stage directories up front, including those for skipped stages. Use the manifest to distinguish completed, skipped and failed work. See the canonical stage sequence and artefact locations, including Stage 80’s evidence inventory.

Named estimator runs also record their resolved contract in 00_run_metadata/estimator_summary.txt and 00_run_metadata/estimator_manifest.yaml. FE writes its transformed within-design screen under 10_pre_diagnostics. CRE writes structural and reference summary-basis screens under 10_pre_diagnostics, a bounded post-fit screen under 20_model_fit, and separate CRE-adjustment decomposition evidence under 40_decomposition.

60_response_curves now includes three complementary curve families:

• saturation-only transformation artefacts
• forward-pass direct contribution artefacts built from scaled observed history
• adstock carryover artefacts

When to use the runner

Choose the runner when you want:

• a reproducible run directory on disk
• structured metadata and manifest files
• staged artefacts for diagnostics and reporting
• a config-driven workflow for repeated runs

If you only need to fit a model interactively in a notebook or script, start with Quickstart: Python API or Quickstart: YAML Builder.