Output Directory Schema

Each pipeline run creates a timestamped directory under the configured output_dir:

<output_dir>/<run_name>_<YYYYMMDD_HHMMSS>_<random_suffix>

The timestamp is generated in UTC. A random suffix and exclusive directory creation keep same-name, same-time runs separate. Use the returned run path; do not infer a path from the timestamp. Existing saved runs remain readable. On POSIX, newly allocated run directories have mode 0700 and manifests have mode 0600.

The runner creates every stage directory up front, then publishes complete run_manifest.json snapshots by atomic replacement as stages start, complete, skip, or fail. Each run has its own manifest and artefacts.

Directory tree

results/
  geo_panel_baseline_20260308_153000_a8k2m7q1/
    run_manifest.json
    00_run_metadata/
    05_prior_sensitivity/
    08_ai_advisor/
    10_pre_diagnostics/
    20_model_fit/
    30_model_assessment/
    35_holdout_validation/
    40_decomposition/
    50_diagnostics/
    55_ai_diagnostics_advisor/
    60_response_curves/
    70_optimisation/
    80_interpretation/
    scenario_planner/
      recipes/
        <recipe>_<timestamp>/

scenario_planner/recipes/ is a post-fit evidence area, not a pipeline stage. It appears only after a user evaluates a retained scenario recipe. Recipe evaluation does not mutate run_manifest.json or refit the model.

Stage directories

Stage Directory Typical artefacts
metadata 00_run_metadata resolved config, model metadata, and estimator contract
prior_sensitivity 05_prior_sensitivity scenario configs, human manifest, and LLM-safe manifest
ai_advisor 08_ai_advisor privacy-safe evidence, rule summary, optional LLM response, and optional patch proposal
preflight 10_pre_diagnostics prior predictive and estimator-specific design evidence
fit 20_model_fit fitted model, trace, posterior summary, and CRE post-fit screen
assessment 30_model_assessment in-sample posterior predictive checks and residual outputs
validation 35_holdout_validation blocked holdout scoring, uncertainty-aware metrics, and residual diagnostics
decomposition 40_decomposition contribution CSVs, CRE reconciliation, and decomposition plots
diagnostics 50_diagnostics raw input screening, MCMC, predictive, and residual diagnostic reports
ai_diagnostics_advisor 55_ai_diagnostics_advisor privacy-safe post-fit diagnostics evidence and optional LLM guidance
curves 60_response_curves saturation-only, forward-pass direct contribution, and adstock NetCDF, summaries, and plots
optimisation 70_optimisation allocation, response, optimisation summary, and bounds audit artefacts
interpretation 80_interpretation evidence inventory and analyst review reminder

See Runner Overview for the stage order and optionality.

Post-fit scenario recipe bundles

Each recipe evaluation creates a new directory under scenario_planner/recipes/. It contains the resolved request, validation evidence, fitted estimator manifest, allocation tables, posterior media-contribution summaries with 94% highest-density intervals, a versioned dashboard payload, and a SHA-256 artefact manifest.

See Comparison Outputs for the complete file contract and interpretation boundary.

Main artefacts by stage

00_run_metadata

Main files:

  • a copy of the original config under its source filename
  • config.original.yaml
  • config.resolved.yaml
  • session_info.txt
  • dataset_metadata.json
  • model_metadata.json
  • data_dictionary.csv
  • design_matrix_manifest.csv
  • spec_summary.csv
  • estimator_summary.txt and estimator_manifest.yaml for named estimators
  • estimator_estimability.csv for the resolved estimator screen summary
  • holiday_feature_manifest.csv when holidays are configured

config.resolved.yaml normalises configured data and holiday paths to absolute paths and records the effective sampler configuration on the model.

05_prior_sensitivity

Main files:

  • scenario_manifest.yaml
  • llm_safe_scenario_manifest.yaml
  • <scenario_name>/config.resolved.yaml for each generated or declared scenario

This stage is optional. When prior_sensitivity is absent or disabled in YAML, the directory still exists and the stage is marked skipped.

scenario_manifest.yaml is the local, human-readable manifest. It can include scenario descriptions and raw override paths. Use llm_safe_scenario_manifest.yaml when passing scenario context to an external LLM because it aliases override paths and avoids free-text scenario prose.

08_ai_advisor

Main files:

  • evidence.json
  • rules_summary.json
  • advisor_status.json
  • advisor_response.json, when an LLM advisor call succeeds
  • <provider>_response.raw.json, when an LLM advisor call succeeds
  • advisor_recommendations.md, when an advisor response is available
  • config_patch_proposal.yaml, when the advisor proposes a YAML patch
  • approval_request.yaml, when the proposed patch is valid and ready for file-based approval
  • advisor_error.json, when the LLM call fails
  • config_patch_error.json, when an advisor patch is invalid or unsafe

This stage is optional. When ai_advisor is absent or disabled in YAML, the directory still exists and the stage is marked skipped.

The advisor evidence is privacy-safe by design: it aliases media channels, prior paths, and prior specs, and excludes raw business values. Deterministic rule checks always run before the optional LLM call. If those rules mark the payload as unsafe, the LLM call is blocked and the status artifact records the reason.

Patch proposals are validated before an approval request is written. Supported patches use an overrides: mapping with safe prior paths only. After a user sets approval_request.yaml to status: approved, the file-based approval CLI writes approved_config.resolved.yaml and approval_record.yaml in this directory.

10_pre_diagnostics

Main files:

  • prior_predictive.nc
  • prior_predictive.png
  • fixed_effects_estimability.csv and fixed_effects_estimability.json for FE
  • cre_structural_estimability.json, cre_reference_estimability.json, and cre_reference_estimability_features.csv for CRE

20_model_fit

Main files:

  • model.nc
  • trace.png
  • posterior_summary.csv
  • cre_postfit_estimability.json for CRE

posterior_summary.csv is intentionally compact. It summarizes structural posterior parameters such as adstock, saturation, seasonality, holiday, and likelihood terms, and omits per-date deterministic series like weekly channel contributions or fitted paths. Use the assessment and decomposition stages for time-indexed fitted or contribution outputs.

30_model_assessment

Main files:

  • posterior_predictive.nc
  • posterior_predictive.png
  • posterior_predictive_summary.csv
  • observed.csv
  • fitted.csv
  • fit_timeseries.png
  • fit_scatter.png
  • residuals.csv
  • residuals_timeseries.png
  • residuals_hist.png
  • residuals_vs_fitted.png

This stage is the in-sample or training-fit assessment. It uses the same data the model was fit on and should not be read as the pipeline’s out-of-sample validation layer.

35_holdout_validation

Main files:

  • validation_metadata.json
  • holdout_posterior_predictive.nc
  • holdout_predictive_summary.csv
  • holdout_predictive_report.json
  • holdout_observed.csv
  • holdout_fitted.csv
  • holdout_residuals.csv
  • holdout_timeseries.png
  • holdout_residuals_acf.png

The holdout summary and report include uncertainty-aware metrics such as crps, bias, and fixed coverage columns for coverage_50, coverage_80, and coverage_94.

This stage is optional. When validation is absent or disabled in YAML, the directory still exists and the stage is marked skipped.

For interpretation guidance and practical rules of thumb, see Blocked Holdout Validation.

40_decomposition

Main files:

  • waterfall_components_decomposition.png
  • weekly_media_contribution.png
  • channel_contributions.csv
  • baseline_contributions.csv
  • mean_contributions_over_time.csv
  • cre_adjustment_contributions.csv for CRE
  • cre_decomposition_reconciliation.csv and cre_decomposition_reconciliation.json for CRE

The CRE adjustment is retained on the baseline or non-incremental side of the decomposition. Do not report it as an incremental media contribution.

50_diagnostics

Main files:

  • design_summary.csv
  • design_report.json
  • vif_report.csv
  • mcmc_summary.csv
  • mcmc_report.json
  • predictive_summary.csv
  • predictive_report.json
  • residual_diagnostics.csv
  • residuals_acf.png
  • diagnostics_report.csv
  • diagnostic_gates.resolved.yaml
  • diagnostics_summary.txt
  • chain_diagnostics.txt

The design-oriented files are raw input screening outputs. In particular, diagnostics_report.csv labels the corresponding phase as raw_input_screening rather than design.

diagnostic_gates.resolved.yaml records the profile name, source, inline override status, exact boundary rule, and effective warn/fail values used for each check. It is the audit trail for the run’s diagnostic decisions.

55_ai_diagnostics_advisor

Main files:

  • diagnostics_evidence.json
  • diagnostics_rules_summary.json
  • diagnostics_advisor_status.json
  • diagnostics_advisor_response.json, when an LLM advisor call succeeds
  • <provider>_diagnostics_response.raw.json, when an LLM advisor call succeeds
  • diagnostics_advisor_recommendations.md, when an advisor response is available
  • diagnostics_advisor_error.json, when the LLM call fails

This stage runs after 50_diagnostics by default for enabled ai_advisor blocks. Set ai_advisor.diagnostics_review_enabled: false to skip it.

The diagnostics advisor evidence is restricted to anonymized channel aliases, convergence counts, normalized predictive metrics, coverage metrics, and scale-free design diagnostics. Raw target-scale fit errors are intentionally excluded from the LLM payload.

The advisor separates computational reliability, raw-design identification risk, predictive evidence, residual structure, prior robustness, and causal identification. Its deterministic decision state is a floor: the LLM can make the result stricter, but cannot soften a failed or warning gate.

60_response_curves

Main files:

  • saturation_curve.nc
  • saturation_curve_summary.csv
  • saturation_curve.png
  • forward_pass_contribution_curve.nc
  • forward_pass_contribution_curve_summary.csv
  • forward_pass_contribution_curve.png
  • adstock_curve.nc
  • adstock_curve_summary.csv
  • adstock_curve.png

These artefacts are intentionally different:

  • saturation_curve.* is the sampled saturation transformation on the scaled channel axis, exported with original-scale contribution values for easier reading. The PNG overlays that saturation-only curve against posterior mean realised contributions.
  • forward_pass_contribution_curve.* is a full-model direct contribution artefact. It rescales the observed historical spend path from 0% to 200%, runs that spend through the fitted adstock and saturation path, and records the resulting total channel contribution in original target units.
  • adstock_curve.* is the sampled carryover-weight profile for one impulse.

70_optimisation

This directory is present for every run, but the stage is skipped unless the YAML config contains an optimization block.

Main files when the stage runs:

  • optimized_allocation.nc
  • optimized_allocation.csv
  • response_distribution.nc (optimised expected media contribution)
  • reference_response_distribution.nc (reference allocation replay)
  • contribution_difference.nc (paired optimised minus reference draws)
  • budget_contribution_difference.csv (total paired difference summary)
  • optimize_result.json
  • budget_summary.csv
  • budget_response_points.csv
  • budget_impact.csv
  • budget_bounds_audit.csv
  • budget_roi_cpa.csv
  • budget_response_curves.csv
  • budget_mroi.csv
  • budget_optimisation.json
  • several PNG plots for allocation, contribution over time, response curves, impact, bounds audit, and ROI or CPA

The two response datasets use identical posterior draws and outcome dates, zero initial history, a shared time profile and no allocation or observation noise. current means a reference allocation replay, not fitted historical attribution. budget_optimisation.json records the comparison contract. budget_impact.csv includes channel-level paired difference HDIs; total difference HDIs come from summed paired draws, not sums of interval bounds.

Pipeline hdi_* fields now use ArviZ single-interval HDIs. Earlier pipeline exports used equal-tailed bounds under these names. Recompute summaries from retained draws; do not relabel historical files as corrected HDIs. Predictive coverage remains a separate equal-tailed calculation.

80_interpretation

Main files:

  • evidence_inventory.md
  • interpretation_report.md

Both files currently contain the same evidence inventory: preceding stage statuses, the holdout validation status and a reminder to review the retained artefacts. They do not provide automated business conclusions. A completed stage records successful report generation, not statistical qualification.

run_manifest.json

The manifest is the machine-readable index for the whole run.

Top-level fields include:

Field Meaning
run_name Effective run name
timestamp UTC run timestamp
config_path Original config path
output_dir Run directory path
status Overall run status
model_class Identifies the model object prepared in Stage 00; does not certify graph completion
data Basic dataset metadata
stages Per-stage manifest records
warnings Run-level warnings
error Run-level failure payload when the pipeline aborts

data includes:

  • x_shape
  • y_length
  • target_column
  • x_columns

Stage records

Each stage record contains:

Field Meaning
directory Stage directory name
status Current stage status
started_at ISO timestamp when the stage started
finished_at ISO timestamp when the stage finished
artifacts Mapping of artefact labels to root-relative paths
warnings Stage warnings
error Error string when the stage fails

The artifacts mapping uses root-relative paths such as 20_model_fit/model.nc.

Stage statuses

Status Meaning
pending Stage has not started yet
running Stage is currently running
completed Stage finished successfully
skipped Stage returned None intentionally
failed Stage raised an exception
not_reached A previous stage failed before this one ran

Common cases:

  • Stage 35 is skipped when validation is missing or disabled from YAML.
  • Stage 70 is skipped when optimization is missing from YAML.
  • Later stages become not_reached after the first failure.

Practical use

Use the run directory when you want:

  • a stable folder for downstream reporting
  • a machine-readable audit trail through run_manifest.json
  • stage-level links to artefacts without hard-coding filenames

If you want to add new artefact types or stages, see Extending the Runner.