Development Setup

This page describes the supported local setup for working on Abacus. The project is maintained with local verification scripts rather than a CI-first workflow, so your development environment needs to be able to run linting, pytest, and the packaging smoke checks directly.

Prerequisites

  • Python 3.12
  • A local environment manager such as Conda
  • A writable temporary directory such as /tmp for PyTensor caches and package verification artefacts

Create the Development Environment

The simplest supported path is the repository environment file:

conda env create -f environment.yml
conda activate abacus-dev
python3 -m pip install -e .

If you know you will be running linting and tests frequently, install the optional extras as well:

python3 -m pip install .[lint,test]

Local Runtime Defaults

Some parts of the stack need writable cache directories. In restricted or sandboxed environments, set the same defaults used by the repo’s local verification scripts:

export PYTENSOR_FLAGS="base_compiledir=/tmp/pytensor,linker=py"
export JAX_PLATFORMS=cpu
export XDG_CACHE_HOME=/tmp

The Makefile already applies these defaults for make test and make smoke_mmm.

Common Commands

Lint and format

make check_lint
make lint
make check_format
make format

These targets check abacus, tests, scripts and runme.py. MyPy uses its configured file list in pyproject.toml; sandbox/ is outside these targets.

Tests

make test
pytest tests/<path>/test_*.py -v

Use targeted pytest first when you are working on a narrow area. Run the wider verification commands before closing substantial changes.

Local verification

make smoke_mmm
make verify_local
make verify_package
make verify_local_all

What these commands do:

  • make smoke_mmm runs the full timeseries demo with its configured main-fit and holdout-refit budgets. It does not reduce sampling. For a small execution check, use the bounded software smoke.
  • make verify_local runs formatting, linting, typing, and the configured test suite in sequence.
  • make verify_package builds package artefacts and validates an installed package smoke path.
  • make verify_local_all runs the local verification matrix and includes the packaging smoke step.

The package verifier builds a source distribution and wheel in a temporary workspace. It creates a clean venv without system or user packages, resolves runtime dependencies, runs pip check, and checks installed imports and assets from outside the checkout. A seeded, tiny model exercises fitting, prediction, metrics and save/load. This is an installation smoke check, not convergence or statistical qualification.

make verify_package runs both reviewed profiles in requirements/:

  • verification-current.txt pins the reviewed direct runtime stack.
  • verification-minimum.txt pins every direct runtime dependency to its declared floor, including NumPy 2.0.0, scikit-learn 1.4.2 and SciPy 1.15.0.

These profiles were exercised on Linux with Python 3.12. Runtime lower bounds now match that tested minimum stack; older versions are no longer declared supported. This does not establish every allowed combination or other Python/platform combinations. Transitive versions are resolved by pip and printed in the log. Retain that output with release verification evidence. Review and rerun both profiles when changing the dependency stack; inference upper bounds remain in pyproject.toml.

To exercise an unconstrained resolution within the package metadata bounds:

python3 scripts/run_package_verification.py

The scikit-learn floor provides the required RMSE API and NumPy 2 support. See the upstream RMSE API and 1.4.2 release notes.

Important Working Files

File Why it matters
Makefile Primary local entry point for lint, test, smoke, and package verification
environment.yml Supported dev environment definition
pyproject.toml Packaging metadata, extras, Ruff, MyPy, and pytest configuration
scripts/run_package_verification.py Package build and installed-wheel smoke verification
ARCHITECTURE.md Contributor-facing module map and dependency rules

Local-Only Areas

The repo contains some directories that are useful locally but are not part of the shipped library surface:

  • .archive/ for archived planning and reference material
  • .planning/standards/ for local documentation and writing standards
  • sandbox/ for ignored local scratch work

Keep temporary scripts in sandbox/ rather than mixing them into the package.

Troubleshooting

PyTensor cache permission errors

If you see errors related to .pytensor lock files or compiledir creation, export the runtime defaults shown above and rerun the command.

Package verification fails because build is missing

Run:

python3 -m pip install build

The make verify_package target does this automatically.

You are not sure which command to run

As a rule:

  • run targeted pytest and ruff while iterating
  • run make verify_local before finishing non-trivial code changes
  • run make verify_package when packaging, imports, or bundled assets changed