API Reference

Catalog

pamir.load_catalog()[source]

Return a DataFrame with one row per dataset.

Columns: id, name, rows, features, defaults, DR, source, license, geography, product, target_definition.

Return type:

DataFrame

pamir.list_datasets()[source]

Return sorted list of dataset ids.

Return type:

List[str]

pamir.dataset_info(dataset_id)[source]

Return full metadata dict for one dataset.

Raises KeyError if the id is not in the catalog.

Return type:

Dict

Parameters:

dataset_id (str)

Data loading

pamir.load_dataset(dataset_id, max_rows=None, auto_download=True)[source]

Load a single dataset from the local cache.

Parameters:

  • dataset_id (str) – One of the PaMIR dataset ids (see pamir.list_datasets()).

  • max_rows (Optional[int]) – Truncate to at most this many rows (in stream order).

  • auto_download (bool) – If the dataset is not cached, fetch and harmonize it from source (default True). Set False to require an explicit pamir.download().

Return type:

Tuple[DataFrame, ndarray, Dict]

Returns:

  • X (DataFrame) – Feature columns exactly as stored: int64, float64, bool and object all occur, and most of the datasets carry at least one non-numeric column. Nothing is encoded or imputed for you — see pamir.encode_features() for the glue most models need.

  • y (ndarray of int) – Binary target (1 = default, 0 = non-default).

  • meta (dict) – Dataset metadata from the catalog, plus n_rows, n_defaults, n_features. The catalog’s notes key is present only on the datasets that required harmonization, so read it with .get().

Downloading data

PaMIR ships no data. These fetch each dataset from its original source and harmonize it locally (see Datasets).

pamir.download(dataset_id, force=False, quiet=False)[source]

Fetch and harmonize one dataset; return the path to the cached parquet.

Parameters:

  • dataset_id (str) – A PaMIR dataset id (see pamir.list_datasets()).

  • force (bool) – Re-fetch and re-harmonize even if a cached parquet exists.

  • quiet (bool) – Suppress progress / validation messages.

Return type:

Path

pamir.cache_dir()[source]

Directory where fetched raw files and harmonized parquet are cached.

Return type:

Path

Baselines

pamir.encode_features(X_train, X_test)[source]

Ordinal-encode non-numeric columns against one shared level set.

Both frames are returned as float, so a level present in only one of them still maps to the same code in both. Numeric missing values are left as NaN — that is information, and filling it is the caller’s decision.

Parameters:

  • X_train (DataFrame) – Feature frames with identical columns, in identical order.

  • X_test (DataFrame) – Feature frames with identical columns, in identical order.

Returns:

(train, test) – New frames; the inputs are not modified.

Return type:

Tuple[DataFrame, DataFrame]

Raises:

ValueError – If the two frames do not carry the same columns in the same order.

pamir.logistic_baseline(X_train, y_train, X_test)[source]

Median-imputed, standardized logistic regression.

The cheap reference point: fast enough to run the whole fleet under the default protocol parameters.

Return type:

ndarray

Parameters:
pamir.gbdt_baseline(X_train, y_train, X_test, max_iter=150, learning_rate=0.1, seed=42)[source]

Histogram gradient boosting, handling missing values natively.

Uses scikit-learn rather than XGBoost so it needs no optional dependency, and so it cannot hit the OpenMP clash documented in the installation notes.

Return type:

ndarray

Parameters:

i.i.d. evaluation

pamir.evaluate_iid(predict_fn, datasets=None, train_frac=0.7, max_n=None, n_seeds=5, verbose=True, on_error='warn')[source]

Run the i.i.d. protocol across the PaMIR fleet.

Return type:

DataFrame

Returns:

  • DataFrame with one row per dataset and the columns in RESULT_COLUMNS.

  • Pass it to pamir.fleet_summary() for the headline numbers.

Parameters:
pamir.evaluate_iid_one(predict_fn, dataset_id, train_frac=0.7, max_n=None, n_seeds=5, on_error='warn')[source]

Evaluate one dataset with a standard i.i.d. train/test split.

Parameters:

  • predict_fn (Callable) – predict_fn(X_train, y_train, X_test) -> np.ndarray of scores (higher = more likely to default).

  • dataset_id (str) – PaMIR dataset id.

  • train_frac (float) – Fraction of data used for training (default 0.7).

  • max_n (Optional[int]) – Truncate dataset before splitting.

  • n_seeds (int) – Number of random shuffles (default 5). The reported AUC is the mean.

  • on_error (str) – What to do when predict_fn raises. See pamir.failures.

Return type:

Dict

Returns:

  • dict with keys (dataset, n_rows, n_defaults, DR, train_frac, n_seeds,)

  • n_calls, n_failures, errors, auc_mean, auc_std, aucs.

Streaming evaluation

pamir.evaluate(predict_fn, datasets=None, lag=1000, k_refit=10, max_n=20000, min_defaults=3, verbose=True, on_error='warn')[source]

Run the streaming protocol across the full PaMIR fleet.

Parameters:
Return type:

DataFrame

Returns:

  • DataFrame with one row per dataset and the columns in RESULT_COLUMNS.

  • Pass it to pamir.fleet_summary() for the headline numbers — a mean

  • taken by hand over this frame silently excludes the datasets the model

  • failed on.

pamir.evaluate_one(predict_fn, dataset_id, lag=1000, k_refit=10, max_n=20000, min_defaults=3, on_error='warn')[source]

Run the streaming protocol on one dataset.

Parameters:

  • predict_fn (Callable) – predict_fn(X_train, y_train, X_test) -> np.ndarray of scores (higher = more likely to default). Called once per refit point. X_train contains only rows whose outcome is resolved; X_test is the remaining unscored stream, and the returned array must have one score per row of it.

  • dataset_id (str) – PaMIR dataset id.

  • lag (int) – Label-maturation delay. Outcome of application at position t is revealed only at position t + lag.

  • k_refit (int) – Refit every k_refit newly resolved defaults. This changes the score as well as the cost — see the protocol documentation — so report it.

  • max_n (Optional[int]) – Truncate the stream to at most this many rows.

  • min_defaults (int) – Minimum resolved defaults before the first refit.

  • on_error (str) – What to do when predict_fn raises. See pamir.failures.

Return type:

Dict

Returns:

  • dict with keys (dataset, n_rows, n_defaults, DR, n_refits, n_calls,)

  • n_failures, errors, auc_final, refit_points (list of dicts with pos,

  • resolved_n, cum_defaults, cum_auc).

Results

pamir.fleet_summary(results)[source]

Summarize a fleet run, withholding the mean when coverage is incomplete.

Parameters:

results (DataFrame) – As returned by pamir.evaluate() or pamir.evaluate_iid().

Return type:

Dict[str, Optional[float]]

Returns:

  • dict with keys

  • n_datasets, n_scored, coverage, complete – How much of the fleet the model actually scored.

  • auc_mean, gini_mean – The headline numbers — None unless every dataset was scored.

  • auc_mean_scored_only – The average over the datasets that did score. Diagnostic only: it is an average over a subset the model chose by failing, so it is not comparable across models.

  • n_failures, failed_datasets – What went wrong and where.

Synthetic augmentation

class pamir.synthetic.SyntheticMixer(generators, weights=None, synthetic_share=None, n_synthetic=None, target=None, sizing='keep_real', n_total=None, stratify=True, synthetic_target_rate=None, oversample=1.5, allow_replacement=False, fidelity=True, fidelity_kwargs=None, pooled_fidelity=True, leakage_probes=True, shuffle=True, random_state=42)[source]

Build a training frame from real rows plus several synthetic engines.

Parameters:

  • generators (Mapping[str, Any]) – {name: generator}. Each value may be a GeneratorAdapter, an SDV synthesizer class or instance (zGAN is one), a pre-generated pool, or any object with fit/sample — as_adapter() wraps it.

  • weights (Optional[Mapping[str, float]]) – How the synthetic part splits across generators, e.g. {"zgan": 0.5, "zedge": 0.5}. Defaults to equal weights.

  • synthetic_share (Optional[float]) – Share of synthetic rows in the mixed frame, in [0, 1). Defaults to 0.5 when neither this nor n_synthetic is given.

  • n_synthetic (Optional[int]) – Exact number of synthetic rows, in place of a share. Under cross-validation this is per fold, so the count is identical in every fold, whereas a share resolves against each fold’s train split and therefore varies with it. Mutually exclusive with synthetic_share.

  • target (Optional[str]) – Target column. Enables stratification, class-balance control and the ML-efficacy fidelity arm.

  • sizing (str) – keep_real adds synthetic rows on top of every real row. fixed_total pins the frame to n_total rows.

  • stratify (bool) – Preserve the target distribution when subsampling real or synthetic rows.

  • synthetic_target_rate (Optional[float]) – Force the synthetic part to this positive rate (e.g. 0.5 to rebalance a 3%-default book). Requires oversampling; raises if the pool cannot supply enough rows of a class.

  • oversample (float) – Multiplier on how many rows each generator is asked for, so that class-balance selection has slack. Skipped for a fixed pool, which cannot generate more rows than it holds.

  • fidelity (bool) – Compute fidelity metrics for each generator’s rows against the real training rows. A full report costs roughly 11 s per generator per fold on a 1k x 20 table, so switch it off for a large fleet sweep and measure fidelity separately. The leakage probes are not switched off with it — see leakage_probes.

  • leakage_probes (bool) – Run the exact-overlap probes whenever a holdout frame is given, whatever fidelity is set to. They cost milliseconds, and a delta reported without them cannot be checked, so switching the battery off for a fleet sweep leaves them running. Setting this to False is the only way to stop them, in either mode.

  • pooled_fidelity (bool) – Also score the concatenation of every generator’s rows as __pooled__. Switch off to save one report per fold.

  • random_state (int) – Seed for every selection step; generators receive derived seeds.

  • n_total (Optional[int])

  • allow_replacement (bool)

  • fidelity_kwargs (Optional[Dict])

  • shuffle (bool)

clone()[source]

A copy with unfitted generators, for the next CV fold.

Return type:

SyntheticMixer

fit(train, fold=None, only=None)[source]

Fit the generators on train — and only on train.

only restricts the work to the named generators. build() passes the ones its plan actually draws from, so a generator weighted to zero rows is not trained for nothing: for a GAN or a diffusion model that is a full training run per fold, thrown away.

Return type:

SyntheticMixer

Parameters:
build(train, holdout=None, fold=None, fit=True)[source]

Fit (optionally), generate, mix, and score.

Parameters:

  • train (DataFrame) – Real training rows including the target column. This is the only data any generator ever sees.

  • holdout (Optional[DataFrame]) – Held-out rows, passed to the fidelity report for leakage probes only — never used to fit, generate or score fidelity itself.

  • fold (Optional[int]) – Fold index, forwarded to pool-based adapters so they pick the pool that belongs to this fold.

  • fit (bool) – Set False to reuse generators already fitted on this same frame.

Return type:

MixResult

class pamir.synthetic.CrossValidatedAugmentation(mixer, n_splits=5, seed=42, predict_fn=None, baseline=True, target='__target__', verbose=True)[source]

Run a SyntheticMixer inside stratified k-fold, without leakage.

Parameters:

  • mixer (SyntheticMixer) – Template. It is cloned per fold; the original is never fitted.

  • n_splits (int) – Folds. 5 matches the upstream ceiling protocol.

  • seed (int) – Seed for the fold split and for the default model arm, so changing it moves the delta on unchanged data — state which seed a run used before comparing two runs. 42 matches public_datasets/ceiling_cv.py; the audit’s scripts/03_ceiling_compare.py uses 0. A predict_fn of your own receives no seed; seed it yourself if it needs one.

  • predict_fn (Optional[Callable]) – predict_fn(X_train, y_train, X_test) -> scores, the same convention as pamir.evaluate(). Defaults to upstream’s XGBoost arm.

  • baseline (bool) – Also fit on the real training rows alone, to report a delta rather than a bare number.

  • target (str)

  • verbose (bool)

run(X, y, dataset=None)[source]

Fit, mix and score fold by fold. X holds features only.

Return type:

CVResult

Parameters:
run_dataset(dataset_id, max_rows=None)[source]

Same, on a PaMIR dataset loaded by id.

Return type:

CVResult

Parameters:

  • dataset_id (str)

  • max_rows (int | None)

class pamir.synthetic.CVResult(folds, oof_augmented, oof_baseline, y, dataset=None)[source]

Pooled out-of-fold outcome of an augmentation run.

Parameters:
fidelity_frame()[source]

Per-fold, per-generator fidelity, one row each.

Return type:

DataFrame

fidelity_headline()[source]

The few fidelity columns worth reading first, per fold and generator.

fidelity_frame returns everything the battery computed — 66 columns on a full SDV run. This is the documented shortlist from HEADLINE_FIDELITY_COLUMNS, narrowed to the columns actually present, so it is also safe on a dependency-free run where most of the battery did not execute.

Return type:

DataFrame

leakage_frame()[source]

The leakage probes alone — the frame to read before believing a delta.

Empty means the probes did not run, never that they ran and found nothing: a frame of fold/generator labels with no leak. column would read as a clean bill of health that nobody issued.

Return type:

DataFrame

pamir.synthetic.run_fleet(mixer, datasets=None, max_rows=20000, **kwargs)[source]

Run the augmentation across PaMIR datasets and return one summary row each.

Return type:

DataFrame

Parameters:
pamir.synthetic.plan_mixture(n_real, synthetic_share=None, weights=None, sizing='keep_real', n_total=None, n_synthetic=None)[source]

Resolve a proportion — or a row count — into exact row counts.

Parameters:

  • n_real (int) – Real training rows available (one fold’s train split, not the table).

  • synthetic_share (Optional[float]) – Target share of synthetic rows in the mixed frame, in [0, 1). 0.7 means 70% synthetic / 30% real. Defaults to 0.5 when neither this nor n_synthetic is given.

  • weights (Optional[Mapping[str, float]]) – Relative weights per generator, e.g. {"zgan": 0.5, "zedge": 0.5}. Need not sum to 1; they are normalized.

  • sizing (str) – keep_real (default) keeps every real row and adds synthetic rows on top until the share is met — the mix never throws real data away. fixed_total pins the mixed frame to n_total rows and subsamples the real part, which is what an ablation at constant training size needs.

  • n_total (Optional[int]) – Required for fixed_total.

  • n_synthetic (Optional[int]) – Exact number of synthetic rows, in place of a share. Under CV this is per fold, so the count is identical in every fold whereas a share is not. Mutually exclusive with synthetic_share: naming both would mean one of them silently losing.

Return type:

MixturePlan

Generator adapters

class pamir.synthetic.GeneratorAdapter(name)[source]

Uniform interface every generator is wrapped in.

Parameters:

name (str)

fit(data, target=None, discrete_columns=None)[source]

Train on real rows — the fold’s training split, target column included.

Return type:

GeneratorAdapter

Parameters:
sample(n, seed=None)[source]

Draw n synthetic rows with the training schema.

Return type:

DataFrame

Parameters:
native_fidelity(real, synthetic)[source]

Metrics shipped by the engine’s own library. Empty when it ships none.

Return type:

Dict[str, float]

Parameters:
clone()[source]

An unfitted copy, so each CV fold trains from scratch.

Return type:

GeneratorAdapter

property fixed_pool: bool

True when the engine can only hand back rows it already holds.

The mixer treats such an engine differently when a class balance is requested: there is no point asking a finite pool for oversampled slack it cannot make, so the whole pool becomes the candidate set instead.

property n_available: int | None

Rows the engine can supply, when that number is bounded. None if not.

class pamir.synthetic.ZganAdapter(repo_path=None, name='zgan', module='app.zgan.zgan_lib', utils_module='app.utils.zgan_utils', synthesizer=None, **params)[source]

zGAN (zyplGANSynthesizer) from the zypl zgan repository.

repo_path is the checkout root; the class is imported from app.zgan.zgan_lib unless module says otherwise. zGAN’s own fidelity helpers in app.utils.zgan_utils are reported through native_fidelity().

Parameters:

  • repo_path (Optional[str])

  • name (str)

  • module (str)

  • utils_module (str)

  • synthesizer (Any)

class pamir.synthetic.ZedgeAdapter(repo_path=None, artifact_dir=None, fold_artifacts=None, name='zedge', mode='pipeline', device='cpu', temperature=2.0, num_steps=50, work_dir=None, oversample=1.2, allow_shared_artifact=False, **pipeline_kwargs)[source]

zEDGE / zGN-LatentDiff (latent diffusion) from the zgn_latdiff repository.

Two ways to run it:

mode="pipeline"

Train on the fold’s split in-process: the training frame is written to a temporary CSV, run_full_pipeline builds the artifact, and generate_samples draws the pool. Needs torch and, realistically, a GPU.

mode="artifact"

Reuse an artifact directory already trained on this fold’s split and only sample from it. fit then verifies nothing but the artifact’s presence, so the caller carries the burden of the leakage contract — which is why fold_artifacts is the keyed form to prefer.

Parameters:

  • repo_path (Optional[str])

  • artifact_dir (Optional[str])

  • fold_artifacts (Optional[Mapping[int, str]])

  • name (str)

  • mode (str)

  • device (str)

  • temperature (float)

  • num_steps (int)

  • work_dir (Optional[str])

  • oversample (float)

  • allow_shared_artifact (bool)

class pamir.synthetic.SDVAdapter(synthesizer, name='sdv', max_categories=20, **params)[source]

Any SDV BaseSingleTableSynthesizer — which is what zGAN subclasses.

Metadata is detected from the fold’s training frame, so nothing about the held-out rows reaches the generator, not even a column’s value range.

Parameters:

  • synthesizer (Any)

  • name (str)

  • max_categories (int)

class pamir.synthetic.PoolAdapter(pool=None, name='pool', fold_pools=None, allow_shared_pool=False, allow_replacement=False)[source]

A pool of rows generated elsewhere — the offline path.

Use it when the engine runs outside this process (a GPU box, the zEDGE service, a nightly job) and all that comes back is a CSV or a DataFrame.

The pool must have been generated from the same fold’s training split. fold_pools keyed by fold index enforces that per fold; a single pool reused across folds is a leak and is rejected unless allow_shared_pool.

Parameters:

  • pool (Any)

  • name (str)

  • fold_pools (Optional[Mapping[int, Any]])

  • allow_shared_pool (bool)

  • allow_replacement (bool)

class pamir.synthetic.CallableAdapter(model, name='generator', init_kwargs=None, fit_kwargs=None, sample_kwargs=None, pass_discrete=False)[source]

Wrap any object exposing fit(frame) and sample(n).

This is the escape hatch for a model class that already follows the usual synthesizer convention but is neither zGAN nor zEDGE.

Each dictionary has exactly one destination, whatever is passed and however many times the adapter is fitted:

init_kwargs

The model’s constructor. Only meaningful when model is a class; given alongside an already-built instance they cannot be applied, so they are refused rather than ignored.

fit_kwargs

model.fit(frame, **fit_kwargs).

sample_kwargs

model.sample(n, **sample_kwargs).

Parameters:

  • model (Any)

  • name (str)

  • init_kwargs (Optional[Dict])

  • fit_kwargs (Optional[Dict])

  • sample_kwargs (Optional[Dict])

  • pass_discrete (bool)

pamir.synthetic.as_adapter(obj, name='generator', **kwargs)[source]

Wrap whatever the caller has into a GeneratorAdapter.

Accepts an adapter (returned unchanged), a DataFrame or CSV path (pool), an SDV synthesizer class or instance, or any object with fit and sample.

Return type:

GeneratorAdapter

Parameters:

Fidelity

pamir.synthetic.fidelity_report(real, synthetic, metadata=None, target=None, holdout=None, heavy=False, max_pairs=60, max_category_cardinality=200, max_contingency_cells=20000, seed=0)[source]

Score synthetic rows against the real rows they were trained on.

Parameters:

  • real (DataFrame) – The real training rows the generator saw — never a test split.

  • synthetic (DataFrame) – Generated rows with the same schema.

  • metadata (Optional[Dict]) – SDV-style {"columns": {name: {"sdtype": ...}}}; inferred when absent.

  • target (Optional[str]) – Target column name. Enables ML-efficacy metrics and the target-rate check.

  • holdout (Optional[DataFrame]) – Held-out real rows, used only for leakage probes (exact-row overlap).

  • heavy (bool) – Also run the slow arms (SVC detection, MLP efficacy, GMLogLikelihood).

  • max_pairs (int) – Cap on the number of column pairs scored, to bound O(p^2) work.

  • max_category_cardinality (int) – A categorical column whose synthetic cardinality exceeds this (and ten times the real cardinality) is reported as a type violation and scored as numerical by the pair, table and aggregate arms. See cardinality_violations().

  • max_contingency_cells (int) – Skip a categorical pair whose contingency table would exceed this many cells. A bound on cost, not a statistical choice.

  • seed (int)

Return type:

FidelityReport

pamir.synthetic.leakage_report(real, synthetic, holdout)[source]

The leakage probes alone, with no distributional metric computed.

The probes cost milliseconds while the full battery costs seconds, so they are deliberately available on their own: switching fidelity off for a fleet sweep must never switch off the check that says whether to believe the resulting delta.

Return type:

FidelityReport

Parameters:
class pamir.synthetic.FidelityReport(overall=<factory>, columns=<factory>, pairs=<factory>, table=<factory>, leakage=<factory>, native=<factory>, type_violations=<factory>, errors=<factory>, n_real=0, n_synthetic=0)[source]

Everything measured about one (real, synthetic) pair.

Parameters:
summary()[source]

Headline numbers, flattened for a results table.

Return type:

Series

column_matrix()[source]

Per-column scores pivoted to column x metric.

Return type:

DataFrame

pamir.synthetic.c2st_auc(real, synth, metadata, seed=0)[source]

Classifier two-sample test: AUC of telling synthetic from real.

The scale is two-sided and runs over [0, 1], not [0.5, 1]:

  • ≈ 0.5 — indistinguishable to a strong learner, which is the goal.

  • > 0.5 — the samples differ; above roughly 0.8 the synthetic rows are trivially identifiable and their value as training data is doubtful.

  • < 0.5 — duplicates, not quality. When synthetic rows repeat real ones, cross-validation puts a row’s twin in the training fold carrying the opposite label, the classifier is systematically wrong on the held-out twin, and the AUC falls below chance. A pure memoriser scores near 0.1. Read it with dcr_zero_share and NewRowSynthesis, and never as “better than indistinguishable”.

fidelity_report() therefore also records C2ST_deviation, the distance |AUC - 0.5|, which is the number to rank generators by: it is 0 for an ideal generator and rises for both failure modes.

Return type:

float

Parameters:
pamir.synthetic.build_metadata(data, target=None, max_categories=20)[source]

SDV single-table metadata as a plain dict (the form every metric takes).

Return type:

Dict

Parameters: