spacr.regression_diagnostics¶

Diagnostics of a screen regression’s design, fit and P-value calibration.

A volcano plot shows the model’s estimates, not whether they are identifiable or calibrated. These checks test that; for example, they flag a design with 824 guides in 587 wells, fitted simultaneously, returning a confident coefficient and P value for every guide from a rank-deficient matrix.

The suite is deliberately split in two:

design_report / plot_design_diagnostics: Properties of the design, computable before any model is fitted, and the ones that decide whether the fit means anything: how many wells per parameter, the rank of the design matrix, its condition number, how many wells each guide appears in, and which guides are so collinear that no method can separate them.
residual_report / plot_residual_diagnostics: Properties of a fitted model: the residual-versus-fitted, scale-location, QQ and leverage panels, plus Cook’s distance.
plot_inference_diagnostics: Properties of the test: the P-value histogram, whose shape tells you whether the null is calibrated, and the observed-versus-expected quantile plot with its genomic inflation factor.

Every function takes plain arrays or frames and returns plain numbers, so they can be asserted on in tests rather than eyeballed.

Functions¶

`collinear_guide_pairs`(→ pandas.DataFrame)	Guide pairs whose well patterns are so alike they cannot be separated.
`design_report`(→ dict)	Measure whether a simultaneous fit on this design is identifiable.
`plot_design_diagnostics`(fractions, *[, block, ...])	Six panels describing the design, before any model is fitted.
`plot_design_identifiability`(fractions, *[, block, ...])	One compact panel reporting wells, predictors, rank and guide support.
`plot_inference_diagnostics`(p_values, *[, adjusted, ...])	Plot the P-value histogram and QQ plot to check null calibration.
`plot_residual_diagnostics`(observed, fitted, *[, ...])	The four classical residual panels, plus Cook's distance.
`residual_report`(→ dict)	Summarise the residuals of a fitted model.
`score_design`(→ object)	Score whether a simultaneous fit on this design is identifiable.
`score_inference`(→ object)	Score whether the P-value null is calibrated or shifted.
`score_residuals`(→ object)	Score whether the residuals support the inference drawn from them.
`variance_inflation_factors`(→ pandas.DataFrame)	Compute per-guide variance inflation factors for a full-rank design.
`write_diagnostic_suite`(→ Mapping[str, str])	Write every diagnostic the supplied inputs can support.

Module Contents¶

spacr.regression_diagnostics.collinear_guide_pairs(fractions: pandas.DataFrame, *, threshold: float = 0.95, limit: int = 500) → pandas.DataFrame[source]¶

Guide pairs whose well patterns are so alike they cannot be separated.

This is the mechanism behind the screen’s false positives: a guide that appears in nearly the same wells as a true hit inherits its signal, and no amount of correction distinguishes them, because the data contain no contrast between them. Reported as a table so the offenders can be named.

Parameters:

fractions – well-by-guide fraction matrix whose nonconstant guide columns are compared pairwise.
threshold – absolute Pearson correlation at or above which a pair is listed.
limit – stop after this many pairs; a wide screen has millions and the first few hundred are the ones worth reading.

spacr.regression_diagnostics.design_report(fractions: pandas.DataFrame, *, block: pandas.Series | None = None, presence_threshold: float = 0.0) → dict[source]¶

Measure whether a simultaneous fit on this design is identifiable.

Parameters:

fractions – well-by-guide matrix, one row per analysed well.
block – optional per-well block labels (normally the plate), which cost one parameter each beyond the first.
presence_threshold – a guide counts as present in a well when its value exceeds this.

Returns:

a dict of scalars. identifiable is the one that matters: it is False when the design has fewer wells than parameters, which is the state in which per-guide coefficients are not unique.

spacr.regression_diagnostics.plot_design_diagnostics(fractions: pandas.DataFrame, *, block: pandas.Series | None = None, save_path=None, save_format=None, presence_threshold: float = 0.0)[source]¶

Six panels describing the design, before any model is fitted.

Parameters:: fractions – well-by-guide fraction matrix, one row per analysed well, as taken by design_report().

spacr.regression_diagnostics.plot_design_identifiability(fractions: pandas.DataFrame, *, block: pandas.Series | None = None, save_path=None, save_format=None, presence_threshold: float = 0.0)[source]¶

One compact panel reporting wells, predictors, rank and guide support.

The six-panel plot_design_diagnostics() remains the detailed audit. This compact version is the publication-facing answer to a narrower question: can one coefficient per guide be identified from these wells?

Parameters:: fractions – well-by-guide fraction matrix, one row per analysed well, as taken by design_report().
Returns:: (written_path, report) in the same form as the other plotters.

spacr.regression_diagnostics.plot_inference_diagnostics(p_values, *, adjusted=None, alpha: float = 0.05, save_path=None, save_format=None, label: str = '')[source]¶

Plot the P-value histogram and QQ plot to check null calibration.

A well-behaved screen gives a flat P-value histogram with a spike at zero. A histogram that slopes or humps in the middle means the test is mis-calibrated, and no correction repairs that.

Parameters:: p_values – raw P values of the tested family; non-finite values are dropped before plotting.

spacr.regression_diagnostics.plot_residual_diagnostics(observed, fitted, *, design: numpy.ndarray | None = None, save_path=None, save_format=None, label: str = '')[source]¶

The four classical residual panels, plus Cook’s distance.

Parameters:

observed – observed response per observation.
fitted – fitted value per observation, aligned with observed; the residuals are observed - fitted.

spacr.regression_diagnostics.residual_report(observed, fitted, *, design: numpy.ndarray | None = None) → dict[source]¶

Summarise the residuals of a fitted model.

Parameters:

observed – observed response values.
fitted – model predictions aligned one-to-one with observed.
design – the model matrix. When given, leverage and Cook’s distance are computed from its hat matrix; without it those keys are omitted rather than guessed.

spacr.regression_diagnostics.score_design(report: Mapping) → object[source]¶

Score whether a simultaneous fit on this design is identifiable.

Parameters:: report – the dict design_report() returns.
Returns:: a PanelVerdict.

Rank deficiency is a failure because a design with fewer independent directions than parameters has no unique coefficient vector. A pseudo-inverse can still return numbers, but individual effects are not identifiable.

The other two rules are the conventional ones and are cited in detail: a design needs more wells than parameters to have any residual degrees of freedom at all, and a guide seen in one well or none has no contrast to be estimated from.

spacr.regression_diagnostics.score_inference(report: Mapping) → object[source]¶

Score whether the P-value null is calibrated or shifted.

Read off the genomic inflation factor: the median observed chi-square over its null median, which is 1.0 when the null behaves. The bands are the ones the GWAS literature uses – above 1.1 a report should explain the inflation, and above 1.2 the p-values are not considered reliable.

A spike of small p is not a fault. It is what a screen with real hits looks like, and lambda is deliberately a median so a handful of true hits barely move it. Scoring the spike itself would flag every successful screen.

Parameters:: report – inference summary with tests, genomic_inflation and optionally pi0 and estimated_non_null, as built by plot_inference_diagnostics(). No tests, or a non-finite inflation factor, scores unknown.

spacr.regression_diagnostics.score_residuals(report: Mapping) → object[source]¶

Score whether the residuals support the inference drawn from them.

Parameters:: report – the dict residual_report() returns.

A diagnostic test’s p reads the other way round: the null is “the assumption holds”, so a large p is the good outcome. Every rule below is written in that direction and says so.

Thresholds are the conventional ones: p >= 0.05 for a diagnostic test, Cook’s distance of 0.5 and 1.0. A threshold invented here would be a number a reviewer cannot check.

spacr.regression_diagnostics.variance_inflation_factors(fractions: pandas.DataFrame, *, max_guides: int = 200) → pandas.DataFrame[source]¶

Compute per-guide variance inflation factors for a full-rank design.

All nonconstant guide columns participate in the calculation. The output is then limited to the guides with the widest well support, so max_guides controls reporting rather than the fitted design.

Parameters:

fractions (pandas.DataFrame) – Well-by-guide design matrix.
max_guides (int, default=200) – Maximum number of guides to return.

Returns:

pandas.DataFrame – Columns guide, vif, and wells_with_guide, ordered by decreasing VIF.

Raises:

ValueError – If the number of wells is not greater than the number of nonconstant guide columns. Use design_report() and collinear_guide_pairs() to inspect that rank-deficient design.

spacr.regression_diagnostics.write_diagnostic_suite(destination, *, fractions=None, block=None, observed=None, fitted=None, design=None, p_values=None, adjusted=None, alpha: float = 0.05, label: str = '', presence_threshold: float = 0.0, formats: Sequence[str] | None = None) → Mapping[str, str][source]¶

Write every diagnostic the supplied inputs can support.

Nothing is required: pass what a given analysis mode has. The permutation test has a design and P values but no fitted values; a simultaneous fit has all of them. Each block is skipped silently when its inputs are absent, and a block that raises is recorded as an error rather than aborting the run – a diagnostic that fails must never take the analysis down with it.

Every sheet is also scored – see score_design(), score_residuals() and score_inference() – and carries its own badge. The verdicts are rows in diagnostic_summary.csv, per sheet and for the suite (its worst), rather than entries in the returned mapping: that mapping’s contract is “key -> a file that exists”.

Parameters:

destination – output directory created as needed for all diagnostic figures, CSV tables, and the optional suite summary.
formats –
force particular formats, one file per format. None, the default, writes one file per panel in the format the user’s figure preference asks for.

Each requested format re-runs the whole panel, and every file written is a separate tile in the gallery.

Nested helpers¶

write_diagnostic_suite._emit(name, function, **kwargs)¶

Write one diagnostic in every captured requested format.

Parameters:

name – stable panel name used for paths and manifest keys.
function – writer returning (written_path, report).
kwargs – diagnostic inputs forwarded to the writer.

Returns:

None. Successful artifacts are keyed by their actual output extension and their report is retained; advisory failures become per-format error entries without stopping the remaining writers.

spacr/regression_diagnostics.py:1005