spacr.group_lasso

Group lasso over guides grouped by gene, selected or dropped as a set.

This model estimates which guides and genes are associated with a response while respecting the grouping of guides within genes.

Ordinary lasso penalises each guide on its own, so from a gene’s four correlated guides it keeps whichever one happens to fit best and drops the rest – which reads as “one guide works and three do not” when the four guides are four measurements of the same gene. Group lasso penalises the gene’s guides as a block:

minimise ||y - Xb||^2 / (2n) + lambda * sum_g sqrt(p_g) * ||b_g||_2

The L2 norm inside the sum is not squared, so its subgradient at zero is a ball rather than a point, and the whole block goes to exactly zero or none of it does. sqrt(p_g) is the standard correction that stops a gene with six guides being penalised more than one with three purely for having more.

For a CRISPR screen it is the penalised analogue of the mixed model with random effects – “which genes are involved, where each gene is measured by several guides” – and it addresses that question with no random effects, no REML fit, and no gene_fraction column. gene_fraction is the sum of a gene’s guide fractions, so a design carrying both blocks is singular by construction. Here the gene enters as a grouping of the guide columns, never as a column of its own, so the design stays full rank.

It also fits where OLS is undefined. This screen is 610 wells and 823 guides; the penalty is what makes the answer well posed at p > n. A penalised fit therefore rests on the assumption that most guides have no effect; the penalty adds no new information.

The model gives no P value. A penalised coefficient’s sampling distribution is not the OLS one, and a naive P value from it is biased toward significance. stability_selection() instead re-fits on many subsamples and reports the fraction of them in which a gene is selected, an error-controlled statement (Meinshausen & Buhlmann) that needs no distributional assumption.

The solver is block coordinate descent with the exact group-soft-threshold proximal step, in numpy. No new dependency: scikit-learn has no group lasso, and the two packages that do are unmaintained.

Functions

choose_lambda(→ float)	Select a group-lasso penalty by held-out prediction error.
describe(→ str)	Return the formula and model description shown in the model tab.
fit(X, y, labels, *[, lam, max_iterations, tolerance, ...])	Fit the group lasso and return (coefficients, intercept, converged).
gene_effects(X, y, labels, **kwargs)	One number per gene: the L2 norm of its guide block.
max_lambda(→ float)	The smallest penalty that zeroes every group.
penalty_path(X, y, labels, *[, points, depth])	Penalties from max_lambda() down, log-spaced.
stability_selection(X, y, labels, *[, lam, n_boot, ...])	Fraction of subsample re-fits in which each gene is selected.

Module Contents

spacr.group_lasso.choose_lambda(X, y, labels, *, folds: int = PATH_FOLDS, points: int = PATH_POINTS, depth: float = PATH_DEPTH, required=None, seed: int = 0, **kwargs) → float

Select a group-lasso penalty by held-out prediction error.

A candidate is eligible only when its fit on the complete dataset selects at least one column allowed by required and its held-out error is finite. If a penalty path contains no eligible candidate, progressively smaller penalties are evaluated for up to PATH_EXTENSIONS additional ranges. If no eligible candidate is found, the smallest penalty evaluated is returned.

Parameters:

• X – two-dimensional design matrix with one row per well and one column per guide.

• y – response values aligned to the rows of X.

• labels – group label for every column of X; guides sharing a label are selected or dropped as one block.

• folds – how many held-out splits. Wells, not guides.

• required – optional boolean mask over columns. A penalty counts as selecting only when one of these columns is non-zero. Defaults to every column.

• seed – fixes the split, so two runs of one screen agree.

Returns:

the chosen penalty, as a float.

spacr.group_lasso.describe(lam: float = 0.05) → str

Return the formula and model description shown in the model tab.

spacr.group_lasso.fit(X, y, labels, *, lam: float = 0.05, max_iterations: int = MAX_ITERATIONS, tolerance: float = TOLERANCE, fit_intercept: bool = True)

Fit the group lasso and return (coefficients, intercept, converged).

Parameters:

• X – the design, one column per guide.

• y – the response, one entry per well.

• labels – the gene each column belongs to, length X.shape[1].

• lam – the penalty. Zero is ordinary least squares and is allowed – it is how a caller checks the solver against a known answer.

• fit_intercept – centre both sides, so the intercept is never penalised. Penalising it would shrink the response’s mean toward zero.

Returns:

coefficients (one per column), intercept, and whether the sweep converged inside max_iterations.

Raises:

ValueError – the shapes disagree, or lam is negative.

spacr.group_lasso.gene_effects(X, y, labels, **kwargs)

One number per gene: the L2 norm of its guide block.

Parameters:

• X – design matrix with one column per guide.

• y – response vector with one value per design row.

• labels – gene label for each design column.

The block is zero or it is not, so the norm is the natural per-gene effect size. Unlike a coefficient it is never negative: group lasso indicates that a gene’s guides move the response together, not in which direction each one does.

spacr.group_lasso.max_lambda(X, y, labels) → float

The smallest penalty that zeroes every group.

Parameters:

• X – design matrix with one column per guide.

• y – response vector with one value per design row.

• labels – gene label for each design column.

Useful as the top of a path: any larger value gives the same all-zero fit, and starting a path above it wastes every iteration spent there.

spacr.group_lasso.penalty_path(X, y, labels, *, points: int = PATH_POINTS, depth: float = PATH_DEPTH)

Penalties from max_lambda() down, log-spaced.

Starting at the ceiling rather than at an absolute number is what makes the path scale-free: a design of fractions and a design of counts have ceilings three orders of magnitude apart, and a fixed grid that suits one is entirely above or entirely below the other.

spacr.group_lasso.stability_selection(X, y, labels, *, lam: float = 0.05, n_boot: int = 100, fraction: float = 0.5, seed: int = 0, **kwargs)

Fraction of subsample re-fits in which each gene is selected.

Parameters:

• X – two-dimensional design matrix with wells in rows and guides in columns.

• y – response values aligned to the rows of X.

• labels – gene or other group label for every guide column.

• n_boot – subsamples.

• fraction – the share of rows in each subsample. Half is Meinshausen & Buhlmann’s choice and the one their error bound is stated for.

Returns:

a DataFrame with a selection_frequency per gene, sorted.

This replaces a P value, which the group lasso does not provide: the sampling distribution of a penalised coefficient is not the OLS one, and a P value computed as though it were is biased toward significance. A selection frequency states how reproducible the selection is and needs no distributional assumption.

Nested helpers

choose_lambda.score(candidates)

(errors, selects) for one path, both aligned to candidates.

The error is held out and the selection is not. Whether a penalty predicts well is a question about unseen wells; whether it selects anything is a question about the fit the caller will get, which is fitted on every well. Reading selection off the folds can choose a penalty at which most folds keep a gene while the full fit keeps none (on the tsg101 screen it chose 0.000183 and the full fit was empty).

spacr/group_lasso.py:249