spacr.projects

Find spaCR projects on disk and summarise each one’s stage, size, last run and staleness.

The summaries answer questions such as which plates have been measured, which finished projects have had their masks re-run since, and which project uses the most disk space, as one list across all project folders.

Every value is read from an existing spaCR function (listed below) rather than computed again here, so the browser reports the same values as the rest of spaCR.

What the browser shows	Where the answer comes from
stage reached	spacr.ports.declared_outputs() — which of a module’s declared outputs are on disk. Not a hand-written ladder of stage names: the order is topologically sorted out of the port graph, so a module registered by a plugin takes its place in it without this file being edited.
size, and what of it	spacr.data_manager.scan_project(). ONE walk of
is unaccounted for	the tree, per-kind attribution, and the registry reconciled against the filesystem. There is no second walker here, and there must not be one.
last run	the registry’s newest created_ns for the project; failing that, the mtime of the outputs themselves. Which of the two answered is reported, because “spaCR recorded this run” and “something wrote into this folder” are different claims.
what is stale	spacr.artifacts.Registry.is_stale(), whose Staleness already carries the reasons, the machine cause codes and the separate missing flag.
what to run next	spacr.chaining.next_steps(), so an offer here is the same offer, with the same readiness check, that the module’s own screen makes.

The project the registry has never seen

A user may copy a colleague’s plate folder onto the machine and open spaCR. A browser that listed only what spaCR itself recorded would show nothing for that folder, and an empty list would look like a valid result.

So a project is found on disk (discover(), looks_like_project()) rather than read out of a registry, and an unrecorded one appears with everything the filesystem can answer — stage, size, files, last write — and with ProjectSummary.known False. It is not reported as “0 stale”, which would read as clean. With no provenance there is nothing to compare against, so staleness is unknown, and ProjectSummary.staleness_known is the flag that says so. ProjectSummary.staleness_note() puts it in words for a table cell.

Nothing here imports Qt, so the same summary is available from a notebook and from a test, and the browser screen is a renderer of it.

Classes

ModuleState	Whether one module has run in a project, judged by what it declares.
ProjectSummary	One project, as the browser lists it.
StaleArtifact	One registered result that no longer matches what made it.

Functions

browse(→ Tuple[ProjectSummary, ...])	Find every project under roots and summarise each one.
discover(→ Tuple[str, ...])	Find project folders under roots.
evidence_ports(→ Tuple[spacr.ports.Port, ...])	Return the produced ports whose presence on disk shows that module ran here.
format_project(→ str)	Render one ProjectSummary as a block of text.
format_projects(→ str)	Render a whole browse as a table, one row per project.
looks_like_project(→ bool)	Whether a folder is a spaCR project, judged without the registry.
module_states(, records, ...])	Judge every producing module against one project folder.
pipeline_order(→ Tuple[str, ...])	Producing modules, upstream first.
producing_modules(→ Tuple[str, ...])	Every declared module that writes something, sorted.
scan(→ ProjectSummary)	Summarise one project folder as a ProjectSummary.

Module Contents

class spacr.projects.ModuleState

Whether one module has run in a project, judged by what it declares.

Parameters:

• module – the module key.

• state – STATE_DONE, STATE_PARTIAL or STATE_ABSENT.

• found – roles whose declared output is on disk — or, when the answer came from the registry, the roles it recorded.

• missing – required roles whose declared output is not there. An optional output that was cleaned up (masks/) is not missing — the declaration already says it may legitimately be absent, and reporting it would make every tidied project look broken.

• optional_missing – optional roles that are absent, kept separate so the difference stays visible without being alarming.

• newest_ns – when the newest of the found outputs was written.

• evidence – what answered — SOURCE_FILESYSTEM, SOURCE_REGISTRY, or "" when nothing did.

• detectable – whether this module has any on-disk signature at all. See evidence_ports(). False plus STATE_ABSENT means “unknown”, not “no”.

describe() → str

One line: the module, its state and what is short.

property ran: bool

True when this module left anything at all behind.

class spacr.projects.ProjectSummary

One project, as the browser lists it.

Parameters:

• root – absolute project root.

• name – its folder name — what the table’s first column shows.

• exists – the folder is there. A registry can outlive the data.

• known – the registry holds at least one artifact for it. False for the folder a user just copied in, and the reason staleness_known exists.

• has_registry – a registry file sits in the project. True with known False when the file exists but records another project — the shared-registry case (spacr.artifacts.ARTIFACTS_DB_ENV).

• stage – the furthest module in pipeline_order() that left anything behind, or "" for a project nothing has been run on.

• modules – every producing module’s ModuleState, in pipeline order.

• size_bytes – every byte under the root, from spacr.data_manager.scan_project().

• n_files – how many files that is.

• unregistered_bytes – of those, how many nothing claims.

• unregistered_files – and how many files.

• n_artifacts – registry rows for this project.

• last_run_ns – when it last produced something.

• last_run_utc – the same instant, ISO-8601, or "".

• last_run_source – SOURCE_REGISTRY or SOURCE_FILESYSTEM — which question was answered. They are different claims, so the browser reports which one applies.

• stale – results that no longer match their inputs.

• missing – registered results whose file is gone.

• next_steps – modules that could run next, ready ones first, as (module, blocked_reason); the reason is "" when it can run.

• errors – paths that could not be read.

• scanned_utc – when this summary was taken.

• usage – the full spacr.data_manager.ProjectUsage, for a detail pane that wants the per-kind breakdown.

__str__() → str

Return the complete human-readable project report.

Returns:

Text produced by format_project() for this summary.

describe() → str

The full report; see format_project().

note() → str

Return the highest-priority note for this project’s table row, or "".

Checked in this order: a folder that is gone, then a project spaCR has no record of, then recorded results no longer on disk, then results that no longer match their inputs, then unregistered bytes making up more than half of the project’s size.

staleness_note() → str

What the stale column should say, in words.

property n_stale: int

How many recorded results are out of date.

property ran: Tuple[str, ...]

Every module that left something behind, in pipeline order.

property stage_label: str

the module, and whether it finished.

Type:

The stage as a table cell

property staleness_known: bool

Whether “is anything stale?” has an answer at all.

False for a project the registry has never seen. Reporting such a project as having zero stale artifacts would read as clean, and it is not clean — it is unexamined.

class spacr.projects.StaleArtifact

One registered result that no longer matches what made it.

A thin projection of spacr.artifacts.Staleness onto the artifact it describes, so a table row has the kind and the producing module beside the reason without the caller re-querying the registry.

Parameters:

• artifact_id – the registry id.

• kind – a spacr.ports kind.

• module – the module that produced it.

• role – its port role.

• path – where it is, or was.

• reasons – the registry’s own sentences.

• causes – the machine cause codes, e.g. "upstream-newer".

• missing – the file is gone. An availability problem, not a provenance one — spacr.artifacts.Staleness keeps the two apart and so does this.

describe() → str

One line, fit for a list under a project.

explain() → str

The causes as one readable clause, via spacr.chaining.

spacr.projects.browse(roots: Iterable[Any], *, depth: int = DEFAULT_DEPTH, registry: spacr.artifacts.Registry | None = None, limit: int = 500, with_next_steps: bool = True, on_progress: Any | None = None) → Tuple[ProjectSummary, ...]

Find every project under roots and summarise each one.

Parameters:

• roots – folders to search, or the projects themselves.

• depth – how deep to look; see discover().

• registry – an open registry to read every project through — the shared-registry case. Omit and each project’s own is used.

• limit – stop after this many projects.

• with_next_steps – passed to scan().

• on_progress – optional fn(done, total, root), called after each project on the thread that runs browse(); an exception it raises is logged and ignored. The GUI runs browse() on a worker thread, so its callback only updates its own counters.

Returns:

summaries, most recently run first, then by name. A project nobody has run sorts last, which is where a browser wants it.

spacr.projects.discover(roots: Iterable[Any], *, depth: int = DEFAULT_DEPTH, limit: int = 500) → Tuple[str, ...]

Find project folders under roots.

Descent stops at a project: a project’s merged/ holds .npy files and would otherwise be reported as a separate project.

Parameters:

• roots – folders to search. A root that is itself a project is returned as one.

• depth – how many levels below each root to look. 0 means “test the roots themselves and nothing under them”.

• limit – stop after this many projects. A browser pointed at a home directory must return something rather than walk a filesystem.

Returns:

absolute paths, sorted, de-duplicated.

spacr.projects.evidence_ports(module: str) → Tuple[spacr.ports.Port, ...]

Return the produced ports whose presence on disk shows that module ran here.

Counting every declared output that is on disk as “stage reached” gives wrong stages. ml_analyze declares one output, measurements/measurements.db, which mask created and measure filled. On that reading, a project that has only been segmented would report classical ML as complete.

So a produced port counts as evidence only when nothing EARLIER in pipeline_order() already explains the same location with a different kind. An earlier writer’s file being there says the earlier module ran, and nothing more. This is the same distinction spacr.artifacts draws when it keeps object-counts and measurements-db apart in one file — “mask creating the file does not make the measurements in it current”.

This has two consequences:

• measure is left with only its optional data/ crops, because mask wrote the database first. A measured project whose crops were never saved therefore reports its stage as mask from the filesystem alone. The registry answers it exactly when it has a record — see module_states().

• A module with no evidence ports at all (ml_analyze) has no on-disk signature whatsoever. ModuleState.detectable is False for it, so the browser can say “no way to tell without a run record” instead of the incorrect “not run”.

Modules that declare an identical set of outputs are interchangeable (mask and timelapse share theirs literally — ports.py passes the same tuple to both). Only the earlier of such a pair carries filesystem evidence, or one segmented project would report that both had run.

Parameters:

module – module key or alias.

Returns:

the subset of module_ports(module).produces that is discriminating, in declaration order.

spacr.projects.format_project(summary: ProjectSummary, *, limit: int = 6) → str

Render one ProjectSummary as a block of text.

Parameters:

• summary – what scan() returned.

• limit – how many stale entries to name.

spacr.projects.format_projects(summaries: Sequence[ProjectSummary]) → str

Render a whole browse as a table, one row per project.

Parameters:

summaries – project summaries to render in their existing order.

spacr.projects.looks_like_project(root: Any) → bool

Whether a folder is a spaCR project, judged without the registry.

Parameters:

root – folder to inspect for registry, artifact, input, or output evidence of a spaCR project.

True when a registry file sits in it, or when any declared output of any producing module is on disk, or when the mask pipeline’s declared input finds raw images there. That last clause is what makes a plate folder somebody just copied in a project: nothing has been run on it yet, and it is exactly the folder a user wants the browser to list.

Reusing the port declarations rather than testing for merged/ by name means a plugin’s module makes its own outputs count as evidence.

spacr.projects.module_states(root: Any, *, modules: Sequence[str] = (), records: Sequence[spacr.artifacts.Artifact] = ()) → Tuple[ModuleState, ...]

Judge every producing module against one project folder.

Two sources, in order of authority. A run record in records settles the question outright — the registry saw the run happen. Without one, the answer comes from the discriminating outputs on disk (see evidence_ports()), which is what makes a project spaCR has never seen readable at all.

Parameters:

• root – the project root.

• modules – which modules to judge, in the order to report them. Defaults to pipeline_order().

• records – artifacts the registry holds for this project.

Returns:

one ModuleState per module, in that order.

spacr.projects.pipeline_order() → Tuple[str, ...]

Producing modules, upstream first.

A topological sort of spacr.ports.PORTS using spacr.ports.upstream_modules(), with an alphabetical tie-break so the answer is stable between runs. Derived, not written down: a module that registers its ports through spacr.ports.register_module_ports() — a plugin, or a module written after this one — takes its place in the ladder without this file changing.

A cycle cannot happen with the declared graph and is not an error worth refusing over, so one is broken alphabetically and logged: a browser that raised rather than showing a slightly odd ordering would be a worse tool.

spacr.projects.producing_modules() → Tuple[str, ...]

Every declared module that writes something, sorted.

A module that only reads — the analysis apps that open measurements.db — cannot be a stage a project has reached, because it leaves nothing behind for anyone to find.

spacr.projects.scan(root: Any, *, registry: spacr.artifacts.Registry | None = None, usage: spacr.data_manager.ProjectUsage | None = None, with_next_steps: bool = True) → ProjectSummary

Summarise one project folder as a ProjectSummary.

Parameters:

• root – the project root. It does not have to be a project, and it does not have to be in the registry.

• registry – an open registry to read through. Omit and the project’s own is opened when a file exists; a project with none is summarised anyway, with ProjectSummary.known False.

• usage – a spacr.data_manager.ProjectUsage already taken for this root, to be reused rather than re-walked. The browser has none; the Data Manager screen, which has just walked the same project, does.

• with_next_steps – compute what could run next. Off skips one spacr.ports.check_ready() per successor, which is the only part of this that globs a second time.

Returns:

a ProjectSummary. Does not raise for a folder that is missing, unreadable or not a project; the summary reports that state so the browser can show it.

Nested helpers

discover._walk(path: str, level: int) → None

Traverse one directory within the captured discovery limits.

Parameters:

• path – directory to inspect for a project or descend through.

• level – distance below the current caller-supplied root.

Returns:

None; discovered project paths are appended to found and resolved directories are recorded in seen. Traversal stops at a project, the depth or result limit, or an unreadable folder.

spacr/projects.py:440