spacr.benchmark

Measure throughput and memory use to recommend a worker count.

spaCR’s worker defaults are arithmetic on the core count – cpu_count() - 4, cpu_count() // 2, -1. On a measurement run the core count is rarely the limiting factor: a field is hundreds of megabytes decompressed, and eight workers on a 16-core laptop with 16 GB will swap long before they saturate the CPU. The limiting quantity is usually the memory one worker needs for one representative field, which depends on the plate and the machine, not on the core count.

The benchmark therefore runs the real work over a few fields, records peak RSS and peak VRAM, and divides the available memory by the memory one worker used.

This measures; it does not tune. It returns a recommendation and the evidence behind it, and never writes a setting. A benchmark that silently changed n_jobs would make a run’s speed depend on when the benchmark last ran, which is the opposite of reproducible.

Qt-free and torch-optional: VRAM is reported when torch is present with a CUDA device and reported as None otherwise, which is not the same as 0.

Classes

Measurement	Item count, wall time and peak memory observed in one benchmark run.
Recommendation	A worker count and the retained evidence supporting it.

Functions

available_memory_bytes(→ int)	Return MemAvailable from /proc/meminfo in bytes, or 0 if unreadable.
benchmark(→ Measurement)	Run work over items serially and record time and peak memory.
format_report(→ str)	Format a benchmark and its recommendation as plain text lines.
recommend_workers(→ Recommendation)	Recommend a worker count from cores, free memory and measured usage.

Module Contents

class spacr.benchmark.Measurement

Item count, wall time and peak memory observed in one benchmark run.

Parameters:

• items – how many representative units were processed.

• seconds – wall clock for all of them.

• peak_rss_bytes – the highest resident set size seen, for the whole process. Not per worker – the benchmark runs serially on purpose, so this is one worker’s requirement.

• peak_vram_bytes – peak CUDA allocation, or None when there is no CUDA device. None and 0 are different answers: one means “not measured”, the other “measured, and it used none”.

• baseline_rss_bytes – RSS before the work started, so the caller can tell the interpreter’s own footprint from the work’s.

• notes – caveats about warm-up or unavailable measurements that must accompany the numeric result.

property items_per_second: float

Return measured throughput, or NaN for a nonpositive duration.

property per_item_seconds: float

Return mean wall time per measured item, or NaN when none ran.

property work_rss_bytes: int

the memory used by the work.

Type:

Return peak RSS minus baseline RSS

class spacr.benchmark.Recommendation

A worker count and the retained evidence supporting it.

Parameters:

• workers – final recommended parallel-worker count; always at least one and bounded by the normalized core count, measured memory capacity when usable, and configured maximum.

• reason – human-readable explanation of the branch that set workers: missing measurement, one-worker fallback, memory bound, core bound, or configured maximum.

• measurement – exact Measurement supplied to recommend_workers(); None means no benchmark was available, while a zero-work-footprint measurement is retained but triggers the core-count fallback.

• cores – effective logical-core ceiling after defaulting from os.cpu_count() and clamping to at least one.

• available_bytes – available-memory snapshot before the configured reserve is subtracted; supplied by the caller or measured by spaCR, and zero when unavailable.

__str__() → str

Return the worker count followed by its sizing explanation.

spacr.benchmark.available_memory_bytes() → int

Return MemAvailable from /proc/meminfo in bytes, or 0 if unreadable.

Available rather than total memory is used because total includes memory already in use; sizing workers against it can exhaust memory mid-run.

spacr.benchmark.benchmark(work: Callable[[Any], Any], items: Sequence[Any], *, warmup: int = 1) → Measurement

Run work over items serially and record time and peak memory.

Items run serially because the recommendation divides by what one worker needs; a parallel run would measure the sum and hide that figure.

Parameters:

• work – callable invoked once for every warm-up and measured item; its return value is ignored because only resource use is measured.

• items – ordered workload to process. At least one item is required, and the final item is always kept in the measured set.

• warmup – items processed before the clock starts. The first field pays for imports, CUDA context creation and page faults that no later field pays again, and counting it makes a short run look far slower than the plate it is predicting.

Raises:

ValueError – no items to measure – a benchmark over nothing would return a per-item cost of NaN and a recommendation built on it.

spacr.benchmark.format_report(measurement: Measurement, recommendation: Recommendation) → str

Format a benchmark and its recommendation as plain text lines.

Parameters:

• measurement – observed serial benchmark costs and throughput.

• recommendation – worker recommendation derived from those costs.

spacr.benchmark.recommend_workers(measurement: Measurement | None = None, *, cores: int | None = None, available_bytes: int | None = None, reserve_bytes: int = 2 * 1024**3, maximum: int = 32) → Recommendation

Recommend a worker count from cores, free memory and measured usage.

The rule, in order:

1. Never more than cores. More workers than cores is contention.

2. Never more than available memory - reserve divided by what one worker measurably needed. This is the term the core-count defaults omit, and it is usually the binding one.

3. Never fewer than 1, and never more than maximum.

Parameters:

reserve_bytes – memory left for everything that is not a worker – the GUI, the page cache the readers depend on, and the operating system. Defaults to 2 GiB.

Returns:

a Recommendation carrying the reason, which states the limit that set the worker count.