Shm pool
cutana.shm_pool
¶
Shared memory pool for streaming orchestrator.
Provides pre-allocated shared memory blocks for zero-copy cutout transfer between worker subprocesses and the orchestrator. Each worker gets its own ShmPool instance; pools are reused across worker lifetimes to avoid repeated allocation/deallocation overhead.
Slot lifecycle
EMPTY -> READY (worker writes cutout and marks slot ready) READY -> EMPTY (orchestrator reads and releases slot)
ShmPoolConfig(slot_shape, dtype, slots_per_worker)
dataclass
¶
ShmPool(config)
¶
Pre-allocated shared memory pool for one worker.
Creates two shared memory blocks: - data_shm: holds cutout pixel data (slots_per_worker * slot_bytes) - control_shm: int32 array with one state value per slot
The orchestrator creates the pool; worker subprocesses attach to it by name and write cutouts into slots.
reset()
¶
Reset all slots to EMPTY for reuse with a new worker.
read_slots(slot_count)
¶
Read cutouts from the first slot_count slots and reset them to EMPTY.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
slot_count
|
int
|
Number of slots to read, starting from slot 0 |
required |
Returns:
| Type | Description |
|---|---|
ndarray
|
numpy array of shape (slot_count, *slot_shape) with cutout data copied out |
cleanup()
¶
Close and unlink both shared memory blocks.
attach_pool(pool_name, control_name, config)
¶
Attach to an existing SHM pool from a worker subprocess.
Returns (data_shm, control_shm). The worker must call detach_pool() when done — never unlink, as the orchestrator owns the memory.
write_cutouts_to_pool(data_shm, config, cutouts)
¶
Write a batch of cutouts into the pool starting at slot 0.
All cutouts must match config.slot_shape and config.dtype. len(cutouts) must not exceed config.slots_per_worker.
mark_slots_ready(control_shm, config, slot_count)
¶
Mark the first slot_count slots as READY after writing cutout data.
detach_pool(data_shm, control_shm)
¶
Detach from SHM pool (worker side). Closes but does NOT unlink.
calculate_pool_config(target_resolution, n_channels, dtype, max_shm_memory_per_worker=None)
¶
Calculate SHM pool configuration from image parameters.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
target_resolution
|
int
|
Cutout size (H=W) |
required |
n_channels
|
int
|
Number of image channels |
required |
dtype
|
dtype
|
Output data type |
required |
max_shm_memory_per_worker
|
int | None
|
Max bytes per worker pool. None = auto-calculate. |
None
|
Returns:
| Type | Description |
|---|---|
ShmPoolConfig
|
ShmPoolConfig with computed slots_per_worker |