Skip to content

Data Structures

leopard_em.pydantic_models.data_structures

Pydantic models for reused data structures across Leopard-EM programs.

OpticsGroup

Bases: BaseModel2DTM

Stores optics group parameters for the imaging system on a microscope.

Currently utilizes the minimal set of parameters for calculating a contrast transfer function (CTF) for a given optics group. Other parameters for future use are included but currently unused.

Attributes:

  • label (str) –

    Unique string (among other optics groups) for the optics group.

  • pixel_size (float) –

    Pixel size in Angstrom.

  • voltage (float) –

    Voltage in kV.

  • spherical_aberration (float) –

    Spherical aberration in mm. Default is 2.7.

  • amplitude_contrast_ratio (float) –

    Amplitude contrast ratio as a unitless percentage in [0, 1]. Default is 0.07.

  • phase_shift (float) –

    Additional phase shift of the contrast transfer function in degrees. Default is 0.0 degrees.

  • defocus_u (float) –

    Defocus (underfocus) along the major axis in Angstrom.

  • defocus_v (float) –

    Defocus (underfocus) along the minor axis in Angstrom.

  • astigmatism_angle (float) –

    Angle of defocus astigmatism relative to the X-axis in degrees.

  • ctf_B_factor (float) –

    B-factor to apply in the contrast transfer function in A^2. Default is 0.0.

Unused Attributes:

chromatic_aberration : float Chromatic aberration in mm. Default is ???. mtf_reference : str | PathLike Path to MTF reference file. mtf_values : list[float] list of modulation transfer functions values on evenly spaced resolution grid [0.0, ..., 0.5]. beam_tilt_x : float Beam tilt X in mrad. beam_tilt_y : float Beam tilt Y in mrad. odd_zernike : Optional[dict[str, float]] Optional dict of odd Zernike moments. Possible keys: "Z31c", "Z31s", "Z33c", "Z33s". even_zernike : Optional[dict[str, float]] Optional dict of even Zernike moments. Possible keys: "Z44c", "Z44s", "Z60". mag_matrix : Optional[list[float]] Optional list of floats of length 4 representing the magnification matrix.

Methods:

  • model_dump

    Returns a dictionary of the model parameters.

mag_matrix_tensor

mag_matrix_tensor: Tensor | None

Convert mag_matrix list to a 2x2 tensor.

Returns:

  • Optional[Tensor]

    A 2x2 tensor representation of the magnification matrix, or None if mag_matrix is None. The matrix is constructed from the list as: [[mag_matrix[0], mag_matrix[1]], [mag_matrix[2], mag_matrix[3]]]

ParticleStackCSV

Bases: _ParticleStackBase

Particle stack whose tabular data is loaded from a CSV file.

Particle images are extracted from the micrograph paths referenced in the CSV at run time. This is the original ParticleStack behavior.

Attributes:

  • df_path (str) –

    Path to the CSV file containing the particle data.

load_df

load_df() -> None

Load and validate the particle DataFrame from df_path.

Raises:

  • ValueError

    If required columns are missing from the CSV.

export_results

export_results(allow_file_overwrite: bool = False) -> None

Write the particle table to df_path as CSV.

Parameters:

  • allow_file_overwrite (bool, default: False ) –

    Whether to overwrite an existing file at df_path. Default is False.

Raises:

  • ValueError

    If the parent directory is not writable, or df_path already exists and allow_file_overwrite is False.

to_hdf5

to_hdf5(hdf5_path: str, allow_file_overwrite: bool = False, include_image_stack: bool = False, include_local_stats: bool = False) -> ParticleStackHDF5

Convert this CSV-backed stack to an HDF5-backed stack and write to disk.

Parameters:

  • hdf5_path (str) –

    Destination path for the HDF5 file.

  • allow_file_overwrite (bool, default: False ) –

    Whether to overwrite an existing file, by default False.

  • include_image_stack (bool, default: False ) –

    Write image_stack to the HDF5 file, by default False. Raises ValueError if the image stack has not been loaded.

  • include_local_stats (bool, default: False ) –

    Write every entry currently in :attr:local_stats to the HDF5 file, by default False. Raises ValueError if :attr:local_stats is empty.

Returns:

ParticleStackHDF5

Bases: _ParticleStackBase

Particle stack stored entirely within a single HDF5 file.

The particle table, optional image stack, and optional per-particle local statistic maps are all held in one .h5 file. Two loading modes are supported — choose one. Mixing them raises errors:

  • Load from referenced files: image_stack and local_stats are computed from the paths stored in the particle table. The HDF5 file stores only the particle table (image_stack_stored=False).
  • Load from HDF5: image_stack and local_stats are read directly from the HDF5 datasets (image_stack_stored=True and/or local_stats_stored=True).

Any subset of the *_path statistic-map columns can be stored as local_stats -- not just correlation average/variance. Populate self.local_stats (e.g. via self.local_stats.update(self.get_local_stat_maps())) before calling to_hdf5(include_local_stats=True), and every entry present at that point is written, each to its own dataset under /local_stats named after its column (e.g. mip_path, correlation_average_path).

HDF5 file layout

::

/ (root)
│  attrs: leopard_em_version, extracted_box_size, original_template_size,
│         image_stack_stored, local_stats_stored,
│         global_whitening_applied, local_whitening_applied,
│         global_normalization_applied, local_normalization_applied
├─ particles/
│      particle_id            (N,)   variable-length str  "{mic_stem}_{idx:05d}"
│      <column>               (N,)   float64 or variable-length str
│      ...
├─ image_stack                (N, box_h, box_w)             float32  [optional]
└─ local_stats/                                                      [optional]
       <column>                (N, valid_h, valid_w)         float32
       ...                                     -- one dataset per entry in
                                                   `local_stats` at write time,
                                                   e.g. `mip_path`,
                                                   `correlation_average_path`

where valid_h = extracted_box_size[0] - original_template_size[0] + 1 and valid_w = extracted_box_size[1] - original_template_size[1] + 1.

Attributes:

  • hdf5_path (str) –

    Path to the HDF5 file.

  • allow_file_overwrite (bool) –

    Whether to permit overwriting an existing file, by default False.

  • image_stack_stored (bool) –

    True when /image_stack is present in the HDF5 file.

  • local_stats_stored (bool) –

    True when /local_stats group is present in the HDF5 file.

load_df

load_df() -> None

Load the particle DataFrame from the HDF5 file at hdf5_path.

Raises:

  • FileNotFoundError

    If hdf5_path does not exist.

export_results

export_results(include_image_stack: bool = False, include_local_stats: bool = False) -> None

Write the particle table (and optional tensors) to hdf5_path.

Alias for to_hdf5, kept for API symmetry with ParticleStackCSV.

to_hdf5

to_hdf5(include_image_stack: bool = False, include_local_stats: bool = False) -> None

Write the particle table and optional tensors to hdf5_path.

Parameters:

  • include_image_stack (bool, default: False ) –

    Write image_stack to /image_stack, by default False. Raises ValueError if image_stack is None.

  • include_local_stats (bool, default: False ) –

    Write every entry currently in :attr:local_stats to its own dataset under /local_stats, by default False.

from_hdf5

from_hdf5(path: str, allow_file_overwrite: bool = True) -> ParticleStackHDF5

Load a ParticleStackHDF5 from an existing HDF5 file.

Parameters:

  • path (str) –

    Path to the HDF5 file written by to_hdf5.

  • allow_file_overwrite (bool, default: True ) –

    Passed to the constructor so that the model validator does not reject the path of the file being loaded, by default True.

Returns:

export_particle_stack

export_particle_stack(df: DataFrame, output_path: str, source_particle_stack: _ParticleStackBase, output_format: Literal['csv', 'hdf5'] | None = None, allow_file_overwrite: bool = False) -> ParticleStackCSV | ParticleStackHDF5

Wrap a particle-result DataFrame in a ParticleStack and write it to disk.

Notes

Used by the refine/optimize/constrained-search managers so that their output back- end matches the back-end of the input particle stack by default, while still allowing an explicit override. The DataFrame is only an intermediate — the returned object is the actual particle stack, reusable directly (e.g. fed into the next program) without re-reading from disk.

Parameters:

  • df (DataFrame) –

    The particle table to write (e.g. a refined result table). Must be a superset of the columns a ParticleStackCSV/ParticleStackHDF5 expects; extra columns (e.g. refined_*) are preserved as-is.

  • output_path (str) –

    Destination file path.

  • source_particle_stack (_ParticleStackBase) –

    The particle stack df was derived from. Supplies the default output format (matches its own back-end) and the shared box-size/pre-processing metadata to carry over to the new instance.

  • output_format (Literal['csv', 'hdf5'] | None, default: None ) –

    Explicit output back-end. If None (default), inferred from type(source_particle_stack): ParticleStackHDF5 -> "hdf5", otherwise "csv".

  • allow_file_overwrite (bool, default: False ) –

    Whether to overwrite an existing file at output_path. Default is False.

Returns:

Raises:

  • ValueError

    If output_format is not one of "csv" or "hdf5".