nimbus_inference package

Submodules

nimbus_inference.cell_analyzer module

class nimbus_inference.cell_analyzer.CellAnalyzer(input_dir, cell_df, output_dir, segmentation_naming_convention, img_width='1000px', context=None)

Bases: object

Cell Analyzer for Nimbus application, can be used to compare and annotate cell images.

Parameters:

  • input_dir (str) – Path to directory containing individual channels of multiplexed images

  • cell_df (pd.DataFrame) – DataFrame with columns ‘pixie_ct’, ‘nimbus_ct’, ‘fov’, ‘label’

  • output_dir (str) – Path to save output annotations

  • segmentation_naming_convention (fn) – Function that maps input path to segmentation path

  • img_width (str) – Width of images in viewer.

  • context (int) – area around the cell to display

cell_id_select_fn(change)

Selects fov to display.

Parameters:

change (dict) – Change dictionary from ipywidgets.

create_composite_image(path_dict, add_boundaries=True)

Creates composite image from input paths.

Parameters:

  • path_dict (dict) – Dictionary of paths to images.

  • add_boundaries (bool) – Whether to add boundaries to multiplex image.

Returns:

Composite image.

Return type:

np.array

create_instance_image(path_dict, cell_id)

Creates composite image from input paths.

Parameters:

  • path_dict (dict) – Dictionary of paths to images.

  • cell_id (int) – id of cell to highlight

Returns:

Composite image.

Return type:

np.array

display()

Displays viewer.

layout()

Creates layout for viewer.

save_annotations_button_click(button)

Updates composite image in viewer when pixie button is clicked.

Parameters:

button (ipywidgets.Button) – Button widget that was clicked.

search_for_similar(select_value)

Searches for similar filename in input directory.

Parameters:

select_value (str) – Filename to search for.

Returns:

Path to similar filename.

Return type:

str

update_button_click(button)

Updates composite image in viewer when update button is clicked.

Parameters:

button (ipywidgets.Button) – Button widget that was clicked.

update_composite()

Updates composite image in viewer.

update_img(image_viewer, composite_image)

Updates image in viewer by saving it as png and loading it with the viewer widget.

Parameters:

  • image_viewer (ipywidgets.Image) – Image widget to update.

  • composite_image (np.array) – Composite image to display.

nimbus_inference.nimbus module

class nimbus_inference.nimbus.Nimbus(dataset: MultiplexDataset, output_dir: str, save_predictions: bool = True, half_resolution: bool = True, batch_size: int = 4, test_time_aug: bool = True, input_shape: list = [1024, 1024], device: str = 'auto')

Bases: Module

Nimbus application class for predicting marker activity for cells in multiplexed images.

Parameters:

  • dataset (MultiplexDataset) – Path to directory containing fovs.

  • output_dir (str) – Path to directory to save output.

  • save_predictions (bool) – Whether to save predictions.

  • half_resolution (bool) – Whether to run model on half resolution images.

  • batch_size (int) – Batch size for model inference.

  • test_time_aug (bool) – Whether to use test time augmentation.

  • input_shape (list) – Shape of input images.

  • suffix (str) – Suffix of images to load.

  • device (str) – Device to run model on, either “auto” (either “mps” or “cuda” , with “cpu” as a fallback), “cpu”, “cuda”, or “mps”. Defaults to “auto”.

check_inputs()

check inputs for Nimbus model

initialize_model(padding='reflect')

Initializes the model and loads the latest weights from Hugging Face Hub if newer weights are available.

Parameters:

padding (str) – Padding mode for model, either “reflect” or “valid”.

predict_fovs()

Predicts cell classification for input data.

Returns:

Predicted cell classification.

Return type:

np.array

predict_segmentation(input_data, preprocess_kwargs)

Predicts segmentation for input data.

Parameters:

  • input_data (np.array) – Input data to predict segmentation for.

  • preprocess_kwargs (dict) – Keyword arguments for preprocessing.

  • batch_size (int) – Batch size for prediction.

Returns:

Predicted segmentation.

Return type:

np.array

prepare_normalization_dict(quantile=0.999, clip_values=(0, 2), n_subset=10, multiprocessing=False, overwrite=False)

Load or prepare and save normalization dictionary for Nimbus model.

Parameters:

  • quantile (float) – Quantile to use for normalization.

  • clip_values (list) – Values to clip images to after normalization.

  • n_subset (int) – Number of fovs to use for normalization.

  • multiprocessing (bool) – Whether to use multiprocessing.

  • overwrite (bool) – Whether to overwrite existing normalization dict.

Returns:

Dictionary of normalization factors.

Return type:

dict

nimbus_inference.nimbus.nimbus_preprocess(image, **kwargs)

Preprocess input data for Nimbus model.

Parameters:

  • image (np.array) – array to be processed

  • **kwargs – keyword arguments for preprocessing: {normalize (bool): whether to normalize the image, marker (str): name of marker, normalization_dict (dict): normalization dictionary, clip_values (tuple): min/max values to clip the image to after normalization}

Returns:

processed image array

Return type:

np.array

nimbus_inference.nimbus.prep_naming_convention(deepcell_output_dir)

Prepares the naming convention for the segmentation data produced with the DeepCell library.

Parameters:

deepcell_output_dir (str) – path to directory where segmentation data is saved

Returns:

function that returns the path to the

segmentation data for a given fov

Return type:

function

nimbus_inference.unet module

class nimbus_inference.unet.ConvBlock(layer_idx, filters_root, kernel_size, padding, activation, up=False, **kwargs)

Bases: Module

Convolutional block consisting of two convolutional layers with same number of filters

and a batch normalization layer in between.

Parameters:

  • layer_idx (int) – index of the layer, used to compute the number of filters

  • filters_root (int) – number of filters in the first convolutional layer

  • kernel_size (int) – size of convolutional kernels

  • padding – padding, either “VALID”, “CONSTANT”, “REFLECT”, or “SYMMETRIC” activation: activation to be used

  • data_format – data format, either “channels_last” or “channels_first”

forward(x)

Apply ConvBlock to inputs.

Parameters:

x (torch.Tensor) – input tensor

Returns:

output tensor

Return type:

torch.Tensor

class nimbus_inference.unet.CropConcatBlock(**kwargs)

Bases: Module

CropConcatBlock that crops spatial dimensions and concatenates filter maps.

Parameters:

data_format (str) – data format, either “channels_last” or “channels_first”

forward(x, down_layer)

Apply CropConcatBlock to inputs.

Parameters:

  • x (torch.Tensor) – input tensor

  • down_layer (torch.Tensor) – tensor from the contracting path

Returns:

output tensor

Return type:

torch.Tensor

class nimbus_inference.unet.Pad2D(padding=(1, 1), mode='constant')

Bases: Module

Padding for 2D input (e.g. images).

Parameters:

  • padding – tuple of 2 ints, how many zeros to add at the beginning and at the end of the 2 padding dimensions (rows and cols)

  • mode – “constant”, “reflect”, or “replicate”

forward(x)

Pad 2D input tensor x.

Parameters:

x (torch.Tensor) – 4D input tensor (B, C, H, W)

Returns:

4D output tensor (B, C, H + 2*padding[0], W + 2*padding[1])

Return type:

torch.Tensor

class nimbus_inference.unet.UNet(num_classes: int = 2, layer_depth: int = 5, filters_root: int = 64, data_format='channels_first', kernel_size: int = 3, pool_size: int = 2, padding: str = 'reflect', activation: str = 'ReLU')

Bases: Module

UNet model with contracting and expanding paths.

Parameters:

  • num_classes (int) – number of classes

  • layer_depth (int) – number of layers

  • filters_root (int) – number of filters in the first convolutional layer

  • data_format – data format, either “channels_last” or “channels_first”

  • kernel_size (int) – size of convolutional kernels

  • pool_size (int) – size of the pooling layer

  • padding (str) – padding, either “VALID”, “CONSTANT”, “REFLECT”, or “SYMMETRIC”

  • activation (str) – activation to be used

forward(x)

Forward pass of the UNet model.

Parameters:

x (torch.Tensor) – input tensor

Returns:

output tensor

Return type:

torch.Tensor

class nimbus_inference.unet.UpconvBlock(layer_idx, filters_root, kernel_size, pool_size, padding, activation, **kwargs)

Bases: Module

Upconvolutional block consisting of an upsampling layer and a convolutional layer.

Parameters:

  • layer_idx (int) – index of the layer, used to compute the number of filters

  • filters_root (int) – number of filters in the first convolutional layer

  • kernel_size (tuple) – size of convolutional kernels

  • pool_size (tuple) – size of the pooling layer

  • padding (str) – padding, either “VALID”, “CONSTANT”, “REFLECT”, or “SYMMETRIC”

  • activation (fn) – activation to be used

  • data_format (str) – data format, either “channels_last” or “channels_first”

forward(x)

Apply UpconvBlock to inputs.

Parameters:

x (torch.Tensor) – input tensor

Returns:

output tensor

Return type:

torch.Tensor

nimbus_inference.unet.maybe_crop(x, target_shape, data_format='channels_first')

Center crops x to target_shape if necessary.

Parameters:

  • x (torch.Tensor) – input tensor

  • target_shape (list) – shape of a reference tensor in BHWC or BCHW format

  • data_format (str) – data format, either “channels_last” or “channels_first”

Returns:

cropped tensor

Return type:

x (torch.Tensor)

nimbus_inference.utils module

class nimbus_inference.utils.HidePrints

Bases: object

Context manager to hide prints

class nimbus_inference.utils.LazyOMETIFFReader(fpath: str)

Bases: OMETIFFReader

Lazy OMETIFFReader class that reads channels only when needed

Parameters:

fpath (str) – path to ome.tif file

get_channel(channel_name: str)

Get an individual channel from the OME-TIFF file by name

Parameters:

channel_name (str) – name of the channel

Returns:

channel image

Return type:

np.array

get_channel_names()

Get the channel names of the OME-TIFF file

Returns:

list of channel names

Return type:

list

get_metadata()

Get the metadata of the OME-TIFF file

Returns:

metadata of the OME-TIFF file

Return type:

dict

get_shape()

Get the shape of the OME-TIFF file array data

Returns:

shape of the array data

Return type:

tuple

class nimbus_inference.utils.MultiplexDataset(fov_paths: list, segmentation_naming_convention: Callable = None, include_channels: list = [], suffix: str = '.tiff', silent=False)

Bases: object

Multiplex dataset class that gives a common interface for data loading of multiplex datasets stored as individual channel images in folders or as multi-channel tiffs.

Parameters:

  • fov_paths (list) – list of paths to fovs

  • segmentation_naming_convention (function) – function to get instance mask path from fov path

  • suffix (str) – suffix of channel images

  • silent (bool) – whether to print messages

check_inputs()

Check inputs for Nimbus model

filter_channels(channels)

Filter channels based on include_channels

Parameters:

channels (list) – list of channel names

Returns:

filtered list of channel names

Return type:

list

get_channel(fov: str, channel: str)

Get a channel from a fov

Parameters:

  • fov (str) – name of a fov

  • channel (str) – channel name

Returns:

channel image

Return type:

np.array

get_channel_single(fov: str, channel: str)

Get a channel from a fov stored as a folder with individual channel images

Parameters:

  • fov (str) – name of a fov

  • channel (str) – channel name

Returns:

channel image

Return type:

np.array

get_channel_stack(fov: str, channel: str)

Get a channel from a multi-channel tiff

Parameters:

  • fov (str) – name of a fov

  • channel (str) – channel name

  • data_format (str) – data format

Returns:

channel image

Return type:

np.array

get_channels()

Get the channel names for the dataset

get_fovs()

Get the fovs in the dataset

get_segmentation(fov: str)

Get the instance mask for a fov

Parameters:

fov (str) – name of a fov

Returns:

instance mask

Return type:

np.array

is_multi_channel_tiff(fov_path: str)

Check if fov is a multi-channel tiff

Parameters:

fov_path (str) – path to fov

Returns:

whether fov is multi-channel

Return type:

bool

nimbus_inference.utils.calculate_normalization(dataset: MultiplexDataset, quantile: float)

Calculates the normalization values for a given ome file

Parameters:

  • dataset (MultiplexDataset) – dataset object

  • quantile (float) – quantile to use for normalization

Returns:

dict with channel names as keys and norm factors as values

Return type:

dict

nimbus_inference.utils.nimbus_preprocess(image, **kwargs)

Preprocess input data for Nimbus model.

Parameters:

image – array to be processed

Returns:

processed image array

Return type:

np.array

nimbus_inference.utils.predict_fovs(nimbus, dataset: MultiplexDataset, normalization_dict: dict, output_dir: str, suffix: str = 'tiff', save_predictions: bool = True, half_resolution: bool = False, batch_size: int = 4, test_time_augmentation: bool = True)

Predicts the segmentation map for each mplex image in each fov

Parameters:

  • nimbus (Nimbus) – nimbus object

  • dataset (MultiplexDataset) – dataset object

  • normalization_dict (dict) – dict with channel names as keys and norm factors as values

  • output_dir (str) – path to output dir

  • suffix (str) – suffix of mplex images

  • save_predictions (bool) – whether to save predictions

  • half_resolution (bool) – whether to use half resolution

  • batch_size (int) – batch size

  • test_time_augmentation (bool) – whether to use test time augmentation

Returns:

cell table with predicted confidence scores per fov and cell

Return type:

pd.DataFrame

nimbus_inference.utils.prepare_input_data(mplex_img, instance_mask)

Prepares the input data for the segmentation model

Parameters:

  • mplex_img (np.array) – multiplex image

  • instance_mask (np.array) – instance mask

Returns:

input data for segmentation model

Return type:

np.array

nimbus_inference.utils.prepare_normalization_dict(dataset: MultiplexDataset, output_dir: str, quantile: float = 0.999, n_subset: int = 10, n_jobs: int = 1, output_name: str = 'normalization_dict.json')

Prepares the normalization dict for a list of ome.tif fovs

Parameters:

  • MultiplexDataset (list) – list of paths to fovs

  • output_dir (str) – path to output directory

  • quantile (float) – quantile to use for normalization

  • n_subset (int) – number of fovs to use for normalization

  • n_jobs (int) – number of jobs to use for joblib multiprocessing

  • output_name (str) – name of output file

Returns:

dict with channel names as keys and norm factors as values

Return type:

dict

nimbus_inference.utils.segment_mean(instance_mask, prediction)

Calculates the mean prediction per instance

Parameters:

  • instance_mask (np.array) – instance mask

  • prediction (np.array) – prediction

Returns:

unique instance ids np.array: mean prediction per instance

Return type:

np.array

nimbus_inference.utils.test_time_aug(input_data, channel, app, normalization_dict, rotate=True, flip=True, batch_size=4)

Performs test time augmentation

Parameters:

  • input_data (np.array) – input data for segmentation model, mplex_img and binary mask

  • channel (str) – channel name

  • app (Nimbus) – segmentation model

  • normalization_dict (dict) – dict with channel names as keys and norm factors as values

  • rotate (bool) – whether to rotate

  • flip (bool) – whether to flip

  • batch_size (int) – batch size

Returns:

predicted segmentation map

Return type:

np.array

nimbus_inference.viewer_widget module

class nimbus_inference.viewer_widget.NimbusViewer(dataset: MultiplexDataset, output_dir: str, img_width='600px', suffix='.tiff', max_resolution=(2048, 2048))

Bases: object

Viewer for Nimbus application.

Parameters:

  • dataset (MultiplexDataset) – dataset object

  • output_dir (str) – Path to directory containing output of Nimbus application.

  • segmentation_naming_convention (fn) – Function that maps input path to segmentation path

  • img_width (str) – Width of images in viewer.

  • suffix (str) – Suffix of images in dataset.

  • max_resolution (tuple) – Maximum resolution of images in viewer.

create_composite_from_dataset(path_dict)

Creates composite image from input paths.

Parameters:

path_dict (dict) – Dictionary of paths to images.

Returns:

Composite image.

Return type:

np.array

create_composite_image(path_dict, add_overlay=True, add_boundaries=False)

Creates composite image from input paths.

Parameters:

path_dict (dict) – Dictionary of paths to images.

Returns:

Composite image.

Return type:

np.array

display()

Displays viewer.

layout()

Creates layout for viewer.

overlay(composite_image, add_boundaries=False, add_overlay=False)

Adds overlay to composite image.

Parameters:

  • composite_image (np.array) – Composite image to add overlay to.

  • boundaries (bool) – Whether to add boundaries to overlay.

Returns:

Composite image with overlay.

Return type:

np.array

search_for_similar(select_value)

Searches for similar filename in input directory.

Parameters:

select_value (str) – Filename to search for.

Returns:

Path to similar filename.

Return type:

str

select_fov(change)

Selects fov to display.

Parameters:

change (dict) – Change dictionary from ipywidgets.

update_button_click(button)

Updates composite image in viewer when update button is clicked.

update_composite()

Updates composite image in viewer.

update_img(image_viewer, composite_image)

Updates image in viewer by saving it as png and loading it with the viewer widget.

Parameters:

  • image_viewer (ipywidgets.Image) – Image widget to update.

  • composite_image (np.array) – Composite image to display.

Module contents