merlin.util package

Submodules

merlin.util.barcodedb module

class merlin.util.barcodedb.BarcodeDB(dataSet, analysisTask)

Bases: object

An abstract class for storing and retrieving barcode information.

For each barcode in the database, the following information is stored:

barcode - the error corrected binary word corresponding to the barcode assigned_barcode - barcode_id - the index of the barcode in the codebook fov - the field of view where the barcode was identified mean_intensity - the mean intensity in the pixels corresponding to

this barcode

max_intensity - the max intensity in the pixels corresponding to

this barcode

area - the number of pixels covered by the barcode mean_distance - the distance between the barcode and the measured

pixel traces averaged for all pixels corresponding to the barcode

min_distance - the minimum distance between the barcode and the measured

pixel traces of all pixels corresponding to the barcode

x,y,z - the average x,y,z position of all pixels covered by the barcode weighted_x, weighted_y, weighted_z - the average x,y,z position of

of all pixels covered by the barcode weighted by the magnitude of each pixel

global_x, global_y, global_z - the global x,y,z position of the barcode cell_index - the cell that contains this barcode intensity_i - the mean intensity across corresponding pixels for

bit i where i is an integer from 0 to the number of bits-1.

abstract empty_database(fov: int = None) → None

Remove all barcodes from this database.

Parameters

fov – index of the field of view. If specified, only barcodes corresponding to the specified fov will be removed. Otherwise, all barcodes will be removed.

abstract get_barcodes(fov: int = None, columnList: List[str] = None, chunksize: int = None)

Get barcodes stored in this database.

Parameters

• fov – index of the field view. If None, barcodes for all fovs are returned.

• columnList – list of columns to extract. If not specified, all columns are returned.

• chunksize – the size of chunks to iterate. If not specified, a pandas dataframe is returned otherwise an iterator over the barcodes is returned.

Returns

if chunksize is not set, a pandas dataframe containing all the

requested barcodes is returned. Otherwise an iterator is returned that iterates over the requested barcodes.

abstract get_filtered_barcodes(areaThreshold: int, intensityThreshold: float, distanceThreshold: float = None, fov: int = None, chunksize: int = None)

Get barcodes from this barcode database that pass the area and intensity thresholds.

Parameters

• areaThreshold – the minimum area threshold. Barcodes that have an area equal to the specified threshold are included in the output.

• intensityThreshold – the minimum value for mean_intenity for the select barcodes

• distanceThreshold – the maximum value for min_distance for the select barcodes

• fov – index of the field view. If None, barcodes for all fovs are returned.

• chunksize – the size of chunks to iterate. If not specified, a pandas dataframe is returned otherwise an iterator over the barcodes is returned.

Returns

if chunksize is not set, a pandas dataframe containing all the

requested barcodes is returned. Otherwise an iterator is returned that iterates over the requested barcodes.

abstract get_intensities_for_barcodes_with_area(area: int) → pandas.core.series.Series

Gets the barcode intensities for barcodes that have the specified area.

abstract write_barcodes(barcodeInformation: pandas.core.frame.DataFrame, fov: int = None) → None

Writes the specified barcodes into the barcode database.

If all the barcodes correspond to the same fov, then fov can be specified to improve performance. This function does not verify that fov is specified incorrectly. If fov is specified but the provided barcodes are not all from the same fov, the barcode database may become corrupted.

Parameters

• barcodeInformation – barcodes to write to the database. The dataframe must have the columns specified for a barcode database.

• fov – the fov of the barcodes if they all correspond to the same fov. If barcodeInformation contains barcodes from different fovs, then fov should be set to None.

get_barcode_intensities() → pandas.core.series.Series

Get mean intensities for all barcodes in this database.

Returns

series containing mean intensity for all barcodes

get_barcode_areas() → pandas.core.series.Series

Get areas for all barcodes in this database.

Returns

series containing areas for all barcodes

get_barcode_distances() → pandas.core.series.Series

Get distances for all barcodes in this database

Returns

series containing distances for all barcodes

class merlin.util.barcodedb.PyTablesBarcodeDB(dataSet: merlin.core.dataset.DataSet, analysisTask)

Bases: merlin.util.barcodedb.BarcodeDB

empty_database(fov: int = None) → None

Remove all barcodes from this database.

Parameters

fov – index of the field of view. If specified, only barcodes corresponding to the specified fov will be removed. Otherwise, all barcodes will be removed.

get_barcodes(fov=None, columnList=None, chunkSize=None) → pandas.core.frame.DataFrame

Get barcodes stored in this database.

Parameters

• fov – index of the field view. If None, barcodes for all fovs are returned.

• columnList – list of columns to extract. If not specified, all columns are returned.

• chunksize – the size of chunks to iterate. If not specified, a pandas dataframe is returned otherwise an iterator over the barcodes is returned.

Returns

if chunksize is not set, a pandas dataframe containing all the

requested barcodes is returned. Otherwise an iterator is returned that iterates over the requested barcodes.

get_filtered_barcodes(areaThreshold: int, intensityThreshold: float, distanceThreshold: float = None, fov: int = None, chunksize: int = None)

Get barcodes from this barcode database that pass the area and intensity thresholds.

Parameters

• areaThreshold – the minimum area threshold. Barcodes that have an area equal to the specified threshold are included in the output.

• intensityThreshold – the minimum value for mean_intenity for the select barcodes

• distanceThreshold – the maximum value for min_distance for the select barcodes

• fov – index of the field view. If None, barcodes for all fovs are returned.

• chunksize – the size of chunks to iterate. If not specified, a pandas dataframe is returned otherwise an iterator over the barcodes is returned.

Returns

if chunksize is not set, a pandas dataframe containing all the

requested barcodes is returned. Otherwise an iterator is returned that iterates over the requested barcodes.

get_intensities_for_barcodes_with_area(area: int) → pandas.core.series.Series

Gets the barcode intensities for barcodes that have the specified area.

write_barcodes(barcodeInformation: pandas.core.frame.DataFrame, fov: int = None) → None

Writes the specified barcodes into the barcode database.

If all the barcodes correspond to the same fov, then fov can be specified to improve performance. This function does not verify that fov is specified incorrectly. If fov is specified but the provided barcodes are not all from the same fov, the barcode database may become corrupted.

Parameters

• barcodeInformation – barcodes to write to the database. The dataframe must have the columns specified for a barcode database.

• fov – the fov of the barcodes if they all correspond to the same fov. If barcodeInformation contains barcodes from different fovs, then fov should be set to None.

merlin.util.binary module

merlin.util.binary.bit_list_to_int(bitList: List[bool]) → int

Converts a binary list to an integer

Parameters

bitList – the binary list to convert

Returns

The integer corresponding to the input bit list

merlin.util.binary.int_to_bit_list(intIn: int, bitCount: int) → List[bool]

Converts an integer to a binary list with the specified number of bits.

Parameters

• intIn – the integer to convert

• bitCount – the number of bits to include in the output bit list

Returns

A list of bit that specifies the input integer. The least significant

bit is first in the list.

merlin.util.binary.k_bit_set(n: int, k: int) → bool

Determine if the k’th bit of integer n is set to 1.

Parameters

• n – the integer to check

• k – the index of the bit to check where 0 corresponds with the least significant bit

Returns

true if the k’th bit of the integer n is 1, otherwise false. If

k is None, this function returns None.

merlin.util.binary.flip_bit(barcode: List[bool], bitIndex: int) → List[bool]

Generates a version of the provided barcode where the bit at the specified index is inverted.

The provided barcode is left unchanged. It is copied before flipping the bit.

Parameters

• barcode – A binary array where the i’th entry corresponds with the value of the i’th bit

• bitIndex – The index of the bit to reverse

Returns

A copy of barcode with bitIndex inverted

merlin.util.decoding module

merlin.util.decoding.normalize(x)

class merlin.util.decoding.PixelBasedDecoder(codebook: merlin.data.codebook.Codebook, scaleFactors: numpy.ndarray = None, backgrounds: numpy.ndarray = None)

Bases: object

decode_pixels(imageData: numpy.ndarray, scaleFactors: numpy.ndarray = None, backgrounds: numpy.ndarray = None, distanceThreshold: float = 0.5176, magnitudeThreshold: float = 1, lowPassSigma: float = 1)

Assign barcodes to the pixels in the provided image stock.

Each pixel is assigned to the nearest barcode from the codebook if the distance between the normalized pixel trace and the barcode is less than the distance threshold.

Parameters

• imageData – input image stack. The first dimension indexes the bit number and the second and third dimensions contain the corresponding image.

• scaleFactors – factors to rescale each bit prior to normalization. The length of scaleFactors must be equal to the number of bits.

• backgrounds – background to subtract from each bit prior to applying the scale factors and prior to normalization. The length of backgrounds must be equal to the number of bits.

• distanceThreshold – the maximum distance between an assigned pixel and the nearest barcode. Pixels for which the nearest barcode is greater than distanceThreshold are left unassigned.

• magnitudeThreshold – the minimum pixel magnitude for which a barcode can be assigned that pixel. All pixels that fall below the magnitude threshold are not assigned a barcode in the decoded image.

• lowPassSigma – standard deviation for the low pass filter that is applied to the images prior to decoding.

Returns

Four results are returned as a tuple (decodedImage, pixelMagnitudes,

normalizedPixelTraces, distances). decodedImage is an image indicating the barcode index assigned to each pixel. Pixels for which a barcode is not assigned have a value of -1. pixelMagnitudes is an image where each pixel is the norm of the pixel trace after scaling by the provided scaleFactors. normalizedPixelTraces is an image stack containing the normalized intensities for each pixel. distances is an image containing the distance for each pixel to the assigned barcode.

extract_barcodes_with_index(barcodeIndex: int, decodedImage: numpy.ndarray, pixelMagnitudes: numpy.ndarray, pixelTraces: numpy.ndarray, distances: numpy.ndarray, fov: int, cropWidth: int, zIndex: int = None, globalAligner=None, segmenter=None, minimumArea: int = 0) → pandas.core.frame.DataFrame

Extract the barcode information from the decoded image for barcodes that were decoded to the specified barcode index.

Parameters

• barcodeIndex – the index of the barcode to extract the corresponding barcodes

• decodedImage – the image indicating the barcode index assigned to each pixel

• pixelMagnitudes – an image containing norm of the intensities for each pixel across all bits after scaling by the scale factors

• pixelTraces – an image stack containing the normalized pixel intensity traces

• distances – an image indicating the distance between the normalized pixel trace and the assigned barcode for each pixel

• fov – the index of the field of view

• cropWidth – the number of pixels around the edge of each image within which barcodes are excluded from the output list.

• zIndex – the index of the z position

• globalAligner – the aligner used for converted to local x,y coordinates to global x,y coordinates

• segmenter – the cell segmenter for assigning a cell for each of the identified barcodes

• minimumArea – the minimum area of barcodes to identify. Barcodes less than the specified minimum area are ignored.

Returns

a pandas dataframe containing all the barcodes decoded with the

specified barcode index

extract_refactors(decodedImage, pixelMagnitudes, normalizedPixelTraces, extractBackgrounds=False) → Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]

Calculate the scale factors that would result in the mean on bit intensity for each bit to be equal.

This code follows the legacy matlab decoder.

If the scale factors for this decoder are not set to 1, then the calculated scale factors are dependent on the input scale factors used for the decoding.

Parameters

imageSet – the image stack to decode in order to determine the scale factors

Returns

a tuple containing an array of the scale factors, an array

of the backgrounds, and an array of the abundance of each barcode determined during the decoding. For the scale factors and the backgrounds, the i’th entry is the scale factor for bit i. If extractBackgrounds is false, the returned background array is all zeros.

Module contents