camfi.datamodel.autoannotation module¶

Defines data structures used during automatic annotation model training and inference. Depends on camfi.util, camfi.datamodel.geometry, and camfi.datamodel.via.

class camfi.datamodel.autoannotation.CamfiDataset(*, root: pathlib.Path, via_project: camfi.datamodel.via.ViaProject, crop: camfi.datamodel.geometry.BoundingBox = None, inference_mode: bool = False, mask_maker: camfi.datamodel.autoannotation.MaskMaker = None, transform: camfi.datamodel.autoannotation.ImageTransform = None, min_annotations: int = None, max_annotations: int = None, box_margin: pydantic.types.PositiveInt = 10, exclude: set = None, keys: list = None)¶

Bases: Generic[torch.utils.data.dataset.T_co]

Defines a camfi dataset, compatible with torch.utils.data.DataLoader, for loading image (and annotation) data for camfi model training and inference.

When indexed with an int, a CamfiDataset object returns an image Tensor and a Target object, which contains the manual annotations for that image. If using in inference mode, the Target will not contain any data (except the image index).

Parameters

root (Path) – Path to directory containing images (in subdirectories). Filenames specified in via_project should be relative paths from this root directory.
via_project (ViaProject) – Via project data, containing all annotation data, and all images to be trained on (or for inference to run on).
crop (Optional[BoundingBox]) – If specified, images will be cropped to this bounding box after loading them from file. This can be useful when working with images taken by trail cameras, as some models burn a information bar onto the bottom of the image, which ideally should be removed before training or inference on that image.
inference_mode (bool) – Flag to indicate whether dataset is being used for inference or training. In training mode, data augmentation etc. will be performed on the dataset.
mask_maker (Optional[MaskMaker]) – Instance of MaskMaker which produces instance segementation masks for model training. Only set if inference_mode = False.
transform (Optional[ImageTransform]) – Image transformations to apply for data augmentation. Only set if inference_mode = False.
min_annotations (int) – Only train on images which have at least this many annotations. Only set if inference_mode = False.
max_annotations (float = inf) – Only train on images which have at most this many annotations. Can be useful if running into memory errors on the GPU, since memory consumption depends on the number of annotated objects in the image. Only set if inference_mode = False.
box_margin (PositiveInt = 10) – Margin to add to bounding boxes of object annotations, for model training. Only set if inference_mode = False.
exclude (set[Path]) – Optionally specify a set of image files to exclude for training or inference. Elements in exclude must match the filenames (ie. relative file paths) of images to exclude.
keys (list[str]) – list of image keys. This field is automatically generated upon instantiation, and should not be set.

__weakref__¶: list of weak references to the object (if defined)

classmethod default_exclude(v)¶: Pydantic validation method, called when instantiating CamfiDataset. sets default for exclude.

classmethod generate_filtered_keys(v, values)¶: Pydantic validation method, called when instantiating CamfiDataset. Generates list of image keys.

metadata(idx: int) → camfi.datamodel.via.ViaMetadata¶

Returns the ViaMetadata object of the image at idx.

Parameters: idx (int) – Index of image in dataset.
Returns: metadata – Annotation data for the image.
Return type: ViaMetadata

classmethod only_set_box_margin_if_not_inference_mode(v, values)¶: Pydantic validation method, called when instantiating CamfiDataset. Ensures that box_margin is only set if in training mode.

classmethod only_set_max_annotations_if_not_inference_mode(v, values)¶: Pydantic validation method, called when instantiating CamfiDataset. Ensures that max_annotations is only set if in training mode.

classmethod only_set_min_annotations_if_not_inference_mode(v, values)¶: Pydantic validation method, called when instantiating CamfiDataset. Ensures that min_annotations is only set if in training mode.

classmethod only_set_transform_if_not_inference_mode(v, values)¶: Pydantic validation method, called when instantiating CamfiDataset. Ensures that transform is only set if in training mode.

classmethod set_iff_not_inference_mode(v, values)¶: Pydantic validation method, called when instantiating CamfiDataset. Ensures that mask_maker is set iff in training mode.

class camfi.datamodel.autoannotation.ImageTransform¶

Bases: pydantic.main.BaseModel, abc.ABC

Abstract base class for transforms on images with segmentation data. Instances of subclasses of ImageTransform are callables which when called, apply a transformation to an image Tensor and an accosiated target_dict.

__call__(image: torch.Tensor, target: camfi.datamodel.autoannotation.Target) → tuple¶: Call self as a function.

abstract apply_to_tensor_dict(image: torch.Tensor, target: dict) → tuple¶

Subclasses of ImageTransform should implement this to return a transformed image and target dict.

Parameters

image (torch.Tensor) – Image tensor on which annotations are made.
target (dict[str, torch.Tensor]) – Specifies annotations on image. Output of Target.to_tensor_dict().

Returns

transformed_image (torch.Tensor) – Transformed image.
transformed_target (dict[str, torch.Tensor]) – Specifies transformed annotations on image. Can be used as input to Target.from_tensor_dict().

class camfi.datamodel.autoannotation.MaskMaker(*, shape: tuple, mask_dilate: pydantic.types.PositiveInt = None)¶

Bases: pydantic.main.BaseModel

Defines methods to generate target instance segmentation masks for training a camfi annotation model.

Parameters

shape (tuple[PositiveInt, PositiveInt]) – Shape of image (height, width). This will also be the shape of the instance segmentation masks.
mask_dilate (Optional[PositiveInt] = None) – sets the amount of morphological dilation to apply to segmentation skeletons to produce instance segmentation masks.

get_circle_mask(circle: camfi.datamodel.geometry.CircleShapeAttributes) → torch.Tensor¶

Produces a mask Tensor for a given circle. Raises a ValueError if centre lies outside self.shape.

Parameters: circle (CirlceShapeAttributes) – Geometry from which to generate instance segmentation mask.
Returns: mask – Instance segmentation mask.
Return type: torch.Tensor

get_mask(shape_attributes: Union[camfi.datamodel.geometry.PointShapeAttributes, camfi.datamodel.geometry.CircleShapeAttributes, camfi.datamodel.geometry.PolylineShapeAttributes]) → torch.Tensor¶

Produces a mask Tensor for a given geometry. Raises a ValueError if any point lies outside self.shape. Calls one of self.get_point_mask, self.get_circle_mask, or self.get_polyline_mask, depending on the type of shape_attributes.

Parameters

shape_attributes (Union[) – PointShapeAttributes, CircleShapeAttributes, PolylineShapeAttributes
] – Geometry from which to generate instance segmentation mask.

Returns

mask – Instance segmentation mask.

Return type

torch.Tensor

get_masks(metadata: camfi.datamodel.via.ViaMetadata) → list¶

Calls self.get_mask on all regions in metadata.

Parameters: metadata (ViaMetadata) – Annotation data for an image.
Returns: masks – list of instance segmentation masks. One for each item in metadata.regions.
Return type: list[torch.Tensor]

get_point_mask(point: camfi.datamodel.geometry.PointShapeAttributes) → torch.Tensor¶

Produces a mask Tensor for a given point. Raises a ValueError if point lies outside self.shape.

Parameters: point (PointShapeAttributes) – Geometry from which to generate instance segmentation mask.
Returns: mask – Instance segmentation mask.
Return type: torch.Tensor

get_polyline_mask(polyline: camfi.datamodel.geometry.PolylineShapeAttributes) → torch.Tensor¶

Produces a mask Tensor for a given polyline. Raises a ValueError if any point lies outside self.shape.

Parameters: polyline (PolylineShapeAttributes) – Geometry from which to generate instance segmentation mask.
Returns: mask – Instance segmentation mask.
Return type: torch.Tensor

class camfi.datamodel.autoannotation.Prediction(*, boxes: list, labels: list, masks: list, scores: list)¶

Bases: camfi.datamodel.autoannotation.TargetPredictionABC

Sub-class of TargetPredictionABC, defining the output of camfi model inference.

Parameters

boxes (list[BoundingBox]) – list of bounding boxes of annotated objects in an image.
labels (list[PositiveInt]) – list of integer class labels of annotated objects in an image. Zero is reserved for the background.
masks (list[torch.Tensor]) – list of instance segmentation masks of annotated objects in an image.
scores (list[NonNegativeFloat]) – list of annotation scores (from 0.0 to 1.0)

__config__¶: alias of camfi.datamodel.autoannotation.Config

classmethod all_fields_have_same_length(values)¶: Pydantic validation method, called when instantiating Prediction. Ensures that all fields have the same length.

classmethod empty() → camfi.datamodel.autoannotation.Prediction¶

Initialises an Prediction with no data.

Returns: prediction – Prediction instance with no data.
Return type: Prediction

Examples

>>> Prediction.empty()
Prediction(boxes=[], labels=[], masks=[], scores=[])

filter_by_score(score_thresh: float) → camfi.datamodel.autoannotation.Prediction¶

Returns a Prediction instance with items with scores below score_thresh removed.

Parameters: score_thresh (float) – Score threshold. Annotations with score below score_thresh are not included in the returned Prediction object.
Returns: prediction – Prediction object with below-score-threshold annotations removed.
Return type: Prediction

classmethod from_tensor_dict(tensor_dict: dict) → camfi.datamodel.autoannotation.Prediction¶

Load Prediction from tensor_dict.

Parameters: tensor_dict (dict[str, torch.Tensor]) – Compatible with pytorch instance segmentation model.
Returns: prediction – For use by camfi.
Return type: Prediction

get_subset_from_index(subset: list) → camfi.datamodel.autoannotation.Prediction¶

Returns a Prediction instance from self with items indexed by the elements of subset.

Parameters: subset (list[NonNegativeInt]) – list of indices to include in subset.
Returns: prediction – Prediction object only including specified subset of annotations.
Return type: Prediction

Examples

>>> prediction = Prediction(
...     boxes=[
...         BoundingBox(x0=0, y0=0, x1=1, y1=1),
...         BoundingBox(x0=1, y0=1, x1=2, y1=2),
...     ],
...     labels=[1, 2],
...     masks=[torch.zeros(2, 2), torch.zeros(2, 2)],
...     scores=[0.0, 1.0],
... )
>>> subset_prediction = prediction.get_subset_from_index([0])
>>> subset_prediction.boxes == prediction.boxes[0:1]
True
>>> subset_prediction.labels == prediction.labels[0:1]
True
>>> len(subset_prediction.masks) == 1
True
>>> subset_prediction.scores == prediction.scores[0:1]
True

to_tensor_dict() → dict¶

Send data to a dict of Tensors.

Returns: tensor_dict – Compatible with pytorch instance segmentation model.
Return type: dict[str, torch.Tensor]

class camfi.datamodel.autoannotation.Target(*, boxes: list, labels: list, masks: list, image_id: pydantic.types.NonNegativeInt)¶

Bases: camfi.datamodel.autoannotation.TargetPredictionABC

Sub-class of TargetPredictionABC, defining the input to camfi model training.

Parameters

boxes (list[BoundingBox]) – list of bounding boxes of annotated objects in an image.
labels (list[PositiveInt]) – list of integer class labels of annotated objects in an image. Zero is reserved for the background.
masks (list[torch.Tensor]) – list of instance segmentation masks of annotated objects in an image.
image_id (NonNegativeInt) – Unique index of image to which Target instance relates.

__config__¶: alias of camfi.datamodel.autoannotation.Config

classmethod empty(image_id: pydantic.types.NonNegativeInt = 0) → camfi.datamodel.autoannotation.Target¶

Initialises a Target with and image_id but no other data.

Parameters: image_id (NonNegativeInt) – Unique image index to initialise Target with.
Returns: target – Target instance with no data (except image_id).
Return type: Target

Examples

>>> Target.empty()
Target(boxes=[], labels=[], masks=[], image_id=0)

classmethod from_tensor_dict(tensor_dict: dict) → camfi.datamodel.autoannotation.Target¶

Load Target from tensor_dict.

Parameters: tensor_dict (dict[str, torch.Tensor]) – Compatible with pytorch instance segmentation model.
Returns: target – For use by camfi.
Return type: Target

to_tensor_dict() → dict¶

Send data to a dict of Tensors.

Returns: tensor_dict – Compatible with pytorch instance segmentation model.
Return type: dict[str, torch.Tensor]

class camfi.datamodel.autoannotation.TargetPredictionABC(*, boxes: list, labels: list, masks: list)¶

Bases: pydantic.main.BaseModel, abc.ABC

Abstract base class for defining Targets (the input to the instance segmentation model training) and Predictions (the ouput of instance segmentation inference).

Parameters

boxes (list[BoundingBox]) – list of bounding boxes of annotated objects in an image.
labels (list[PositiveInt]) – list of integer class labels of annotated objects in an image. Zero is reserved for the background.
masks (list[torch.Tensor]) – list of instance segmentation masks of annotated objects in an image.

class Config¶

Bases: object

Pydantic configuration for TargetPredictionABC.

__dict__ = mappingproxy({'__module__': 'camfi.datamodel.autoannotation', '__doc__': 'Pydantic configuration for TargetPredictionABC.', 'arbitrary_types_allowed': True, '__dict__': <attribute '__dict__' of 'Config' objects>, '__weakref__': <attribute '__weakref__' of 'Config' objects>, '__annotations__': {}})¶

__weakref__¶: list of weak references to the object (if defined)

__config__¶: alias of camfi.datamodel.autoannotation.Config

classmethod all_fields_have_same_length(values)¶: Pydantic validation method, called when instantiating subclasses of TargetPredictionABC. Ensures that all fields have the same length.

classmethod all_masks_same_shape(v)¶: Pydantic validation method, called when instantiating subclasses of TargetPredictionABC. Ensures that all masks have the same shape.

abstract classmethod from_tensor_dict(tensor_dict: dict) → camfi.datamodel.autoannotation.TargetPredictionABC¶

Load Target or Prediction from tensor_dict.

Parameters: tensor_dict (dict[str, torch.Tensor]) – Compatible with pytorch instance segmentation model.
Returns: target_or_prediction – Sub-classes of TargetPredictionABC should define this to return an instance of themselves.
Return type: TargetPredictionABC

abstract to_tensor_dict() → dict¶

Send data to a dict of Tensors.

Returns: tensor_dict – Compatible with pytorch instance segmentation model.
Return type: dict[str, torch.Tensor]