camfi.annotator module

Defines procedures for training, and evaluation automatic camfi annotation models, and for using them for making automatic annotations (inference). Depends on camfi.util, camfi.datamodel.autoannotation, camfi.datamodel.geometry, camfi.datamode.via, as well as ._torchutils and ._models.

class camfi.annotator.AnnotationValidationResult(*, ious: list = [], polyline_hausdorff_distances: list = [], length_differences: list = [], true_positives: list = [], false_positives: list = [], false_negatives: pydantic.types.NonNegativeInt = 0)

Bases: pydantic.main.BaseModel

Contains various metrics for assessing the quality of a set of automatically obtained annotations of flying insects.

Parameters

  • ious (list[tuple[NonNegativeFloat, NonNegativeFloat]]) – list of (iou, score) pairs. iou is the Intersection over Union of the bounding boxes of true positives to their matched ground truth annotation. All matched annotations are included.

  • polyline_hausdorff_distances (list[tuple[NonNegativeFloat, NonNegativeFloat]]) – list of (h_dist, score) pairs. h_dist is the hausdorff distance of a true positive polyline annotation, where the annotation is matched to a polyline ground truth annotation. Only polyline annotations which matched to a polyline ground truth annotation are included.

  • length_differences (list[tuple[float, NonNegativeFloat]]) – list of (l_diff, score) pairs. l_diff is calculated as the length of a true positive polyline annotation minus the length of it’s matched ground truth annotation. Only polyline annotations which matched to a polyline ground truth annotation are included.

  • true_positives (list[NonNegativeFloat]) – list of scores.

  • false_positives (list[NonNegativeFloat]) – list of scores. Score is the prediction score of the automatic annotation.

  • false_negatives (int) – Number of false negative annotations.

class camfi.annotator.Annotator(*, dataset: camfi.datamodel.autoannotation.CamfiDataset, model: torchvision.models.detection.mask_rcnn.MaskRCNN = 'release', device: Union[str, torch.device] = 'cpu', backup_device: Optional[Union[str, torch.device]] = None, backup_model: torchvision.models.detection.mask_rcnn.MaskRCNN = None, split_angle: pydantic.types.PositiveFloat = 15.0, poly_order: pydantic.types.PositiveInt = 2, endpoint_method: Callable[[...], tuple] = <function endpoint_truncate>, endpoint_extra_args: list = [10], score_thresh: float = 0.4, overlap_thresh: float = 0.4, edge_thresh: pydantic.types.NonNegativeInt = 20, backup_model_used: int = 0)

Bases: pydantic.main.BaseModel

Provides methods for automatically annotating images of flying insects using a pre-trained instance segmentation model.

Parameters

  • dataset (CamfiDataset) – Dataset to annotate.

  • model (Union[str, Path, MaskRCNN]) – Either a path to state dict file which defines the segmentation model, or a url pointing to a model to download, or one of the model names defined in camfi.models.model_urls. Alternatively, a MaskRCNN instance can be given directly.

  • device (Union[str, torch.device]) – Specifies device to run inference on. E.g. set to “cuda” to use an Nvidia GPU.

  • backup_device (Optional[Union[str, torch.device]]) – Specifies device to run inference on when a runtime error occurs while using device. Probably only makes sense to set this to “cpu” if device=”cuda”. This option enables the annotator to leverage a GPU with limited memory capacity without crashing if a difficult image is encountered.

  • backup_model (Optional[MaskRCNN]) – Defines the backup model. Will be automatically generated if backup_device is set. Should not be set manually.

  • split_angle (PositiveFloat) – Approximate maximum angle between polyline segments in degrees. Note that this will immediately be converted to radians upon instantiation of Annotator.

  • poly_order (PositiveInt) – Order of polynomial used for fitting motion blur paths.

  • endpoint_method (Callable[[np.ndarray, ...], tuple[NonNegativeInt, NonNegativeInt]]) – Method to find endpoints of motion blurs. The first argument to this method should be a cropped mask np.ndarray.

  • endpoint_extra_args (list[Any]) – Extra arguments to pass to endpoint_method.

  • score_thresh (float) – Score threshold between 0.0 and 1.0 for automatic annotations to be kept.

  • overlap_thresh (float) – Minimum proportion of overlap (weighted intersection over minimum) between two instance segmentation masks to infer that one of the masks should be discarded.

  • edge_thresh (NonNegativeInt) – Minimum distance an annotation has to be from the edge of the image before it is converted from a polyline annotation to a circle annotation.

annotate(disable_progress_bar: Optional[bool] = True) → camfi.datamodel.via.ViaProject

Calls self.annotate_img on all images and returns a ViaProject instance. Copies the via_attributes and via_settings fields from self.dataset.via_project, and just replaces the via_img_metadata field.

Parameters

disable_progress_bar (Optional[bool]) – If True (default), progress bar is disabled. If set to None, disable on non-TTY.

Returns

project – With automatic annotations made.

Return type

ViaProject

annotate_img(img_idx: int) → list

Calls self.get_prediction, self.filter_annotations, and self.fit_poly to produce annotations for an image specified with img_idx.

Parameters

img_idx (int) – Index of image in via project.

Returns

regions – list of annotations for image.

Return type

list[ViaRegion]

convert_to_circle(polyline: camfi.datamodel.geometry.PolylineShapeAttributes, img_shape: tuple) → Union[camfi.datamodel.geometry.PolylineShapeAttributes, camfi.datamodel.geometry.CircleShapeAttributes]

Checks if a polyline annotation is close to the edge of an image, and if so, converts it to a circle annotation by computing the smallest enclosing circle of all points in the polyline.

Parameters

  • polyline (PolylineShapeAttributes) – Shape to convert if too close to edge.

  • img_shape (tuple[int, int]) – Height and width of image.

Returns

shape_attributes – Geometry of annotation after (possible) conversion. If polyline does not go too close to the edge of the image, then polyline is returned unchanged. Else, a circle annotation is returned.

Return type

Union[PolylineShapeAttributes, CircleShapeAttributes]

filter_annotations(prediction: camfi.datamodel.autoannotation.Prediction) → camfi.datamodel.autoannotation.Prediction

Applies self.score_thresh and self.overlap_thresh to filter out poor quality annotations.

Parameters

prediction (Prediction) – Output of model prediction.

Returns

filtered_prediction – Filtered prediction.

Return type

Prediction

fit_poly(box: camfi.datamodel.geometry.BoundingBox, mask: torch.Tensor) → Optional[Union[camfi.datamodel.geometry.PolylineShapeAttributes, camfi.datamodel.geometry.CircleShapeAttributes]]

Uses polynomial regression to fit a polyline annotation to the provided segmentation mask.

Parameters

  • box (BoundingBox) – Fully contains the object to be annotated.

  • mask (tensor or array) – Segmentation mask of instance with shape (image_width, image_height).

Returns

shape_attributes – Geometry of automatic annotation.

Return type

Union[PolylineShapeAttributes, CircleShapeAttributes, None]

get_prediction(img_idx: pydantic.types.NonNegativeInt) → camfi.datamodel.autoannotation.Prediction

Run predicion on a single image. First tries to use the model on self.device, and falls back to the model on self.backup_device if a RuntimeError is caught (if set).

Parameters

img_idx (int) – Index of image in via project.

Returns

prediction – Output of model prediction.

Return type

Prediction

camfi.annotator.copy_annotation_model(model: torchvision.models.detection.mask_rcnn.MaskRCNN) → torchvision.models.detection.mask_rcnn.MaskRCNN

Copies a camfi annotation model.

Parameters

model (MaskRCNN) – Model to copy.

Returns

model_copy – Copy of model.

Return type

MaskRCNN

camfi.annotator.load_annotation_model(model_path_or_url: Union[pathlib.Path, str]) → torchvision.models.detection.mask_rcnn.MaskRCNN

Loads a camfi annotation model. Accepts any model key provided in camfi.models, a Path object, or a URL str.

Parameters

model_path_or_url (Union[Path, str]) – Path to .pth file specifying model parameters, model name defined in camfi.models.model_urls, or url to model to download from the internet.

Returns

model – Instance segmentation model used for automatic annotation.

Return type

MaskRCNN

camfi.annotator.train_model(dataset: camfi.datamodel.autoannotation.CamfiDataset, load_pretrained_model: Optional[Union[pathlib.Path, str]] = None, device: Union[str, torch.device] = 'cpu', batch_size: int = 5, num_workers: int = 2, num_epochs: int = 10, outdir: pydantic.types.DirectoryPath = PosixPath('.'), model_name: Optional[str] = None, save_intermediate: bool = False) → pathlib.Path

Trains a camfi instance segmentation annotation model on specified dataset, saving to trained model to outdir.

Parameters

  • dataset (CamfiDataset) – Dataset on which to train the model.

  • load_pretrained_model (Optional[Union[Path, str]]) – Path or url to model parameters file. If set, will load the pretrained parameters. By default, will start with a model pre-trained on the Microsoft COCO dataset.

  • device (Union[str, torch.device]) – E.g. “cpu” or “cuda”. Training is typically much faster on a GPU. Use “cuda” for Nvidia GPUs.

  • batch_size (int) – Number of images to load at once.

  • num_workers (int) – Number of worker processes for data loader to spawn.

  • num_epochs (int) – Number of epochs to train.

  • outdir (DirectoryPath) – Path to directory where to save model(s).

  • model_name (Optional[str]) – Identifier to include in model save file. By default the current date in YYYYmmdd format.

  • save_intermediate (bool) – If True, model is saved after each epoch, not just after all epoch are complete. This is recommended, especially if training on a service which could terminate unpredicatbly (e.g. Google Colab).

Returns

model_path – Path to saved model.

Return type

Path

camfi.annotator.validate_annotations(auto_annotations: camfi.datamodel.via.ViaProject, ground_truth: camfi.datamodel.via.ViaProject, iou_thresh: float = 0.5, subset_functions: Optional[dict] = None, disable_progress_bar: Optional[bool] = True) → list

Compares automatic annotations against a ground-truth annotations for validation puposes. Validation data is stored in an AnnotationValidationResult object.

Parameters

  • auto_annotations (ViaProject) – Automatically obtained annotations to assess.

  • ground_truth (ViaProject) – Manually created ground-truth annotations.

  • iou_thresh (float) – Threshold of intersection-over-union of bounding boxes to be considered a match. Typically, this is 0.5.

  • subset_functions (Optional[dict[str, Callable[[ViaMetadata], bool]]]) – Mapping from subset name to subset function. If set, validation will be repeated multiple times with different subsets, once for each element.

  • disable_progress_bar (Optional[bool]) – If True (default), progress bar is disabled. If set to None, disable on non-TTY.

Returns

validation_results – list containing instances of AnnotationValidationResult. If subset_functions is set, then validation_results will have len(subset_functions) elements. By default it will just contain one element.

Return type

list[AnnotationValidationResult]