camfi.projectconfig module

Provides camfi project config parser.

class camfi.projectconfig.AnnotatorConfig(*, crop: camfi.datamodel.geometry.BoundingBox = None, training: camfi.projectconfig.TrainingConfig = None, inference: camfi.projectconfig.InferenceConfig = None, validation: camfi.projectconfig.ValidationConfig = None)

Bases: pydantic.main.BaseModel

Contains settings for automatic annotation training and inference.

Parameters

  • crop (Optional[BoundingBox]) – If specified, images will be cropped to this bounding box after loading them from file. This can be useful when working with images taken by trail cameras, as some models burn a information bar onto the bottom of the image, which ideally should be removed before training or inference on that image.

  • training (Optional[TrainingConfig]) – Contains settings for camfi annotator model training.

  • inference (Optional[InferenceConfig]) – Contains settings for camfi annotator inference.

exception camfi.projectconfig.AnnotatorConfigUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised if AnnotatorConfig is required but is not specified.

class camfi.projectconfig.CameraConfig(*, camera_time_to_actual_time_ratio: float = None, line_rate: pydantic.types.PositiveFloat = None)

Bases: pydantic.main.BaseModel

Camera hardware-related configuration.

exception camfi.projectconfig.CameraConfigUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised when a method requies a camera config but none was supplied.

class camfi.projectconfig.CamfiConfig(*, root: pydantic.types.DirectoryPath = PosixPath('.'), disable_progress_bar: bool = None, via_project_file: Optional[Union[list, pydantic.types.FilePath]] = None, day_zero: datetime.date = None, output_tz: camfi.util.Timezone = None, default_output: pathlib.Path = None, filters: camfi.projectconfig.FilterConfig = None, camera: camfi.projectconfig.CameraConfig = None, time: camfi.datamodel.locationtime.LocationTimeCollector = None, place: camfi.datamodel.weather.LocationWeatherStationCollector = None, wingbeat_extraction: camfi.wingbeat.WingbeatExtractorConfig = None, annotator: camfi.projectconfig.AnnotatorConfig = None)

Bases: pydantic.main.BaseModel

Defines structure of Camfi’s config.json files, and provides methods for loading data from various sources.

annotate() → camfi.datamodel.via.ViaProject

Performs automatic annotation on all the images in project. Saves project to a file if self.annotator.inference.output_path is set.

Returns

project – With automatic annotations made.

Return type

ViaProject

apply_image_filters() → None

Applies image filter defined in self.filters to self.via_project. Operates in-place.

apply_region_filters() → None

Applies region filters defined in self.filters to self.via_project. Operates in-place.

extract_all_wingbeats() → None

Calls extract_all_wingbeats on self.via_project with parameters taken from configuration.

filelist() → list

lists the images in self.via_project.

Returns

image_files – list of filepaths.

Return type

list[Path]

get_autoannotated_via_project()

Loads automatic annotations from file.

Returns

project – ViaProject with automatically aquired annotations.

Return type

ViaProject

get_image_dataframe() → pandas.core.frame.DataFrame

Calls self.via_project.to_image_dataframe(tz=self.output_tz), returning the result.

get_sun_time_dataframe(days: Union[str, dict]) → pandas.core.frame.DataFrame

Calls self.place.get_sun_time_dataframe

get_weather_dataframe() → pandas.core.frame.DataFrame

Calls self.place.get_weather_dataframe(), returning the result.

property inference_dataset: camfi.datamodel.autoannotation.CamfiDataset

Gets CamfiDataset suitable for automatic annotaton.

load_all_exif_metadata() → None

Calls self.via_project.load_all_exif_metadata with appropriate arguments, set by config. Operates in place.

classmethod parse_json_file(document_path: pathlib.Path, **replace_fields) → camfi.projectconfig.CamfiConfig

Parses JSON document read from file and returns a CamfiConfig instance.

Parameters

  • document_path (Path) – Path to file containing yaml document.

  • **replace_fields – If set, fields will be replaced with those set in replace_fields before parsing.

Returns

config – CamfiConfig instance with settings defined in document.

Return type

CamfiConfig

classmethod parse_yaml(document: str, **replace_fields) → camfi.projectconfig.CamfiConfig

Parses YAML document and returns a CamfiConfig instance.

Parameters

  • document (str) – StrictYAML document string.

  • **replace_fields – If set, fields will be replaced with those set in replace_fields before parsing.

Returns

config – CamfiConfig instance with settings defined in document.

Return type

CamfiConfig

classmethod parse_yaml_file(document_path: pathlib.Path, **replace_fields) → camfi.projectconfig.CamfiConfig

Parses YAML document read from file and returns a CamfiConfig instance.

Parameters

  • document_path (Path) – Path to file containing yaml document.

  • **replace_fields – If set, fields will be replaced with those set in replace_fields before parsing.

Returns

config – CamfiConfig instance with settings defined in document.

Return type

CamfiConfig

train_model() → pathlib.Path

Calls camfi.annotator.train_model with appropriate arguments from self.

property training_dataset: camfi.datamodel.autoannotation.CamfiDataset

Gets CamfiDataset suitable for training camfi automatic annotation model.

validate_annotations() → list

Validates automatically aquired annotations against ground-truth annotations.

Returns

validation_results – Results from validation.

Return type

list[AnnotationValidationResult]

property via_project

Loads ViaProject from file. Raises ViaProjectUnspecifiedError if self.via_project is None.

Returns

via_project – VIA project loaded from self.via_project file.

Return type

ViaProject

write_project() → None

Writes self.via_project as json to self.default_output (if set) or stdout.

yaml(**kwargs) → str

Serialises self to yaml string.

Parameters

**kwargs – Passed to self.json(). E.g. exclude_unset. Note that by default, exclude_none=True to avoid YAMLSerializationError. Also, exclude_unset=True by default, as this makes more sense for CamfiConfig. This differs from the defaults of CamfiConfig.json(). Other defaults are the same.

Returns

document – String containing yaml document.

Return type

str

zip_images() → None

Makes a zip archive of all the images in the VIA project file. Requires self.default_output to be set.

class camfi.projectconfig.FilterConfig(*, image_filters: camfi.projectconfig.ImageFilterConfig = None, region_filters: camfi.datamodel.region_filter_config.RegionFilterConfig = None)

Bases: pydantic.main.BaseModel

Contains settings for filtering images and/or regions (annotations) from a VIA project.

exception camfi.projectconfig.FiltersUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised if filters is required but is not specified.

class camfi.projectconfig.ImageFilterConfig(*, min_annotations: int = None, exclude_images: list = None, include_images: list = None)

Bases: pydantic.main.BaseModel

Contains options for filtering images from a VIA project.

class camfi.projectconfig.InferenceConfig(*, output_path: pathlib.Path = None, model: Union[pathlib.Path, str] = 'release', device: str = 'cpu', backup_device: str = None, split_angle: pydantic.types.PositiveFloat = 15.0, poly_order: pydantic.types.PositiveInt = 2, endpoint_method: str = 'truncate', endpoint_extra_args: list = [10], score_thresh: float = 0.0, overlap_thresh: float = 0.4, edge_thresh: pydantic.types.NonNegativeInt = 20)

Bases: pydantic.main.BaseModel

Contains settings for camfi annotator inference.

Parameters

  • output_path (Optional[Path]) – If set, automatically generated annotations will be saved to file.

  • model (Union[Path, str]) – Path to .pth file specifying model parameters, model name defined in camfi.models.model_urls, or url to model to download from the internet.

  • device (str) – Specifies device to run inference on. E.g. set to “cuda” to use an Nvidia GPU.

  • backup_device (Optional[str]) – Specifies device to run inference on when a runtime error occurs while using device. Probably only makes sense to set this to “cpu” if device=”cuda”. This option enables the annotator to leverage a GPU with limited memory capacity without crashing if a difficult image is encountered.

  • split_angle (PositiveFloat) – Approximate maximum angle between polyline segments in degrees. Note that this will immediately be converted to radians upon instantiation of Annotator.

  • poly_order (PositiveInt) – Order of polynomial used for fitting motion blur paths.

  • endpoint_method (str) – Method to find endpoints of motion blurs. One of camfi.utils.endpoint_methods.

  • endpoint_extra_args (list[Any]) – Extra arguments to pass to endpoint method function.

  • score_thresh (float) – Score threshold between 0.0 and 1.0 for automatic annotations to be kept.

  • overlap_thresh (float) – Minimum proportion of overlap (weighted intersection over minimum) between two instance segmentation masks to infer that one of the masks should be discarded.

  • edge_thresh (NonNegativeInt) – Minimum distance an annotation has to be from the edge of the image before it is converted from a polyline annotation to a circle annotation.

exception camfi.projectconfig.InferenceConfigUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised if InferenceConfig is required but is not specified.

exception camfi.projectconfig.MaskMakerUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised if MaskMaker is required but is not specified.

exception camfi.projectconfig.OutputUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised if output is required but is not specified.

exception camfi.projectconfig.ParameterUnspecifiedError

Bases: Exception

Base exception called when a parameter which needs to be specified is not.

__weakref__

list of weak references to the object (if defined)

exception camfi.projectconfig.PlaceUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised when CamfiConfig.place is requested, but it is unspecified.

class camfi.projectconfig.TrainingConfig(*, mask_maker: camfi.datamodel.autoannotation.MaskMaker = None, min_annotations: int = None, max_annotations: int = None, box_margin: pydantic.types.PositiveInt = 10, test_set_file: pydantic.types.FilePath = None, test_set: list = [], load_pretrained_model: Optional[Union[pathlib.Path, str]] = None, device: str = 'cpu', batch_size: int = 5, num_workers: int = 2, num_epochs: int = 10, outdir: pydantic.types.DirectoryPath = PosixPath('.'), model_name: str = None, save_intermediate: bool = False)

Bases: pydantic.main.BaseModel

Contains settings for camfi annotator model training.

Parameters

  • mask_maker (Optional[MaskMaker]) – Instance of MaskMaker which produces instance segementation masks for model training.

  • min_annotations (int) – Only train on images which have at least this many annotations.

  • max_annotations (float) – Only train on images which have at most this many annotations. Can be useful if running into memory errors on the GPU, since memory consumption depends on the number of annotated objects in the image.

  • box_margin (PositiveInt) – Margin to add to bounding boxes of object annotations, for model training.

  • test_set_file (Optional[FilePath]) – Alternative to setting test_set directly, load it from a file.

  • test_set (list[Path]) – list of images to exclude from training.

  • load_pretrained_model (Optional[Union[Path, str]]) – Path or url to model parameters file. If set, will load the pretrained parameters. By default, will start with a model pre-trained on the Microsoft COCO dataset.

  • device (str) – E.g. “cpu” or “cuda”. Training is typically much faster on a GPU. Use “cuda” for Nvidia GPUs.

  • batch_size (int) – Number of images to load at once.

  • num_workers (int) – Number of worker processes for data loader to spawn.

  • num_epochs (int) – Number of epochs to train.

  • outdir (DirectoryPath) – Path to directory where to save model(s).

  • model_name (Optional[str]) – Identifier to include in model save file. By default the current date in YYYYmmdd format.

  • save_intermediate (bool) – If True, model is saved after each epoch, not just after all epoch are complete. This is recommended, especially if training on a service which could terminate unpredicatbly.

exception camfi.projectconfig.TrainingConfigUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised if TrainingConfig is required but is not specified.

class camfi.projectconfig.ValidationConfig(*, autoannotated_via_project_file: pathlib.Path = None, iou_thresh: float = 0.5, image_sets: List[camfi.projectconfig.ConstrainedStrValue] = ['all'], output_dir: pydantic.types.DirectoryPath = None, output_stem: str = 'validation')

Bases: pydantic.main.BaseModel

Contains settings for camfi annotator validation.

Parameters

  • autoannotated_via_project_file (Optional[Path]) – Path to file containing VIA Project with annotations to validate.

  • iou_thresh (float) – Threshold of intersection-over-union to match annotations.

exception camfi.projectconfig.ValidationConfigUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised if ValidationConfig is required but is not specified.

exception camfi.projectconfig.ViaProjectUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised by CamfiConfig.load_via_project.

exception camfi.projectconfig.WingbeatExtractorConfigUnspecifiedError

Bases: camfi.projectconfig.ParameterUnspecifiedError

Raised when a WingbeatExtractorConfig is needed, but was not supplied.