vak.config.eval.EvalConfig¶

class vak.config.eval.EvalConfig(checkpoint_path, output_dir, model, batch_size, dataset: DatasetConfig, trainer: TrainerConfig, labelmap_path=None, frames_standardizer_path=None, post_tfm_kwargs: dict | None = None, num_workers=2)[source]¶

Bases: object

Class that represents [vak.eval] table in configuration file.

checkpoint_path¶

path to directory with checkpoint files saved by Torch, to reload model

Type:: str

output_dir¶

Path to location where .csv files with evaluation metrics should be saved.

Type:: str

model¶

The model to use: its name, and the parameters to configure it. Must be an instance of vak.config.ModelConfig

Type:: vak.config.ModelConfig

batch_size¶

number of samples per batch presented to models during training.

Type:: int

dataset¶

The dataset to use: the path to it, and optionally a path to a file representing splits, and the name, if it is a built-in dataset. Must be an instance of vak.config.DatasetConfig.

Type:: vak.config.DatasetConfig

trainer¶

Configuration for lightning.pytorch.Trainer. Must be an instance of vak.config.TrainerConfig.

Type:: vak.config.TrainerConfig

num_workers¶

Number of processes to use for parallel loading of data. Argument to torch.DataLoader. Default is 2.

Type:: int

labelmap_path¶

path to ‘labelmap.json’ file.

Type:: str

frames_standardizer_path¶

path to a saved vak.transforms.FramesStandardizer object used to standardize (normalize) frames. If spectrograms were normalized and this is not provided, will give incorrect results.

Type:: str

post_tfm_kwargs¶

Keyword arguments to post-processing transform. If None, then no additional clean-up is applied when transforming labeled timebins to segments, the default behavior. The transform used is vak.transforms.frame_labels.PostProcess`. Valid keyword argument names are 'majority_vote' and 'min_segment_dur', and should be appropriate values for those arguments: Boolean for ``majority_vote, a float value for min_segment_dur. See the docstring of the transform for more details on these arguments and how they work.

Type:: dict

__init__(checkpoint_path, output_dir, model, batch_size, dataset: DatasetConfig, trainer: TrainerConfig, labelmap_path=None, frames_standardizer_path=None, post_tfm_kwargs: dict | None = None, num_workers=2) → None¶: Method generated by attrs for class EvalConfig.

Methods

`__init__`(checkpoint_path, output_dir, model, ...)	Method generated by attrs for class EvalConfig.
`from_config_dict`(config_dict)	Return `EvalConfig` instance from a `dict`.

Attributes

`checkpoint_path`
`output_dir`
`model`
`batch_size`
`dataset`
`trainer`
`labelmap_path`
`frames_standardizer_path`
`post_tfm_kwargs`
`num_workers`

classmethod from_config_dict(config_dict: dict) → EvalConfig[source]¶

Return EvalConfig instance from a dict.

The dict passed in should be the one found by loading a valid configuration toml file with vak.config.parse.from_toml_path(), and then using key eval, i.e., EvalConfig.from_config_dict(config_dict['eval']).