vak.eval.parametric_umap.eval_parametric_umap_model#

vak.eval.parametric_umap.eval_parametric_umap_model(model_name: str, model_config: dict, dataset_path: str | Path, checkpoint_path: str | Path, output_dir: str | Path, batch_size: int, num_workers: int, transform_params: dict | None = None, dataset_params: dict | None = None, split: str = 'test', device: str | None = None) → None[source]#

Evaluate a trained model.

Parameters:

  • model_name (str) – Model name, must be one of vak.models.registry.MODEL_NAMES.

  • model_config (dict) – Model configuration in a dict, as loaded from a .toml file, and used by the model method from_config.

  • dataset_path (str, pathlib.Path) – Path to dataset, e.g., a csv file generated by running vak prep.

  • checkpoint_path (str, pathlib.Path) – Path to directory with checkpoint files saved by Torch, to reload model

  • output_dir (str, pathlib.Path) – Path to location where .csv files with evaluation metrics should be saved.

  • batch_size (int) – Number of samples per batch fed into model.

  • num_workers (int) – Number of processes to use for parallel loading of data. Argument to torch.DataLoader. Default is 2.

  • transform_params (dict, optional) – Parameters for data transform. Passed as keyword arguments. Optional, default is None.

  • dataset_params (dict, optional) – Parameters for dataset. Passed as keyword arguments. Optional, default is None.

  • split (str) – Split of dataset on which model should be evaluated. One of {‘train’, ‘val’, ‘test’}. Default is ‘test’.

  • device (str) – Device on which to work with model + data. Defaults to ‘cuda’ if torch.cuda.is_available is True.