Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
vak documentation
Logo
  • Getting Started
    • Installation
    • Automated Annotation
  • How-to guides
    • How do I prepare datasets of annotated vocalizations for use with vak?
    • How do I use my own vocal annotation format?
    • How do I use my own spectrograms?
  • Frequently Asked Questions (FAQs)
  • Reference
    • vak command-line interface
    • Configuration files
    • File naming conventions
    • Spectrogram file format
  • Development
    • Contributors Guide
  • API Reference
    • vak.cli.cli
      • vak.cli.cli.cli
      • vak.cli.cli.eval
      • vak.cli.cli.learncurve
      • vak.cli.cli.predict
      • vak.cli.cli.prep
      • vak.cli.cli.train
    • vak.cli.eval
      • vak.cli.eval.eval
    • vak.cli.learncurve
      • vak.cli.learncurve.learning_curve
    • vak.cli.predict
      • vak.cli.predict.predict
    • vak.cli.prep
      • vak.cli.prep.prep
      • vak.cli.prep.purpose_from_toml
    • vak.cli.train
      • vak.cli.train.train
    • vak.prep
      • vak.prep.audio_dataset
        • vak.prep.audio_dataset.prep_audio_dataset
      • vak.prep.constants
      • vak.prep.dataset_df_helper
        • vak.prep.dataset_df_helper.add_split_col
        • vak.prep.dataset_df_helper.get_dataset_csv_filename
        • vak.prep.dataset_df_helper.get_dataset_csv_path
      • vak.prep.frame_classification
        • vak.prep.frame_classification.assign_samples_to_splits
        • vak.prep.frame_classification.frame_classification
          • vak.prep.frame_classification.frame_classification.prep_frame_classification_dataset
        • vak.prep.frame_classification.learncurve
          • vak.prep.frame_classification.learncurve.make_index_vectors_for_each_subset
          • vak.prep.frame_classification.learncurve.make_subsets_from_dataset_df
          • vak.prep.frame_classification.learncurve.Sample
        • vak.prep.frame_classification.make_splits
          • vak.prep.frame_classification.make_splits.argsort_by_label_freq
          • vak.prep.frame_classification.make_splits.make_splits
          • vak.prep.frame_classification.make_splits.Sample
        • vak.prep.frame_classification.source_files
          • vak.prep.frame_classification.source_files.get_or_make_source_files
        • vak.prep.frame_classification.validators
          • vak.prep.frame_classification.validators.validate_and_get_frame_dur
      • vak.prep.parametric_umap
        • vak.prep.parametric_umap.dataset_arrays
          • vak.prep.parametric_umap.dataset_arrays.move_files_into_split_subdirs
        • vak.prep.parametric_umap.parametric_umap
          • vak.prep.parametric_umap.parametric_umap.prep_parametric_umap_dataset
      • vak.prep.prep_
        • vak.prep.prep_.prep
      • vak.prep.sequence_dataset
        • vak.prep.sequence_dataset.has_unlabeled_segments
        • vak.prep.sequence_dataset.where_unlabeled_segments
      • vak.prep.spectrogram_dataset
        • vak.prep.spectrogram_dataset.audio_helper
          • vak.prep.spectrogram_dataset.audio_helper.files_from_dir
          • vak.prep.spectrogram_dataset.audio_helper.make_spectrogram_files_from_audio_files
        • vak.prep.spectrogram_dataset.prep
          • vak.prep.spectrogram_dataset.prep.prep_spectrogram_dataset
        • vak.prep.spectrogram_dataset.spect
          • vak.prep.spectrogram_dataset.spect.butter_bandpass
          • vak.prep.spectrogram_dataset.spect.butter_bandpass_filter
          • vak.prep.spectrogram_dataset.spect.spectrogram
        • vak.prep.spectrogram_dataset.spect_helper
          • vak.prep.spectrogram_dataset.spect_helper.make_dataframe_of_spect_files
      • vak.prep.split
        • vak.prep.split.algorithms
          • vak.prep.split.algorithms.bruteforce
            • vak.prep.split.algorithms.bruteforce.brute_force
            • vak.prep.split.algorithms.bruteforce.unique_set_from_labels
            • vak.prep.split.algorithms.bruteforce.validate_labels
          • vak.prep.split.algorithms.validate
            • vak.prep.split.algorithms.validate.validate_split_durations
        • vak.prep.split.split
          • vak.prep.split.split.frame_classification_dataframe
          • vak.prep.split.split.train_test_dur_split_inds
          • vak.prep.split.split.unit_dataframe
      • vak.prep.unit_dataset
        • vak.prep.unit_dataset.unit_dataset
          • vak.prep.unit_dataset.unit_dataset.abspath
          • vak.prep.unit_dataset.unit_dataset.prep_unit_dataset
          • vak.prep.unit_dataset.unit_dataset.save_spect
          • vak.prep.unit_dataset.unit_dataset.spectrogram_from_segment
          • vak.prep.unit_dataset.unit_dataset.Segment
          • vak.prep.unit_dataset.unit_dataset.SpectToSave
    • vak.prep.frame_classification
      • vak.prep.frame_classification.assign_samples_to_splits
      • vak.prep.frame_classification.frame_classification
        • vak.prep.frame_classification.frame_classification.prep_frame_classification_dataset
      • vak.prep.frame_classification.learncurve
        • vak.prep.frame_classification.learncurve.make_index_vectors_for_each_subset
        • vak.prep.frame_classification.learncurve.make_subsets_from_dataset_df
        • vak.prep.frame_classification.learncurve.Sample
      • vak.prep.frame_classification.make_splits
        • vak.prep.frame_classification.make_splits.argsort_by_label_freq
        • vak.prep.frame_classification.make_splits.make_splits
        • vak.prep.frame_classification.make_splits.Sample
      • vak.prep.frame_classification.source_files
        • vak.prep.frame_classification.source_files.get_or_make_source_files
      • vak.prep.frame_classification.validators
        • vak.prep.frame_classification.validators.validate_and_get_frame_dur
    • vak.prep.parametric_umap
      • vak.prep.parametric_umap.dataset_arrays
        • vak.prep.parametric_umap.dataset_arrays.move_files_into_split_subdirs
      • vak.prep.parametric_umap.parametric_umap
        • vak.prep.parametric_umap.parametric_umap.prep_parametric_umap_dataset
    • vak.prep.spectrogram_dataset
      • vak.prep.spectrogram_dataset.audio_helper
        • vak.prep.spectrogram_dataset.audio_helper.files_from_dir
        • vak.prep.spectrogram_dataset.audio_helper.make_spectrogram_files_from_audio_files
      • vak.prep.spectrogram_dataset.prep
        • vak.prep.spectrogram_dataset.prep.prep_spectrogram_dataset
      • vak.prep.spectrogram_dataset.spect
        • vak.prep.spectrogram_dataset.spect.butter_bandpass
        • vak.prep.spectrogram_dataset.spect.butter_bandpass_filter
        • vak.prep.spectrogram_dataset.spect.spectrogram
      • vak.prep.spectrogram_dataset.spect_helper
        • vak.prep.spectrogram_dataset.spect_helper.make_dataframe_of_spect_files
    • vak.prep.unit_dataset
      • vak.prep.unit_dataset.unit_dataset
        • vak.prep.unit_dataset.unit_dataset.abspath
        • vak.prep.unit_dataset.unit_dataset.prep_unit_dataset
        • vak.prep.unit_dataset.unit_dataset.save_spect
        • vak.prep.unit_dataset.unit_dataset.spectrogram_from_segment
        • vak.prep.unit_dataset.unit_dataset.Segment
        • vak.prep.unit_dataset.unit_dataset.SpectToSave
    • vak.prep.audio_dataset
      • vak.prep.audio_dataset.prep_audio_dataset
    • vak.prep.constants
    • vak.prep.dataset_df_helper
      • vak.prep.dataset_df_helper.add_split_col
      • vak.prep.dataset_df_helper.get_dataset_csv_filename
      • vak.prep.dataset_df_helper.get_dataset_csv_path
    • vak.prep.prep_
      • vak.prep.prep_.prep
    • vak.prep.sequence_dataset
      • vak.prep.sequence_dataset.has_unlabeled_segments
      • vak.prep.sequence_dataset.where_unlabeled_segments
    • vak.prep.split.algorithms.bruteforce
      • vak.prep.split.algorithms.bruteforce.brute_force
      • vak.prep.split.algorithms.bruteforce.unique_set_from_labels
      • vak.prep.split.algorithms.bruteforce.validate_labels
    • vak.prep.split.algorithms.validate
      • vak.prep.split.algorithms.validate.validate_split_durations
    • vak.prep.split.split
      • vak.prep.split.split.frame_classification_dataframe
      • vak.prep.split.split.train_test_dur_split_inds
      • vak.prep.split.split.unit_dataframe
    • vak.train.train_
      • vak.train.train_.train
    • vak.train.frame_classification
      • vak.train.frame_classification.get_split_dur
      • vak.train.frame_classification.get_train_callbacks
      • vak.train.frame_classification.get_trainer
      • vak.train.frame_classification.train_frame_classification_model
    • vak.train.parametric_umap
      • vak.train.parametric_umap.get_split_dur
      • vak.train.parametric_umap.get_trainer
      • vak.train.parametric_umap.train_parametric_umap_model
    • vak.eval.eval_
      • vak.eval.eval_.eval
    • vak.eval.frame_classification
      • vak.eval.frame_classification.eval_frame_classification_model
    • vak.eval.parametric_umap
      • vak.eval.parametric_umap.eval_parametric_umap_model
    • vak.predict.predict_
      • vak.predict.predict_.predict
    • vak.predict.frame_classification
      • vak.predict.frame_classification.predict_with_frame_classification_model
      • vak.predict.frame_classification.AnnotationDataFrame
    • vak.predict.parametric_umap
      • vak.predict.parametric_umap.predict_with_parametric_umap_model
    • vak.learncurve.curvefit
      • vak.learncurve.curvefit.fit_learning_curve
      • vak.learncurve.curvefit.residual_two_functions
    • vak.learncurve.dirname
      • vak.learncurve.dirname.replicate_dirname
      • vak.learncurve.dirname.train_dur_dirname
    • vak.learncurve.frame_classification
      • vak.learncurve.frame_classification.learning_curve_for_frame_classification_model
    • vak.learncurve.learncurve
      • vak.learncurve.learncurve.learning_curve
    • vak.config.config
      • vak.config.config.Config
    • vak.config.dataset
      • vak.config.dataset.DatasetConfig
    • vak.config.eval
      • vak.config.eval.are_valid_post_tfm_kwargs
      • vak.config.eval.convert_post_tfm_kwargs
      • vak.config.eval.EvalConfig
    • vak.config.learncurve
      • vak.config.learncurve.LearncurveConfig
    • vak.config.load
    • vak.config.model
      • vak.config.model.ModelConfig
    • vak.config.predict
      • vak.config.predict.PredictConfig
    • vak.config.prep
      • vak.config.prep.are_valid_dask_bag_kwargs
      • vak.config.prep.duration_from_toml_value
      • vak.config.prep.is_valid_duration
      • vak.config.prep.PrepConfig
    • vak.config.spect_params
      • vak.config.spect_params.freq_cutoffs_validator
      • vak.config.spect_params.is_valid_transform_type
      • vak.config.spect_params.SpectParamsConfig
    • vak.config.train
      • vak.config.train.TrainConfig
    • vak.config.trainer
      • vak.config.trainer.is_valid_accelerator
      • vak.config.trainer.is_valid_devices
      • vak.config.trainer.TrainerConfig
    • vak.config.validators
      • vak.config.validators.are_keys_valid
      • vak.config.validators.are_table_keys_valid
      • vak.config.validators.are_tables_valid
      • vak.config.validators.is_a_directory
      • vak.config.validators.is_a_file
      • vak.config.validators.is_annot_format
      • vak.config.validators.is_audio_format
      • vak.config.validators.is_spect_format
      • vak.config.validators.is_valid_model_name
    • vak.datapipes.frame_classification
      • vak.datapipes.frame_classification.constants
      • vak.datapipes.frame_classification.helper
        • vak.datapipes.frame_classification.helper.inds_in_sample_array_filename_for_subset
        • vak.datapipes.frame_classification.helper.load_frames
        • vak.datapipes.frame_classification.helper.sample_ids_array_filename_for_subset
      • vak.datapipes.frame_classification.infer_datapipe
        • vak.datapipes.frame_classification.infer_datapipe.InferDatapipe
      • vak.datapipes.frame_classification.metadata
        • vak.datapipes.frame_classification.metadata.is_valid_audio_format
        • vak.datapipes.frame_classification.metadata.is_valid_dataset_csv_filename
        • vak.datapipes.frame_classification.metadata.is_valid_spect_format
        • vak.datapipes.frame_classification.metadata.Metadata
      • vak.datapipes.frame_classification.train_datapipe
        • vak.datapipes.frame_classification.train_datapipe.get_window_inds
        • vak.datapipes.frame_classification.train_datapipe.TrainDatapipe
    • vak.datapipes.parametric_umap
      • vak.datapipes.parametric_umap.metadata
        • vak.datapipes.parametric_umap.metadata.is_valid_audio_format
        • vak.datapipes.parametric_umap.metadata.is_valid_dataset_csv_filename
        • vak.datapipes.parametric_umap.metadata.is_valid_spect_format
        • vak.datapipes.parametric_umap.metadata.Metadata
      • vak.datapipes.parametric_umap.parametric_umap
        • vak.datapipes.parametric_umap.parametric_umap.get_graph_elements
        • vak.datapipes.parametric_umap.parametric_umap.get_umap_graph
        • vak.datapipes.parametric_umap.parametric_umap.Datapipe
    • vak.metrics.classification.classification
      • vak.metrics.classification.classification.Accuracy
    • vak.metrics.classification.functional
      • vak.metrics.classification.functional.accuracy
    • vak.metrics.distance.distance
      • vak.metrics.distance.distance.CharacterErrorRate
      • vak.metrics.distance.distance.Levenshtein
    • vak.metrics.distance.functional
      • vak.metrics.distance.functional.character_error_rate
      • vak.metrics.distance.functional.levenshtein
    • vak.models.convencoder_umap
    • vak.models.decorator
      • vak.models.decorator.model
      • vak.models.decorator.ModelDefinitionValidationError
    • vak.models.definition
      • vak.models.definition.validate
      • vak.models.definition.ModelDefinition
    • vak.models.ed_tcn
    • vak.models.frame_classification_model
      • vak.models.frame_classification_model.FrameClassificationModel
    • vak.models.get
    • vak.models.parametric_umap_model
      • vak.models.parametric_umap_model.ParametricUMAP
      • vak.models.parametric_umap_model.ParametricUMAPDatamodule
      • vak.models.parametric_umap_model.ParametricUMAPModel
    • vak.models.registry
      • vak.models.registry.model_family
      • vak.models.registry.register_model
    • vak.models.tweetynet
    • vak.nets.conv_encoder
      • vak.nets.conv_encoder.ConvEncoder
    • vak.nets.ed_tcn
      • vak.nets.ed_tcn.ED_TCN
    • vak.nets.tweetynet
      • vak.nets.tweetynet.TweetyNet
    • vak.nn.loss
      • vak.nn.loss.crossentropy
        • vak.nn.loss.crossentropy.CrossEntropyLoss
      • vak.nn.loss.dice
        • vak.nn.loss.dice.dice_loss
        • vak.nn.loss.dice.DiceLoss
      • vak.nn.loss.umap
        • vak.nn.loss.umap.compute_cross_entropy
        • vak.nn.loss.umap.convert_distance_to_probability
        • vak.nn.loss.umap.umap_loss
        • vak.nn.loss.umap.UmapLoss
    • vak.nn.functional
      • vak.nn.functional.one_hot
    • vak.plot.annot
      • vak.plot.annot.annotation
      • vak.plot.annot.plot_labels
      • vak.plot.annot.plot_segments
    • vak.plot.learncurve
      • vak.plot.learncurve.frame_error_rate
      • vak.plot.learncurve.get_all_results_list
      • vak.plot.learncurve.syllable_error_rate
    • vak.plot.spect
    • vak.transforms.defaults
      • vak.transforms.defaults.frame_classification
        • vak.transforms.defaults.frame_classification.InferItemTransform
        • vak.transforms.defaults.frame_classification.TrainItemTransform
    • vak.transforms.frame_labels.functional
      • vak.transforms.frame_labels.functional.boundary_inds_from_boundary_labels
      • vak.transforms.frame_labels.functional.from_segments
      • vak.transforms.frame_labels.functional.postprocess
      • vak.transforms.frame_labels.functional.remove_short_segments
      • vak.transforms.frame_labels.functional.segment_inds_list_from_boundary_labels
      • vak.transforms.frame_labels.functional.segment_inds_list_from_class_labels
      • vak.transforms.frame_labels.functional.take_majority_vote
      • vak.transforms.frame_labels.functional.to_labels
      • vak.transforms.frame_labels.functional.to_segments
    • vak.transforms.frame_labels.transforms
      • vak.transforms.frame_labels.transforms.FromSegments
      • vak.transforms.frame_labels.transforms.PostProcess
      • vak.transforms.frame_labels.transforms.ToLabels
      • vak.transforms.frame_labels.transforms.ToSegments
    • vak.transforms.functional
      • vak.transforms.functional.add_channel
      • vak.transforms.functional.pad_to_window
      • vak.transforms.functional.standardize_spect
      • vak.transforms.functional.to_floattensor
      • vak.transforms.functional.to_longtensor
      • vak.transforms.functional.view_as_window_batch
    • vak.transforms.transforms
      • vak.transforms.transforms.AddChannel
      • vak.transforms.transforms.FramesStandardizer
      • vak.transforms.transforms.PadToWindow
      • vak.transforms.transforms.ToFloatTensor
      • vak.transforms.transforms.ToLongTensor
      • vak.transforms.transforms.ViewAsWindowBatch
    • vak.common.accelerator
      • vak.common.accelerator.get_default
    • vak.common.annotation
      • vak.common.annotation.audio_filename_from_path
      • vak.common.annotation.files_from_dir
      • vak.common.annotation.format_from_df
      • vak.common.annotation.from_df
      • vak.common.annotation.has_unlabeled
      • vak.common.annotation.map_annotated_to_annot
      • vak.common.annotation.AudioFilenameNotFoundError
      • vak.common.annotation.MapUsingExtensionError
      • vak.common.annotation.MapUsingNotatedPathError
    • vak.common.constants
    • vak.common.converters
      • vak.common.converters.bool_from_str
      • vak.common.converters.expanded_user_path
      • vak.common.converters.labelset_to_set
      • vak.common.converters.range_str
    • vak.common.files
      • vak.common.files.files
        • vak.common.files.files.find_fname
        • vak.common.files.files.from_dir
      • vak.common.files.spect
        • vak.common.files.spect.find_audio_fname
        • vak.common.files.spect.is_valid_set_of_spect_files
        • vak.common.files.spect.load
        • vak.common.files.spect.timebin_dur
    • vak.common.labels
      • vak.common.labels.from_df
      • vak.common.labels.multi_char_labels_to_single_char
      • vak.common.labels.to_map
      • vak.common.labels.to_set
    • vak.common.learncurve
      • vak.common.learncurve.get_train_dur_replicate_subset_name
    • vak.common.logging
      • vak.common.logging.config_logging_for_cli
      • vak.common.logging.log_version
    • vak.common.paths
      • vak.common.paths.generate_results_dir_name_as_path
    • vak.common.tensorboard
      • vak.common.tensorboard.events2df
      • vak.common.tensorboard.get_summary_writer
    • vak.common.timebins
      • vak.common.timebins.timebin_dur_from_vec
    • vak.common.timenow
      • vak.common.timenow.get_timenow_as_str
    • vak.common.typing
    • vak.common.validators
      • vak.common.validators.column_or_1d
      • vak.common.validators.is_a_directory
      • vak.common.validators.is_a_file
      • vak.common.validators.row_or_1d
  • About vak
  • Poems
    • The Bird May Die
    • What Does the Bird Care?
    • Unknown Bird
Back to top
View this page

vak.cli.cli.eval¶

vak.cli.cli.eval(toml_path)[source]¶
Next
vak.cli.cli.learncurve
Previous
vak.cli.cli.cli
Copyright © 2017-present, David Nicholson, Yarden Cohen
Made with Sphinx and @pradyunsg's Furo
On this page
  • vak.cli.cli.eval
    • eval()