vak.prep.frame_classification.source_files.get_or_make_source_files#
- vak.prep.frame_classification.source_files.get_or_make_source_files(data_dir: str | Path, input_type: str, audio_format: str | None = None, spect_format: str | None = None, spect_params: dict | None = None, spect_output_dir: str | Path | None = None, annot_format: str | None = None, annot_file: str | Path | None = None, labelset: set | None = None, audio_dask_bag_kwargs: dict | None = None) DataFrame [source]#
Get source files for a dataset, or make them.
Gets either audio or spectrogram files from
data dir
, possibly paired with annotation files.If
input_type
is'audio'
, then this function will look for files with the extension foraudio_format
indata_dir
. Ifinput_type
is'spectrogram'
, andspect_format
is specified, then this function will look for files with the extension for that format indata_dir
. Ifinput_type
is spectrogram, andaudio_format
is specified, this function will look for audio files with that extension and then generate spectrograms for them usingspect_params
. If anannot_format
is specified, this function will additionally look for annotation files for the audio or spectrogram files. If all annotations are in a single file, this can be specified with theannot_file
parameter, and that will be used instead of looking for other annotation files.- Parameters:
data_dir (str, Path) β Path to directory with files from which to make dataset.
input_type (str) β The type of input to the neural network model. One of {βaudioβ, βspectβ}.
audio_format (str) β Format of audio files. One of {βwavβ, βcbinβ}. Default is
None
, but eitheraudio_format
orspect_format
must be specified.spect_format (str) β Format of files containing spectrograms as 2-d matrices. One of {βmatβ, βnpzβ}. Default is None, but either audio_format or spect_format must be specified.
spect_params (dict, vak.config.SpectParams) β Parameters for creating spectrograms. Default is
None
.spect_output_dir (str) β Path to location where spectrogram files should be saved. Default is None. If
input_type
is'spect'
, thenspect_output_dir
defaults todata_dir
.annot_format (str) β Format of annotations. Any format that can be used with the :module:`crowsetta` library is valid. Default is
None
.annot_file (str) β Path to a single annotation file. Default is
None
. Used when a single file contains annotates multiple audio or spectrogram files.audio_dask_bag_kwargs (dict) β Keyword arguments used when calling
dask.bag.from_sequence()
insidevak.io.audio()
, where it is used to parallelize the conversion of audio files into spectrograms. Option should be specified in config.toml file as an inline table, e.g.,audio_dask_bag_kwargs = { npartitions = 20 }
. Allows for finer-grained control when needed to process files of different sizes.labelset (str, list, set) β Set of unique labels for vocalizations. Strings or integers. Default is
None
. If notNone
, then files will be skipped where the associated annotation contains labels not found inlabelset
.labelset
is converted to a Pythonset
usingvak.converters.labelset_to_set()
. See help for that function for details on how to specifylabelset
.
- Returns:
source_files_df β Source files that will become the dataset, represented as a pandas.DataFrame. Each row corresponds to one sample in the dataset, either an audio file or spectrogram file, possibly paired with annotations.
- Return type: