% add agate to the path
addpath(genpath('C:\Users\User.Name\Documents\MATLAB\agate'))
% initialize agate
% make sure configuration file now has updated WISPR Settings section
% (not required during mission so may not be set yet)
CONFIG = agate('agate_mission_config.cnf');Convert acoustic files
Guide for converting raw acoustic .dat files to .wav
The Seaglider can be equipped with a variety of acoustic systems. Two - the PMARXL and WISPR systems - both record raw data as .dat files that need to be converted to .wav or .flac for analysis. agate includes conversion utilities for both systems. Example workflows for each system are provided in the agate\example_workflows folder and detailed instructions are below.
Following a mission in Spring 2023, a bug in the WISPR2 firmware was discovered that led to variable gain over the duration of the deployment. We developed a systematic way to ‘fix’ this gain in an attempt to standardize it across all files. In the name of posterity, that process is documented on the WISPR gain fix page.
Convert WISPR raw files
Dependency references
Embedded Ocean Systems, the WISPR developer, has MATLAB and Python tools to work with raw WISPR data. These tools can be found on GitHub - wispr3.
Some pieces of the MATLAB tools have been modified to better interface with agate. Those modified functions are included within agate but more detailed info on the modifications can be found in a forked version: sfregosi-noaa/wispr3 sf branch.
An agate user does not need to download or clone these repositories for the below conversion to work, but these links are provided for reference.
Example workflow
See workflow_convertWISPR.m in the example_workflows folder for the basic lines needed to convert a directory (that can include subdirectories) of WISPR .dat files into either .flac or .wav files.
Like other example workflows, agate must first be initialized with a mission-level configuration file (e.g., agate_config_example.cnf). For the task of converting WISPR files, this can be empty. But, optionally the CONFIG.ws.inDir and CONFIG.ws.outDir settings in the OPTIONAL - acoustics sections can be set to streamline processing and avoid manual selection of the input and output folders.
The convertWispr function will actually run the conversion. The output file type is set with the 'outExt' argument; it can be either '.flac' or '.wav'. Setting the 'showProgress' argument to true will print each file name to the Command Window as it is processed. This information is automatically also stored in a log file called conversionLog.txt that is saved in the output folder. The log will track the start and end time and state if any files have to be skipped due to being invalid/empty.
% process all the files!
convertWispr(CONFIG, 'showProgress', true, 'outExt', '.flac');If the process is interrupted at any point, it is possible to restart at a specified subdirectory. WISPR typically saves raw .dat files in folders by date, named with date in the format YYMMDD, so enter the name of the dated subdirectory as a six digit string to restart there. It will start at the beginning of the directory so any already processed files in that folder will just be reprocessed. Note that the fileheaders.txt file will just append after the restart so some file headers will be repeated. It is possible to just manually delete the to-be-reprocessed files from fileheaders.txt if you don’t want any duplicates.
% restart the process at folder 240919
convertWispr(CONFIG, 'showProgress', true, 'outExt', '.flac', ...
'restartDir', '240919');Convert PMAR raw files
Dependency references
A function-ized and modified version of the convertPmar.m script and associated function pmarIn (written by Dave Mellinger and available in the Mathworks File Exchange: convertPmar) are included in agate.
Example workflow
The script and function have been modified and combined to work within the agate toolbox. The input CONFIG files are populated from a PMAR-specific configuration (.cnf) file. An example can be found in the agate/settings folder.
See workflow_convertPMAR.m in the example_workflows folder for a simple example workflow for converting a mission’s PMAR .dat files to .wav.
Like other workflows in agate it must first be initialized with a mission-level configuration file (e.g., agate_config_example.cnf). Only the top REQUIRED and CONFIG.pm settings in the OPTIONAL - acoustics sections of the configuration file are needed for file conversion.
Additionally, a PMAR conversion configuration file (e.g., pmarConvert_example.cnf) is needed. This contains the specific paths and is where you can set your desired conversion settings. Detailed descriptions of each parameter setting are in the example configuration file or can be found on the Configuration - PMAR conversion page.