% add agate to the path
addpath(genpath('C:\Users\User.Name\Documents\MATLAB\agate'))
% initialize agate
% use WISPR Settings section to set in and out paths
CONFIG = agate('agate_mission_config.cnf');Convert acoustic files
Guide for converting raw acoustic .dat files
agate includes utilities for converting raw acoustic files (.dat) recorded with either the PMARXL or WISPR acoustic systems into a more easily readable .wav or .flac format. Example workflows for each system are provided in the agate\example_workflows folder and detailed instructions are below.
WISPR2 gain bug
Following a Seaglider mission in Spring 2023, a bug in the WISPR2 firmware was discovered that led to variable gain over the duration of the deployment. We developed a systematic way to ‘fix’ this gain in an attempt to standardize it across all files. In the name of posterity, that process is documented on the WISPR gain fix page.
Convert WISPR raw files
Dependency references
Embedded Ocean Systems, the WISPR developer, has MATLAB and Python tools to work with raw WISPR data available on GitHub: wispr3. An agate user does not need to download or clone this repository but the link is provided here for reference.
Some functions from wispr3 have been modified to better interface with agate; the modified functions are included within agate and more detailed info on the modifications can be found in a forked version: sfregosi-noaa/wispr3 sf branch.
Example workflow
See workflow_convertWISPR.m in the example_workflows folder for the basic steps needed to convert a directory (that can include subdirectories) of Seaglider-collected WISPR .dat files into either .flac or .wav files.
Oceanscout or Slocum gliders
Use the workflow_convertWISPR_hefring.m script for a simplified example to convert WISPR data collected by a Hefring Oceanscout or TWR Slocum glider. This workflow uses a blank or very basic configuration file and ignores some Seaglider specific settings.
Like other example workflows, agate must first be initialized with a mission-level configuration file (e.g., agate_config_example.cnf). For the task of converting WISPR files, this can be empty (use agate_config_empty.cnf). But, optionally the CONFIG.ws.inDir and CONFIG.ws.outDir settings in the OPTIONAL - acoustics sections can be set to streamline processing and avoid manual selection of the input and output folders.
Currently, the input directory must either be a directory named with the date or a directory full of subdirectories where each subdirectory is named with the date using the format ‘YYMMDD’, (e.g., 230504 for 4 May 2023).
Use the convertWispr function to run the conversion. The output file type is set with the 'outExt' argument; it can be either '.flac' or '.wav'. Set the 'showProgress' argument to true to print each file name to the Command Window as it is processed. This information is automatically stored in a log file called conversionLog.txt that is saved in the output folder. The log tracks the processing start and end time and will record any files that have to be skipped due to being invalid/empty.
% process all the files!
convertWispr(CONFIG, 'showProgress', true, 'outExt', '.flac');If the process is interrupted at any point, it is possible to restart at a specified subdirectory. WISPR typically saves raw .dat files in folders by date, named with date in the format YYMMDD, so enter the name of the dated subdirectory as a six digit string to restart there. It will start at the beginning of the directory so any already processed files in that folder will just be reprocessed. Note that the fileheaders.txt file will just append after the restart so some file headers will be repeated. It is possible to just manually delete the to-be-reprocessed files from fileheaders.txt if you don’t want any duplicates.
% restart the process at folder 240919
convertWispr(CONFIG, 'showProgress', true, 'outExt', '.flac', ...
'restartDir', '240919');Convert PMAR raw files
Dependency references
A function-ized and modified version of the convertPmar.m script and associated function pmarIn (written by Dave Mellinger and available in the Mathworks File Exchange: convertPmar) are included in agate.
Example workflow
The script and function have been modified and combined to work within the agate toolbox. The input CONFIG files are populated from a PMAR-specific configuration (.cnf) file. An example can be found in the agate/settings folder.
See workflow_convertPMAR.m in the example_workflows folder for a simple example workflow for converting a mission’s PMAR .dat files to .wav.
Like other workflows in agate it must first be initialized with a mission-level configuration file (e.g., agate_config_example.cnf). Only the top REQUIRED and CONFIG.pm settings in the OPTIONAL - acoustics sections of the configuration file are needed for file conversion.
Additionally, a PMAR conversion configuration file (e.g., pmarConvert_example.cnf) is needed. This contains the specific paths and is where you can set your desired conversion settings. Detailed descriptions of each parameter setting are in the example configuration file or can be found on the Configuration - PMAR conversion page.
WAV to FLAC and FLAC to WAV conversion
The tools within agate allow conversion from raw .dat files directly to either .wav. or .flac. Some users may later want to convert between .wav. and .flac. Rather than re-converting the raw .dat files, it is much faster to use flac.exe to encode/decode to/from .flac. There are several ways to do this (free GUIs, command line, MATLAB audioread/audiowrite).
One simple and efficient way is to use a MATLAB wrapper-type function available from github.com/sfregosi/myUtils. These two functions (wav2flac and flac2wav) call flac.exe via command line but have the added functionality to operate over a directory of files, check for proper paths, and track progress.
After downloading wav2flac.m and flac2wav.m from myUtils (in the main folder, or can clone the whole repository), add them to the MATLAB path and follow the below example code:
% set path to flac software, input directory of WAV files, output directory for FLAC files
path_flac = 'C:\Users\User.Name\programs\flac-1.5.0-win\Win64\flac';
% to convert from WAV to FLAC
inDir = 'F:\wavFiles';
outDir = 'F:\flacFiles\'; % important it ends in slash! Function has built in check for this
wav2flac(path_flac, inDir, outDir)
% to convert from FLAC to WAV
inDir = 'F:\flacFiles';
outDir = 'F:\wavFiles\'; % important it ends in slash! Function has built in check for this
flac2wav(path_flac, inDir, outDir)