Convert raw WISPR files

Guide for converting raw WISPR .dat files

agate includes utilities for converting raw acoustic files (.dat) recorded with the WISPR acoustic systems into a more easily readable .wav or .flac format. An example workflow workflow_convertWISPR.m is provided in the agate\example_workflows folder and detailed instructions are below.

WISPR2 gain bug

Following a Seaglider mission in Spring 2023, a bug in the WISPR2 firmware was discovered that led to variable gain over the duration of the deployment. We developed a systematic way to ‘fix’ this gain in an attempt to standardize it across all files. In the name of posterity, that process is documented on the WISPR gain fix page.

_{Back to top}

Convert WISPR raw files

Example workflow

See workflow_convertWISPR.m in the example_workflows folder for the basic steps needed to convert a directory (that can include subdirectories) of Seaglider-collected WISPR .dat files into either .flac or .wav files.

Oceanscout or Slocum gliders

Use the workflow_convertWISPR_hefring.m script for a simplified example to convert WISPR data collected by a Hefring Oceanscout or TWR Slocum glider. This workflow uses a blank or very basic configuration file and ignores some Seaglider specific settings.

Like other example workflows, agate must first be initialized with a mission-level configuration file (e.g., agate_config_example.cnf). For the task of converting WISPR files, this can be empty (use agate_config_empty.cnf). But, optionally the CONFIG.ws.inDir and CONFIG.ws.outDir settings in the OPTIONAL - acoustics sections can be set to streamline processing and avoid manual selection of the input and output folders.

Currently, the input directory must either be a directory named with the date or a directory full of subdirectories where each subdirectory is named with the date using the format ‘YYMMDD’, (e.g., 230504 for 4 May 2023).

% add agate to the path
addpath(genpath('C:\Users\User.Name\Documents\MATLAB\agate'))

% initialize agate
% use WISPR Settings section to set in and out paths
CONFIG = agate('agate_mission_config.cnf');

Use the convertWispr function to run the conversion. The output file type is set with the 'outExt' argument; it can be either '.flac' or '.wav'. Set the 'showProgress' argument to true to print each file name to the Command Window as it is processed. This information is automatically stored in a log file called conversionLog.txt that is saved in the output folder. The log tracks the processing start and end time and will record any files that have to be skipped due to being invalid/empty.

% process all the files!
convertWispr(CONFIG, 'showProgress', true, 'outExt', '.flac');

If the process is interrupted at any point, it is possible to restart at a specified subdirectory. WISPR typically saves raw .dat files in folders by date, named with date in the format YYMMDD, so enter the name of the dated subdirectory as a six digit string to restart there. It will start at the beginning of the directory so any already processed files in that folder will just be reprocessed. Note that the fileheaders.txt file will just append after the restart so some file headers will be repeated. It is possible to just manually delete the to-be-reprocessed files from fileheaders.txt if you don’t want any duplicates.

% restart the process at folder 240919
convertWispr(CONFIG, 'showProgress', true, 'outExt', '.flac', ...
    'restartDir', '240919');

_{Back to top}

Dependency references

Embedded Ocean Systems, the WISPR developer, has MATLAB and Python tools to work with raw WISPR data available on GitHub: wispr3. An agate user does not need to download or clone this repository but the link is provided here for reference.

Some functions from wispr3 have been modified to better interface with agate; the modified functions are included within agate and more detailed info on the modifications can be found in a forked version: sfregosi-noaa/wispr3 sf branch.