Processing and QAQC

Recommended QAQC Tests

Flag meanings are based on the QARTOD flag meanings. Descriptions of each flag can be found in the acspype flag documentation. Tests are considered QARTOD tests if they were coded in a way that would satisfy the requirements for the generic QARTOD tests found in each manual (e.g. gap test, syntax test, gross range test, etc.). Test that are not QARTOD tests are custom tests developed from information in the ACS manual or are modified forms of a related QARTOD test.

Note: If using acspype and a flag of 2 (NOT_EVALUATED) is assigned in these tests, that indicates there was a programmatic failure in the test and an issue should be raised on GitHub. A flag of 9 (MISSING_DATA) indicates that the input data or ancillary data was missing.

Test Name

Test Description

Recommended Settings

Is QARTOD?

Possible Results

Notes

Gap Test

The first stage is to compare the data acquisition time to the host computer clock time. The second stage is to compare the known record length against the number of bytes in the serial buffer.

Time Increment = 0.25 (s)

Yes

4, 1

This test is generally not performed on archived datasets. If using archived data, users should reach out to the data provider to verify the timestamps or data acquisition protocols.

Syntax Test

The first stage checks if the registration bytes occur at the beginning of the packet. The second stage checks if the packet contains a pad byte at the end. The third stage compares the length of the packet against the size of the packet descriptor. The fourth stage compares the record length against the length of the packet. The fifth stage verifies the checksum.

Not Applicable

Yes

4, 1

This test is generally not performed on archived datasets, although it can be recreated when processing ACS packets that have been dumped into binary files.

Elapsed Time Test

Check the elapsed time in each ACS sample to see if it is collected at a reasonable time since the instrument received power. The manufacturer states that the ACS takes up to 10 minutes to warm-up and that data may be questionable during this time. On moorings where power is a commodity, this is may not be possible.

Fail Threshold = 45000 (ms)
Suspect Threshold = 240000 (ms)

No (Minimum Side Gross Range Test)

4, 3, 1

The fail and suspect values used in this test depend on the ACS datasets. For moored or profiler time-series, a lower fail/suspect threshold may be best, otherwise significant quantities of the data may be flagged. Users should understand that data collected at the beginning of a power start up sequence may be excessively noisy or contain inaccurate data. This data should be carefully reviewed.

Internal Temperature Test

Check the internal temperature of the ACS to see if it is within the range specified in the device file. Values outside the device calibration range are flagged as suspect.

Defined in device file and automatically applied.

No (Gross Range Test)

3, 1

Data outside the calibrated temperature bins may be suspect simply because it exists outside of the calibrated range. A typical calibration range is 2-35 degC. If experiencing waters outside that range, users should request a larger calibration range from SBS.

Inf Nan Test

Check the uncorrected values for Inf and NaN values. Inf and Nan values can appear in uncorrected measurements for a number of reasons. A common reason is that the one or multiple reference counts contain the value of 0, which propagate to NaN when performing a log operation and remain at each processing step.

Not Applicable

No (Custom test)

4, 1

If a NaN exists in the spectrum, it can probably be discarded. Users should review the surrounding spectrum and the same wavelength bin to determine if the issue is a one-off. Repeated NaNs (or zero reference counts) in uncorrected values may indicate that factory recalibration is needed or an instrument malfunction.

Gross Range Test

On TS and scattering corrected spectrum, run the gross range test, which will assign a flag for each wavelength in a spectra.

Fail Threshold = [0, 10]
Suspect Threshold = [0.001,8.5]

Yes

4, 3, 1

Some may find that this results in fail flags in the red wavelengths, even after correction. This

Blanket Gross Range Test

Assigns a blanket flag to the entire spectrum if a certain percentage of coefficients across the spectrum are flagged as fail. This is useful for identifying spectra that are not suitable for further processing.

Ignore Wavelengths = [700, 755]
Fail Threshold = 10
Suspect Threshold = 5

No (Modified Gross Range Test)

4, 3, 1

This may be useful for identifying poor quality spectra. Users should review the neighboring spectrum samples to see if the issue is persistent or a one-off. If only one spectrum is found as failed while the neighboring spectrum are ok, then it could probably be removed before analysis is performed without a significant impact on the end result.

A Greater Than C Test

Check if scattering corrected absorption is greater than ts-corrected attenuation. Consecutive values in a spectrum and over time may indicate bubbles or a stuck object in the flow cell.

Not Applicable

No (Modified Minimum Side Gross Range Test)

3, 1

This test should be used to consider data suspect, but should not be used to immediately remove data.

Rolling Variance Test

Assigns a flag at each wavelength bin of spectrum if the variance over time exceeds a percentage of the mean. A rolling centered window is used to calculate the variance and a flag is assigned to the central sample.

Percentage = 25
Window Size = 240

No (Custom Test)

3, 1

Suspect flags may indicate bubbles or debris in the flow cell. NOT CURRENTLY IMPLEMENTED.

Discontinuity Offset Test

Check if the absolute value of the discontinuity offset exceeds a fail threshold or a multiple of the median for the time-series.

Fail Threshold = 10
Median Multiplier = 3

No (Custom Test)

4, 3, 1

Discontinuity offsets that are larger than the fail threshold are flagged as fail. The maximum sensing range of the ACS is used as the default for the fail threshold. In general, discontinuity offsets should be small and within the typical precision of the sensor (+/- 0.003 m^-1). A second option for defining a median multiplier is used to flag data that exceed X times the median as suspect.

Reference Material

Manuals

The most up-to-date manuals and protocols for the ACS can be found on the Sea-Bird Scientific ACS Downloads Page.

Literature

Below is a small list of ACS, absorption, attenuation, or ACS data product related literature.

Author

Year

Title

Link

Zaneveld et al.

1994

Scattering error correction of reflecting-tube absorption meters

https://doi.org/10.1117/12.190095

Bricaud et al.

1995

Variability in the chlorophyll-specific absorption coefficients of natural phytoplankton: Analysis and parameterization

https://doi.org/10.1029/95JC00463

Davis et al.

1997

Reducing the effects of fouling on chlorophyll estimates derived from long-term deployments of optical instruments

https://doi.org/10.1029/96JC02430

Pegau et al.

1997

Absorption and attenuation of visible and near-infrared light in water: dependence on temperature and salinity

https://doi.org/10.1364/AO.36.006035

Pegau et al.

2003

Ocean Optics Protocols for Satellite Ocean Color Sensor Validation, Revision 4, Volume IV

https://ntrs.nasa.gov/api/citations/20030093642/downloads/20030093642.pdf

Behrenfeld and Boss

2006

Beam attenuation and chlorophyll concentration as alternative optical indices of phytoplankton biomass.

https://doi.org/10.1357/002224006778189563

Gardner et al.

2006

Global POC concentrations from in-situ and satellite data

https://doi.org/10.1016/j.dsr2.2006.01.029

Sullivan et al.

2006

Hyperspectral temperature and salt dependencies of absorption by water and heavy water in the 400–750 nm spectral range

https://doi.org/10.1364/AO.45.005294

Boss et al.

2007

Measurements of spectral optical properties and their relation to biogeochemical variables and processes in Crater Lake, Crater Lake National Park, OR

https://doi.org/10.1007/978-1-4020-5824-0_9

Stramski et al.

2008

Relationships between the surface concentration of particulate organic carbon and optical properties in the eastern South Pacific and eastern Atlantic Oceans

https://doi.org/10.5194/bg-5-171-2008

Slade et al.

2010

Underway and Moored Methods for Improving Accuracy in Measurement of Spectral Particulate Absorption and Attenuation

https://doi.org/10.1175/2010JTECHO755.1

Cetenic et al.

2012

Particulate organic carbon and inherent optical properties during 2008 North Atlantic Bloom Experiment

https://doi.org/10.1029/2011JC007771

Rottgers et al.

2013

Evaluation of scatter corrections for ac-9 absorption measurements in coastal waters

https://doi.org/10.1016/j.mio.2013.11.001

Roesler and Barnard

2013

Optical proxy for phytoplankton biomass in the absence of photophysiology: Rethinking the absorption line height

https://doi.org/10.1016/j.mio.2013.12.003

Boss et al.

2013

The characteristics of particulate absorption, scattering and attenuation coefficients in the surface ocean; Contribution of the Tara Oceans expedition

https://doi.org/10.1016/j.mio.2013.11.002

Stockley et al.

2017

Assessing uncertainties in scattering correction algorithms for reflective tube absorption measurements made with a WET Labs ac-9

https://doi.org/10.1364/OE.25.0A1139

Boss et al.

2019

Ocean Optics and Biogeochemistry Protocols for Satellite Ocean Colour Sensor Validation, Volume 2.0. Beam Transmission and Attenuation Coefficients: Instruments, Characterization, Field Measurements and Data Analysis Protocols.

http://dx.doi.org/10.25607/OBP-458

Goni et al.

2021

Wintertime particulate organic matter distributions in surface waters of the northern California current system

https://doi.org/10.1016/j.csr.2020.104312

Mobley

2022

The Ocean Optics Book

http://dx.doi.org/10.25607/OBP-1710

Software

Below is a list of other code sets that provide ACS processing scripts or other related functionality.

Author

Title

Link

Bausell, Jesse

acsPROCESS_INTERACTIVE

https://github.com/JesseBausell/acsPROCESS_INTERACTIVE/

Bourdin, Guillaume

InLineAnalysis

https://github.com/OceanOptics/InLineAnalysis

Haëntjens, Nils

Inlinino

https://github.com/OceanOptics/Inlinino

Haëntjens, Nils

pyACS

https://github.com/OceanOptics/pyACS

Wingard

ooi-data-explorations

https://github.com/oceanobservatories/ooi-data-explorations