Processing Equations

Equation 1

Generally, the total attenuation (loss) of light is the result of the total absorption and total scattering of light by seawater and its contents.

$$a+b=c$$

where $a$ is the absorption coefficient, $b$ is the scattering coefficient, and $c$ is the attenuation coefficient, all with units of inverse meters ($m^{-1}$).

Absorption Processing

Total Absorption Constituents

$$a_t = a_w + a_p + a_g$$ where $a_t$ is the total absorption of seawater and its constituents, $a_w$ is the absorption by water, $a_p$ is the absorption by all particulates, and $a_g$ is the absorption by all gelbstoff (dissolved) constituents.

Uncorrected Absorption from Raw Counts

$$ a(lambda){mathrm{uncorr}} = frac{1}{x} cdot ln!left( frac{a(lambda){mathrm{signal}}}{a(lambda){mathrm{reference}}} right) $$ where $a(lambda){mathrm{uncorr}}$ is the uncorrected absorption, x is the path length in meters (typically 0.25m), $a(lambda){mathrm{signal}}$ is raw signal counts for absorption, and $a(lambda){mathrm{reference}}$ is raw reference counts for absorption.

Measured Absorption, Corrected for Internal Temperature, Uncorrected for Spectra Discontinuity

$$a(lambda)_{m_discontinuity} = a(lambda){offset}-a(lambda){uncorr}-Delta Ta$$ where $a(lambda){m_discontinuity}$ is the measured absorption not corrected for the inherent discontinuity, $a(lambda){offset}$ is the absorption offset from the device file, $a(lambda){uncorr}$ is the uncorrected absorption from Equation 3a, and $Delta T_a$ is the linearly interpolated internal temperature correction.

Measured Absorption, Corrected for Spectra Discontinuity Using Second Half Shift Method

$$ a(lambda){text{m}} = a(lambda){text{m_discontinuity}} + mathrm{offset}_d left{ begin{array}{lll} lambda0 rightarrow lambda{text{discontinuityidx}} &,& 0 lambda{text{discontinuity_idx}+1} rightarrow lambda_n &,& text{offset}d end{array} right. $$ where $a(lambda){m_discontinuity}$ is from Equation 4a, and $offset_d$ is a scalar offset applied to the latter half of a spectra, derived using a cubic spline.

Temperature Salinity Corrected Absorption

$$ a(lambda)_{mts} = a(lambda)_m - bigl[psit , cdot (T - T{mathrm{ref}}) + psi_{sa},cdot Sbigr] $$

where $a(lambda)_{mts}$ is the measured absorption, corrected for temperature and salinity dependence, $a(lambda)_m$ is derived from Equation 5a, $psit$ and $psi{sa}$ are derived from the TS4.cor file, and $T$ and $S$ are derived from an ancillary instrument, $T_{ref}$ is from the device file.

Baseline Scattering Correction

$$ a(lambda){mts*{baseline}} = a(lambda)*{mts} - a(lambda{ref}){mts} $$ where $a(lambda){mts_{baseline}}$ is the absorption corrected for scattering using the baseline method, $a(lambda){mts}$ is from Equation 6a and $lambda{ref}$ is the absorption at the reference wavelength, typically 715nm.

Fixed Scattering Correction

$$ a(lambda){mts*{fixed}} = a(lambda)*{mts} - varepsilon cdot (c(lambda){mts} - a(lambda){mts}) $$ where $a(lambda){mts_{fixed}}$ is the absorption corrected for scattering using the fixed method, $a(lambda){mts}$ and $c(lambda){mts}$ are from Equation 6a/6c and $varepsilon$ is an empirically derived coefficient.

Proportional Scattering Correction

$$ a(lambda)_{text{mts_proportional}} = a(lambda)_{mts} - left(left( frac{a(mathrm{lambdaref}){mts}}

{c(mathrm{lambdaref}){mts} - a(mathrm{lambdaref}){mts}}

right) cdot bigl( c(lambda){mts} - a(lambda){mts} bigr)right) $$ where $a(lambda)_{text{mts_proportional}}$ is the absorption corrected for scattering using the proportional method, $lambda_{ref}$ is the absorption/attenuation at the reference wavelength, typically 715nm.

Attenuation Processing

Total Attenuation Constituents

$$c_t = c_w + c_p + c_g$$ where $c_t$ is the total attenuation of seawater and its constituents, $c_w$ is the attenuation by water, $c_p$ is the attenuation by all particulates, and $c_g$ is the attenuation by all gelbstoff (dissolved) constituents.

Uncorrected Attenuation From Raw Counts

$$ c(lambda){mathrm{uncorr}} = frac{1}{x} cdot ln!left( frac{c(lambda){mathrm{signal}}}{c(lambda){mathrm{reference}}} right) $$ where $c(lambda){mathrm{uncorr}}$ is the uncorrected attenuation, x is the path length, $c(lambda){mathrm{signal}}$ is raw signal counts for attenuation, and $c(lambda){mathrm{reference}}$ is raw reference counts for attenuation.

Measured Attenuation, Corrected for Internal Temperature, Uncorrected for Spectra Discontinuity

$$c(lambda)_{m_discontinuity} = c(lambda){offset}-c(lambda){uncorr}-Delta Tc$$ where $c(lambda){m_discontinuity}$ is the measured attenuation not corrected for the inherent discontinuity, $c(lambda){offset}$ is the attenuation offset from the device file, $c(lambda){uncorr}$ is the uncorrected attenuation from Equation 3c, and $Delta T_c$ is the linearly interpolated internal temperature correction.

Measured Attenuation, Corrected for Spectra Discontinuity Using Second Half Shift Method

$$ c(lambda){text{m}} = c(lambda){text{m_discontinuity}} + mathrm{offset}_d left{ begin{array}{lll} lambda0 rightarrow lambda{text{discontinuityidx}} &,& 0 lambda{text{discontinuity_idx}+1} rightarrow lambda_n &,& text{offset}d end{array} right. $$ where $c(lambda){m_discontinuity}$ is from Equation 4c, and $offset_d$ is a scalar offset applied to the latter half of a spectra, derived using a cubic spline.

Temperature Salinity Corrected Attenuation

$$ c(lambda)_{mts} = c(lambda)_m - bigl[psit , cdot (T - T{mathrm{ref}}) + psi{sc},cdot Sbigr] $$ where $c(lambda){mts}$ is the measured attenuation, corrected for temperature and salinity dependence, $c(lambda)_m$ is derived from Equation 5c, $psit$ and $psi{sc}$ are derived from the TS4.cor file, and $T$ and $S$ are derived from an ancillary instrument, $T_{ref}$ is from the device file.

Processing and QAQC

The ACS communicates over RS232 at 115200-8-N-1.

Step

Step Description

Notes

0.0

Find Packet

Read the serial stream and parse out a full ACS packet.

0.1

Split/Parse Packet

Split a packet into its raw constituents as defined by the ACS manual. (e.g. record_length, a_signal, raw_internal_temperature, etc).

0.2

Run Syntax Test

Verify that the packet length matches the record length and verify the checksum.

0.3

Run Gap Test

Check the assigned packet timestamp and the serial port buffer for pile up.

0.4

Import ACS Device File

The serial number from the parsed packet can be used to automatically identify a corresponding device file.

0.5

Calibrate Packet

Calibrate the packet using the device file to produce products in engineering units (e.g. a_m, internal_temperature)

0.6

Import TSCor Data

TS-correction does take time to perform each loop, but can be done in a single thread.

0.7

TS Correct Packet

OPTIONAL: Apply TS correction to data in the packet, if ancillary temperature and salinity are available. (e.g. a_mts)

0.8

Run Additional QAQC Tests

OPTIONAL: Run elapsed time test, internal temperature test, and Inf Nan Test on uncorrected data. These test can probably be done during data acquisition on a single thread.

Note: Many of the steps in the table below are described further in the Sea-Bird Scientific ACS Protocol Document (Rev Q).

GSW is the Pythonic implementation of Gibbs-SeaWater for TEOS-10.

Step

Step Description

Supported in *acspype*?

Description

Notes

1.0

Import ACS Device File and TS Correction Coefficients

Yes

Import a corresponding ACS device file (ACS-XXX.dev) and TS-correction coefficients (TS4.cor). These files are provided by Sea-Bird Scientific.

The TS4.cor file from SBS should be the same for all ACS. acspype has these coefficients hardcoded. It is strongly recommended to rename the .dev file to ACS-XXXXX_YYYY-MM-DD.dev, where XXXXX is the zero padded serial number (e.g. 00011) and YYYY-MM-DD is the date of calibration in the device file, to avoid overwriting device files if stored in the same directory.

1.1

Compute Internal Temperature

Yes

Compute internal temperature of the sensor from raw counts. This data is needed for calculation of the measured (a_m, c_m) coefficients.

The external temperature isn’t used in processing ACS data, but can also be calculated at this stage.

1.2

OPTIONAL: Calculate Absolute Salinity and Conservative Temperature

No (Use GSW)

The use of TEOS-10 salinity/temperature over EOS-80 salinity/temperature variable should not impact the TS-correction of spectra significantly. However, it is recommended, particularly if other physical properties of seawater of interest in analysis (e.g. density).

This requires a co-located CTD or TSG.

1.3

Compute Uncorrected Absorption (a_uncorrected) and Attenuation (c_uncorrected)

Yes

Compute the uncorrected absorption and attenuation in inverse meters (m^-1).

This is the first step in the processing chain where the ACS data is converted from raw counts to engineering units. Comparing uncorrected values may be useful in identifying debris or bubbles in the flow path.

1.4

Compute Measured Absorption (a_m_discontinuity) and Attenuation (c_m)

Yes

Compute the measured absorption and attenuation in inverse meters (m^-1).

acspype uses the measured (m) subscript because the filtration state of the medium is unknown. Use of the particulate-gelbstoff (pg), particulate (p), and gelbstoff (g) subscripts should be applied later at user discretion.

1.5

OPTIONAL, RECOMMENDED: Apply User-Collected Pure Water Coefficients

No

Apply an additional pure water offset derived by the user.

This may require multiple calibrations. Linear interpolation between two or more user created offsets will help assess and remove sensor drift.

1.6

OPTIONAL, RECOMMENDED: Correct Measured Absorption (a_m) and Attenuation (c_m) for Discontinuity

Yes

Correct the measured absorption and attenuation for the observed discontinuity jump that occurs roughly halfway in each spectra.

This step applies a scalar offset to the second half of the spectra to create a smoother spectrum.

1.7

Time Lag Correction

No

Using known volume, flow rate, and co-located sensors, shift ancillary variables to match the time the same water enters the ACS.

This requires a nearby CTD or TSG and knowledge of the flow rate of water pumped through the ACS.

1.8

Compute TS-Corrected Absorption (a_mts) and Attenuation (c_mts)

Yes

Using ancillary temperature (conservative temperature) and salinity (absolute salinity), correct spectra for the effect of temperature and salinity.

This requires a nearby CTD or TSG. DO NOT USE THE ACS EXTERNAL TEMPERATURE.

1.9

Zero Shift Correction Absorption (a_mts) and Attenuation (c_mts)

Yes

Shift values between -0.005 and 0 to be 0.

“”

2.0

OPTIONAL: A-C Lag Correction

No

This step is only necessary if flow cells are plumbed in series and if there analysis is done on full resolution data.

OPTIONAL: This only applies if the flow cells are plumbed in series and you do not intend to resample to coarser time bins.

3.0

Interpolate to Common Wavelengths

Yes

Linearly interpolate absorption and attenuation variables to common wavelength bins.

Linear interpolation between wavelength bins is not common in ACS literature. However, this makes it significantly easier to perform scattering correction and for comparison between absorption and attenuation.

3.1

Correct Absorption for Scattering Using Baseline and Proportional Methods

Yes

Perform a user selected scattering correction to absorption.

Baseline, Fixed, and Proportional Scattering Correction are provided by acspype. For scattering correction that requires a reference wavelength, 715nm is commonly used. However, acspype provides a function that will estimate an optimal reference wavelength by using the first red wavelength (>700nm) that is closes to zero.

4.0

Run QAQC Tests

Yes

Run QAQC tests (see Recommended QAQC Tests table for recommended tests).

Elapsed Time Test, Internal Temperature Test, Inf Nan Test, Gross Range Test

4.1

Remove Poor Quality Spectra Based on QAQC Test Results

No (Use Xarray)

Remove poor quality spectra from analysis. The results of some may tests provide indication of instrument malfunction, occurrence of debris or bubble in flow, or that the data is suspect because of defined limits in the sensor file.

“”

5.0

Split into measured (a_pg, c_pg), particulate (a_p, c_p) and gelbstoff (a_g, c_g) Measurements

No

At this stage, data can be renamed to match its filtration state. Generally p = pg - g

This stage requires knowledge of the filtration state of the seawater.

6.0

Filter and Resample

No

At this stage, data can be passed through a series of smoothing filters and resampled to user-defined bins.

This stage requires careful consideration of the required resolution for analysis.

7.0

Compute Advanced Data Products

Yes (Ongoing Development)

Corrected absorption and attenuation products are used to compute advanced data products (e.g. chlorophyll-a from absorption line height and particulate organic carbon).

“”

Flag meanings are based on the QARTOD flag meanings. Descriptions of each flag can be found in the acspype flag documentation. Tests are considered QARTOD tests if they were coded in a way that would satisfy the requirements for the generic QARTOD tests found in each manual (e.g. gap test, syntax test, gross range test, etc.). Test that are not QARTOD tests are custom tests developed from information in the ACS manual or are modified forms of a related QARTOD test.

Note: If using acspype and a flag of 2 (NOT_EVALUATED) is assigned in these tests, that indicates there was a programmatic failure in the test and an issue should be raised on GitHub. A flag of 9 (MISSING_DATA) indicates that the input data or ancillary data was missing.

Test Name

Test Description

Recommended Settings

Is QARTOD?

Possible Results

Notes

Gap Test

The first stage is to compare the data acquisition time to the host computer clock time. The second stage is to compare the known record length against the number of bytes in the serial buffer.

Time Increment = 0.25 (s)

Yes

4, 1

This test is generally not performed on archived datasets. If using archived data, users should reach out to the data provider to verify the timestamps or data acquisition protocols.

Syntax Test

The first stage checks if the registration bytes occur at the beginning of the packet. The second stage checks if the packet contains a pad byte at the end. The third stage compares the length of the packet against the size of the packet descriptor. The fourth stage compares the record length against the length of the packet. The fifth stage verifies the checksum.

Not Applicable

Yes

4, 1

This test is generally not performed on archived datasets, although it can be recreated when processing ACS packets that have been dumped into binary files.

Elapsed Time Test

Check the elapsed time in each ACS sample to see if it is collected at a reasonable time since the instrument received power. The manufacturer states that the ACS takes up to 10 minutes to warm-up and that data may be questionable during this time. On moorings where power is a commodity, this is may not be possible.

Fail Threshold = 45000 (ms)
Suspect Threshold = 240000 (ms)

No (Minimum Side Gross Range Test)

4, 3, 1

The fail and suspect values used in this test depend on the ACS datasets. For moored or profiler time-series, a lower fail/suspect threshold may be best, otherwise significant quantities of the data may be flagged. Users should understand that data collected at the beginning of a power start up sequence may be excessively noisy or contain inaccurate data. This data should be carefully reviewed.

Internal Temperature Test

Check the internal temperature of the ACS to see if it is within the range specified in the device file. Values outside the device calibration range are flagged as suspect.

Defined in device file and automatically applied.

No (Gross Range Test)

3, 1

Data outside the calibrated temperature bins may be suspect simply because it exists outside of the calibrated range. A typical calibration range is 2-35 degC. If experiencing waters outside that range, users should request a larger calibration range from SBS.

Inf Nan Test

Check the uncorrected values for Inf and NaN values. Inf and Nan values can appear in uncorrected measurements for a number of reasons. A common reason is that the one or multiple reference counts contain the value of 0, which propagate to NaN when performing a log operation and remain at each processing step.

Not Applicable

No (Custom test)

4, 1

If a NaN exists in the spectrum, it can probably be discarded. Users should review the surrounding spectrum and the same wavelength bin to determine if the issue is a one-off. Repeated NaNs (or zero reference counts) in uncorrected values may indicate that factory recalibration is needed or an instrument malfunction.

Gross Range Test

On TS and scattering corrected spectrum, run the gross range test, which will assign a flag for each wavelength in a spectra.

Fail Threshold = [0, 10]
Suspect Threshold = [0.001,8.5]

Yes

4, 3, 1

Some may find that this results in fail flags in the red wavelengths, even after correction. This

Blanket Gross Range Test

Assigns a blanket flag to the entire spectrum if a certain percentage of coefficients across the spectrum are flagged as fail. This is useful for identifying spectra that are not suitable for further processing.

Ignore Wavelengths = [700, 755]
Fail Threshold = 10
Suspect Threshold = 5

No (Modified Gross Range Test)

4, 3, 1

This may be useful for identifying poor quality spectra. Users should review the neighboring spectrum samples to see if the issue is persistent or a one-off. If only one spectrum is found as failed while the neighboring spectrum are ok, then it could probably be removed before analysis is performed without a significant impact on the end result.

A Greater Than C Test

Check if scattering corrected absorption is greater than ts-corrected attenuation. Consecutive values in a spectrum and over time may indicate bubbles or a stuck object in the flow cell.

Not Applicable

No (Modified Minimum Side Gross Range Test)

3, 1

This test should be used to consider data suspect, but should not be used to immediately remove data.

Rolling Variance Test

Assigns a flag at each wavelength bin of spectrum if the variance over time exceeds a percentage of the mean. A rolling centered window is used to calculate the variance and a flag is assigned to the central sample.

Percentage = 25
Window Size = 240

No (Custom Test)

3, 1

Suspect flags may indicate bubbles or debris in the flow cell. NOT CURRENTLY IMPLEMENTED.

Discontinuity Offset Test

Check if the absolute value of the discontinuity offset exceeds a fail threshold or a multiple of the median for the time-series.

Fail Threshold = 10
Median Multiplier = 3

No (Custom Test)

4, 3, 1

Discontinuity offsets that are larger than the fail threshold are flagged as fail. The maximum sensing range of the ACS is used as the default for the fail threshold. In general, discontinuity offsets should be small and within the typical precision of the sensor (+/- 0.003 m^-1). A second option for defining a median multiplier is used to flag data that exceed X times the median as suspect.

Reference Material

The most up-to-date manuals and protocols for the ACS can be found on the Sea-Bird Scientific ACS Downloads Page.

Below is a small list of ACS, absorption, attenuation, or ACS data product related literature.

Author

Year

Title

Link

Zaneveld et al.

1994

Scattering error correction of reflecting-tube absorption meters

https://doi.org/10.1117/12.190095

Bricaud et al.

1995

Variability in the chlorophyll-specific absorption coefficients of natural phytoplankton: Analysis and parameterization

https://doi.org/10.1029/95JC00463

Davis et al.

1997

Reducing the effects of fouling on chlorophyll estimates derived from long-term deployments of optical instruments

https://doi.org/10.1029/96JC02430

Pegau et al.

1997

Absorption and attenuation of visible and near-infrared light in water: dependence on temperature and salinity

https://doi.org/10.1364/AO.36.006035

Pegau et al.

2003

Ocean Optics Protocols for Satellite Ocean Color Sensor Validation, Revision 4, Volume IV

https://ntrs.nasa.gov/api/citations/20030093642/downloads/20030093642.pdf

Behrenfeld and Boss

2006

Beam attenuation and chlorophyll concentration as alternative optical indices of phytoplankton biomass.

https://doi.org/10.1357/002224006778189563

Gardner et al.

2006

Global POC concentrations from in-situ and satellite data

https://doi.org/10.1016/j.dsr2.2006.01.029

Sullivan et al.

2006

Hyperspectral temperature and salt dependencies of absorption by water and heavy water in the 400–750 nm spectral range

https://doi.org/10.1364/AO.45.005294

Boss et al.

2007

Measurements of spectral optical properties and their relation to biogeochemical variables and processes in Crater Lake, Crater Lake National Park, OR

https://doi.org/10.1007/978-1-4020-5824-0_9

Stramski et al.

2008

Relationships between the surface concentration of particulate organic carbon and optical properties in the eastern South Pacific and eastern Atlantic Oceans

https://doi.org/10.5194/bg-5-171-2008

Slade et al.

2010

Underway and Moored Methods for Improving Accuracy in Measurement of Spectral Particulate Absorption and Attenuation

https://doi.org/10.1175/2010JTECHO755.1

Cetenic et al.

2012

Particulate organic carbon and inherent optical properties during 2008 North Atlantic Bloom Experiment

https://doi.org/10.1029/2011JC007771

Rottgers et al.

2013

Evaluation of scatter corrections for ac-9 absorption measurements in coastal waters

https://doi.org/10.1016/j.mio.2013.11.001

Roesler and Barnard

2013

Optical proxy for phytoplankton biomass in the absence of photophysiology: Rethinking the absorption line height

https://doi.org/10.1016/j.mio.2013.12.003

Boss et al.

2013

The characteristics of particulate absorption, scattering and attenuation coefficients in the surface ocean; Contribution of the Tara Oceans expedition

https://doi.org/10.1016/j.mio.2013.11.002

Stockley et al.

2017

Assessing uncertainties in scattering correction algorithms for reflective tube absorption measurements made with a WET Labs ac-9

https://doi.org/10.1364/OE.25.0A1139

Boss et al.

2019

Ocean Optics and Biogeochemistry Protocols for Satellite Ocean Colour Sensor Validation, Volume 2.0. Beam Transmission and Attenuation Coefficients: Instruments, Characterization, Field Measurements and Data Analysis Protocols.

http://dx.doi.org/10.25607/OBP-458

Goni et al.

2021

Wintertime particulate organic matter distributions in surface waters of the northern California current system

https://doi.org/10.1016/j.csr.2020.104312

Mobley

2022

The Ocean Optics Book

http://dx.doi.org/10.25607/OBP-1710

Below is a list of other code sets that provide ACS processing scripts or other related functionality.

Author

Title

Link

Bausell, Jesse

acsPROCESS_INTERACTIVE

https://github.com/JesseBausell/acsPROCESS_INTERACTIVE/

Bourdin, Guillaume

InLineAnalysis

https://github.com/OceanOptics/InLineAnalysis

Haëntjens, Nils

Inlinino

https://github.com/OceanOptics/Inlinino

Haëntjens, Nils

pyACS

https://github.com/OceanOptics/pyACS

Wingard

ooi-data-explorations

https://github.com/oceanobservatories/ooi-data-explorations