acspype code

acspype.stream module

This module contains functions and classes for reading and parsing ACS data over RS232.

class acspype.stream.ACSStream(port: str, baudrate: int = 115200, timeout: float = 1)[source]

Bases: object

BAUDRATE: int = 115200

BYTESIZE: int = 8

FLOWCONTROL: int = 0

PARITY: str = 'N'

STOPBITS: int = 1

TIMEOUT: float = 1

calibrate_packet(parsed_packet, dev: ACSDev) → DeviceCalibratedPacket[source]: ACSStream wrapper for the calibrate_packet function inherited from the packet module.

clear_packet_buffer()[source]: Clear the packet buffer that is currently in memory. :return: None

clear_serial_buffers() → None[source]: Clear the input and output serial buffers on the class assigned port. :return: None

connect() → None[source]

Attempt to connect to the port assigned at class instantiation.

Returns:: None

disconnect() → None[source]

Disconnect from the port assigned at class instantiation.

Returns:: None

find_packet() → ACSPacket[source]

Find the first full ACS packet within buffered serial data.

Returns:: Raw ACS Packet data stored in a custom NamedTuple.

parse_packet(acs_packet: ACSPacket) → ParsedPacket[source]: ACSStream wrapper for the parse_packet function inherited from the packet module.

read_stream() → None[source]

Read the current number of bytes waiting in the serial port buffer and append them to a byte array in memory.

Returns:: None

reset_serial(port: str, baudrate: int, bytesize: int = 8, parity: str = 'N', stopbits: int = 1, flowcontrol: int = 0, timeout: float = 1)[source]

Reset or instantiate serial object using the assigned settings.

Parameters:

port – A string representing the COM port the ACS is connected to. e.g. ‘COM1’ or ‘/dev/ttyUSB0’
baudrate – The baudrate the ACS communicates at. All ACS sensors come from the factory set to 115200 bps.
bytesize – The number of bits in a byte. Default is 8.
parity – The parity setting for the ACS. Default is ‘N’ for no parity.
stopbits – The number of stop bits. Default is 1.
flowcontrol – The flow control setting. Default is 0 for no flow control.
timeout – The number of seconds to wait before timeout. Default is 1 second.

Returns:

None

ts_correct_packet(dev_cal_packet, temperature, salinity, dev: ACSDev) → TSCorrectedPacket[source]: ACSStream wrapper for the ts_correct_packet function inherited from the packet module.

acspype.structures module

This module contains NamedTuple structures that ACS data can be dumped into for easy tracking and exporting. Each NamedTuple is assigned methods that allows for the export of data to a dictionary or to a formatted xarray.Dataset.

class acspype.structures.ACSPacket(daq_time: datetime, full_packet: bytes | bytearray | None)[source]

Bases: NamedTuple

A class for representing a time-stamped ACS packet.

Parameters:

daq_time – The host computer time the packet was acquired.
full_packet – The full packet, including the registration bytes and the checksum + pad bytes.

daq_time: datetime: Alias for field number 0

full_packet: bytes | bytearray | None: Alias for field number 1

to_dict() → dict[source]

Export the ACSPacket as a dictionary.

Returns:: A dict representation of the ACSPacket, where keys are attributes and values are attribute values.

to_xarray() → Dataset[source]

Export the ACSPacket as an xarray.Dataset.

Returns:: An xarray.Dataset representation of the ACSPacket, where coordinates are the daq_time and the data variable is the full_packet.

Bases: NamedTuple

A class for representing a device file-calibrated ACS packet.

Parameters:

daq_time – The host computer time the packet was acquired, usually passed from an ParsedPacket.
a_wavelength – The wavelengths for the absorption channel.
c_wavelength – The wavelengths for the attenuation channel.
sn_hexdec – The serial number of the device represented as a hexadecimal string.
serial_number – The serial number of the device represented as a string.
internal_temperature – The internal temperature of the device in degrees Celsius.
external_temperature – The external temperature of the device in degrees Celsius.
a_uncorrected – The uncorrected absorption signal in inverse meters.
c_uncorrected – The uncorrected attenuation signal in inverse meters.
discontinuity_index – The index of the discontinuity in both channels.
a_discontinuity_offset – The offset for the absorption channel discontinuity.
c_discontinuity_offset – The offset for the attenuation channel discontinuity.
a_m_discontinuity – The absorption coefficient without discontinuity correction applied.
c_m_discontinuity – The attenuation coefficient without discontinuity correction applied.
a_m – The measured absorption coefficient in inverse meters.
c_m – The measured attenuation coefficient in inverse meters.

a_discontinuity_offset: float: Alias for field number 10

a_m: Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]: Alias for field number 14

a_m_discontinuity: Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]: Alias for field number 12

a_uncorrected: Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]: Alias for field number 7

a_wavelength: tuple[float, ...]: Alias for field number 1

c_discontinuity_offset: float: Alias for field number 11

c_m: Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]: Alias for field number 15

c_m_discontinuity: Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]: Alias for field number 13

c_uncorrected: Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]: Alias for field number 8

c_wavelength: tuple[float, ...]: Alias for field number 2

daq_time: datetime: Alias for field number 0

discontinuity_index: int: Alias for field number 9

external_temperature: float: Alias for field number 6

internal_temperature: float: Alias for field number 5

serial_number: str: Alias for field number 4

sn_hexdec: str: Alias for field number 3

to_dict() → dict[source]

Export the DeviceCalibratedPacket as a dictionary.

Returns:: A dict representation of the DeviceCalibratedPacket, where keys are attrs and values are attr values.

to_xarray() → Dataset[source]

Export the DeviceCalibratedPacket as an xarray.Dataset.

Returns:: An xarray.Dataset representation of the DeviceCalibratedPacket.

class acspype.structures.ParsedPacket(daq_time: datetime, record_length: int, packet_type: int, sn_int: int, a_reference_dark: int, a_signal_dark: int, raw_external_temperature: int, raw_internal_temperature: int, c_reference_dark: int, c_signal_dark: int, elapsed_time: int, number_of_wavelengths: int, c_reference: tuple[int, ...], a_reference: tuple[int, ...], c_signal: tuple[int, ...], a_signal: tuple[int, ...], checksum: int)[source]

Bases: NamedTuple

A class for representing a parsed ACS packet.

Parameters:

daq_time – The host computer time the packet was acquired, usually passed from an ACSPacket.
record_length – The length of the packet in bytes.
packet_type – The type of packet.
sn_int – The serial number of the device represented as an integer.
a_reference_dark – The dark reference counts for the absorption channel.
a_signal_dark – The dark signal counts for the absorption channel.
raw_external_temperature – The raw external temperature reading in counts.
raw_internal_temperature – The raw internal temperature reading in counts.
c_reference_dark – The dark reference counts for the attenuation channel.
c_signal_dark – The dark signal counts for the attenuation channel.
elapsed_time – The elapsed time since power was supplied to the ACS, in milliseconds.
number_of_wavelengths – The number of wavelengths for each channel.
c_reference – The attenuation reference in counts.
a_reference – The absorption reference in counts.
c_signal – The attenuation signal in counts.
a_signal – The absorption signal in counts.
checksum – The checksum of the packet.

a_reference: tuple[int, ...]: Alias for field number 13

a_reference_dark: int: Alias for field number 4

a_signal: tuple[int, ...]: Alias for field number 15

a_signal_dark: int: Alias for field number 5

c_reference: tuple[int, ...]: Alias for field number 12

c_reference_dark: int: Alias for field number 8

c_signal: tuple[int, ...]: Alias for field number 14

c_signal_dark: int: Alias for field number 9

checksum: int: Alias for field number 16

daq_time: datetime: Alias for field number 0

elapsed_time: int: Alias for field number 10

number_of_wavelengths: int: Alias for field number 11

packet_type: int: Alias for field number 2

raw_external_temperature: int: Alias for field number 6

raw_internal_temperature: int: Alias for field number 7

record_length: int: Alias for field number 1

sn_int: int: Alias for field number 3

to_dict() → dict[source]

Export the ParsedPacket as a dictionary.

Returns:: A dict representation of the ParsedPacket, where keys are attributes and values are attribute values.

to_xarray() → Dataset[source]

Export the ParsedPacket as an xr.Dataset.

Returns:: An xarray.Dataset representation of the ParsedPacket.

Bases: NamedTuple

A class for representing a temperature and salinity-corrected ACS packet.

Parameters:

daq_time – The host computer time the packet was acquired, usually passed from a DeviceCalibratedPacket.
a_wavelength – The wavelengths for the absorption channel.
c_wavelength – The wavelengths for the attenuation channel.
temperature – The temperature of the water in degrees Celsius.
salinity – The salinity of the water in practical salinity units (PSU).
a_mts – The temperature and salinity-corrected absorption coefficient in inverse meters.
c_mts – The temperature and salinity-corrected attenuation coefficient in inverse meters.

a_mts: Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]: Alias for field number 5

a_wavelength: tuple[float, ...]: Alias for field number 1

c_mts: Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]: Alias for field number 6

c_wavelength: tuple[float, ...]: Alias for field number 2

daq_time: datetime: Alias for field number 0

salinity: float: Alias for field number 4

temperature: float: Alias for field number 3

to_dict() → dict[source]

Export the TSCorrectedPacket as a dictionary.

Returns:: A dict representation of the TSCorrectedPacket, where keys are attributes and values are attr values.

to_xarray() → Dataset[source]

Export the TSCorrectedPacket as an xarray.Dataset.

Returns:: An xarray.Dataset representation of the TSCorrectedPacket.

acspype.packet module

This module contains functions for working with raw ACS byte packets and converting them into more readable data structures. The ACSStream class has wrapper functions around the functions in this module. Users may want to use these functions separately from the ACSStream class if reprocessing from a file that contains raw packets, such as the WETView generated .dat files or OOI CSPP .acs files.

acspype.packet.calibrate_packet(parsed_packet: ParsedPacket, dev: ACSDev) → DeviceCalibratedPacket[source]

Calibrate a parsed packet using the device calibration parameters. This function is intended for use on a single: ParsedPacket object. If attempting to implement with Xarray, you will need to create a separate function or attempt to wrap this function in an xarray.apply_ufunc.

Parameters:

parsed_packet – A ParsedPacket object containing the parsed data.
dev – An ACSDev-like object containing the device calibration parameters.

Returns:

A DeviceCalibratedPacket object containing data corrected/calibrated to the values in the ACSDev object.

acspype.packet.parse_packet(acs_packet: ACSPacket) → ParsedPacket[source]

Parse an ACS packet into a ParsedPacket object. This function is intended for use on a single ACSPacket object.: If attempting to implement with Xarray, you will need to create a separate function or attempt to wrap this function in an xr.apply_ufunc.

Parameters:: acs_packet – An ACSPacket object containing timestamp and the full packet data. Note: Users looking to only use this to parse the packet without caring about the daq_time can use an ACSPacket with a dummy daq_time.
Returns:: A ParsedPacket object containing the parsed data.

acspype.packet.ts_correct_packet(device_calibrated_packet: DeviceCalibratedPacket, temperature: float, salinity: float, dev: ACSDev) → TSCorrectedPacket[source]

Apply TS-correction to a device calibrated packet.

Parameters:

device_calibrated_packet – A DeviceCalibratedPacket object containing the device calibrated data.
temperature – The temperature or conservative temperature of the water sample in degrees Celsius.
salinity – The practical or absolute salinity of the water sample.
dev – A ACSDev-like object containing the device calibration parameters.

Returns:

A TSCorrectedPacket object containing data that has been corrected for dependence on temperature and salinity.

acspype.packet.unpack_packet(full_packet: bytes | bytearray) → tuple[tuple, int][source]

Unpack a full ACS packet into raw bytes and the checksum. This function is intended for use on a single packet.: If attempting to implement with Xarray, you will need to create a separate function or attempt to wrap this function in an xarray.apply_ufunc.

Parameters:: full_packet – A bytes or bytesarray object containing the full ACS packet, including the registration bytes and the pad byte.
Returns:: A tuple containing the raw bytes and the checksum.

acspype.processing module

This module contains functions that are used to process data from the ACS in real-time or as a batch process.

Most functions are intended to be used with scalar values/1D arrays (e.g. data produced in real-time) or xarray DataArrays (e.g. mass processing of an ACS deployment), or both. Scalar/1D inputs are intended to be used in the processing of a single packet or spectrum from the ACS. xr.DataArray implementation is intended to be used in the processing of multiple spectra already in an xarray.Dataset. For computations that are not immediately builtin to xarray,the xarray.apply_ufunc function is used to apply the function that is denoted with a prepended underscore.

It is strongly recommended that your xarray.Dataset contain the coordinates/dimensions of: time, a_wavelength, c_wavelength

For a list of recommended coordinate/variable names for ACS data products, please see https://github.com/IanTBlack/acspype/blob/main/info/NAMING_CONVENTIONS.md

acspype.processing.apply_discontinuity_offset(measured: ndarray[tuple[Any, ...], dtype[float]] | DataArray, disc_off: float | ndarray[tuple[Any, ...], dtype[float]], disc_idx: int, wavelength_dim: str, shift_method: str = 'halfway') → ndarray[tuple[Any, ...], dtype[float]] | DataArray[source]

Apply a discontinuity offset to a measured coefficient. Vectorized version of _apply_discontinuity_offset, where applicable.

Parameters:

measured – Measured absorption or attenuation coefficient.
disc_off – The discontinuity offset of the absorption and attenuation spectra.
disc_idx – The index of the discontinuity in the absorption and attenuation spectra.
wavelength_dim – The name of the wavelength dimension if inputting an xarray DataArray.

Returns:

The spectrum with the offset applied.

acspype.processing.baseline_plus_scattering_correction(a_mts: DataArray, a_mts_715: DataArray) → DataArray[source]

Perform baseline plus scattering correction for the absorption coefficient. This method originates from Rottgers et al. (2013). https://doi.org/10.1016/j.mio.2013.11.001 True absorption at 715nm is empirically determined in Equation 5.

Parameters:

a_mts – The TS-corrected absorption coefficient.
a_mts_715 – The absorption at the reference wavelength. Historically, 715nm is used.

Returns:

Baseline plus scattering corrected absorption coefficient.

acspype.processing.baseline_scattering_correction(a_mts: ndarray[tuple[Any, ...], dtype[float]] | DataArray, reference_a: float | DataArray) → ndarray[tuple[Any, ...], dtype[float]] | DataArray[source]

Perform baseline scattering correction on the absorption coefficient. This is the baseline method from Zaneveld et al. (1994).

Parameters:

a_mts – The TS-corrected absorption coefficient across all wavelengths.
reference_a – The TS-corrected absorption coefficient at a reference wavelength.

Returns:

The baseline scattering-corrected absorption coefficient.

acspype.processing.compute_depth(counts: int | DataArray, scale_factor: float, offset: float, return_uncertainty: bool = False, unc: float = nan) → float | DataArray[source]

Compute depth for ACS instruments that have a pressure sensor. The depth is calculated from a scale factor and offset found in the .dev file. Formula: depth = scale_factor * counts + offset

The offset and scale_factor are in the Depth calibration line in the dev file, in that particular order.

Parameters:

counts – The pressure counts from the ACS.
scale_factor – The scale factor found in the ACS dev file.
offset – The offset found in the ACS dev file.

Returns:

The depth in meters.

acspype.processing.compute_discontinuity_offset(measured: ndarray[tuple[Any, ...], dtype[float]] | DataArray, wavelength: ndarray[tuple[Any, ...], dtype[float]] | DataArray, disc_idx: int, wavelength_dim: str) → float | DataArray[source]

Compute the scalar discontinuity offset for a measured coefficient.

Parameters:

measured – Absorption or attenuation coefficient.
wavelength – The wavelengths of the absorption or attenuation spectra.
disc_idx – The index of the discontinuity in the absorption and attenuation spectra.
wavelength_dim – The name of the wavelength dimension if inputting an xarray DataArray.

Returns:

The discontinuity offset of the absorption and attenuation spectra as a float.

acspype.processing.compute_external_temperature(counts: int | DataArray, return_uncertainty: bool = False, unc: float = 0.1) → float | DataArray[source]

Compute the external temperature of the ACS from the counts of the external temperature sensor. Not used in processing if the sensor is not placed in-situ or if there is a better _source that is indicative of the temperature of the water sampled by the ACS.

Parameters:: counts – The counts of the external temperature sensor.
Returns:: The external temperature in degrees Celsius.

acspype.processing.compute_internal_temperature(counts: int | DataArray, return_uncertainty: bool = False, unc: float = 0.1) → float | DataArray[source]

Compute the internal temperature of the ACS from the counts of the internal temperature sensor.

Parameters:

counts – Values of the internal temperature sensor, in counts.
return_uncertainty – If True, return the uncertainty of the internal temperature sensor as an uncertainties.unumpy object.
unc – The unc of the internal temperature sensor.

Returns:

A float value if the input was a scalar counts value or an xr.DataArray of the internal temperature if input as an xr.DataArray. Units of return are degrees Celsius.

acspype.processing.compute_measured(uncorrected: ndarray[tuple[Any, ...], dtype[float]] | DataArray, internal_temperature: float | DataArray, offset: ndarray[tuple[Any, ...], dtype[float]], func_delta_t: Callable, return_uncertainty: bool = False) → ndarray[tuple[Any, ...], dtype[float]] | DataArray[source]

Compute the measured absorption or attenuation coefficient from the uncorrected coefficient and the internal temperature.

Parameters:

uncorrected – The uncorrected absorption or attenuation coefficient. Typically computed via compute_uncorrected().
internal_temperature – The internal temperature of the ACS in degrees Celsius. Typically computed via compute_internal_temperature().
offset – The coefficient offset from the device file.
func_delta_t – The interpolation function for correcting for internal temperature variation. This function is built by default when using an ACSDev object.

Returns:

Measured absorption or attenuation coefficient in m^-1, corrected for internal temperature variation.

acspype.processing.compute_uncorrected(signal_counts: tuple[int, ...] | ndarray[tuple[Any, ...], dtype[float]] | DataArray, reference_counts: tuple[int, ...] | ndarray[tuple[Any, ...], dtype[float]] | DataArray, path_length: float = 0.25, return_uncertainty: bool = False, unc: float = 0.01044030650891055) → ndarray[tuple[Any, ...], dtype[float]] | DataArray[source]

Compute the uncorrected coefficient from the signal and reference counts.

Parameters:

signal_counts – The absorption or attenuation channel signal counts.
reference_counts – The absorption or attenuation channel reference counts.
path_length – The path length of the ACS in meters. May be denoted by “x” in equations and literature. Default is 0.25m, but it is recommended to input the path length that is in the device file.

Returns:

Uncorrected absorption or attenuation coefficient in m^-1.

acspype.processing.convert_sn_hexdec(sn_int: int | DataArray) → str | DataArray[source]

Converts an integer representing an ACS serial number to the hexadecimal representation. The function vectorizes _convert_sn_hexdec for xr.DataArray inputs.

Parameters:: sn_int – An integer or xr.DataArray of integers representing the ACS serial number.
Returns:: The hexadecimal representation of the serial number, not really as readable as the integer representation.

acspype.processing.convert_sn_str(sn_int: int | DataArray) → str | DataArray[source]

Converts an integer representing an ACS serial number to a string representation. This function vectorizes _convert_sn_str for xr.DataArray inputs.

Parameters:: sn_int – An integer or xr.DataArray of integers representing the ACS serial number.
Returns:: The string representation of the serial number.

acspype.processing.discontinuity_correction(measured: DataArray, discontinuity_index: int, wavelength_dim: str, shift_method: str = 'halfway') → tuple[DataArray, DataArray][source]

Compute the discontinuity offset and apply it to the measured coefficient. NOTE: This function only works with xarray DataArrays.

Parameters:

measured – The measured absorption or attenuation coefficient.
discontinuity_index – The index of the discontinuity in the absorption and attenuation spectra.
wavelength_dim – The wavelength dimension of the measured coefficient if inputting an xarray DataArray.

Returns:

A tuple of xr.DataArrays (corrected measured coefficient, offset)

acspype.processing.estimate_reference_wavelength(a_mts: DataArray, wavelength_dim: str) → float[source]

Estimate a reference wavelength for an ND absorption dataset. The index closest to zero in the last 1/5 of each spectrum sample is determined. The index with the most repeats across the dataset estimated to be the reference wavelength.

Parameters:

a_mts – TS-corrected absorption coefficient.
wavelength_dim – The wavelength dimension name for a_mts.

Returns:

The reference wavelength in nm.

acspype.processing.find_discontinuity_index(a_wavelength: ndarray[tuple[Any, ...], dtype[float]], c_wavelength: ndarray[tuple[Any, ...], dtype[float]], min_wvl: float = 550.0, max_wvl: float = 600.0) → int[source]

Find the wavelength index of the discontinuity in the absorption and attenuation spectra. The discontinuity is in the range of 535-600nm.

Parameters:

a_wavelength – The absorption wavelengths of the ACS.
c_wavelength – The attenuation wavelengths of the ACS.
min_wvl – Default is 535 nm. Can be adjusted if the discontinuity is not in the expected range.
max_wvl – Default is 600 nm. Can be adjusted if the discontinuity is not in the expected range.

Returns:

The index of the discontinuity in the absorption and attenuation spectra.

acspype.processing.fixed_scattering_correction(a_mts: ndarray[tuple[Any, ...], dtype[float]] | DataArray, c_mts: ndarray[tuple[Any, ...], dtype[float]] | DataArray, epsilon: float = 0.14) → ndarray[tuple[Any, ...], dtype[float]] | DataArray[source]

Perform fixed scattering correction on the absorption coefficient. This is the fixed correction method from Pegau et al. 2003.

Parameters:

a_mts – TS-corrected absorption.
c_mts – TS-corrected attenuation.
epsilon – A default value of 0.14 is used for biological particle dominant waters. A value of 0.18 is recommended for waters dominated by sediments.

Returns:

Fixed scattering corrected absorption coefficient.

acspype.processing.interpolate_common_wavelengths(ds: Dataset, a_wavelength_dim: str = 'a_wavelength', c_wavelength_dim: str = 'c_wavelength', new_wavelength_dim: str = 'wavelength', wavelength_range: list | str = 'infer', step: int = 1) → Dataset[source]

This function interpolates the absorption and attenuation spectra to a common wavelength range and step size. Only works on xarray Datasets. Applies interpolation to all variables on the a_wavelength and c_wavelength dimensions.

Parameters:

ds – The input dataset with two wavelength dimensions, one for absorption and the other for attenuation.
a_wavelength_dim – The name of the absorption wavelength dimension.
c_wavelength_dim – The name of the attenuation wavelength dimension.
new_wavelength_dim – The name of the new wavelength dimension to be created. Default is ‘wavelength’, which is the recommended name for the common wavelength dimension.
wavelength_range – The wavelength range to interpolate along. Default is ‘infer’, which will use the minimums and maximums of the absorption and attenuation wavelengths to identify a common range for both channels.
step – The wavelength step size to interpolate to. Default is 1 nm.

Returns:

A dataset interpolated to the common wavelength range and step size. If there are variables that you do not want to interpolate along (e.g. a_signal, a_reference, c_signal, c_reference), it is recommended that you subset before interpolation.

acspype.processing.proportional_plus_scattering_correction(a_mts: DataArray, c_mts: DataArray, a_mts_715: DataArray, c_mts_715: DataArray, e_c: float = 0.56) → DataArray[source]

Perform proportional plus scattering correction for the absorption coefficient. This method originates from Rottgers et al. (2013). https://doi.org/10.1016/j.mio.2013.11.001 True absorption at 715nm is empirically determined in Equation 5. The value e_c represents attenuation correction.

This function only works with xarray DataArrays that share a common ‘wavelength’ dimension, thus the dataset must be linearly interpolated to a common wavelength range and bin size.

Parameters:

a_mts – The TS-corrected absorption coefficient.
c_mts – The TS-corrected attenuation coefficient.
a_mts_715 – The absorption at the reference wavelength. Historically, 715nm is used.
c_mts_715 – The attenuation at the reference wavelength. Historically, 715nm is used.
e_c – The attenuation correction factor. Normally it is 1 (no attenuation correction), but Rottgers et al. (2013) suggest a value of 0.56 when running wavelength independent correction (which this function does).

Returns:

Proportional plus scattering corrected absorption coefficient.

acspype.processing.proportional_scattering_correction(a_mts: ndarray[tuple[Any, ...], dtype[float]] | DataArray, c_mts: ndarray[tuple[Any, ...], dtype[float]] | DataArray, reference_a: ndarray[tuple[Any, ...], dtype[float]] | DataArray, reference_c: ndarray[tuple[Any, ...], dtype[float]] | DataArray) → ndarray[tuple[Any, ...], dtype[float]] | DataArray[source]

Perform proportional scattering correction on the absorption coefficient. This is the proportional method from Zaneveld et al. (1994).

Parameters:

a_mts – TS-corrected absorption.
c_mts – TS-corrected attenuation.
reference_a – The TS-corrected absorption coefficient at a reference wavelength. Historically 715nm.
reference_c – The TS-corrected attenuation coefficient at a reference wavelength. Historically 715nm.

Returns:

Proportional scattering corrected absorption coefficient.

acspype.processing.ts_correction(measured: ndarray[tuple[Any, ...], dtype[float]] | DataArray, temperature: float | DataArray, salinity: float | DataArray, psi_temperature: ndarray[tuple[Any, ...], dtype[float]] | DataArray, psi_salinity: ndarray[tuple[Any, ...], dtype[float]] | DataArray, tcal: float, sigma_psi_temperature: ndarray[tuple[Any, ...], dtype[float]] | DataArray | None = None, sigma_psi_salinity: ndarray[tuple[Any, ...], dtype[float]] | DataArray | None = None, return_uncertainty: bool = False, unc_temperature: float = 0.0, unc_salinity: float = 0.0) → ndarray[tuple[Any, ...], dtype[float]] | DataArray[source]

Correct the measured absorption or attenuation coefficient for temperature and salinity. This function works on singletons and xarray DataArrays.

Parameters:

measured – The measured absorption or attenuation coefficient.
temperature – The temperature of the water in degrees Celsius.
salinity – The salinity of the water in PSU.
psi_temperature – The temperature coefficients from ACSTSCor, subset by wavelength equivalent to the corresponding measured coefficient wavelengths.
psi_salinity – The temperature coefficients from ACSTSCor, subset by wavelength equivalent to the corresponding measured coefficient wavelengths.
tcal – The tcal value from the device file.

Returns:

TS-corrected absorption or attenuation coefficient in m^-1.

acspype.processing.zero_shift_correction(mts: ndarray[tuple[Any, ...], dtype[float]] | DataArray) → ndarray[tuple[Any, ...], dtype[float]] | DataArray[source]

Shift values between [-0.005, 0] to 0. This function works both for 1D arrays and xarray DataArrays.

Parameters:: mts – The TS-corrected absorption or attenuation coefficient
Returns:: The shifted TS-corrected absorption or attenuation coefficient.

acspype.qaqc module

This module contains functions for ACS data quality assurance and quality control (QAQC). QAQC tests use QARTOD-style flagging and meanings, unless an alternative is specified.

class acspype.qaqc.FLAG[source]

Bases: object

Flag values for the ACS QAQC tests. The following flags follow the QARTOD flag meanings.

Parameters:

OK – Indicates that the data passed the test.
PASS – Indicates that the data passed the test.
NOT_EVALUATED – Indicates that the data was not evaluated. Appearance of this flag indicates a problem with the test function and not the data. It defeats the purpose of running a QAQC if the data is not evaluated.
SUSPECT – Indicates that the data is suspect. This flag indicates that the data should be reviewed more closely.
HIGH_INTEREST – Indicates that the data is of high interest. Although it uses the same flag value as SUSPECT, the context of the test and environmental conditions should be considered when reviewing the data.
FAIL – Indicates that the data failed the test conditions.
BAD – Indicates that the data failed the test conditions.
MISSING_DATA – Indicates that the input data or ancillary data is missing (NaN), resulting in data that cannot be used.

BAD: int = 4

FAIL: int = 4

HIGH_INTEREST: int = 3

MISSING_DATA: int = 9

NOT_EVALUATED: int = 2

OK: int = 1

PASS: int = 1

SUSPECT: int = 3

acspype.qaqc.a_gt_c_test(absorption: DataArray, attenuation: DataArray) → DataArray[source]

Assess if the absorption is greater than the attenuation. Having absorption (a) greater than attenuation (c) is (mostly) impossible. This test can be run on uncorrected data to help assess sensor drift. It is also recommended that this test be run on scattering corrected absorption with ts-corrected attenuation. This test is not related to QARTOD, but is a custom test based on reality checks in the ACS protocol document. This test is only intended for use with xarray.DataArray objects.

Parameters:

absorption – Absorption data. Either uncorrected a or a_mts_scatcorr.
attenuation – Attenuation data. Either uncorrected c or ts corrected c.

Returns:

Flag indicating if absorption is greater than attenuation.

acspype.qaqc.blanket_gross_range_test(nd_gross_range_results: DataArray, wavelength_dim: str, suspect_threshold: float, fail_threshold: float, include_suspect_flags: bool = False) → DataArray[source]

Assess the validity of the gross range test results as a blanket flag for the spectra. This is an experimental test. The test considers the number of wavelength bins that were previously flagged as suspect or fail for exceeding the gross range limits. If X percent of wavelength bins were flagged as fail (or fail/suspect) and exceed the user-defined fail threshold, then the entire spectrum is flagged as fail. If the percentage is between the suspect and fail thresholds then the spectra is flagged as suspect. The threshold represents the percentage threshold between 0-1 (0-100%).

Parameters:

nd_gross_range_results – The N-dimensional results from the gross range test. Effectively a single flag for every value along the time/wavelength dimension.
wavelength_dim – The wavelength dimension of the results.
suspect_threshold – The percentage threshold for suspect data.
fail_threshold – The percentage threshold for fail data.
include_suspect_flags – If True, include suspect flags along the spectra in the total fail.

Returns:

A blanket flag for the spectra, which maintains the same shape as the time dimension.

acspype.qaqc.discontinuity_offset_test(discontinuity_offset: DataArray, median_multiplier: float = 3, fail_threshold: float = 10) → DataArray[source]

Flag a discontinuity offset value as pass or suspect. This is a custom test that uses a multiple of the median of the offset to assess if the offset is acceptable. Spectra with significantly large discontinuity offsets are likely to be of poor quality, but should be assessed using other QAQC tests before being discarded, such as the a_gt_c_test, which may likely confirm the quality of the data.

This function is only intended for use with xarray.DataArray objects.

Parameters:

discontinuity_offset – The discontinuity offset value for absorption or attenuation.
median_multiplier – The multiplier of the median. Values within the multiplier range are deemed acceptable.
fail_threshold – The threshold for flagging the offset as fail. Default is 10, or the maximum value of the ACS range.

Returns:

The flag of the discontinuity offset, which maintains the same size as the time dimension.

acspype.qaqc.elapsed_time_test(elapsed_time: int | DataArray, fail_threshold: int = 60000, suspect_threshold: int = 180000) → int | DataArray[source]

Assess the elapsed time since the instrument was turned on. The elapsed time is always output from the ACS as the number of milliseconds since the instrument first received power. This is a custom test that is based on information in the ACS manual. Generally, the ACS takes 5-10 minutes to warm-up. Waiting this long may not be practical in some situations. It is recommended to flag the first minute of data as poor quality and then minutes 2-3 as suspect in applications where the ACS is on for less than 10 minutes at a time.

Parameters:

elapsed_time – The elapsed time parsed from an ACS packet. Represents the number of milliseconds that have passed since the ACS was turned on.
fail_threshold – The amount of time, from initial instrument power on in milliseconds, where data is considered to be of poor quality.
suspect_threshold – The amount of time in milliseconds between the fail and suspect threshold where data is considered to be suspect.

Returns:

Flag indicating pass, suspect, or fail.

acspype.qaqc.gap_test(now: datetime, time_stmp: datetime, record_length: int, buffer_length: int | None, time_inc: float = 0.25) → int[source]

Assess the gap between the current time and the timestamp of the packet. This is a modified form of the generic QARTOD gap test.

Parameters:

now – The time at which the test is being run. Must be compatibile with a datetime.datetime object.
time_stmp – The timestamp of the packet. Must be compatible with a datetime.datetime object.
buffer_length – The number of bytes in the serial buffer. If None, the corresponding section of the test is skipped. Determined through the protected attribute “_buffer” in the ACSStream class.
record_length – The record length of the packet. This is the expected number of bytes in the packet that is within each ACS packet header.
time_inc – The time increment to use for assessing timestamp gaps. Default is 0.25 seconds, which is the approximate time that it takes the ACS to send a complete packet.

Returns:

A flag indicating pass or fail.

acspype.qaqc.gross_range_test(mts: DataArray, sensor_min: float = -0.005, sensor_max: float = 12.0, user_min: float = 0.001, user_max: float = 10) → DataArray[source]

Assess if the TS-corrected data are within the limitations of the instrument. From the ACS manual (Rev I), values of 12 m^-1 may occur at shorter wavelengths. The valid dynamic range of the sensor is 0-10 m^-1.

The return is a set of flags in the same shape as the input data. Note that fail flags may occur in the red wavelengths of absorption spectrum consecutively toward the end of each spectrum, which does not necessarily make the entire spectrum of poor quality. Users should consider the pass/fail state of the rest of the spectrum before making a decision on the validity of the data.

Parameters:

mts – TS-corrected absorption or attenuation.
sensor_min – The minimum sensor range. Default is -0.005, which is effectively 0 as described in the manual.
sensor_max – The maximum sensor range. Default is 12.00, which is a value that is mentioned in Rev I of the user’s guide.
user_min – Operator set minimum for flagging suspect data. Default is 0.001 which is the minimum of the dynamic range.
user_max – Operator set maximum for flagging suspect data. Default is 10 which is the maximum of the dynamic range.

Returns:

Flag indicating pass, fail, or suspect.

acspype.qaqc.inf_nan_test(uncorrected: ndarray[tuple[Any, ...], dtype[float]] | DataArray) → int | DataArray[source]

Assess if the uncorrected data contains any NaN or Inf values. This is a custom test that can be used to identify instances where the reference counts are zero, which produces mathematical errors in the conversion to geophysical units. Zero reference counts happen occasionally and can probably be discarded during analysis. Repeated samples containing zero reference counts may indicate a problem with the instrument.

Parameters:: uncorrected – Uncorrected a or c data.
Returns:: Flag indicating pass or fail.

acspype.qaqc.internal_temperature_test(internal_temperature: float | DataArray, dev: ACSDev) → float | DataArray[source]

Assess the internal temperature of the ACS. This is a custom test that is based on information within the manual and the device file. The ACS is calibrated at specific temperature bins, which are listed in the device file. Typically, the internal temperature correction is linearly interpolated from the delta T values in the device file. If the internal temperature of the sensor exceeds the range of those temperature bins, then data are considered to be suspect.

Parameters:

internal_temperature – The internal temperature of the ACS.
dev – The device file for the ACS as an ACSDev object.

Returns:

Flag indicating pass or suspect.

acspype.qaqc.rolling_variance_test(mts: DataArray, use_mean: str = 'rolling', window_size: int = 241, exceedance_threshold: float = 0.25, min_periods: int | None = 1) → DataArray[source]

Apply a rolling variance test to a measured absorption or attenuation coefficient.

Parameters:

mts – The measured absorption or attenuation coefficient. For absorption the recommendation is to use a scattering corrected measurement (e.g. a_mts_proportional). For attenuation the recommendation is to use the TS-corrected measurement.
use_mean – Indicates whether to use the timeseries mean or a rolling window mean along the time dimension. If ‘rolling’, the rolling mean at each wavelength bin along the window size is used. If ‘timeseries’, the timeseries mean at each wavelength bin is used.
window_size – The centered window size for the variance window and for the mean window, if use_mean = ‘timeseries’.
exceedance_threshold – The percentage of the mean that the variance must exceed to be flagged as suspect. Represented as a value between 0-1 (0-100%).
min_periods – The minimum number of periods to apply this test. Only applies if use_mean = ‘rolling’. Same functionality as the min_periods argument for xarray.DataArray.rolling. The default is 1, which generally means that this test is not applied to the first window in the timeseries.

Returns:

A flag indicating SUSPECT or PASS.

acspype.qaqc.syntax_test(full_packet: bytearray) → int[source]

Test the syntax of an ACS packet. This test expands on the QARTOD syntax test. Failure of this test indicates that the packet is malformed or incomplete.

Parameters:: full_packet – A full packet from the ACS, including registration bytes and the pad byte. See acspype.core for the static syntax of the header and tail of each packet. The unpack_packet function in the acspype.packet module shows how to dynamically unpack the packet using the packet header.
Returns:: Flag indicating pass or fail.

acspype.experimental module

acspype.experimental.compute_alh_676(a_p_650: DataArray, a_p_676: DataArray, a_p_715: DataArray) → DataArray[source]

Compute absorption line height at 676 nm via Boss et al, 2007. https://link.springer.com/chapter/10.1007/978-1-4020-5824-0_9

Parameters:

a_p_650 – a_p data at 650 nm.
a_p_676 – a_p data at 676 nm.
a_p_715 – a_p data at 715 nm.

Returns:

Absorption line height at 676 nm in m^-1.

acspype.experimental.estimate_chl(a_p_650: DataArray, a_p_676: DataArray, a_p_715: DataArray, alh_coeff: float = 0.0104) → DataArray[source]

Estimate chlorophyll-a from absorption line height via Roesler and Barnard 2013. https://doi.org/10.1016/j.mio.2013.12.003

This function has only been tested on xarray.Datasets.

Parameters:

a_p_650 – a_p data at 650 nm.
a_p_676 – a_p data at 676 nm.
a_p_715 – a_p data at 715 nm.
alh_coeff – (a_p line height coefficient value, default is 0.0104, which is the average from Table 1 in Roesler and Barnard 2013.

Returns:

Chlorophyll-a concentration in mg/m^3.

acspype.experimental.estimate_poc(c_p_660: DataArray, slope_offset: str | tuple | list) → DataArray[source]

Compute particulate organic carbon from particle beam attenuation at 660nm using a provided slope and offset. If the slope_offset value is a tuple or list, the first value is the slope and the second value is the offset.

If the slope_offset value is a string, a predefined slope and offset will be used. Strings represent the first author and the year of the publication.

Note: If supplying your own slope and offset, please ensure that the units are correct.

Parameters:

c_p_660 – particulate attenuation (c_p) at 660 nm.
slope_offset – A string indicating the literary source for the slope and offset or a tuple or list containing the slope and offset values.

Returns:

Estimated particulate organic carbon in mg/m3.

acspype.dev module

This module contains functions and classes for parsing and accessing data from ACS .dev files. Currently, only structure version 3 is supported.

class acspype.dev.ACSDev(filepath: str)[source]

Bases: object

A class for parsing ACS .dev files and putting them into a format that is easier to work with for larger or multiple file datasets.

Generally, users will not call individual functions, but rather use the class to obtain attributes, which are created at class instantiation or convert the data to an xarray dataset using the to_xarray function.

to_xarray() → Dataset[source]

Convert the parsed .dev file files to an xarray dataset

Returns:: An appropriately dimensioned xarray dataset containing device file files.

acspype.tscor module

class acspype.tscor.ACSTS4CorReader(filepath: str)[source]

Bases: object

A class for parsing ACS TS4.cor files and putting them into a format that is easier to work with for larger or multiple file datasets.

This class can be used if users want to import the TS4.cor file themselves and don’t want to use values interpolated by acspype.

to_xarray()[source]

Convert the TS4COR class attributes to an xarray.Dataset.

Returns:: A dimensioned xarray.Dataset.

class acspype.tscor.ACSTSCor[source]

Bases: object

A wrapper class for the TS4COR and TS4COR_SIGMA dictionaries that are included with acspype.

to_xarray()[source]

Convert the TS4COR class attributes to an xarray.Dataset.

Returns:: A dimensioned xarray.Dataset.

acspype.utils.core module

This module contains functions that are beneficial to use alongside acspype, but do not support acspype directly.

acspype.utils.core.find_acs_port(baudrate: int = 115200, timeout: int = 1, check_length: int = 1) → str[source]

Iterate through available serial ports and check the incoming data for the ACS registration bytes. The first port with ACS registration bytes is returned. This function has not been tested with multiple ACS sensors connected to the same computer.

Parameters:

baudrate – The baudrate for the ACS connection. Default is 115200 bps and does not need to be changed.
timeout – The timeout for the serial connection. Default is 1 second.
check_length – The amount of time in seconds to collect data from each serial port in hopes of receiving the ACS registration bytes.

Returns:

The operating system serial port as a string. Can be used by pyserial for future connections.

acspype.utils.core.list_available_ports() → list[source]

List available serial ports for the native operating system.

Returns:: A list of available serial port names.

acspype.utils.ooi module

This module contains helper functions for accessing and working with OOI ACS data. The OOI has named all their ACS sensors OPTAA.

acspype.utils.ooi.download_and_load_goldcopy(thredds_fileserver_url: str, save_dir: str = 'ooi_data/') → Dataset[source]

Download and load a dataset from the OOI THREDDS GoldCopy catalog. Sometimes data accessed through the OpenDAP URL is incomplete, so downloading is recommended.

Parameters:

thredds_fileserver_url – A THREDDS GoldCopy URL. Must contain fileServer, not dodsC.
save_dir – The directory to save the dataset to.

Returns:

The xarray dataset.

acspype.utils.ooi.get_ooi_optaa_cal(ds: Dataset) → object[source]

Find an OOI OPTAA calibration for a given dataset.

Parameters:: ds – The OOI OPTAA dataset. It must contain data from a single deployment. It is recommended to not change the format of the file until after running this function.
Returns:: A ACSDev-like object containing the calibration data.

acspype.utils.ooi.reformat_ooi_optaa(ds: Dataset) → Dataset[source]

Reformat an OOI OPTAA dataset to make it more compatible with acspype.

Parameters:: ds – An OOI OPTAA dataset.
Returns:: A reformatted and renamed OOI OPTAA dataset.