kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
AbspectroscoPY, a Python toolbox for absorbance-based sensor data in water quality monitoring
KTH, School of Engineering Sciences (SCI), Engineering Mechanics.ORCID iD: 0000-0002-2460-578X
Show others and affiliations
2022 (English)In: Environmental Science: Water Research & Technology, ISSN 2053-1400, E-ISSN 2053-1419, Vol. 8, no 4, p. 836-848Article in journal (Refereed) Published
Abstract [en]

The long-term trend of increasing natural organic matter (NOM) in boreal and north European surface waters represents an economic and environmental challenge for drinking water treatment plants (DWTPs). High-frequency measurements from absorbance-based online spectrophotometers are often used in modern DWTPs to measure the chromophoric fraction of dissolved organic matter (CDOM) over time. These data contain valuable information that can be used to optimise NOM removal at various stages of treatment and/or diagnose the causes of underperformance at the DWTP. However, automated monitoring systems generate large datasets that need careful preprocessing, followed by variable selection and signal processing before interpretation. In this work we introduce AbspectroscoPY (“Absorbance spectroscopic analysis in Python”), a Python toolbox for processing time-series datasets collected by in situ spectrophotometers. The toolbox addresses some of the main challenges in data preprocessing by handling duplicates, systematic time shifts, baseline corrections and outliers. It contains automated functions to compute a range of spectral metrics for the time-series data, including absorbance ratios, exponential fits, slope ratios and spectral slope curves. To demonstrate its utility, AbspectroscoPY was applied to 15-month datasets from three online spectrophotometers in a drinking water treatment plant. Despite only small variations in surface water quality over the time period, variability in the spectrophotometric profiles of treated water could be identified, quantified and related to lake turnover or operational changes in the DWTP. This toolbox represents a step toward automated early warning systems for detecting and responding to potential threats to treatment performance caused by rapid changes in incoming water quality. 

Place, publisher, year, edition, pages
Royal Society of Chemistry (RSC) , 2022. Vol. 8, no 4, p. 836-848
Keywords [en]
Automation, Biogeochemistry, Curve fitting, Data handling, High level languages, Large dataset, Monitoring, Organic compounds, Potable water, Signal processing, Spectroscopic analysis, Surface waters, Time series, Time series analysis, Water quality, Water treatment, Water treatment plants, Absorbances, Dissolved organic matters, Drinking water treatment plants, Economic challenges, Environmental challenges, High-frequency measurement, Long-term trend, Natural organic matters, Sensors data, Water quality monitoring, Python, drinking water, organic matter, pollutant removal, removal experiment, surface water, water treatment plant
National Category
Oceanography, Hydrology and Water Resources
Identifiers
URN: urn:nbn:se:kth:diva-323505DOI: 10.1039/d1ew00416fISI: 000759677000001Scopus ID: 2-s2.0-85129062546OAI: oai:DiVA.org:kth-323505DiVA, id: diva2:1734468
Note

QC 20230206

Available from: 2023-02-06 Created: 2023-02-06 Last updated: 2024-09-24Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Kern, J. Simon

Search in DiVA

By author/editor
Kern, J. Simon
By organisation
Engineering Mechanics
In the same journal
Environmental Science: Water Research & Technology
Oceanography, Hydrology and Water Resources

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 112 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf