High throughput correlative analysis of instrumentation used in atmospheric chemistry

Colleen Marciel F. Rosales, Ph.D((.)^(1, 2))
(( )^1)University of California in Davis
((  )^2)OpenAQ
cfrosales@ucdavis.edu / colleen@openaq.org

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

Outline

Scientific Introduction

Air Pollution:  Problem Description

Data:  Air quality data & atmospheric chemistry instrumentation

Data organization

Dataset Functionality

Data visualization

Inter-instrumental comparisons

Internal checks: inter-elemental comparisons

Statistical analysis: Correlations

Air Pollution

Air pollution is the second leading cause of premature mortality.

1 in 8 global deaths are linked to air pollution.

Particulate matter ((Subscript[PM, 2.5])): mixture of solid and liquid “fine inhalable particles” in the air

Image: US EPA (PM Pollution basics)

Air quality data diversity is an example of how environmental science is moving to “big data”.

Access to Real-time Air Quality Data

The largest open-source air quality data platform whose mission is to fight air inequality through open data.

Chemical Speciation Network (CSN)

A long-term (since 2000) national monitoring program of the EPA (focused on public health effects)

Regulatory Compliance:  1997 (Subscript[PM, 2.5]) National Ambient Air Quality Standards and 1999 Regional Haze Rule

Laboratory analysis performed by the UC Davis Air Quality Research Center

More than 160+ urban sampling sites across the US

Sample collection for (Subscript[PM, 2.5]):  24 hours in filter medium every three or six days

Composition:  ions, organic and elemental carbon, elements

Chemical analysis of particulate matter ((Subscript[PM, 2.5]))

Chemical Speciation Network

Elemental analysis using two multi-elemental* techniques

*one sample = 33+ elements can be detected at once

Energy Dispersive X-ray Fluorescence Spectroscopy (ED-XRF)

Inductively Coupled Plasma Mass Spectrometry (ICP-MS)

How well can one instrument reproduce measurements from the other?

Q: How well can one instrument reproduce measurements from the other instrument?

Raw Data:  Disorganized or Multi-sourced
		↓
		Data organization using Dataset
		↓
		Visualization + Statistical Analysis
		      ↓
		       Element-by-element Instrumental Comparisons

Data Organization

“Big” (Multidimensional) Data:  355 samples x 29 elements x 2 instruments

->  Make use of Dataset

Allows for human-readable keyword access rather than (or in addition to) numerical indexing

Similar to dictionaries in other programming languages

Data Readability

Data Computability

Verify the organization is as intended, element-by-element:

Table[allValsTb[[i]][All,"Parameter"][[1]],{i,1,Length@allValsTb}]//PositionIndex

Extract the data for one of the elements (e.g. Sodium):

allValsTb[[1]][All,"wholeFilterConcAQMT"]//Normal

Metadata Analysis

For a given Dataset, assess each data point’s collection site:

SortBy[samplingLocations,"SiteName"]

Assess geographic diversity by tally:

ReverseSortBy[Tally@samplingLocations[All,"SiteName"],Last]

Data visualization

Data organization using Dataset
	↓
 	Inter-instrumental (ED-XRF vs ICPMS) ← Visualization + stat. analysis → Inter-elemental

I. Sanity Check:  Inter-elemental Visual Comparison

Certain elements are known to be correlated (co-emitted) in the environment, e.g., for traffic-related dust:

Cu, Zn, Ba, Fe, Ti from brake wear

Zn and Fe from tire wear

1. Create a Correlation MatrixPlot

MatrixPlot[Correlation[matrixICPBatch28elem],]

2. Visualize the Underlying Data:  WFR - PairwiseScatterPlot

Bulk-visualize the raw data, element-wise, using a WFR function:

PairwiseScatterPlot[logMatrixICPBatch28elem, ]

3. Combine individual ListLogLogPlots (Row[], Column[], Grid[])

Create custom correlative plots using System functions:

II. Inter-Instrumental Comparisons

Visually compare 1:1 reproducibility between techniques for specific elements:

Black dashed lines are unity lines.  However, we need a more quantitative way to compare the two instruments.

Inter-instrumental Correlation

Q: How well can one instrument reproduce another’s?

Theoretically, instruments with 1:1 reproducibility will lie along the unity line, but each instrument is subject to systematic and random errors. What then is the optimal technique to look at this correlation?

1. Model I Technique - Ordinary Least Squares (Fit, LinearModelFit, etc.)

Preliminary look, but assumes strictly “Y on X” -- not true in this case (Y and X are interchangeable)

2. Model II Techniques - Symmetric “Y/X” - Wolfram Function Repository

Theil-Sen (TheilSenLine Resource Function from the Wolfram Function Repository)

Wolfram Alpha Math team is working on WFRs for:

Standard (Reduced) Major Axis

Major Axis Regression

Bartlett Three-Group Method

Key Takeaways

The WL & WFR contain many broad computational and scientific functionality supporting and atmospheric chemistry “big data” analyses:

Data Organization: Dataset

Data Visualization: MatrixPlot, ListLogLogPlot, PairwiseScatterPlot

Statistical Analysis:  Regression Tools

System:  Fit, LinearModelFit, etc.

WFR:  Standard Major Axis, Major Axis, BartlettThreeGroupFit, TheilSenLine

Relevant talks & workshops:

Using Resource Functions for K–College Math (Oct 19, 11:30-12 PM)

Active Learning Notebooks for College Mathematics (Oct 19, 11:00 AM - 12:00 PM)

Probability and Statistics Office Hour (Oct 19, 3:30 - 4:30 PM)

Wolfram Alpha Notebook Edition Office Hour (Oct 20, 3:30 PM)