I am a student, currently working on a project comparing automotive paint samples aged in water samples over a month. I've been given access to ATR-FTIR (using transmittance) to test the epoxy clear coat layer of these chips. The issue is, I have about 300 chip samples, and I've just found out I have no access to a database of references for these samples, or access to any CSV data files from the software we use. I have hundreds of FTIR spectra and I'm at a loss for how to process this raw data into something that can convey the amount of chemical degeneration of the epoxy over time, that can be presented in a report. From what I can see, the most I can do is compare peaks, but overall the ageing has not massively impacted peaks, there's still peaks where I would expect them to be comparing controls to samples, the only difference is minor changes in the peak location and height. I've not been given any resources in my course this far outside of identification of functional groups, nothing to say of how to further process any results. I was hoping to get CSV files to process but that's not really an option.

Any help would be appreciated, I've been racking my brain for months over this, I'm a Biology major, so I don't have much experience.

The linear response observed in high-end instrumentation should be a baseline expectation for scientific measurement. However, in the commercial spectroscopy market, vendors often "gloss over" non-linear hardware performance by promising a software-side fix.

FIGURE 1. Linearity Comparison under varying exposure.PART A: TCD1304 with High-Slew AFE and 16-bit Differential ADC. This demonstrates an inherently linear response driven by signal chain integrity. Even with standard timer-driven clocks, the high-bandwidth AFE prevents the dV/dt \"rounding\" of peaks, ensuring the data remains metrologically honest (0.19% INL).PART B: Popular Commercial USB Spectrometer. This displays the characteristic slew-limited \"S-curve.\" This illustrates a hardware bottleneck where the internal signal chain cannot track rapid transitions—a failure of their design that global software correction cannot truly rectify.

1. The Mathematical "Canard"

As seen in the figure above, the "S-curve" response on the right (b) is often presented as a minor calibration hurdle. This is a metrological "canard" that ignores the failure of their design. While vendors claim these errors can be fixed in post-processing, they are actually masking a signal chain that cannot keep up with the physical reality of the light being measured.

There is a widespread practice of claiming that hardware non-linearity can be corrected through the pixel-wise application of a global polynomial:

S(Aₙ) = a₀ + a₁Aₙ + a₂Aₙ² + a₃Aₙ³ + ...

In this model, Aₙ is the raw intensity of pixel n, and the coefficients aᵢ are assumed to be constant across the entire array. While mathematically convenient, this model is fundamentally out of sync with the reality of any instrument where the response is limited by bandwidth or slew.

2. The Smoking Gun: The Mercury Line Test

To demonstrate the failure of this global model, we evaluated the coefficients required to linearize the response of the commercial instrument shown in (b) using two mercury (Hg) lines: the broad 546nm peak and the narrow 435nm peak.

Inconsistency of Polynomial Correction

Spectral Line	a0​ (Constant)	a1​ (x1)	a2​ (x2)	a3​ (x3)
Broad Peak (546nm)	-0.0112	1.1692	-0.6160	0.6320
Narrow Peak (435nm)	-0.0343	1.5597	-2.0373	1.7793

The results are stark: the narrow, sharper peak requires second- and third-order coefficients that are nearly triple those of the broader peak. Furthermore, the 23.1 mV gap in the constant term (a0​) confirms that the "correction" is not a static property of the pixel, but a dynamic function of the signal’s geometry.

3. The Physical "Why": Slew-Limited Response

The divergence in these coefficients proves that the error is tied to the steepness (the dV/dt) of the spectral line. In a slew-limited instrument, non-linearity is a dynamic result of how the signal changes across a set of pixels. Because a global polynomial is static, it is mathematically irreconcilable with the physical behavior of the sensor signal chain.

4. Hardware Integrity Over Software "Gloss"

The goal of our open-science project is to provide metrological and radiometric truth at the hardware level, eliminating the need for mathematical "guesses". By using a hardware-locked state machine architecture and a sub-millivolt noise floor, we have achieved 0.19% Integral Non-Linearity (INL) without software trickery.

Our mission is to provide affordable, high-fidelity hardware to under-resourced scientists who cannot afford to rely on "black box" commercial corrections that fail under scrutiny.

5. The Proof of Radiometric Truth

As demonstrated in Figure 2, the difference between a high-integrity signal chain and a software-dependent one is unmistakable. In our instrument (a), normalizing the spectral traces by exposure time results in a perfect overlay across the dynamic range. This suggests that the system is capturing radiometric truth; the data scales linearly with the photon count because the high-slew AFE and 16-bit differential ADC preserve the peak geometry without "funny business" in the baseline. Conversely, the commercial unit (b) shows significant divergence. This inconsistency confirms that their "S-curve" is not a simple scaling factor but a dynamic distortion that varies with intensity, making it physically impossible to reconcile through standard normalization or global polynomial correction.

FIGURE 2. Intensity Normalization and Baseline Stability.PART A: TCD1304 with High-Slew AFE/ADC. When normalized by exposure time, all spectral traces OVERLAY PERFECTLY across the entire dynamic range. This confirms the signal chain is metrologically honest, maintaining a stable dark floor and linear scaling (0.19% INL) without \"funny business\" in the baseline.PART B: Popular Commercial USB Spectrometer. Even with dark subtraction, the normalized intensities remain HIGHLY INCONSISTENT. The failure of the internal design to track peak geometry leads to a divergent response where the data cannot be reconciled simply by dividing by exposure time.

6. Closing: Open Science in Practice

The goal of this project has been to raise the bar and provide high-fidelity, linear spectroscopy in an open-source design at a fraction of the cost of commercial units. By prioritizing hardware integrity and radiometric truth, we have moved away from metrological fictions and toward an instrument that any scientist can trust.

Full documentation, including the hardware-locked state machine architecture and 0.19% INL performance data, is available on GitHub for the community to pick up, use in their research and advance the standard of open-source instrumentation:https://github.com/drmcnelson/TCD1304-Sensor-Device-with-Linear-Response-and-16-Bit-Differential-ADC