Analysis of Brent Crude Futures Data Visualization

Built this in matplotlib using 1.89 million rows of 1-minute Brent crude futures data (2019–2026), resampled to daily closes. A few deliberate design decisions worth discussing.

Color Encoding

The central choice was year-based color encoding rather than a single continuous line. On a traditional monochrome chart, the COVID crash and Ukraine spike read as anomalies disrupting an otherwise continuous series. With year coloring, something more interesting emerges: the post-COVID recovery (2020–2021) and the war-premium unwind (2022–2023) cluster visually as distinct regimes rather than noise around a trend. The structure of how prices moved — not just where — becomes legible.

Background and Canvas

Dark background was deliberate. Commodity chart colors tend to be muted on white; dark canvas lets the year hues breathe without competing with gridlines.

Layout and Volume Analysis

Dual-panel layout (price above, volume below, color-matched by year): volume tells a different story than price alone. The 2022 spike looks dramatic on price; volume in that period was actually thinner than 2020, which changes the interpretation.

Key Anchors

• Period low $19.60 on April 21, 2020 — COVID demand collapse.
• Period high $124.56 on March 8, 2022 — Ukraine invasion.
• 2023–2024 was a slow grind from ~$95 back toward $70.
• 2026 creeping above $100 again.

Suggestions for Improvement

One thing I'd change: the vertical event markers are labeled with raw date strings rather than short annotations — they add precision but reduce at-a-glance readability.

Discussion Question

What's your rule of thumb for when categorical color encoding earns its complexity cost versus when it just adds visual noise?

Resampling Minute-by-Minute Data into Hourly Summaries

If you've ever needed to convert minute-by-minute data into hourly summaries, that's called resampling — aggregating fine-grained time series data into coarser time buckets. It sounds simple. It's not always.

Here's the trap I fell into. I had a table with timestamps, a category column (Product A and Product B in the same dataset), and a price column. I wanted hourly aggregates per category. The catch: a new category sometimes starts mid-hour. If you just group by the time bucket, you silently mix rows from two different categories into the same aggregate. No errors. Wrong numbers.

In pandas, I was doing a groupby + resample combo and getting subtle corruptions at exactly those category-switch boundaries. Took me embarrassingly long to notice.

Polars has a clean answer: group_by_dynamic with a group_by parameter. It partitions by your category column first, then builds time windows independently per category. No bleed-over.

>

Output correctly starts a fresh 1-hour window for each category even if the switch happens at 14:23 instead of 14:00.

I knew pandas well before Polars, and this is where Polars' explicit API saved me from a silent bug pandas would have happily let through.

What Polars gotchas have tripped you up? Especially curious if anyone else has hit silent correctness issues like this one.

Methodological Issue Inquiry

I ran into a methodological issue I'd love input on.

I had a variable showing Pearson correlation of roughly 0.12 with my outcome variable modest but consistent across the sample. Based on that alone, it looked like a potentially useful predictor.

The problem appeared when I introduced a one-step time delay: using the value at t-1 to predict at t, the relationship essentially disappeared. The correlation was contemporaneous , it described the current state of the system well, but carried no forward-looking information once you respected the temporal ordering of the data.

This got me thinking about a distinction I'm not sure how to formalize: the difference between a variable that's correlated with the current state of a system versus one that's genuinely predictive of future state transitions. In my case, the variable seems to be the former: a descriptor, not a predictor.

I looked into Granger causality as a framework for this, but didn't fully apply it, partly because the setup didn't cleanly fit the assumptions, and partly because I wasn't sure it addresses this specific distinction or just formalizes precedence.

Is there a standard statistical test or framework for diagnosing this? Something that goes beyond checking lagged correlations and more formally separates "state variable" from "predictive signal"?

Polars Pipeline Bug Report

Built a computed column in a Polars pipeline. Downstream metrics looked clean for a week — consistent, trending right, passing all existing tests.

Then I found the bug. Fixed version: metric collapses. The pipeline was technically running, just producing confidently wrong numbers the entire time.

The column was a run-length counter: consecutive rows meeting a condition, grouped by a category key, resetting to zero on a miss. Standard pattern.

import polars as pl

df = pl.DataFrame({
    "group": ["A", "A", "A", "A"],
    "cond": [True, True, False, True],
})

df = df.with_columns(
    pl.when(pl.col("cond"))
    .then(pl.lit(1).cum_sum().over("group"))
    .otherwise(0)
    .alias("run_len")
)

Expected after the False row: reset to 0, then count from 1. Actual: cum_sum carries through the False row, counter continues from 3.

No exception. No warning. No null. Just wrong values that are systematically larger than correct ones — which made every downstream metric look better than it was.

This is the failure mode that kills pipeline trust: not a crash, not a null propagation, not a schema mismatch. A logically valid operation that silently produces plausible-but-wrong intermediate state, which flows downstream unchallenged.

What's your actual line of defense here? Unit tests on the transformation logic, dbt tests on output columns, Great Expectations range checks, property-based tests with Hypothesis? Specifically curious how people handle validation of intermediate computed columns — the ones that never surface to end consumers but everything downstream depends on.

Background: I work in systematic energy research. Over the past several months, I've been building out a Brent-specific research framework, partly because every published term structure model I could find was calibrated on WTI. The assumption baked into most of this literature is that Brent and WTI curve dynamics are interchangeable enough that WTI findings generalize. I wanted to test whether that held up empirically. Short answer: it doesn't, and the divergence is larger than I expected.

A few specific observations from the period I examined (not going to drop numbers since this is ongoing work, but I can speak to the directional findings):

• Front-month basis behavior differs materially between the two benchmarks. WTI front-month reacts to Cushing-specific storage dynamics that simply don't apply to Brent. Treating them as interchangeable loses this.
• Seasonal norm deviations in Brent carry different information than in WTI. The magnitude and persistence differ across the strip.
• Regime transitions (contango/backwardation shifts) don't sync between the two in the way most public models imply.

Part of what surfaced this was using an LLM-based approach to classify curve shape regimes rather than relying solely on a threshold rule. The regime labels it produces don't match what a volatility-normalized time-series model would give you — and in back-examination, the LLM labels were more consistent with what actually mattered for forward returns.

I'm not aware of published work that does this specifically on Brent curve shape (most of the LLM + commodity literature uses sentiment scoring for directional forecasting, not regime classification on curve geometry). Happy to be corrected if someone knows of something I missed.

For those working on systematic energy: does the WTI-Brent non-transferability show up in your live models, or is this a research-only observation that gets priced away in execution?