u/triton_one

Why Solana historical data is slow (and how we fixed it)

Why Solana historical data is slow (and how we fixed it)

Most people assume slow Solana history is just how it works.

It's not. It's a storage problem.

Bigtable has been the default backend for Solana historical data across most of the industry since genesis, but it was never built for Solana's access patterns. Random lookups are slow, adapting it is nearly impossible, and the egress costs at scale get passed straight to the builders.

After enough time working around Bigtable's limitations, we decided the only real fix was to replace it entirely. 

Hydrant is Triton’s new Clickhouse-based Rust backend, designed for how Solana history actually gets queried:

  • Storage sorted for each query pattern so requests skip irrelevant data entirely
  • Materialised views built at write time, pre-sorted for common lookups before your query hits the database
  • Dedicated routing for high-traffic accounts like USDC so they don't bottleneck everyone else
  • In-memory cache returning recent data at 0.57 ms P50

Three methods live on our infrastructure now with flat pricing across the full ledger: $0.08/GB + $10/M requests, regardless of epoch. 

Soon going open-source under AGPL.

Deep dive with the full architecture → https://blog.triton.one/introducing-hydrant-solanas-full-ledger-queryable-in-milliseconds/

https://preview.redd.it/xyaqooht6vwg1.png?width=2460&format=png&auto=webp&s=9cfdf491383411b46b5f17a7292e09e3f175429a

reddit.com
u/triton_one — 18 hours ago
▲ 6 r/solana

NEW! Steamboat: 20x faster getProgramAccounts from indexed lookups on shared infrastructure

If you've ever called getProgramAccounts on a program with millions of accounts, you know the pain. Every call scans the entire account set, and you get throttled or timed out.

"Build an index" is a great solution, but it ignores the cost. Until now, you needed a dedicated Agave node at $3,000+/month just to serve reads, and most teams don't have that budget.

We built Steamboat to solve this. It decouples account reads from the validator entirely and serves getProgramAccounts, getTokenAccountsByOwner, and getTokenAccountsByDelegate from PostgreSQL-based indexes on shared infrastructure.

How it works:

  1. Bootstraps from a full Solana snapshot, so the account state is complete from the first query
  2. Subscribes to a live gRPC stream (Yellowstone Dragon's Mouth) to keep the state current
  3. Stores everything in PostgreSQL, with superseded account versions garbage-collected on finalisation
  4. A query tracker watches your traffic and automatically creates targeted indexes when a filter pattern crosses a frequency threshold
  5. A stateless JSON-RPC layer serves your standard calls from there

Token queries (gTABO, gTABD) get fast-paths via denormalised columns, so they skip the generic scan path entirely.

Now available on Triton infrastructure. Open-sourcing under AGPL soon as part of the Yellowstone project.

Full breakdown: https://blog.triton.one/introducing-steamboat-indexed-account-reads-for-every-app-on-solana/

https://preview.redd.it/ie2snue2kowg1.png?width=2460&format=png&auto=webp&s=3f88d625f6f3cd231115fa982a8fce242431367d

reddit.com
u/triton_one — 1 day ago