TicklyDB
Tick queries, compiled. Write as-of joins, OHLCV bars and snapshots
in Python; they compile ahead of time into one native binary that
streams your own Parquet — measured faster than DuckDB and Polars on
the three query shapes shown, with no joined table ever built, no
server, no cluster and nothing to rewrite in q. Every answer is
checked against DuckDB and Polars, and against itself at every
thread count, before any speed is claimed.
The problem
Tick data outgrows something eventually — the budget, the machine,
the migration, or the person who wrote the fast path. The fast
commercial engines charge per core before a core has done any work.
DuckDB and pandas both build the joined table before they can use it,
so a single day of quotes turns an interactive question into an
overnight job. Migrating off q is a rewrite whose hard part is
proving the new answer matches the old. And a hand-written C++ or
Rust kernel has to be re-entered on every strategy change and re-read
on every review.
The solution
Three query shapes a tick desk runs every day: an as-of join pricing
trades against the quote standing when they printed; OHLCV and VWAP
bars; and a per-symbol snapshot of the latest trade with its
prevailing quote. Each is written as an ordinary dataframe expression
and compiled ahead of time into one optimised pass of C++ over
Parquet, with the generated source kept beside the binary for anyone
who needs to audit what ran.
Nothing is licensed per core, the join never has to fit in memory,
the same query runs as DuckDB SQL for a second opinion, and the fast
path is generated rather than maintained by hand.
What it measured
On 12 million trades against 120 million quotes, at four threads, the
compiled binary ran the as-of join 2.53× faster than DuckDB and 1.80×
faster than Polars; the market snapshot 2.72× and 2.08×; and bar
building 1.19× and 1.29× — a margin narrow enough to sit inside the
session-to-session variance we measured and could not explain, which
the page says on the row it belongs to. Every run was verified
against DuckDB and Polars, and against its own output at one, two and
four threads, before it was timed. A run that disagrees is never
published.
Interactive content
The measured bars — time and peak memory for each engine, at one, two
and four threads — and the three queries as written need JavaScript to
draw. Everything claimed above is in them.
Contact
A benchmark on your data, a walkthrough, or a straight answer on
whether we can compile your workload yet — same address for all
three: contact@ticklydb.com
No tracking, no cookies, no analytics.