Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions components/blog/AnalyticsArchitecture.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
import { Globe, BarChart3, Database, Server, Gauge, ArrowRight } from 'lucide-react'
import { cn } from '@/lib/utils'
import type { ComponentType } from 'react'

/**
* Diagram for the ClickHouse analytics blog post: the path a pageview takes from
* the browser through Plausible CE into ClickHouse, with PostgreSQL and the
* separate Core Web Vitals store shown as satellites rather than in the hot path.
*
* Server-rendered, theme-aware via the fd-* tokens. The arrows are decorative
* (aria-hidden); the node labels carry the meaning, and the whole figure has an
* aria-label that reads the flow as one sentence for screen readers.
*/

function Node({
icon: Icon,
title,
subtitle,
muted = false,
}: {
icon: ComponentType<{ className?: string }>
title: string
subtitle: string
muted?: boolean
}) {
return (
<div
className={cn(
'flex flex-1 flex-col items-center gap-2 rounded-lg border p-4 text-center',
muted ? 'border-dashed border-fd-border bg-transparent' : 'border-fd-border bg-fd-card',
)}
>
<Icon
className={cn('h-6 w-6', muted ? 'text-fd-muted-foreground/70' : 'text-fd-foreground')}
/>
<div className="text-sm font-medium leading-tight">{title}</div>
<div className="text-xs leading-snug text-fd-muted-foreground">{subtitle}</div>
</div>
)
}

function Arrow() {
return (
<ArrowRight
aria-hidden="true"
className="h-5 w-5 shrink-0 rotate-90 self-center text-fd-muted-foreground/60 sm:rotate-0"
/>
)
}

export function AnalyticsArchitecture() {
return (
<figure
className="not-prose my-8"
aria-label="Data flow: the browser sends events to Plausible Community Edition, which writes them to ClickHouse Cloud. Plausible keeps its site and user configuration in PostgreSQL. A separate ClickHouse instance stores Core Web Vitals."
>
<div className="rounded-xl border border-fd-border bg-fd-card/40 p-4 sm:p-6">
<div className="flex flex-col items-stretch gap-3 sm:flex-row sm:items-stretch">
<Node icon={Globe} title="Your browser" subtitle="~1 KB script, no cookies" />
<Arrow />
<Node icon={BarChart3} title="Plausible CE" subtitle="open-source analytics app" />
<Arrow />
<Node icon={Database} title="ClickHouse Cloud" subtitle="columnar event store" />
</div>

<div className="mt-3 grid grid-cols-1 gap-3 sm:grid-cols-2">
<Node
icon={Server}
title="PostgreSQL"
subtitle="sites, users, settings — not events"
muted
/>
<Node
icon={Gauge}
title="Core Web Vitals"
subtitle="separate ClickHouse, separate tenant"
muted
/>
</div>
</div>
<figcaption className="mt-3 text-center text-sm text-fd-muted-foreground">
Every pageview crosses three hops. The interesting one is the last.
</figcaption>
</figure>
)
}
116 changes: 116 additions & 0 deletions components/blog/SiteStats.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
import { cn } from '@/lib/utils'

/**
* Live-ish stats card for the ClickHouse analytics blog post.
*
* When PLAUSIBLE_STATS_API_KEY is set, this server component queries the
* self-hosted Plausible Stats API (v2) at request time, cached for an hour, and
* shows the real librechat.ai numbers. Without the key — local dev, preview
* builds, anyone forking the repo — it falls back to a dated snapshot of the
* figures pulled straight from ClickHouse, so the post always renders and never
* fails a build on a missing secret.
*
* No secret is ever shipped to the client: the fetch runs on the server and only
* the rendered numbers reach the browser.
*/

type Stat = { label: string; value: string }
type Stats = { mode: 'live' | 'snapshot'; period: string; items: Stat[] }

// Real figures read from ClickHouse at migration time (2026-06-25). Used when no
// Plausible API key is configured. Honest and dated rather than invented-live.
const SNAPSHOT: Stats = {
mode: 'snapshot',
period: 'snapshot · June 2026',
items: [
{ label: 'Events stored', value: '1.9M' },
{ label: 'Sessions', value: '222K' },
{ label: 'On disk', value: '≈28 MiB' },
],
}

const compact = new Intl.NumberFormat('en-US', { notation: 'compact', maximumFractionDigits: 1 })

async function getLiveStats(): Promise<Stats | null> {
const key = process.env.PLAUSIBLE_STATS_API_KEY
if (!key) return null

const siteId = process.env.PLAUSIBLE_SITE_ID ?? 'librechat.ai'
const baseUrl = process.env.PLAUSIBLE_BASE_URL ?? 'https://plausible.librechat.ai'

try {
const res = await fetch(`${baseUrl}/api/v2/query`, {
method: 'POST',
headers: {
Authorization: `Bearer ${key}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
site_id: siteId,
metrics: ['visitors', 'pageviews', 'visits'],
date_range: '30d',
}),
signal: AbortSignal.timeout(4000),
// Cache for an hour so a popular post doesn't hammer the analytics box.
next: { revalidate: 3600 },
})

if (!res.ok) return null
const data = (await res.json()) as { results?: { metrics?: number[] }[] }
const metrics = data.results?.[0]?.metrics
if (!metrics || metrics.length < 3) return null

const [visitors, pageviews, visits] = metrics
return {
mode: 'live',
period: 'last 30 days · live',
items: [
{ label: 'Unique visitors', value: compact.format(visitors) },
{ label: 'Pageviews', value: compact.format(pageviews) },
{ label: 'Visits', value: compact.format(visits) },
],
}
} catch {
// Network error, timeout, bad JSON — fall back to the snapshot.
return null
}
}

export async function SiteStats() {
const stats = (await getLiveStats()) ?? SNAPSHOT
const isLive = stats.mode === 'live'

return (
<div className="not-prose my-8 rounded-xl border border-fd-border bg-fd-card p-5 sm:p-6">
<div className="mb-5 flex items-center justify-between gap-3">
<div>
<div className="text-sm font-medium">librechat.ai</div>
<div className="text-xs text-fd-muted-foreground">{stats.period}</div>
</div>
<span className="inline-flex items-center gap-1.5 rounded-full border border-fd-border px-2.5 py-1 text-xs text-fd-muted-foreground">
<span
aria-hidden="true"
className={cn(
'h-1.5 w-1.5 rounded-full',
isLive ? 'animate-pulse bg-emerald-500' : 'bg-fd-muted-foreground/50',
)}
/>
{isLive ? 'Live via Plausible' : 'Snapshot'}
</span>
</div>

<dl className="grid grid-cols-3 gap-3 sm:gap-6">
{stats.items.map((stat) => (
<div key={stat.label}>
<dt className="text-xs text-fd-muted-foreground">{stat.label}</dt>
<dd className="mt-1 text-2xl font-semibold tabular-nums sm:text-3xl">{stat.value}</dd>
</div>
))}
</dl>

<p className="mt-5 text-xs text-fd-muted-foreground">
Counted by Plausible, stored in ClickHouse. No cookies, no cross-site tracking.
</p>
</div>
)
}
103 changes: 103 additions & 0 deletions content/blog/2026-06-25_clickhouse-analytics.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
title: 'ClickHouse behind the scenes: how we count visitors to librechat.ai'
date: 2026-06-25
description: "Our website analytics run on Plausible, with ClickHouse as the engine underneath. Why we built it that way, and what happens when you point a LibreChat agent at a few million pageviews and just ask it questions in plain English."
tags:
- clickhouse
- plausible
- analytics
- agents
- self-hosted
- privacy
- guide
author: berry
ogImage: /images/blog/2026-06-25_clickhouse-analytics.png
category: guide
---

I run the infrastructure behind LibreChat, including this website. Like anyone who ships a site, I want to know which pages people read, where they come from, and whether a release landed. What I do not want is to follow readers around the internet to find that out, or to greet them with a cookie banner before they have read a sentence.

So librechat.ai runs on [Plausible](https://plausible.io/). It is open source, it counts visits without cookies, and it never builds a profile of anyone. The dashboard gives me the handful of numbers I actually use and nothing I would be uncomfortable collecting. That part is not very interesting to write about, though. The interesting part is what sits underneath it.

<SiteStats />

Those numbers are not living in Plausible. Plausible is the app you log into. The data is in ClickHouse, and that is the part worth a blog post.

## Why the storage engine matters here

Analytics is a particular shape of problem. You write a lot of small, similar rows, almost never update them, and then ask broad questions across all of them: how many visits last week, top ten pages this month, which referrer sent the most people. A normal row-oriented database is built for the opposite job, reading and writing whole records one at a time, and it shows the strain once the table gets big.

ClickHouse is a column store. Each column lives together on disk, so a query that only needs `timestamp` and `pathname` reads just those two columns and skips everything else. Similar values sitting next to each other also compress extremely well, which is the detail that surprised me most when I looked at our own data: the main events table holds about **1.9 million rows in roughly 28 MiB on disk.** That is the entire pageview history of the site, and it would fit on a floppy disk twice over. Aggregations that touch all of it come back in well under a second.

That combination, cheap to store and fast to scan, is exactly why Plausible uses ClickHouse as its backend rather than the relational database it uses for everything else.

## The path a pageview takes

When you open a page here, a script of about a kilobyte fires a single request. It carries no cookie and no identifier that follows you to the next site. Plausible Community Edition receives that event, does its session bookkeeping, and writes it into ClickHouse.

<AnalyticsArchitecture />

Plausible keeps two kinds of state. The configuration, which sites exist, who can log in, what the settings are, lives in PostgreSQL, because that is a small relational problem and Postgres is the right tool for it. The events and the session rollups, the part that grows forever, live in ClickHouse. Keeping those two stores separate is the whole design: the settings barely change, the events pile up by the million. And once a few million events are sitting in a database this fast, you can start asking them questions you would never bother taking to a normal dashboard.

## Asking it questions in plain English

This is the part I did not expect to enjoy. A Plausible dashboard gives you the usual charts: top pages, top sources, visitors over time. They are useful, but you are stuck with the questions someone already built a chart for.

Underneath, though, it is just a fast SQL table. So I gave a LibreChat agent read-only access to it over MCP and started asking it things the way I would ask a coworker who had read every row. Not "draw me a chart," but actual questions:

> When is the best day to ship a release, and when are people actually around to see it?

It answers in plain language, not graphs. A few of the things it turned up, none of which I had a dashboard for:

| What I asked | What it found |
| ----------------------- | ------------------------------------------------------------------------- |
| Best day to ship | Monday or Wednesday — the widest reach and the most engaged workday crowd |
| Most engaged readers | Tuesday, longest sessions (~225s), mostly people deep in the docs |
| When everyone's online | A clear peak from 07:00 to 15:00 UTC, every weekday |
| Do weekends matter | Saturday and Sunday are ~27% of weekly pageviews — not the rounding error I'd assumed |
| Mobile vs desktop | Mobile share climbs ~9 points on Fri–Sat versus mid-week |
| Where readers are | Germany is our #2 country, which is a real argument for German docs |
| What people read about | Agents and MCP sit in the top ten pages literally every day |
| How people install | Docker out-pulls npm by 2–3× every single day |

I did not build a single one of those. I asked, it scanned a few million rows, and it answered in about the time it took to read the question. That speed is the whole reason this sits in ClickHouse instead of a spreadsheet, or a database that starts sweating at a million rows. The prompts I used are [at the end](#the-prompts), if you want to point an agent at your own.

## What it doesn't know about you

Now read that list again. Best release day, peak hours, weekend mobile, Germany at #2. Every line is about the audience as a whole, and not one of them needed to know who you are.

That is on purpose. Plausible sets no cookie, hands you no persistent ID, and never trails you to the next site. There is no profile of you in that ClickHouse table, because nobody ever built one. The agent can tell that the Docker page is busy on a Saturday morning without the faintest idea that it was you reading it.

This usually gets framed as a trade-off: real analytics on one side, leaving people alone on the other, pick one. The table above is the whole counter-argument. You can learn a lot about how a site gets used without learning anything about the people using it.

## Where this is going

There is a fitting twist to writing this now. We have leaned on ClickHouse for the site's analytics quietly for a while, and as of recently LibreChat and ClickHouse are joining forces — there is a banner about it at the top of this very page. The database that has been counting these pageviews in the background is becoming part of LibreChat itself.

We even moved the entire analytics store onto a fresh ClickHouse Cloud cluster while this post was being written. It took an afternoon and about five minutes of downtime, and that is the most boring sentence here, which is exactly how a database move should feel.

The numbers at the top of the page come out of all of it: counted without a cookie, kept somewhere that treats a few million events as a rounding error, and now close enough to just ask in plain English.

## The prompts

Want to talk to your own analytics this way? Point a LibreChat agent at your ClickHouse (read-only) and start here.

Agent instructions:

> You have read-only access to our Plausible analytics in ClickHouse (database `plausible_events_db`; `events_v2` holds pageviews and events, `sessions_v2` holds sessions). Only run read queries. Answer with whole-audience aggregates only, never anything that could single out a visitor. Prefer clear takeaways and small tables over raw rows, and always say which date range you used.

The weekly digest, the one that produced the table above:

> From the last 30 days of `events_v2` and `sessions_v2`, give me a "Key Insights" table: best weekday to publish, when readers are most engaged, peak hours in UTC, the weekend share of traffic, the mobile/desktop split by day, top countries, the most-read feature pages, and the most popular install method. One row per insight, with a short "why."

Best time to ship:

> What day and hour (UTC) should I publish a release to reach the most engaged readers? Use the last 8 weeks of sessions, weight by average session duration, and show the top three windows.

A shareable, privacy-safe snapshot:

> Write a short public traffic summary I can post: total visits, pageviews, top 5 pages, and top 5 countries for last month. Whole-audience numbers only.

---

_Plausible Community Edition v3.2.1 · ClickHouse 26.2 · librechat.ai._
4 changes: 4 additions & 0 deletions lib/mdx-components.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ import { TrackedLink, TrackedAnchor } from '@/components/TrackedLink'
import { CredentialsGeneratorMDX } from '@/components/tools/CredentialsGeneratorMDX'
import { YAMLValidatorMDX } from '@/components/tools/YAMLValidatorMDX'
import { CompatibilityMatrix } from '@/components/CompatibilityMatrix'
import { AnalyticsArchitecture } from '@/components/blog/AnalyticsArchitecture'
import { SiteStats } from '@/components/blog/SiteStats'
import type { ReactNode } from 'react'

function mapCalloutType(type?: string): 'info' | 'warn' | 'error' {
Expand Down Expand Up @@ -256,6 +258,8 @@ export const mdxComponents = {
Accordions,
OptionTable,
CompatibilityMatrix,
AnalyticsArchitecture,
SiteStats,
Frame,
ThemeImage,
Video,
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading