Trillion Characters

Typists online:

x:

y:

Info

'Trillion Characters' is a realtime collaborative experiment built with Datastar.

Click anywhere on the canvas to place your cursor and start typing. You can see other people's cursors typing as well.

This page uses no clientside JavaScript except for Datastar & Datastar attributes.

Datastar is a lightweight JavaScript framework (11kB) with a completely radical architecture:
Compared to other JS frameworks (e.g. React, Vue, Svelte), Datastar eliminates state and logic on the client-side.
Instead, all interactive logic is moved to the backend. Page updates are streamed to the client as server-rendered templates over a persistent SSE stream.
This is a bit like streaming a movie, except the movie is HTML and you receive new content from updates and UI interactions.
The client's browser is effectively reduced to being a rendering viewport only capable of displaying raw HTML, yet still offering full interactivity.

Server-sent events (SSE) are not a special type of connection. Instead, it's just an HTTP response type.
Sending full page updates may raise concerns about network bandwidth. However, use of SSE streams enables compression across an entire session as compared to mere individual requests. Using the browser's built-in Brotli compression, extra bytes are sent over the wire only if the content is changed. This achieves total compression ratio's upwards of 50-4000x, exceeding what is commonly attainable from gzip'ed responses.
Under the hood, a sophisticated DOM-morphing algorithm is used to make updates performant and seamless.

Traditional web servers are based on REPR (request endpoint response).
Instead, 'Trillion Characters' uses Query Command Request Segregation (CQRS), an architectural pattern that turns any program into realtime and collaborative by default. Reads & writes are separated. Commands from all clients (e.g. typing characters) are queued and written by the server on a fixed update rate. The page is rendered in HTML templates as a function of the current state: `f(db state)`.
The render is customized for each client to apply chunk-based culling, sending only the chunks visible to a given client.

Character are stored as 45x45 chunks in LMDB, a very fast memory-mapped key-value store database. LZ4 compression is applied on a chunk-basis to eliminate whitespace and repeating characters. Chunk coordinates are used as keys and stored in Z-order with morton codes, translating locality from 2D into 1D space for the sorted order of the storage B-tree.

Reads and writes to the databases are full ACID with no relaxations. Every character you see on the canvas has been atomically saved to disk.

This application runs on a €5,52 VPS in Germany, on the cheapest public cloud tier available.

Made with 🚀 by Elias de Jong

https://github.com/eliasdejong

Ok

Cancel

'Trillion Characters' is a real-time collaborative experiment built upon the Datastar framework, designed to facilitate simultaneous typing interactions where users can view the cursors of others. The core innovation lies in Datastar's radical architectural approach, which deliberately eliminates state and logic on the client side, shifting all interactive logic to the backend. Page updates are transmitted to the client via Server-Sent Events (SSE), which functions as an HTTP response type used for streaming content. This method, combined with the use of the browser's built-in Brotli compression, allows for session-wide compression, potentially achieving compression ratios of 50 to 4000 times that of gzip, by sending only changed content, underpinned by a sophisticated DOM-morphing algorithm to ensure performant and seamless updates.

The system employs the Query Command Request Segregation (CQRS) architectural pattern, which fundamentally turns the application into a real-time and collaborative system by default, separating reads and writes. Commands originating from all clients, such as typing characters, are queued and processed by the server at a fixed update rate before the page is rendered as a function of the current database state, f(db state). To optimize network usage, this rendering process incorporates chunk-based culling, ensuring that only the specific data chunks visible to an individual client are transmitted.

Data storage is handled efficiently using LMDB, a fast memory-mapped key-value store database. Characters are segmented into 45x45 chunks, and LZ4 compression is applied at the chunk level to remove whitespace and recurring characters. To manage storage locations efficiently, chunk coordinates are indexed using Z-order with Morton codes, which translates two-dimensional spatial locality into one-dimensional space for ordering within the storage B-tree. All read and write operations to these databases maintain full ACID properties, guaranteeing that every character displayed on the canvas is atomically saved to disk. This entire system is presented as an example of modern, distributed architecture, developed by Elias de Jong.