SQLite is All You Need for Durable Workflows
- Blog

Obelisk

Features
Install
Blog
Docs
GitHub

SQLite is All You Need for Durable Workflows
2026-05-29
DBOS recently argued that
Postgres
is all you need for durable execution: if you already trust your database, you do not need a
separate orchestration tier. I agree with the direction, and I think the idea can be pushed further.
For a large class of durable systems, SQLite is all you need.

The Durable Part
Durable execution is often discussed as if it requires durable infrastructure. In many cases it does
not. The durable part is the workflow state. The compute can stay cheap and disposable.
That is a natural fit for Obelisk: workflow progress lives in an execution log, workflows replay
from persisted history, and activities can be retried. What matters most is keeping the workflow
state around and easy to inspect.
Why SQLite Fits
SQLite is appealing because it gives you transactional durable state without introducing a separate
database service. There is no network hop, no extra control plane, and no new operational surface
area just to keep workflow progress safe. For many systems, a local database file is exactly the
right level of machinery.
Litestream Makes It Portable
The obvious concern is what to do with those SQLite files once experiments start to accumulate. That
is where Litestream helps. It can stream SQLite
changes asynchronously to S3-compatible object storage. That gives you a simple way to keep working
state close to the runtime while still copying databases out for backup, migration, and inspection.
The caveat is that Litestream replication is asynchronous. A restore can miss the newest local
writes if the SQLite volume disappears before they are copied. That is fine for many AI and
experimentation workflows, but it is not the same thing as a highly available shared database.
That still leads to a useful operating model: run an Obelisk server with a SQLite database, back it
up with Litestream, and let an observer pull interesting databases when needed. The same file can be
used for local replay, debugging, and understanding what an agent actually did.
Why This Works Well For Agents
This is especially attractive for AI agents and AI-generated workflows. Those systems are often
bursty, experimental, and easier to reason about when each agent or tenant has a small
self-contained unit of state. A fleet of tiny servers in micro VMs or containers, each with its own
SQLite database and object storage backup, is often a better fit than one large always-on shared
system. It is simpler, cheaper, and gives better fault isolation.
When To Use Postgres Instead
SQLite is not the answer to every deployment shape. Obelisk also supports Postgres, and that is the
right choice when you need higher availability, broader shared scalability, or other deployment
properties that are better served by a network database. It is also the better fit when asynchronous
replication to object storage is not the durability model you want.
Many workflow systems do not need that on day one and should not start with more infrastructure than
their state actually demands.
In a large set of cases, a local SQLite database plus Litestream backup to S3 is enough. Add cheap
workers around it and you get a durable system with very little infrastructure. For the world of AI
agents, that may be the most sensible default.

« Back to Blog List

SQLite is presented as a viable solution for managing durable workflows, arguing that it is sufficient for a large class of systems, positioning it as an alternative to requiring a separate orchestration tier, a direction that the author agrees with but suggests can be further developed. The concept of durable execution is often confused with the necessity for heavy, durable infrastructure; the crucial element is the workflow state, which can be managed by relying on execution logs, replayable history, and retriable activities. SQLite is appealing because it inherently provides transactional durable state without introducing the overhead of a separate database service, thereby eliminating the need for network hops, an extra control plane, or additional operational surface area to secure workflow progress. This intrinsic simplicity suggests that a local database file is often the appropriate level of machinery for many applications.

To address the concern regarding the persistence of SQLite files that accumulate during experimentation, the system incorporates Litestream, which facilitates asynchronous streaming of SQLite changes to S3-compatible object storage. This mechanism allows users to maintain working state locally while simultaneously creating a means to back up, migrate, and inspect databases. It is important to note that the replication process is asynchronous, meaning a restoration operation might miss the newest local writes if the local volume disappears before the data is copied. This trade-off is deemed acceptable for systems like AI and experimentation workflows, which prioritize flexibility over the immediate consistency of a highly available shared database. The suggested operating model involves running a system with an SQLite database, using Litestream for backup, and allowing observers to pull necessary databases for debugging and understanding agent activities.

This architecture is particularly advantageous for AI agents and AI-generated workflows because these systems are often characterized as bursty and experimental, benefiting from small, self-contained units of state for each agent or tenant. Consequently, deploying a fleet of small servers or containers, each with its own SQLite database and associated object storage backup, offers superior fault isolation, simplicity, and cost-effectiveness compared to maintaining one large, always-on shared system. Conversely, the text specifies that SQLite is not universally applicable; Obelisk also supports Postgres, which is the preferable choice when requirements shift toward needing higher availability, broader shared scalability, or a durability model that mandates asynchronous replication to object storage. For workflow systems that do not require such guarantees from the outset, starting with the minimum necessary infrastructure based on state demand is recommended. In many scenarios, the combination of a local SQLite database backed by Litestream to S3, supplemented by cheap worker infrastructure, provides a durable system with minimal overhead, making it a sensible default, especially in the context of AI agent systems.