--- title: "Architecture" sidebarTitle: "Architecture" description: "How LanceDB is put together, and what changes between running it embedded and running it distributed." icon: cubes keywords: ["architecture", "embedded", "distributed", "object storage", "compute", "deployment"] --- LanceDB has one storage layer and two ways to put compute on top of it. Which one you run decides where queries execute, what runs on your behalf, and what you have to operate — and almost nothing else. The data is the same [Lance](/basics/lance) tables either way. ## One storage layer {#storage} Everything durable lives in object storage: table data, manifests, indexes. No machine owns a dataset, and no server has to be running for the data to exist. ``` ┌──────────────────────────────────────────┐ │ object storage │ │ table data · manifests · indexes │ └──────────────────────────────────────────┘ ``` That is the whole persistence story, and it is why the two deployment shapes can differ so much above it while staying compatible below it. A table written embedded is opened by a deployment, and the reverse, without conversion. ## Embedded {#embedded} ``` your process ┌────────────────────────────────┐ │ application │ │ LanceDB library │ ← plans, executes, indexes, compacts └────────────────────────────────┘ │ ┌─────────▼─────────┐ │ object storage │ └───────────────────┘ ``` `lancedb.connect("./data")` or `connect("s3://bucket/path")` gives you a **library**, not a client. Your process opens the tables, plans the queries, runs the scans, builds the indexes, and writes the new versions. There is no server in the picture at all. **What follows from that.** Nothing runs unless you call it — no background compaction, no server-side [jobs](/basics/jobs), no [SQL endpoint](/basics/sql). Scale is your process's scale: a query gets the cores and memory your process has. Access control is whatever your object store enforces, since there is no other gatekeeper. **What it is good at.** Development, notebooks, single-node pipelines, embedding LanceDB inside an application, and anything where "no service to operate" is worth more than elasticity. Comfortably handles millions of vectors on one node -- disk-based indexes keep it performant without extra infrastructure. ## Distributed {#distributed} Enterprise ``` your process ┌──────────────┐ │ application │ │ LanceDB SDK │ ← a thin client └──────┬───────┘ │ db:// (REST / Flight SQL) ┌──────▼──────────────────────────────────────┐ │ control plane config · identity · policy │ ├─────────────────────────────────────────────┤ │ data plane query serving · writes │ │ indexers indexing · compaction │ └──────┬──────────────────────────────────────┘ │ ┌──────▼────────────┐ │ object storage │ └───────────────────┘ ``` `connect("db://...")` gives you a **client**. Planning and execution move to a cluster, and the work splits across tiers that scale independently: a control plane governing configuration, identity and policy; a data plane serving reads and writes; and indexers building indexes and compacting outside the request path. **What follows from that.** Compute and storage are separated, so a query fleet scales for interactive traffic without also scaling indexing capacity, and heavy background work does not steal resources from user queries. Because a server exists, so do the things that need one: [SQL](/sql), [functions](/management/function), {/* if functions launched */} server-side jobs, and [authorization](/governance/authorization) per object rather than per bucket. **What it is good at.** Many concurrent queries, continuous ingestion and indexing, multi-team access with real permissions, and workloads too large for one machine -- customers regularly operate on billions of rows and tens of terabytes. ## What changes, precisely {#differences} | | Embedded | Distributed | | --- | --- | --- | | What you connect to | a library in your process | a cluster | | Where queries plan and execute | your process | the data plane | | Table handle | `LanceTable` | `RemoteTable` | | Indexing and compaction | when you call it | dedicated indexers | | [Jobs](/basics/jobs) | run in-process, no server record | real jobs: id, status, events, cancel | | [SQL](/sql) | unavailable | Flight SQL endpoint | | [Functions](/management/function) {/* if functions launched */} | unavailable | run on workers | | Access control | object-store permissions | [per-object grants](/governance/authorization) | | Scale bound | your process | the cluster | | What you operate | nothing | a deployment | Both expose the same core table API — create, open, mutate, query — so that code moves between them by changing the URI. Code that depends on a row in the right-hand column does not; [Moving to LanceDB Enterprise](/basics/sdk#moving-to-enterprise) lists what changes. ## Choosing, and changing your mind {#choosing} Start embedded. It has no operational cost and the API is the same one you will keep using. Move when you hit something in the right-hand column — usually concurrent serving, ingestion that outgrows one machine, or a permissions requirement. Because the storage layer is identical, moving is a connection-string change plus a data migration, not a rewrite. The one thing to decide early is **stable row ids**: the SDK can give them to a table only when it creates one on a local path or an object store, and a deployment may require them on a [materialized view](/management/materialized-view) source. See [Limits](/basics/limits#schema). ## Where to go next {#next} What a write promises, and what a reader sees. The open format underneath both shapes. The distributed tiers in detail. The bounds you can actually hit.