docs: Updates and refactor (#683)

This PR makes incremental changes to the documentation.

* Closes #697 
* Closes #698

## Chores
- [x] Add dark mode
- [x] Fix headers in navbar
- [x] Add `extra.css` to customize navbar styles
- [x] Customize fonts for prose/code blocks, navbar and admonitions
- [x] Inspect all admonition boxes (remove redundant dropdowns) and
improve clarity and readability
- [x] Ensure that all images in the docs have white background (not
transparent) to be viewable in dark mode
- [x] Improve code formatting in code blocks to make them consistent
with autoformatters (eslint/ruff)
- [x] Add bolder weight to h1 headers
- [x] Add diagram showing the difference between embedded (OSS) and
serverless (Cloud)
- [x] Fix [Creating an empty
table](https://lancedb.github.io/lancedb/guides/tables/#creating-empty-table)
section: right now, the subheaders are not clickable.
- [x] In critical data ingestion methods like `table.add` (among
others), the type signature often does not match the actual code
- [x] Proof-read each documentation section and rewrite as necessary to
provide more context, use cases, and explanations so it reads less like
reference documentation. This is especially important for CRUD and
search sections since those are so central to the user experience.

## Restructure/new content 
- [x] The section for [Adding
data](https://lancedb.github.io/lancedb/guides/tables/#adding-to-a-table)
only shows examples for pandas and iterables. We should include pydantic
models, arrow tables, etc.
- [x] Add conceptual tutorial for IVF-PQ index
- [x] Clearly separate vector search, FTS and filtering sections so that
these are easier to find
- [x] Add docs on refine factor to explain its importance for recall.
Closes #716
- [x] Add an FAQ page showing answers to commonly asked questions about
LanceDB. Closes #746
- [x] Add simple polars example to the integrations section. Closes #756
and closes #153
- [ ] Add basic docs for the Rust API (more detailed API docs can come
later). Closes #781
- [x] Add a section on the various storage options on local vs. cloud
(S3, EBS, EFS, local disk, etc.) and the tradeoffs involved. Closes #782
- [x] Revamp filtering docs: add pre-filtering examples and redo headers
and update content for SQL filters. Closes #783 and closes #784.
- [x] Add docs for data management: compaction, cleaning up old versions
and incremental indexing. Closes #785
- [ ] Add a benchmark section that also discusses some best practices.
Closes #787

---------

Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com>
Co-authored-by: Will Jones <willjones127@gmail.com>

docs/src/javascript/README.md

@@ -33,7 +33,7 @@ const results = await table.search([0.1, 0.3]).limit(20).execute();
 console.log(results);
 ```
 
-The [examples](./examples) folder contains complete examples.
+The [examples](../examples/index.md) folder contains complete examples.
 
 ## Development
 

docs/src/javascript/modules.md

@@ -1,15 +1,78 @@
 [vectordb](README.md) / Exports
 
-# vectordb
+# JavaScript API Reference
 
-## Table of contents
+This section contains the API reference for the OSS JavaScript / Node.js API.
 
-### Enumerations
+## Installation
 
-- [MetricType](enums/MetricType.md)
-- [WriteMode](enums/WriteMode.md)
+```bash
+npm install vectordb
+```
 
-### Classes
+This will download the appropriate native library for your platform. We currently
+support x86_64 Linux, aarch64 Linux, Intel MacOS, and ARM (M1/M2) MacOS. We do not
+yet support Windows or musl-based Linux (such as Alpine Linux).
+
+## Usage
+
+### Basic Example
+
+```javascript
+const lancedb = require('vectordb');
+const db = await lancedb.connect('data/sample-lancedb');
+const table = await db.createTable("my_table",
+      [{ id: 1, vector: [0.1, 1.0], item: "foo", price: 10.0 },
+      { id: 2, vector: [3.9, 0.5], item: "bar", price: 20.0 }])
+const results = await table.search([0.1, 0.3]).limit(20).execute();
+console.log(results);
+```
+
+The [examples](../examples/index.md) folder contains complete examples.
+
+## Development
+
+To build everything fresh:
+
+```bash
+npm install
+npm run tsc
+npm run build
+```
+
+Then you should be able to run the tests with:
+
+```bash
+npm test
+```
+
+### Rebuilding Rust library
+
+```bash
+npm run build
+```
+
+### Rebuilding Typescript
+
+```bash
+npm run tsc
+```
+
+### Fix lints
+
+To run the linter and have it automatically fix all errors
+
+```bash
+npm run lint -- --fix
+```
+
+To build documentation
+
+```bash
+npx typedoc --plugin typedoc-plugin-markdown --out ../docs/src/javascript src/index.ts
+```
+
+## Classes
 
 - [DefaultWriteOptions](classes/DefaultWriteOptions.md)
 - [LocalConnection](classes/LocalConnection.md)
@@ -17,7 +80,12 @@
 - [OpenAIEmbeddingFunction](classes/OpenAIEmbeddingFunction.md)
 - [Query](classes/Query.md)
 
-### Interfaces
+## Enumerations
+
+- [MetricType](enums/MetricType.md)
+- [WriteMode](enums/WriteMode.md)
+
+## Interfaces
 
 - [AwsCredentials](interfaces/AwsCredentials.md)
 - [CleanupStats](interfaces/CleanupStats.md)
@@ -35,10 +103,6 @@
 - [VectorIndex](interfaces/VectorIndex.md)
 - [WriteOptions](interfaces/WriteOptions.md)
 
-### Type Aliases
-
-- [VectorIndexParams](modules.md#vectorindexparams)
-
 ### Functions
 
 - [connect](modules.md#connect)