feat: libscope/lite — embeddable semantic search library (#451)

CLAUDE.md

@@ -75,6 +75,12 @@ src/
 ├── mcp/           # MCP server
 │   ├── server.ts  # MCP entry point
 │   └── tools/     # Each file exports registerTools(server, db, provider)
+├── lite/          # libscope/lite — embeddable semantic search (no CLI/MCP/connectors)
+│   ├── index.ts              # Public entrypoint — exports LibScopeLite + types
+│   ├── core.ts               # LibScopeLite class (index, search, getContext, ask, rate)
+│   ├── types.ts              # LiteOptions, LiteDoc, LiteSearchResult, etc.
+│   ├── normalize.ts          # Raw input → markdown (dispatches to core/parsers/)
+│   └── chunker-treesitter.ts # Optional tree-sitter code chunker (TS/JS/Python)
 ├── core/          # Business logic (documents, search, indexing, packs, topics, etc.)
 │   └── parsers/   # File format parsers (markdown, pdf, docx, html, epub, pptx, csv, yaml, json)
 ├── api/           # REST API server (routes, middleware, openapi spec)

README.md

@@ -175,7 +175,9 @@ Search uses sqlite-vec for vector similarity when available, with FTS5 full-text
 
 ### Programmatic SDK
 
-LibScope also exports a `LibScope` class for use as a library:
+LibScope exports two embeddable APIs:
+
+**`LibScope`** — full SDK with all features (connectors, topics, packs, etc.):
 
 ```ts
 import { LibScope } from "libscope";
@@ -186,7 +188,36 @@ const results = await scope.search("query");
 scope.close();
 ```
 
-See the [Programmatic Usage](/guide/programmatic-usage) guide for details on the SDK, batch search, and document TTL/expiry.
+**`LibScopeLite`** — lightweight embeddable class for external applications. No CLI, no MCP server, no connectors. Designed for embedding semantic search directly into other tools (MCP servers, VS Code extensions, CI scripts):
+
+```ts
+import { LibScopeLite } from "libscope/lite";
+
+const lite = new LibScopeLite({ dbPath: ":memory:" });
+
+// Index documents (or code files via tree-sitter chunking)
+await lite.indexBatch(docs, { concurrency: 4 });
+
+// Hybrid vector + FTS5 search
+const results = await lite.search("how to authenticate");
+
+// Get RAG context for injection into an external LLM prompt
+const context = await lite.getContext("How does auth work?");
+
+lite.close();
+```
+
+Tree-sitter powered code indexing splits TypeScript, JavaScript, and Python files at function/class boundaries:
+
+```ts
+import { TreeSitterChunker } from "libscope/lite";
+
+const chunker = new TreeSitterChunker();
+const chunks = await chunker.chunk(sourceCode, "typescript");
+// Each chunk is a complete function or class with 1-based line numbers
+```
+
+See the [LibScope Lite guide](https://libscope.com/guide/lite) and [API reference](https://libscope.com/reference/lite-api) for the full documentation.
 
 ## Organizing Content
 

docs/.vitepress/config.ts

@@ -61,6 +61,8 @@ export default defineConfig({
               text: "Programmatic Usage",
               link: "/guide/programmatic-usage",
             },
+            { text: "LibScope Lite", link: "/guide/lite" },
+            { text: "Code Indexing", link: "/guide/code-indexing" },
           ],
         },
         {
@@ -86,6 +88,7 @@ export default defineConfig({
             { text: "CLI Commands", link: "/reference/cli" },
             { text: "MCP Tools", link: "/reference/mcp-tools" },
             { text: "REST API", link: "/reference/rest-api" },
+            { text: "LibScope Lite API", link: "/reference/lite-api" },
             { text: "Registry", link: "/reference/registry" },
             { text: "Configuration", link: "/reference/configuration" },
           ],

docs/guide/architecture.md

@@ -7,28 +7,44 @@ This guide explains how LibScope is structured internally. It is intended for co
 LibScope is organized into four distinct layers:
 
 ```
-┌─────────────────────────────────────────────┐
-│           Entry Points                       │
-│   CLI (Commander.js)  MCP Server  REST API   │
-└──────────────────┬──────────────────────────┘
-                   │
-┌──────────────────▼──────────────────────────┐
-│           Core Business Logic                │
-│  indexing · search · rag · documents · ...   │
-└──────────────────┬──────────────────────────┘
-                   │
-┌──────────────────▼──────────────────────────┐
-│          Infrastructure                      │
-│   db/ (SQLite)      providers/ (embeddings)  │
-└─────────────────────────────────────────────┘
+┌─────────────────────────────────────────────────────────┐
+│                    Entry Points                          │
+│  CLI (Commander.js)  MCP Server  REST API  LibScopeLite  │
+└──────────────────────────┬──────────────────────────────┘
+                           │
+┌──────────────────────────▼──────────────────────────────┐
+│                 Core Business Logic                      │
+│     indexing · search · rag · documents · parsers · …   │
+└──────────────────────────┬──────────────────────────────┘
+                           │
+┌──────────────────────────▼──────────────────────────────┐
+│                    Infrastructure                        │
+│      db/ (SQLite + sqlite-vec)    providers/ (embeddings)│
+└─────────────────────────────────────────────────────────┘
 ```
 
-**Entry points** (`src/cli/`, `src/mcp/`, `src/api/`) are thin adapters. They parse input, call core functions, and format output. They contain no business logic.
+**Entry points** (`src/cli/`, `src/mcp/`, `src/api/`, `src/lite/`) are thin adapters. They parse input, call core functions, and format output. They contain no business logic.
 
-**Core** (`src/core/`) contains all business logic. Core modules are plain TypeScript functions — they don't know whether they were called from the CLI, an MCP tool, or the REST API.
+**Core** (`src/core/`) contains all business logic. Core modules are plain TypeScript functions — they don't know whether they were called from the CLI, an MCP tool, the REST API, or `LibScopeLite`.
 
 **Infrastructure** (`src/db/`, `src/providers/`) handles persistence and external services. The database layer uses better-sqlite3 (synchronous). The provider layer abstracts embedding models behind a common interface.
 
+### LibScope Lite Layer
+
+`src/lite/` is a separate entry point that exposes a minimal embeddable API built on top of the same core and infrastructure modules:
+
+```
+libscope/lite  →  src/lite/index.ts  →  LibScopeLite class
+                                         ├── core/indexing.ts
+                                         ├── core/search.ts
+                                         ├── core/rag.ts
+                                         ├── core/ratings.ts
+                                         ├── db/connection.ts
+                                         └── providers/
+```
+
+`LibScopeLite` deliberately omits connectors, topics, packs, webhooks, and registry — keeping the API surface small and the import footprint minimal for embedding in external applications.
+
 ## Module Map
 
 ```
@@ -113,6 +129,12 @@ src/
 │   ├── onenote.ts            # Microsoft Graph API sync
 │   ├── http-utils.ts         # shared retry logic with exponential backoff
 │   └── sync-tracker.ts       # sync history and status in database
+├── lite/
+│   ├── index.ts              # public entrypoint — exports LibScopeLite + types
+│   ├── core.ts               # LibScopeLite class implementation
+│   ├── types.ts              # LiteOptions, LiteDoc, LiteSearchResult, etc.
+│   ├── normalize.ts          # raw input → markdown (dispatches to core/parsers/)
+│   └── chunker-treesitter.ts # optional tree-sitter code chunker (TS/JS/Python)
 ├── config.ts                 # loadConfig() — merges env, project, user, defaults
 ├── errors.ts                 # LibScopeError hierarchy
 ├── logger.ts                 # pino logger with child logger support