feat: extract @libscope/core — search engine, indexing, RAG, and DB

[RobertLD]

Summary

Part of #488. Extract the core search engine, indexing pipeline, RAG, database layer, and embedding providers into a standalone @libscope/core package. This is the kernel that all other packages depend on.

Problem / Motivation

Today src/core/index.ts re-exports API servers, connectors, and CLI utilities alongside search/index APIs — making the public surface impossible to understand and forcing library consumers to pull in server and connector code. Extracting and scoping down the core package gives library consumers a clean, minimal dependency for semantic search.

Proposed Solution

Move the following into packages/core/:

• src/db/ — SQLite schema, connection, migrations
• src/errors.ts, src/logger.ts, src/config.ts
• src/providers/ — EmbeddingProvider interface + local/ollama/openai impls
• src/core/ — indexing, search, RAG, documents, chunks, ratings, topics, tags, links, analytics, dedup, versioning, batch, bulk, saved-searches, export

Scope down src/core/index.ts to export only: indexing, search, RAG, documents, topics, tags, links, ratings, analytics, bulk operations, and saved searches. Remove re-exports of API servers, web server, connectors, and scheduler.

Acceptance Criteria

• @libscope/core builds and tests pass independently on Node 20 and 22
• The published package does not include or depend on commander, @modelcontextprotocol/sdk, connector HTTP clients, or web server code
• src/core/index.ts (now packages/core/src/index.ts) exports only search/index/document/topic/tag/RAG APIs — no server or connector re-exports
• All downstream packages (@libscope/lite, @libscope/mcp, @libscope/server, @libscope/connectors) update their imports to use @libscope/core
• Existing libscope CLI behaviour is unchanged
• CI passes on Node 20 and 22

Out of Scope

• Changing the search algorithm or embedding provider implementations
• Removing src/core/scheduler.ts coupling — that is handled in the @libscope/connectors issue

Technical Notes

• src/core/scheduler.ts imports connectors directly — this coupling must be resolved in the connectors extraction (separate issue) before core can be fully clean
• Key deps to keep: better-sqlite3, sqlite-vec, zod, @xenova/transformers, openai, @anthropic-ai/sdk
• src/LibScope.ts (the main orchestration class) moves to the libscope CLI package, not core