Implement Vector Search with sqlite-vec Extension

Makefile

@@ -9,7 +9,7 @@
 ### export GIT_VERSION=3.x.y-dev
 ### ```
 
-GIT_VERSION ?= $(shell git describe --long --abbrev=7)
+GIT_VERSION ?= $(shell git describe --long --abbrev=7 2>/dev/null || git describe --long --abbrev=7 --always)
 ifndef GIT_VERSION
     $(error GIT_VERSION is not set)
 endif

deps/Makefile

@@ -246,7 +246,11 @@ sqlite3/sqlite3/sqlite3.o:
 	cd sqlite3/sqlite3 && ${CC} ${MYCFLAGS} -fPIC -c -o sqlite3.o sqlite3.c -DSQLITE_ENABLE_MEMORY_MANAGEMENT -DSQLITE_ENABLE_JSON1 -DSQLITE_DLL=1
 	cd sqlite3/sqlite3 && ${CC} -shared -o libsqlite3.so sqlite3.o
 
-sqlite3: sqlite3/sqlite3/sqlite3.o
+sqlite3/sqlite3/vec.o: sqlite3/sqlite3/sqlite3.o
+	cd sqlite3/sqlite3 && cp ../sqlite-vec-source/sqlite-vec.c . && cp ../sqlite-vec-source/sqlite-vec.h .
+	cd sqlite3/sqlite3 && ${CC} ${MYCFLAGS} -fPIC -c -o vec.o sqlite-vec.c -DSQLITE_CORE -DSQLITE_VEC_STATIC -DSQLITE_ENABLE_MEMORY_MANAGEMENT -DSQLITE_ENABLE_JSON1 -DSQLITE_DLL=1
+
+sqlite3: sqlite3/sqlite3/sqlite3.o sqlite3/sqlite3/vec.o
 
 
 libconfig/libconfig/out/libconfig++.a:

deps/sqlite3/README.md

@@ -0,0 +1,95 @@
+# SQLite-vec Integration in ProxySQL
+
+This directory contains the integration of [sqlite-vec](https://github.com/asg017/sqlite-vec) - a SQLite extension that provides vector search capabilities directly within SQLite databases.
+
+## What is sqlite-vec?
+
+sqlite-vec is an extension that enables SQLite to perform vector similarity searches. It provides:
+- Vector storage and indexing
+- Distance calculations (cosine, Euclidean, etc.)
+- Approximate nearest neighbor (ANN) search
+- Support for multiple vector formats (JSON, binary, etc.)
+
+## Integration Details
+
+### Directory Structure
+- `sqlite-vec-source/` - Source files for sqlite-vec (committed to repository)
+- `sqlite3/` - Build directory where sqlite-vec gets compiled during the build process
+
+### Integration Method
+
+The integration uses **static linking** to embed sqlite-vec directly into ProxySQL:
+
+1. **Source Storage**: sqlite-vec source files are stored in `sqlite-vec-source/` to persist across builds
+2. **Compilation**: During build, sources are copied to the build directory and compiled with static linking flags:
+   - `-DSQLITE_CORE` - Compiles as part of SQLite core
+   - `-DSQLITE_VEC_STATIC` - Enables static linking mode
+3. **Embedding**: The compiled `vec.o` object file is included in `libproxysql.a`
+4. **Auto-loading**: The extension is automatically registered when any SQLite database is opened
+
+### Modified Files
+
+#### Build Files
+- `../Makefile` - Updated to ensure git version is available during build
+- `../deps/Makefile` - Added compilation target for sqlite-vec
+- `../lib/Makefile` - Modified to include vec.o in libproxysql.a
+
+#### Source Files
+- `../lib/Admin_Bootstrap.cpp` - Added extension loading and auto-registration code
+
+### Database Instances
+
+The extension is enabled in all ProxySQL SQLite databases:
+- **Admin database** - Management interface
+- **Stats database** - Runtime statistics
+- **Config database** - Configuration storage
+- **Monitor database** - Monitoring data
+- **Stats disk database** - Persistent statistics
+
+## Usage
+
+Once ProxySQL is built and restarted, you can use vector search functions in any SQLite database:
+
+```sql
+-- Create a vector table
+CREATE VIRTUAL TABLE my_vectors USING vec0(
+    vector float[128]
+);
+
+-- Insert vectors with JSON format
+INSERT INTO my_vectors(rowid, vector)
+VALUES (1, json('[0.1, 0.2, 0.3, ..., 0.128]'));
+
+-- Perform similarity search
+SELECT rowid, distance
+FROM my_vectors
+WHERE vector MATCH json('[0.1, 0.2, 0.3, ..., 0.128]')
+LIMIT 10;
+```
+
+## Compilation Flags
+
+The sqlite-vec source is compiled with these flags:
+- `SQLITE_CORE` - Integrate with SQLite core
+- `SQLITE_VEC_STATIC` - Static linking mode
+- `SQLITE_ENABLE_MEMORY_MANAGEMENT` - Memory management features
+- `SQLITE_ENABLE_JSON1` - JSON support
+- `SQLITE_DLL=1` - DLL compatibility
+
+## Benefits
+
+- **No runtime dependencies** - Vector search is embedded in the binary
+- **Automatic loading** - No need to manually load extensions
+- **Full compatibility** - Works with all ProxySQL SQLite databases
+- **Performance** - Native SQLite virtual table implementation
+
+## Building
+
+The integration is automatic when building ProxySQL. The sqlite-vec sources are compiled and linked as part of the normal build process.
+
+## Verification
+
+To verify that sqlite-vec is properly integrated:
+1. Build ProxySQL: `make`
+2. Check symbols: `nm src/proxysql | grep vec`
+3. Should see symbols like `sqlite3_vec_init`, `vec0_*`, `vector_*`, etc.

deps/sqlite3/sqlite-vec-source/README.md

@@ -0,0 +1,111 @@
+# sqlite-vec - Vector Search for SQLite
+
+This directory contains the source files for [sqlite-vec](https://github.com/asg017/sqlite-vec), an SQLite extension that provides vector search capabilities directly within SQLite databases.
+
+## What is sqlite-vec?
+
+sqlite-vec is an open-source SQLite extension that enables SQLite to perform vector similarity searches. It implements vector search as a SQLite virtual table, providing:
+
+### Features
+- **Vector Storage**: Store vectors directly in SQLite tables
+- **Vector Indexing**: Efficient indexing for fast similarity searches
+- **Distance Functions**:
+  - Cosine distance
+  - Euclidean distance
+  - Inner product
+  - And more...
+- **Approximate Nearest Neighbor (ANN)**: High-performance approximate search
+- **Multiple Formats**: Support for JSON, binary, and other vector formats
+- **Batch Operations**: Efficient bulk vector operations
+
+### Vector Search Functions
+```sql
+-- Create a vector table
+CREATE VIRTUAL TABLE my_vectors USING vec0(
+    vector float[128]
+);
+
+-- Insert vectors
+INSERT INTO my_vectors(rowid, vector)
+VALUES (1, json('[0.1, 0.2, 0.3, ..., 0.128]'));
+
+-- Search for similar vectors
+SELECT rowid, distance
+FROM my_vectors
+WHERE vector MATCH json('[0.1, 0.2, 0.3, ..., 0.128]')
+LIMIT 10;
+```
+
+## Source Files
+
+### sqlite-vec.c
+The main implementation file containing:
+- Virtual table interface (vec0)
+- Vector distance calculations
+- Search algorithms
+- Extension initialization
+
+### sqlite-vec.h
+Header file with:
+- Function declarations
+- Type definitions
+- API documentation
+
+### sqlite-vec.h.tmpl
+Template for generating the header file.
+
+## Integration in ProxySQL
+
+These source files are integrated into ProxySQL through static linking:
+
+### Compilation Flags
+In ProxySQL's build system, sqlite-vec is compiled with these flags:
+- `-DSQLITE_CORE` - Compile as part of SQLite core
+- `-DSQLITE_VEC_STATIC` - Enable static linking mode
+- `-DSQLITE_ENABLE_MEMORY_MANAGEMENT` - Memory management features
+- `-DSQLITE_ENABLE_JSON1` - JSON support
+- `-DSQLITE_DLL=1` - DLL compatibility
+
+### Integration Process
+1. Sources are stored in this directory (committed to repository)
+2. During build, copied to the build directory
+3. Compiled with static linking flags
+4. Linked into `libproxysql.a`
+5. Auto-loaded when SQLite databases are opened
+
+## Licensing
+
+sqlite-vec is licensed under the [MIT License](LICENSE). Please refer to the original project for complete license information.
+
+## Documentation
+
+For complete documentation, examples, and API reference, see:
+- [sqlite-vec GitHub Repository](https://github.com/asg017/sqlite-vec)
+- [sqlite-vec Documentation](https://sqlite-vec.github.io/)
+
+## Building Standalone
+
+To build sqlite-vec standalone (outside of ProxySQL):
+```bash
+# Download source
+git clone https://github.com/asg017/sqlite-vec.git
+cd sqlite-vec
+
+# Build the extension
+gcc -shared -fPIC -o libsqlite_vec.so sqlite_vec.c -I/path/to/sqlite/include \
+  -DSQLITE_VEC_STATIC -DSQLITE_ENABLE_JSON1
+```
+
+## Performance Considerations
+
+- Use appropriate vector dimensions for your use case
+- Consider the trade-offs between exact and approximate search
+- Batch operations are more efficient than single-row operations
+- Indexing improves search performance for large datasets
+
+## Contributing
+
+This is a third-party library integrated into ProxySQL. For bugs, features, or contributions:
+1. Check the [sqlite-vec repository](https://github.com/asg017/sqlite-vec)
+2. Report issues or contribute to the sqlite-vec project
+3. ProxySQL-specific integration issues should be reported to the ProxySQL project