NanoQdrant: Build your own Vector Database from Scratch in Rust

A high-performance, lightweight vector database written in Rust. Open-Source, Github: https://github.com/abdibrokhim/nanoQdrant Features

• ✨ Vector Similarity Search — Fast similarity search using multiple distance metrics
• 📊 Multiple Distance Metrics — Cosine, Euclidean, and Dot Product
• 🗂️ Collection Management — Organize vectors into collections
• 🏷️ Payload Support — Attach JSON metadata to vectors
• 🔍 Filtering — Search with payload-based filters
• 💾 Persistence — Optional disk persistence
• 🚀 REST API — Full-featured HTTP API
• ⚡ In-Memory Storage — Lightning-fast operations

Quick Start

Installation

1. Clone the repository

git clone <repository-url> cd rustvecdb

1. Build the project

cargo build --release

Running the Server

1. Run with in-memory storage (default)

cargo run

1. Run with persistent storage

STORAGE_PATH=./data/storage.json cargo run

1. Custom host and port

HOST=0.0.0.0 PORT=8080 cargo run The server will start at http://127.0.0.1:6333 by default.

API Usage Health Check curl http://localhost:6333/health

Create a Collection curl -X PUT http://localhost:6333/collections/my_collection \

 -H "Content-Type: application/json" \
 -d '{
   "name": "my_collection",
   "distance": "Cosine"
 }'

Distance options: Cosine, Euclidean, DotProduct

Insert Points (Vectors) Single Point curl -X PUT http://localhost:6333/collections/my_collection/points/550e8400-e29b-41d4-a716-446655440000 \

 -H "Content-Type: application/json" \
 -d '{
   "vector": [0.1, 0.2, 0.3, 0.4],
   "payload": {
     "name": "Item 1",
     "category": "electronics"
   }
 }'

Batch Insert curl -X POST http://localhost:6333/collections/my_collection/points \

 -H "Content-Type: application/json" \
 -d '[
   {
     "vector": [0.1, 0.2, 0.3, 0.4],
     "payload": {"name": "Item 1", "category": "electronics"}
   },
   {
     "vector": [0.5, 0.6, 0.7, 0.8],
     "payload": {"name": "Item 2", "category": "books"}
   }
 ]'

Search for Similar Vectors curl -X POST http://localhost:6333/collections/my_collection/points/search \

 -H "Content-Type: application/json" \
 -d '{
   "vector": [0.1, 0.2, 0.3, 0.4],
   "limit": 10
 }'

Search with Filter curl -X POST http://localhost:6333/collections/my_collection/points/search \

 -H "Content-Type: application/json" \
 -d '{
   "vector": [0.1, 0.2, 0.3, 0.4],
   "limit": 10,
   "filter": {
     "category": "electronics"
   }
 }'

Get a Point curl http://localhost:6333/collections/my_collection/points/550e8400-e29b-41d4-a716-446655440000

List All Points curl http://localhost:6333/collections/my_collection/points

Delete a Point curl -X DELETE http://localhost:6333/collections/my_collection/points/550e8400-e29b-41d4-a716-446655440000

List Collections curl http://localhost:6333/collections

Get Collection Info curl http://localhost:6333/collections/my_collection

Delete Collection curl -X DELETE http://localhost:6333/collections/my_collection

Get Storage Stats curl http://localhost:6333/stats

Distance Metrics Cosine Similarity Measures the cosine of the angle between vectors. Range: [-1, 1], where 1 means identical direction. similarity = (A · B) / (||A|| * ||B||)

Euclidean Distance Straight-line distance between vectors. Lower is more similar. distance = sqrt(Σ(a_i - b_i)²)

Dot Product Sum of element-wise multiplication. Higher is more similar. dot_product = Σ(a_i * b_i)

Example: Building a Simple Recommendation System import requests import json BASE_URL = "http://localhost:6333"

1. 1. Create a collection

requests.put(f"{BASE_URL}/collections/movies", json={

   "name": "movies",
   "distance": "Cosine"

})

1. 2. Add movie embeddings

movies = [

   {
       "id": "550e8400-e29b-41d4-a716-446655440000",
       "vector": [0.1, 0.2, 0.3, 0.4],
       "payload": {"title": "The Matrix", "genre": "sci-fi", "year": 1999}
   },
   {
       "id": "550e8400-e29b-41d4-a716-446655440001",
       "vector": [0.15, 0.25, 0.35, 0.45],
       "payload": {"title": "Inception", "genre": "sci-fi", "year": 2010}
   },
   {
       "id": "550e8400-e29b-41d4-a716-446655440002",
       "vector": [0.8, 0.1, 0.2, 0.3],
       "payload": {"title": "The Notebook", "genre": "romance", "year": 2004}
   }

]

for movie in movies:

   requests.put(
       f"{BASE_URL}/collections/movies/points/{movie['id']}",
       json={"vector": movie["vector"], "payload": movie["payload"]}
   )

1. 3. Search for similar movies

response = requests.post(f"{BASE_URL}/collections/movies/points/search", json={

   "vector": [0.1, 0.2, 0.3, 0.4],  # The Matrix embedding
   "limit": 3

}) results = response.json() print("Similar movies:", json.dumps(results, indent=2))

1. 4. Search with genre filter

response = requests.post(f"{BASE_URL}/collections/movies/points/search", json={

   "vector": [0.1, 0.2, 0.3, 0.4],
   "limit": 3,
   "filter": {"genre": "sci-fi"}

}) filtered_results = response.json() print("Sci-fi movies:", json.dumps(filtered_results, indent=2))

Development

Running Tests cargo test

Building for Production cargo build --release ./target/release/rustvecdb

Environment Variables

• HOST - Server host (default: 127.0.0.1)
• PORT - Server port (default: 6333)
• STORAGE_PATH - Path to persistence file (optional)
• RUST_LOG - Log level (info, debug, warn, error)

Architecture Diagram ┌────────────────────────────────────────────┐
│ REST API Layer │
│ (Actix-web Handlers) │
│ /collections /points /search /health │
└──────────────────────┬──────────────────────┘
┌──────────────────────▼──────────────────────┐
│ Storage Layer │
│ (VectorStorage + RwLock) │
│ — Collection Management │
│ — Thread-safe access │
│ — Persistence (JSON) │
└──────────────────────┬──────────────────────┘
┌──────────────────────▼──────────────────────┐
│ Collection Layer │
│ (HashMap<UUID, Point>) │
│ — Vector storage │
│ — Dimension validation │
│ — Search operations │
└──────────────────────┬──────────────────────┘
┌──────────────────────▼──────────────────────┐
│ Vector Operations │
│ — Cosine similarity │
│ — Euclidean distance │
│ — Dot product │
└─────────────────────────────────────────────┘

Project Structure rustvecdb/ ├── Cargo.toml # Project dependencies and metadata ├── Cargo.lock # Locked dependency versions ├── .gitignore # Git ignore rules ├── Makefile # Build and run commands │ ├── README.md # Main documentation ├── QUICK_START.md # Quick start guide ├── ARCHITECTURE.md # Architecture documentation ├── PROJECT_STRUCTURE.md # This file │ ├── src/ │ ├── main.rs # Application entry point │ ├── lib.rs # Library exports │ ├── models.rs # Data structures and types │ ├── vector.rs # Vector similarity operations │ ├── collection.rs # Collection management │ ├── storage.rs # Storage layer │ └── api.rs # REST API handlers │ └── examples/

   ├── basic_usage.rs        # Rust example
   └── api_client.py         # Python API client example

File Descriptions Root Files

• Cargo.toml: Rust package manifest with dependencies
• Cargo.lock: Locked versions for reproducible builds
• .gitignore: Ignores target/, data/, *.log files
• Makefile: Convenient commands (make run, make test, etc.)

Documentation

• README.md: Comprehensive guide with API documentation and examples
• QUICK_START.md: Get started in 5 minutes
• ARCHITECTURE.md: Detailed architecture and design decisions
• PROJECT_STRUCTURE.md: This file describing the project layout

Source Code (src/) main.rs - Application Entry Point

• Initializes the HTTP server
• Configures logging
• Sets up routes
• Reads environment variables

lib.rs - Library Exports

• Public API for the library
• Module declarations
• Re-exports main types

models.rs - Data Structures Contains:

• Point: Vector with ID and payload
• DistanceMetric: Cosine, Euclidean, DotProduct
• SearchRequest: Search query parameters
• SearchResult: Search results with scores
• ApiResponse<T>: Generic API response wrapper
• CollectionInfo: Collection metadata

vector.rs - Vector Operations Functions:

• cosine_similarity(): Angle-based similarity
• euclidean_distance(): Straight-line distance
• dot_product(): Element-wise multiplication sum
• magnitude(): Vector norm
• normalize(): Convert to unit vector
• calculate_similarity(): Dispatch based on metric

Includes unit tests for all functions. collection.rs - Collection Management

• Collection struct manages vectors in a collection
• Methods:
• upsert_point(): Insert or update vector
• get_point(): Retrieve by ID
• delete_point(): Remove vector
• search(): Find similar vectors
• search_with_filter(): Search with payload filters

Includes unit tests. storage.rs - Storage Layer

• VectorStorage struct manages multiple collections
• Thread-safe with Arc<RwLock<>>
• Methods:
• create_collection(): Create new collection
• get_collection(): Retrieve collection
• delete_collection(): Remove collection
• update_collection(): Modify collection
• persist_to_disk(): Save to JSON file
• load_from_disk(): Load from JSON file

api.rs - REST API Handlers Actix-web handlers:

• health_check(): GET /health
• get_stats(): GET /stats
• create_collection(): PUT /collections/{name}
• list_collections(): GET /collections
• get_collection(): GET /collections/{name}
• delete_collection(): DELETE /collections/{name}
• upsert_point(): PUT /collections/{name}/points/{id}
• batch_upsert_points(): POST /collections/{name}/points
• get_point(): GET /collections/{name}/points/{id}
• delete_point(): DELETE /collections/{name}/points/{id}
• search_points(): POST /collections/{name}/points/search
• get_all_points(): GET /collections/{name}/points

Examples (examples/) basic_usage.rs Demonstrates:

• Creating a collection
• Inserting vectors with payloads
• Searching for similar vectors
• Filtering by payload

Run with: cargo run --example basic_usage api_client.py

Python script demonstrating REST API usage:

• Health check
• Creating collections
• Batch inserting points
• Searching with and without filters
• Getting statistics

Run with: python examples/api_client.py (server must be running)

Build Artifacts (not in repo) target/ ├── debug/ # Debug builds │ └── rustvecdb # Debug binary └── release/ # Release builds

   └── rustvecdb      # Optimized binary

Data Files (not in repo) data/ └── storage.json # Persistent storage file Created when using STORAGE_PATH=./data/storage.json

Dependencies Summary

Production

• actix-web: Web framework
• tokio: Async runtime
• serde: Serialization
• uuid: ID generation
• nalgebra: Linear algebra
• env_logger: Logging

Development

• reqwest: HTTP client for tests

Key Design Patterns

• Repository Pattern: VectorStorage acts as repository
• Builder Pattern: Actix-web app configuration
• Strategy Pattern: Different distance metrics
• Wrapper Pattern: ApiResponse<T> wraps all responses

API Design Philosophy

• RESTful: HTTP methods map to CRUD operations
• JSON: All data exchanged as JSON
• Consistent: Same response format for all endpoints
• Idempotent: PUT operations are idempotent

Testing Strategy

• Unit tests in vector.rs and collection.rs
• Integration tests via examples
• Manual testing with Python client
• Run with: cargo test

Performance Considerations

• In-Memory Storage: All data is kept in RAM for fast access
• Linear Search: Current implementation uses brute-force search (O(n) complexity)
• Persistence: Writes entire dataset to disk on updates

Future Optimizations

• HNSW (Hierarchical Navigable Small World) index for faster search
• Product Quantization for memory efficiency
• Sharding for horizontal scaling
• Write-Ahead Log (WAL) for safer persistence
• gRPC API for better performance
• Sparse vector support
• Advanced filtering with query DSL