In 2026, cloud-based vector databases are everywhere. They are powerful, but they also bring network latency, API costs, subscription models, and privacy concerns. If you are building a personal note search tool, a local knowledge base, or an internal enterprise catalog, you probably don't need a cloud cluster. You can run everything offline, on your own machine, using a single-file database.

This guide shows you how to implement local semantic search using two simple, free tools: Ollama (for generating vector embeddings locally) and SQLite (with its new native vector extensions or standard cosine similarity tables) to build a zero-cost search engine that works completely offline.

Why SQLite and Ollama?

This combination is the ultimate setup for private, low-overhead software development:

  • Zero API Costs: Ollama runs the models on your CPU/GPU. No OpenAI bill at the end of the month.
  • Absolute Privacy: Your text never leaves your computer. Perfect for sensitive notes or proprietary code.
  • Single-File Simplicity: SQLite stores your data and vectors in a single database file. Easy to back up, share, and manage.
  • High Performance: Queries on local tables take milliseconds rather than round-tripping to cloud APIs.

Step 1: Install Ollama and Retrieve Embeddings

First, install Ollama and download an embedding model. The model nomic-embed-text is small, incredibly fast, and specifically tuned for high-quality text retrieval.

# Start Ollama and download the model
ollama pull nomic-embed-text

We can generate an embedding vector (which is a list of 768 floating-point numbers representing the meaning of our text) using a simple HTTP POST request to Ollama's local port (11434):

curl http://localhost:11434/api/embeddings -d '{
  "model": "nomic-embed-text",
  "prompt": "SQLite is a lightweight, single-file database engine."
}'

Step 2: Designing the SQLite Database

In 2026, SQLite supports official vector extensions like sqlite-vec. If you are using a standard SQLite build without external modules, you can also store vectors as binary Blobs (floats) or as JSON arrays, and run a simple custom function or compute similarity in memory. Here is a simple schema that stores the document along with its embedding:

CREATE TABLE documents (
      id INTEGER PRIMARY KEY,
      title TEXT NOT NULL,
      content TEXT NOT NULL,
      embedding BLOB NOT NULL  -- 768 floating point numbers stored as binary data
  );

Step 3: Calculating Similarity

Semantic search works by taking the user's query, generating its embedding via Ollama, and finding the documents in the database whose embeddings have the highest cosine similarity (or lowest cosine distance) to the query embedding.

Here is a complete, dependency-free Python snippet to query the SQLite database and rank the results:

import sqlite3
import json
import urllib.request
import struct
import numpy as np

# Generate embedding from local Ollama service
def get_embedding(text):
    req = urllib.request.Request(
        "http://localhost:11434/api/embeddings",
        data=json.dumps({"model": "nomic-embed-text", "prompt": text}).encode('utf-8'),
        headers={'Content-Type': 'application/json'}
    )
    with urllib.request.urlopen(req) as res:
        response = json.loads(res.read().decode('utf-8'))
        return response['embedding']

# Convert list of floats to binary BLOB for storage
def serialize_vector(vector):
    return struct.pack(f'{len(vector)}f', *vector)

# Convert binary BLOB back to numpy array
def deserialize_vector(blob):
    num_floats = len(blob) // 4
    return np.array(struct.unpack(f'{num_floats}f', blob))

# Cosine similarity function
def cosine_similarity(v1, v2):
    return np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2))

# Search the local database
def search_notes(query_text, db_path="notes.db"):
    query_vector = np.array(get_embedding(query_text))
    conn = sqlite3.connect(db_path)
    cursor = conn.cursor()
    
    cursor.execute("SELECT id, title, content, embedding FROM documents")
    results = []
    for doc_id, title, content, emb_blob in cursor.fetchall():
        doc_vector = deserialize_vector(emb_blob)
        score = cosine_similarity(query_vector, doc_vector)
        results.append((score, title, content))
        
    # Sort by highest similarity score
    results.sort(key=lambda x: x[0], reverse=True)
    return results[:3]

Conclusion

Using this simple setup, you can build search interfaces that find matches by meaning rather than exact keywords. Searching for "how do I store structured data locally" will easily pull up your note titled "SQLite introduction" even if it doesn't contain the word "locally" or "structured". Best of all, it consumes negligible memory, runs instantaneously, and costs absolutely nothing.