Local Vector Search with QQL Edge in Python and Node

Introduction

Local vector search with QQL Edge is a pattern where pyqql-edge or @veristamp/nqql-edge parses QQL queries, embeds text using fastembed-rs, and searches an embedded qdrant-edge storage shard in-process without requiring a remote Qdrant server.¹²

Running this execution path on Linux confirms that query processing stays local to the host process. Initial execution downloads ONNX model weights to a local cache directory. Subsequent vector inference runs directly on host hardware without outbound network calls. Using remote HTTP embedding endpoints represents a distinct architecture requiring external network connectivity.

Key Definitions & Operational Concepts

An In-Process Edge Executor (local_executor) is an embedded session instance running fastembed-rs inference and qdrant-edge storage inside host application memory.
The fastembed-rs Engine is a synchronous Rust library (fastembed crate) using @pykeio/ort for ONNX Runtime execution and @huggingface/tokenizers for fast text encoding without Tokio overhead.
A Fail-Closed Topology Check refers to returning explicit QQL-MISSING-USING error codes when querying multi-vector collections without specifying target vector fields.
An ONNX Local Model Cache (FASTEMBED_CACHE_DIR / HF_HOME) is a host file directory (cache_dir) that stores downloaded model weights to guarantee offline vector inference.
A Process File Lock Invariant represents single-process file access constraints protecting edge storage directories from concurrent write corruption.

1. How does QQL Edge execute local vector search in-process?

Integrating local vector search into Python applications requires installing pyqql-edge version 0.1.5:

pip install pyqql-edge==0.1.5

1.1 In-Process Python Setup with fastembed-rs

The local_executor constructor initializes the underlying storage engine and configures local fastembed-rs model loading:

from pathlib import Path
import pyqql_edge

# Configure storage directory and local embedding cache
storage_directory = Path("./qql-edge-data")
executor = pyqql_edge.local_executor(
    str(storage_directory),
    on_disk_payload=False,
    model="bge-small-en-v1.5",
    cache_dir="./fastembed-cache",
)

try:
    # 1. Create collection with hybrid dense + sparse support
    create_result = executor.execute("CREATE COLLECTION docs HYBRID")
    print("Collection Creation:", create_result)

    # 2. Insert point record with UUID identifier and payload metadata
    upsert_result = executor.execute("""
        UPSERT INTO docs VALUES {
          id: '550e8400-e29b-41d4-a716-446655440001',
          text: 'Refunds are available within thirty days of purchase.',
          category: 'billing'
        }
    """)
    print("Point Ingestion:", upsert_result)

    # 3. Perform local vector search query
    query_result = executor.execute(
        "QUERY TEXT 'refund window' FROM docs USING dense LIMIT 5",
        on_error="stop",
    )
    print("Search Output:", query_result)
finally:
    executor.close()

1.2 Execution Log Inspection

Inspecting execution reports confirms local operation details:

CREATE COLLECTION → ok=True, operation=CREATE_COLLECTION, succeeded=1
UPSERT            → ok=True, "Upserted 1 point(s)", count=1
QUERY             → ok=True, "Found 1 hits"
                    id=550e8400-e29b-41d4-a716-446655440001
                    score≈0.743
                    payload.category=billing

This workflow executes full text-to-vector embedding and similarity scoring without opening socket connections to localhost:6333. Always invoke executor.close() before releasing process resources or modifying storage paths.

In-process vector search offers significant operational advantages for desktop applications, edge microservices, and command-line utilities. By embedding vector storage directly inside application processes, developers eliminate the latency overhead associated with network socket serialization and remote database connections.

Eliminating network overhead improves response times for local retrieval workloads. In applications performing high-frequency vector lookups or offline data processing, in-process execution avoids TCP handshake overhead and payload serialization delays.

1.3 Node.js In-Process Binding

The Node.js implementation shares an identical API structure:

npm install @veristamp/[email protected]

const { localExecutor } = require("@veristamp/nqql-edge");

async function runLocalEdgeSearch() {
  const client = localExecutor("./qql-edge-data", {
    model: "bge-small-en-v1.5",
    onDiskPayload: false,
  });

  try {
    await client.execute("CREATE COLLECTION docs HYBRID");
    await client.execute(`
      UPSERT INTO docs VALUES {
        id: '550e8400-e29b-41d4-a716-446655440001',
        text: 'Refunds are available within thirty days of purchase.',
        category: 'billing'
      }
    `);

    const report = await client.execute(
      "QUERY TEXT 'refund window' FROM docs USING dense LIMIT 5",
      { onError: "stop" }
    );
    console.log("Node Execution Report:", report);
  } finally {
    await client.close();
  }
}

runLocalEdgeSearch();

Node.js developers benefit from native C++ binding integrations that bypass external service dependencies. Applications running inside AWS Lambda, Cloudflare Workers, or desktop Electron apps can perform local vector similarity scoring without managing separate database infrastructure.

Embedded vector engines simplify deployment pipelines in serverless environments. Rather than configuring complex network VPC peering rules or managing database connection pools, serverless functions package local edge binaries directly into deployment artifacts for immediate startup execution.

In-process execution also reduces cloud infrastructure costs for edge workloads. By avoiding dedicated vector database instances for small-scale datasets, edge microservices achieve low-latency vector similarity matching while running on standard compute instances.

2. Which error codes safeguard against ambiguous topology and invalid IDs?

QQL Edge avoids masking server errors with silent fallbacks. When statements violate storage contracts or query specifications, the engine returns explicit error codes:

2.1 Edge Runtime Error Matrix

Error Scenario	Specific Error Code	Runtime Root Cause
`QUERY TEXT 'refund' FROM docs LIMIT 5`	`QQL-MISSING-USING`	Ambiguous topology containing dense and sparse vectors; explicit `USING` target required
`QUERY TEXT 'x' FROM docs GROUP BY category`	`QQL-EDGE-QUERY-GROUPS`	Grouping aggregations require distributed cluster support
`UPSERT INTO docs VALUES { id: 'doc-1', ... }`	`QQL-EDGE-INVALID-POINT-ID`	Point identifiers must be integers or valid UUID strings
`QUERY TEXT 'x' FROM docs SHARD 'acme'`	`QQL-EDGE-SHARD-UNSUPPORTED`	Physical sharding applies only to distributed Qdrant clusters

2.2 Resolving Ambiguous Vector Topology

Executing a query without a USING spec on a hybrid collection produces an explicit failure:

[QQL-MISSING-USING] Collection 'docs' has an ambiguous vector topology.
Add USING <vector_name>. Available vectors: dense, sparse

To resolve this failure, explicitly declare the target vector index:

-- Query dense vector index
QUERY TEXT 'refund policy' FROM docs USING dense LIMIT 5;

-- Query sparse BM25 vector index
QUERY TEXT 'refund policy' FROM docs USING sparse LIMIT 5;

Custom SHARD clauses are also blocked on edge runners. Physical partition routing belongs on distributed Qdrant nodes, whereas tenant isolation is managed via inject_filter() as detailed in our 0.1.5 release notes and AST filter injection post.

Fail-closed error codes prevent subtle bugs during development. When an application attempts to perform un-targeted queries on hybrid collections containing multiple vector indices, QQL Edge surfaces QQL-MISSING-USING immediately rather than picking an arbitrary vector space.

Surfacing explicit error codes during local development ensures that query bugs are caught before code reaches production environments. When developers test queries against local edge executors, error codes enforce strict compliance with QQL syntax specifications.

Detailed error messaging accelerates local debugging workflows. By providing practical error codes and field suggestions, QQL Edge helps developers identify missing vector target specifiers or malformed point identifiers without consulting external documentation.

Providing explicit diagnostic error messages eliminates guesswork during application development. When query execution fails due to invalid parameters, developers receive clear error context specifying the exact field or clause that caused the execution failure.

3. How does local fastembed-rs execution differ from remote HTTP embedding?

Local edge execution requires four components working within host boundaries:

Your Application Process (pyqql-edge / nqql-edge)
Edge Storage Directory (qdrant-edge shard)
QQL Runtime Engine (qql-core / qql-plan)
Local fastembed-rs Engine & Model Files (cached in .fastembed_cache or FASTEMBED_CACHE_DIR)

When these elements are present, vector search operates offline without network communication.³

3.1 Local ONNX vs Remote HTTP Architecture

Contrast this with remote embedding architectures:

# Remote HTTP Embedding Architecture (requires network connectivity)
import pyqql

http_embedder = pyqql.HttpEmbedder(
    endpoint="https://embeddings.example.net/v1/embeddings",
    model="managed-model-v1",
    dimension=768,
    api_key="secret-api-key",
)

In air-gapped environments, model weights should be pre-bundled into container images during build stages. Startup validation scripts should confirm model cache availability (FASTEMBED_CACHE_DIR or HF_HOME) before serving traffic.

Local fastembed-rs execution keeps document contents and search queries strictly inside host memory, providing complete data privacy by design.

3.2 fastembed-rs CPU SIMD & Synchronous ort Engine Mechanics

Underneath qql-edge, local vector generation is handled by fastembed-rs (the Rust fastembed crate by Anush008). Unlike Python embedding wrappers that drag in heavy PyTorch or AsyncIO event loop runtimes, fastembed-rs is engineered specifically for fast, synchronous systems performance:⁴

Synchronous Execution: Operates synchronously without requiring a Tokio async runtime context, eliminating async task spawning overhead for batch embedding calls.
ort ONNX Runtime Integration: Uses @pykeio/ort (the Rust ONNX Runtime binding) to leverage CPU SIMD instructions (AVX-512, NEON) for sub-millisecond 384-dimensional vector inference.
tokenizers Integration: Uses @huggingface/tokenizers for high-throughput, thread-safe text encoding.
Quantized Model Support: Supports 8-bit quantized enums (such as EmbeddingModel::BGESmallENV15Q or EmbeddingGemma300MQ4) to reduce model memory footprints by up to 75% on edge hardware.
Multi-Modal & Sparse Support: Supports dense text models (BAAI/bge-small-en-v1.5, nomic-embed-text-v1.5), sparse models (SPLADEPPV1, bge-m3), image models (clip-ViT-B-32-vision), and cross-encoder rerankers (bge-reranker-base).

Pre-caching ONNX model weights in deployment artifacts guarantees deterministic startup performance. Industrial edge deployments operating in remote or bandwidth-constrained environments avoid runtime network dependencies by mounting cached model files during container initialization.

Using local fastembed-rs models protects user data privacy. Because text embedding occurs within host memory boundaries, sensitive document contents and search queries are never transmitted across external networks to third-party embedding APIs.

4. How do you verify disk persistence across process restarts?

Confirming data persistence requires writing records, closing the executor, and reopening the storage path in a fresh process instance:

4.1 Automated Process Restart Test Script

from pathlib import Path
import pyqql_edge

storage_path = Path("./qql-edge-data")

# Step 1: Write record and close executor session
ex1 = pyqql_edge.local_executor(str(storage_path), model="bge-small-en-v1.5")
ex1.execute("CREATE COLLECTION docs HYBRID")
ex1.execute("""
    UPSERT INTO docs VALUES {
      id: '550e8400-e29b-41d4-a716-446655440001',
      text: 'Persistent vector storage test.',
      status: 'active'
    }
""")
ex1.close()

# Step 2: Reopen storage directory in a new executor instance
ex2 = pyqql_edge.local_executor(str(storage_path), model="bge-small-en-v1.5")
try:
    report = ex2.execute(
        "QUERY TEXT 'persistent vector' FROM docs USING dense LIMIT 5",
        on_error="stop",
    )
    assert report["ok"] and report["succeeded"] == 1, "Persistence assertion failed!"
    print("Disk persistence successfully verified across process restart.")
finally:
    ex2.close()

4.2 Storage Troubleshooting Checklist

If the secondary executor yields zero hits, verify that temporary directories were not used and that identical embedding models were specified across both sessions. Modifying embedding models requires re-indexing existing payloads.

Verifying persistence across process restarts ensures that embedded storage engines retain payload metadata and vector index structures across application deployments or container restarts.

Automated persistence tests protect against unexpected storage data loss. Incorporating process restart assertions into integration test suites confirms that local database transactions flush to disk correctly before sessions terminate.

Persistent disk storage allows edge applications to maintain long-lived knowledge bases. Applications running on local servers or desktop operating systems can continuously ingest new points, flush index updates to disk, and query stored vectors across system reboots.

5. What platform binaries and lifecycle rules apply to edge runners?

Deploying QQL Edge across infrastructure targets requires following key operational practices:

5.1 Native Binary Distribution Rules

Platform Binary Support: Pre-built native binaries support Linux x64, Windows x64 (with optional DirectML GPU acceleration via ort), and macOS arm64 (Apple Silicon). macOS x86_64 legacy hardware requires building from source due to ONNX Runtime dependencies.¹²⁴
Model Identifier Consistency: Record embedding model versions alongside application code. Switching models without re-indexing produces inaccurate similarity scores.
Point Identifier Formatting: Ensure point keys use unsigned 64-bit integers or valid UUID strings (550e8400-e29b-41d4-a716-446655440001). Custom text strings (doc-101) are rejected.
Resource Closure: Always invoke close() on executor instances prior to application shutdown to flush index buffers to disk.
Browser Parsers: For browser-only query compilation without local vector storage, use the WASM playground pattern.

5.2 Resource Cleanup and Memory Limits

Managing native binary dependencies across CI/CD build environments requires verifying target architecture triples. When building container images for ARM64 server architectures or Apple Silicon laptops, ensure that the appropriate native ONNX execution library is packaged into your deployment bundle.

Proper lifecycle management prevents database index corruption. Calling close() explicitly flushes in-memory vector index updates to disk and releases file system locks, ensuring that subsequent process initializations acquire clean storage handles.

Monitoring memory consumption on edge nodes is essential for sustained performance.⁵ Embedded vector storage engines consume host RAM to cache payload indices and fastembed-rs ONNX session memory, so engineering teams should allocate memory limits based on expected vector dataset sizes. For gateway architectures, consult our QQL Gateway guide and QQL Go Retrieval Operations. For structured extraction, review our Schema-First LLM Wiki post.

Establishing clear operational guidelines for edge deployments prevents common runtime failures. By standardizing binary distribution, point key formatting, and resource cleanup across your infrastructure, QQL Edge delivers reliable in-process vector retrieval.

Summary and Key Takeaways

pyqql-edge and @veristamp/nqql-edge bring declarative vector search into local application processes. By combining fastembed-rs ONNX inference with embedded qdrant-edge storage, developers can build fast, offline-capable vector search applications.

In-Process Performance: Executes text embedding and vector search without server daemons using fastembed-rs.
Synchronous ONNX Execution: Powered by @pykeio/ort and @huggingface/tokenizers with zero Tokio dependency.
Explicit Error Invariants: Rejects ambiguous topologies (QQL-MISSING-USING) and invalid IDs (QQL-EDGE-INVALID-POINT-ID).
Verified Persistence: Preserves collections and payloads across process restarts.
Cross-Language Consistency: Delivers identical QQL syntax support across Python, Node.js, and Rust.

Explore our related posts on QQL 0.1.5 cross-runtime release, AST filter injection, and browser playground tooling to complete your vector search architecture.

Building applications with QQL Edge combines the performance of in-process C++ execution with the expressive power of declarative query syntax. Standardizing your retrieval logic on QQL ensures your codebase remains clean, maintainable, and ready for production scale.

QQL. pyqql-edge / qql-edge package repository and Python API exports. GitHub(opens in a new tab) · PyPI(opens in a new tab). ↩ ↩²
QQL. @veristamp/nqql-edge Node.js package distribution. npm(opens in a new tab). ↩ ↩²
Qdrant. “What is Qdrant Edge?” Embedded storage architecture documentation. Qdrant Docs(opens in a new tab). ↩
Anush008. fastembed-rs Rust library for generating vector embeddings and reranking locally via ort ONNX Runtime and tokenizers. crates.io/crates/fastembed(opens in a new tab) · GitHub(opens in a new tab). ↩ ↩²
pykeio. ort: Rust wrapper for Microsoft ONNX Runtime with CPU, DirectML, and CUDA execution providers. GitHub(opens in a new tab). ↩