QQL inject_filter: AST Isolation Proven Before Execution

Introduction

QQL filter injection is a typed Abstract Syntax Tree (AST) transformation, executed via inject_filter(), that merges mandatory tenant comparisons into parsed query statements prior to plan generation or database execution.¹ The injection mechanism does not handle user authentication, validate OAuth tokens, or configure Qdrant collections. Instead, it transforms statement syntax trees under the direct control of the process managing your application API boundary.

Verifying security enforcement requires testing against live parser evaluation and Rust test suites designed to detect logical bypass vectors, such as OR expansion, NOT negation, subquery CTE recursion, and UPSERT payload modification.¹² While strong security architectures require API gateways (detailed in our QQL Gateway guide), filter injection represents the core structural primitive invoked immediately after authorization.

---

Key Definitions & Security Concepts

• An AST Filter Transformation (inject_filter) is a host-level query compilation step that merges mandatory tenant comparison nodes into parsed query trees before REST/gRPC serialization.
• A Root AND Container is the top-level logical node that enforces injected tenant predicates across all evaluated sub-clauses.
• A Logical Bypass Vector refers to query inputs (such as WHERE true OR tenant = 'other') designed to escape payload filtering when naive string concatenation is used.
• Payload Provenance Stamping means writing execution metadata (batch IDs, timestamps, worker IDs) into point payload dictionaries during UPSERT ingestion.

---

1. How does the filter injection execution lifecycle work?

Production security pipelines enforce a strict sequence of execution stages before statements reach the database engine:

authenticate → authorize → parse → inject_filter → inspect → plan → execute

1.1 The Seven-Stage Security Pipeline

Attempting to enforce policy through text manipulation fails because string formatters cannot evaluate nested logical operators, Common Table Expressions (CTEs), or prefetch query branches. Typed AST transformation parses input text into structured nodes before applying structural transformations:

import pyqql
import json

raw_query = (
    "QUERY TEXT 'price adjustment policy' FROM knowledge "
    "USING dense "
    "WHERE visibility = 'public' OR visibility = 'partner' "
    "LIMIT 10"
)

# Parse raw string into Abstract Syntax Tree
statement = pyqql.parse(raw_query)[0]

# Inject mandatory security filter at the AST root
pyqql.inject_filter(statement, "deleted", "=", False)

# Print transformed AST filter representation
print("Transformed Filter AST:")
print(json.dumps(statement.to_dict()["Query"]["filter"], indent=2))

# Inspect execution plan
print("\nGenerated Execution Plan:")
print(pyqql.explain(statement)["plan"])

1.2 Structural AST Inspection Before Execution

Parsing input text into a typed Abstract Syntax Tree guarantees that structural query boundaries are established before security filters are injected. String formatting techniques operate on raw characters, making them vulnerable to unexpected quote escaping or parenthesis imbalance. In contrast, AST transformation operates on validated node trees, ensuring that security predicates are merged into exact logical positions.

Applying security policies at the AST layer allows host processes to inspect query structure before sending compilation plans to database engines. Security teams can log transformed AST representations to prove that mandatory tenant boundaries were applied to every outgoing request.

---

2. Why does outer AND defeat logical OR and NOT bypasses?

Before filter injection, the AST filter consists of an Or node combining two visibility conditions. Invoking inject_filter(statement, "deleted", "=", False) wraps the existing expression inside a root And node:

2.1 AST Structural Transformation Output

{
  "And": {
    "operands": [
      {
        "Or": {
          "operands": [
            { "Compare": { "field": "visibility", "op": "Eq", "value": { "Str": "public" } } },
            { "Compare": { "field": "visibility", "op": "Eq", "value": { "Str": "partner" } } }
          ]
        }
      },
      {
        "Compare": {
          "field": "deleted",
          "op": "Eq",
          "value": { "Bool": false }
        }
      }
    ]
  }
}

2.2 Proof of Resistance Against Negation Attacks

The generated execution plan reports Filter: present. User-supplied conditions occupy operand position 0 inside the root And node, while injected security policies occupy operand position 1.

The identical structural guarantee applies to negation attacks. If a caller submits WHERE NOT tenant_id = 'acme', injecting tenant_id = 'acme' produces an explicit logical contradiction (NOT tenant_id = 'acme' AND tenant_id = 'acme'), yielding zero match results and preventing unauthorized record access. This security invariant is verified by core integration tests (injection_resists_negation_bypass).²

Supported filter comparison operators include =, >, >=, <, and <=. Non-deterministic or broad operators such as !=, IN, or MATCH are restricted from injection to keep policy primitives predictable.¹

Evaluating policy nodes inside a root And container ensures that user-supplied logical conditions can never expand the search boundary beyond injected constraints. Even if a malicious input constructs elaborate logical trees containing nested OR conditions or inverted NOT expressions, the root And wrapper guarantees that injected tenant constraints remain mandatory across all evaluated branches.

---

3. How does inject_filter recurse across CTEs and prefetches?

Complex retrieval queries frequently use Common Table Expressions (WITH blocks) and subquery prefetch clauses to fetch candidate vectors across multiple indices:

3.1 CTE and Prefetch Query Structure

import pyqql

query_with_ctes = """
WITH
  dense_candidates AS (QUERY TEXT 'security audit' FROM knowledge USING dense LIMIT 100),
  sparse_candidates AS (QUERY TEXT 'security audit' FROM knowledge USING sparse LIMIT 100)
QUERY TEXT 'security audit' FROM knowledge
USING dense
PREFETCH (dense_candidates, sparse_candidates)
LIMIT 10
"""

statement = pyqql.parse(query_with_ctes)[0]

# Inject policy filter across main query and all subqueries
pyqql.inject_filter(statement, "project_id", "=", "proj_9921")

ast_tree = statement.to_dict()["Query"]

# Verify root query filter
assert ast_tree["filter"]["Compare"]["field"] == "project_id"

# Verify recursive injection into CTE subqueries
assert ast_tree["ctes"][0]["query"]["filter"]["Compare"]["value"]["Str"] == "proj_9921"
assert ast_tree["ctes"][1]["query"]["filter"]["Compare"]["value"]["Str"] == "proj_9921"
print("Recursive CTE filter injection verified successfully.")

3.2 Preventing Candidate Leakage in Hybrid Search

Restricting filter injection to the outer query block allows candidate generation subqueries to scan unrestricted index spaces. Recursive AST traversal ensures mandatory tenant conditions are applied to every subquery branch within the statement tree.

Subquery prefetch clauses used in hybrid retrieval workflows are equally vulnerable to tenant isolation leaks if left un-policed. When candidate generation subqueries retrieve vectors without tenant constraints, unauthorized candidate vectors enter fusion scoring pipelines. Applying inject_filter() recursively across all CTE definitions and prefetch trees eliminates candidate leakage across multi-stage retrieval pipelines.

---

4. How does equality injection stamp UPSERT point provenance?

When executing point ingestion via UPSERT, equality filter injection modifies point payload dictionaries directly, writing specified key-value pairs without altering record identifiers:

4.1 Provenance Payload Ingestion Example

import pyqql

upsert_query = "UPSERT INTO documents VALUES {id: 402, text: 'Q3 Security Roadmap'}"
statement = pyqql.parse(upsert_query)[0]

# Inject ingestion provenance tag into point payload
pyqql.inject_filter(statement, "ingestion_batch", "=", "2026-07-30-import-01")

point_payload = statement.to_dict()["Upsert"]["points"][0]["payload"]
print("Updated Point Payload:", point_payload)
# Output contains: ["ingestion_batch", {"Str": "2026-07-30-import-01"}]

4.2 Fail-Closed Validation Matrix across Statement Types

Production ingestion pipelines should stamp immutable execution IDs into point payloads rather than mutable labels, ensuring ingestion history can be audited across storage layers.

Stamping provenance metadata during UPSERT operations simplifies data lifecycle management and auditing. When data pipelines import points from external feeds, injecting batch identifiers, pipeline versions, or ingestion timestamps ensures that records can be tracked or purged selectively using payload matching filters.

QQL 0.1.5 implements strict fail-closed enforcement when inject_filter() is invoked against unsupported statement types:

Statement Category	AST Transformation Behavior	Operational Result
QUERY	Merges policy filter into root And node; recurses CTEs and prefetches	Filter applied to retrieval
SCROLL, COUNT	Merges policy filter into query filter payload	Filter applied to aggregation
DELETE, UPDATE PAYLOAD, CLEAR PAYLOAD	Merges policy filter into deletion selector	Operation scoped by policy
UPSERT (Equality on Non-ID Field)	Stamps key-value pair into point payload dictionary	Ingestion provenance recorded
DDL / SHOW / UPDATE VECTOR	Rejects statement with QQL-VALIDATION-FILTER-INJECT error	Operation blocked immediately

---

5. How do isolation filters differ from physical shard routing?

A common architectural misunderstanding equates physical database sharding with tenant isolation. In QQL 0.1.5, security filters and node routing are explicitly decoupled:

5.1 System Layer Responsibility Separation

System Layer	Operational Tool	Wire Mechanism
Security Isolation	inject_filter() or WHERE clause	Qdrant Filter payload matching
Physical Routing	SHARD 'key' clause or stmt.shard_key	REST shard_key / gRPC ShardKeySelector
Index Layout	CREATE INDEX … is_tenant = true	Payload index optimization
Partition Management	CREATE SHARD KEY 'key'	Cluster administration API

5.2 Programmatic Partition & Isolation Workflow

import pyqql

statement = pyqql.parse("QUERY TEXT 'audit' FROM docs USING dense LIMIT 10")[0]

# Security Isolation: Mandatory filter enforced via AST
pyqql.inject_filter(statement, "tenant_id", "=", "acme")

# Physical Routing: Target node selector (valid on custom-sharded collections)
statement.shard_key = "acme"

There is no inject_shard_key function in QQL 0.1.5. Node routing improves query performance and data locality, but security isolation depends entirely on inject_filter() and WHERE conditions. Distributed clusters handle physical sharding, whereas local edge runners evaluate filters in-process as detailed in our local edge search post.

Decoupling isolation filters from physical node routing protects applications against data leakage during cluster rebalancing or partition migrations. Node routing directs network traffic to specific hardware nodes, but inject_filter() guarantees that query execution remains constrained by payload security predicates across all nodes. For complete gateway patterns, review our QQL Gateway guide and QQL Go Retrieval Operations.

---

6. What host security policies can engineering teams implement?

Filter injection supports diverse security and operational constraints beyond tenant isolation:

6.1 Enterprise Security Policy Specifications

Security & Governance Policy	Applied Filter Specification	Engineering Purpose
Soft Delete Enforcement	deleted = false	Excludes archived records from retrieval
Environment Scoping	environment = 'production'	Prevents staging record exposure in production
Multi-Region Compliance	region = 'eu-central-1'	Restricts data access to authorized geographic regions
Content Moderation	moderation_status = 'approved'	Filters unverified user content
Multi-Tenant Isolation	tenant_id = 'org_4412'	Enforces multi-tenant data boundaries
Pipeline Ingestion Provenance	ingested_by = 'etl_worker_v3'	Stamps audit metadata during UPSERT operations

6.2 Sequential Policy Stacking in CI/CD

Multiple filter calls can be stacked sequentially. Engineering teams should include automated snapshot assertions in CI pipelines, verifying transformed AST structures for both simple queries and complex CTE subqueries across all target language bindings (Rust, Python, Node, WASM).

Stacking multiple security policies enables fine-grained access control models. For example, an enterprise retrieval endpoint can enforce tenant isolation, soft-delete filtering, and region compliance simultaneously by applying sequential inject_filter() transformations to a single parsed statement tree before execution. For governed workflows, consult our Governed Code Mode guide and Schema-First LLM Wiki post.

---

Summary and Key Takeaways

inject_filter() provides a verified, fail-closed mechanism for applying security policies to QQL queries. By operating directly on Abstract Syntax Trees rather than text strings, filter injection guarantees that security rules remain unbreakable regardless of user input structure.

• AST Structural Invariants: Outer And wrapping prevents OR and NOT query bypasses.
• Recursive Subquery Scoping: Automatically applies security filters across CTE expressions and prefetch branches.
• Payload Provenance Stamping: Writes audit key-value pairs into point payloads during UPSERT operations.
• Fail-Closed Validation: Blocks unsupported DDL statements with QQL-VALIDATION-FILTER-INJECT errors.

Combine inject_filter() with our QQL Go Security Gateway pattern and explore QQL 0.1.5 release features to establish zero-trust vector retrieval across your application stack.

Footnotes

1. QQL. Host isolation architecture and filter injection specification. docs/inject_filter.md(opens in a new tab). ↩ ↩² ↩³

2. QQL. Core AST transformation and logical bypass test suites (injection_resists_logical_or_bypass, CTE recursion, UPSERT payload stamping). transform_tests.rs(opens in a new tab). ↩ ↩²