{
  "slug": "text-to-sql-agents",
  "title": "Text-to-SQL and Database Agents",
  "description": "How agents answer questions over structured data by generating and executing SQL: schema context, few-shot prompting, self-correction, safety constraints, benchmarks (Spider, BIRD-SQL), and tooling (LangChain SQLDatabaseToolkit, LlamaIndex NLSQLTableQueryEngine, Vanna, MCP Postgres server).",
  "category": "Guide",
  "tags": [
    "text-to-sql",
    "sql",
    "database",
    "schema-linking",
    "agents",
    "rag",
    "security",
    "benchmarks",
    "langchain",
    "llamaindex"
  ],
  "updated": "2026-07-10",
  "premium": false,
  "canonical": "https://changegamer.ai/resources/text-to-sql-agents",
  "markdown": "https://changegamer.ai/resources/text-to-sql-agents.md",
  "outline": [
    {
      "depth": 2,
      "text": "Key facts",
      "anchor": "key-facts"
    },
    {
      "depth": 2,
      "text": "Why it is hard",
      "anchor": "why-it-is-hard"
    },
    {
      "depth": 2,
      "text": "Core techniques",
      "anchor": "core-techniques"
    },
    {
      "depth": 2,
      "text": "Safety",
      "anchor": "safety"
    },
    {
      "depth": 2,
      "text": "Transparency",
      "anchor": "transparency"
    },
    {
      "depth": 2,
      "text": "Benchmarks",
      "anchor": "benchmarks"
    },
    {
      "depth": 2,
      "text": "Tooling",
      "anchor": "tooling"
    },
    {
      "depth": 2,
      "text": "Practical checklist",
      "anchor": "practical-checklist"
    },
    {
      "depth": 2,
      "text": "Verified sources",
      "anchor": "verified-sources"
    }
  ],
  "related": [
    {
      "slug": "chunking-strategies-for-rag",
      "title": "Chunking Strategies for RAG",
      "description": "Practitioner reference for chunking documents before embedding: fixed-size, recursive, semantic, late chunking, and contextual retrieval — with a strategy comparison table, chunk-size and overlap tradeoffs, code/table/Markdown handling, embedding model context limits, and evaluation methods.",
      "url": "https://changegamer.ai/resources/chunking-strategies-for-rag"
    },
    {
      "slug": "agent-identity-authentication",
      "title": "Agent Identity and Authentication",
      "description": "How autonomous agents prove who they are and get authorized to act: workload identity vs. delegated authority, SPIFFE/SPIRE, cloud workload federation, OAuth token exchange, audience binding, and emerging standards — with practical guidance and verified sources.",
      "url": "https://changegamer.ai/resources/agent-identity-authentication"
    },
    {
      "slug": "agent-memory-context",
      "title": "Agent Memory and Context Management",
      "description": "Architecture reference for agent memory: types (working, long-term, episodic, semantic, procedural), context-management techniques (summarization, RAG, sliding windows, prompt caching), storage substrates, and memory frameworks — with security notes and cross-links to related guides.",
      "url": "https://changegamer.ai/resources/agent-memory-context"
    },
    {
      "slug": "agentic-browsers",
      "title": "Agentic AI Browsers: Comet, Atlas, and the Prompt-Injection Attack Surface",
      "description": "What agentic browsers (Perplexity Comet, the now-sunsetting ChatGPT Atlas, Microsoft Edge Copilot Mode, Opera Neon) are, how they differ from developer-facing computer-use APIs, and the documented prompt-injection attacks — CometJacking, indirect injection, hidden-text/screenshot instructions — that target the whole product category.",
      "url": "https://changegamer.ai/resources/agentic-browsers"
    }
  ],
  "furtherReading": [
    {
      "slug": "mcp-tool-description-injection",
      "title": "Defending MCP Clients Against Tool Description and Output Injection",
      "description": "Two distinct MCP injection surfaces — a tool description at connect-time and a tool's return value at call-time — and the client-side architectural patterns (Dual LLM, Action-Selector, Context-Minimization) that contain each one.",
      "url": "https://changegamer.ai/articles/mcp-tool-description-injection"
    },
    {
      "slug": "fraud-and-abuse-from-agent-traffic",
      "title": "Fraud and Abuse from AI Agent Traffic: What a Seller Should Detect",
      "description": "How a seller of APIs, content, or tools to AI agents spots and mitigates abuse once access is already granted — key sharing, over-scope scraping, spend-ceiling circumvention, spoofed identity, and rate-limit evasion patterns specific to autonomous agents.",
      "url": "https://changegamer.ai/articles/fraud-and-abuse-from-agent-traffic"
    }
  ],
  "body": "A text-to-SQL agent accepts a natural-language question, generates a SQL query, executes it against a real database, and synthesizes an answer from the result. The gap between a demo and a production system is wide: schema size, ambiguity, dialect differences, multi-table joins, and query safety all require deliberate engineering.\n\n## Key facts\n\n- Building a production text-to-SQL agent means confronting large schemas, imprecise natural language, differences between database engines, and the fact that a query can execute cleanly yet still return a wrong answer.\n- Giving the model table definitions, column descriptions, and a handful of sample rows per table — not table names alone — does more for accuracy than any other single change; on large schemas, a retrieval step first narrows the set down to only the relevant tables and columns.\n- Dynamically retrieving the few-shot examples closest to each new question beats a static example set, and one round of feeding a database error back to the model resolves most fixable mistakes, bounded to a small number of attempts.\n- A read-only database role is treated as a hard safety boundary rather than a prompt instruction; a security research team documented a real case in which an archived reference Postgres MCP server allowed a stacked SQL statement to slip past its read-only guard before it was fixed with prepared statements.\n- Mandatory controls beyond the read-only role include row limits on generated queries, an allowlist restricting which tables are exposed, banning statement stacking in favor of parameterized queries, and requiring human approval before any write.\n- On an enterprise-grade successor benchmark with real large schemas, the best models manage only roughly one-fifth correct, a sharp contrast with the original benchmark's now-routine above-90% execution accuracy.\n- On another leading benchmark, scores climbed from a baseline typical earlier in the year to past 80% by mid-2026, still short of the human execution-accuracy mark, with the gains attributed mainly to elaborate multi-step pipelines rather than raw single-model improvement.\n\n## Why it is hard\n\n**Schema understanding** — large databases have hundreds of tables and thousands of columns. Feeding the entire schema into the context window is often infeasible and noisy. The model must identify which tables and columns are relevant to the question (schema linking) before writing correct SQL.\n\n**Ambiguity** — natural language is imprecise. \"Sales last quarter\" requires knowing the calendar convention, time-zone handling, and whether \"sales\" means orders, invoices, or revenue rows.\n\n**Dialect differences** — SQLite, PostgreSQL, MySQL, BigQuery, Snowflake, and DuckDB differ in date functions, string operators, window syntax, and quoting rules. A prompt tuned for one dialect degrades on another.\n\n**Correctness** — a query can execute without error yet return a wrong answer (wrong JOIN condition, missing filter, off-by-one aggregation). Execution success does not imply semantic correctness.\n\n## Core techniques\n\n**Schema context injection** — provide the model with DDL (CREATE TABLE statements), column descriptions, and 2–3 sample rows per table. This is the single highest-leverage intervention. For large schemas, use retrieval to select only the relevant subset of tables and columns — see /resources/rag-retrieval-for-agents for the retrieval patterns.\n\n**Few-shot examples** — include 3–10 verified question/SQL pairs in the prompt. Examples calibrate the model to your dialect, naming conventions, and common query patterns. Dynamic few-shot (retrieving the most similar examples for each incoming question) consistently outperforms static few-shot on held-out questions.\n\n**Self-correction (run → read error → fix)** — after generating a query, execute it; if it returns a database error, feed the error message back to the model with the original question and schema and ask it to produce a corrected query. One correction round captures the majority of fixable errors. Limit to 2–3 attempts to avoid infinite loops.\n\n**Query validation before execution** — for higher-stakes pipelines, add a validation step: parse the SQL with a library (e.g., `sqlglot`) and check that all referenced table/column names exist in the schema before executing. This catches hallucinated identifiers without a live database round-trip.\n\n**Read-only constraint** — connect to the database with a read-only role (no INSERT/UPDATE/DELETE privileges). This is a hard safety boundary, not a prompt instruction.\n\n## Safety\n\nText-to-SQL agents combine two injection attack surfaces: the natural-language input (prompt injection) and the generated SQL (SQL injection). A user can craft a question that instructs the model to emit destructive SQL, or stacked statements that escape a read-only transaction. Datadog Security Labs documented a real example: the reference Anthropic Postgres MCP server allowed statement stacking that bypassed its read-only transaction guard (`COMMIT; DROP SCHEMA public CASCADE;` executed successfully), patched by switching to prepared statements.\n\n**Mandatory controls:**\n\n- **Read-only DB role** — enforce at the database connection level, not in the prompt.\n- **Row limits** — add `LIMIT N` to every generated query (or enforce via query rewriting) to prevent runaway full-table scans.\n- **Allowlist / schema restriction** — expose only the tables the agent needs; deny access to system catalogs and sensitive tables.\n- **No statement stacking** — use parameterized queries or prepared statement APIs; do not pass raw model output as a multi-statement string to the database driver.\n- **Human approval for writes** — if the use case requires write access, add an explicit human-in-the-loop confirmation step before executing any INSERT/UPDATE/DELETE.\n\nSee /resources/agentic-security-checklist (prompt injection mitigations) and /resources/code-execution-sandboxing (database sandboxing patterns).\n\n## Transparency\n\nReturn the generated SQL to the caller alongside the result. Agents that hide the SQL make it impossible to audit wrong answers or catch injection attempts. Logged SQL is also the primary signal for fine-tuning and few-shot improvement.\n\n## Benchmarks\n\n**Spider** (Yale LILY Lab) — 10,181 questions over 200 databases across 138 domains, with complex cross-domain SQL. The original benchmark where models now exceed 90% execution accuracy, making it a baseline rather than a frontier test. Official site: https://yale-lily.github.io/spider\n\n**Spider 2.0** (ICLR 2025 Oral) — enterprise-grade successor using real Snowflake, BigQuery, and SQLite databases with large, complex schemas. The best models score roughly 21% success rate, highlighting the gap between toy benchmarks and production workloads. GitHub: https://github.com/xlang-ai/Spider2\n\n**BIRD-SQL** — 12,751 question-SQL pairs over 95 large databases (33.4 GB total) spanning 37 professional domains including dirty/null data and external-knowledge requirements. Evaluates execution accuracy (EX): whether the generated query returns the correct result set. Human performance is ~93% EX; the leaderboard moved fast through mid-2026 — Google's Gemini-SQL2 (built on Gemini 3.1 Pro) posted 80.04% EX in June 2026, and Agentar-Scale-SQL reported 81.67% EX shortly after, both well past the ~72% figure typical earlier in the year. Gains have come mainly from elaborate multi-step pipelines, not raw single-model capability. BIRD-Interact accepted to ICLR 2026 (Oral). Official site: https://bird-bench.github.io/\n\n## Tooling\n\n**LangChain SQLDatabaseToolkit** — a set of tools (`QuerySQLDatabaseTool`, `InfoSQLDatabaseTool`, `ListSQLDatabaseTool`) used with `create_react_agent` (LangGraph). The toolkit handles schema introspection, query execution, and error feedback. Scope credentials to read-only before connecting. Docs: https://python.langchain.com/docs/modules/agents/toolkits/sql_database\n\n**LlamaIndex NLSQLTableQueryEngine** — generates SQL from natural language over a SQLAlchemy- connected database. For schemas too large to fit the context window, `SQLTableRetrieverQueryEngine` dynamically selects relevant tables at query time. Docs: https://developers.llamaindex.ai/python/examples/index_structs/struct_indices/sqlindexdemo/\n\n**Vanna (vanna-ai/vanna)** — open-source Python RAG framework for text-to-SQL. Trains on DDL, documentation strings, and verified question-SQL pairs stored in a vector store; uses retrieved examples as few-shot context at inference time. GitHub: https://github.com/vanna-ai/vanna\n\n**MCP Postgres server** — the reference `@modelcontextprotocol/server-postgres` (now archived at `modelcontextprotocol/servers-archived`) provided read-only PostgreSQL access as an MCP tool. Community successors with configurable access include `crystaldba/postgres-mcp`. All queries should run in READ ONLY transactions; see the Datadog SQL-injection case study for why prepared statements are required even in read-only mode. Archived repo: https://github.com/modelcontextprotocol/servers-archived/tree/main/src/postgres\n\n## Practical checklist\n\n1. Give the model the schema (DDL + column descriptions + sample rows), not just table names.\n2. Use dynamic few-shot: retrieve the 3–5 most similar verified examples per question.\n3. Connect with a read-only, schema-scoped database role — enforce at the driver, not the prompt.\n4. Add a `LIMIT` clause to every generated query.\n5. Validate SQL syntax and referenced identifiers before execution.\n6. Run the query; on error, feed the error back and retry (max 2–3 rounds).\n7. Return the SQL in the response for auditability.\n8. For writes, require explicit human approval.\n\nCross-links: /resources/rag-retrieval-for-agents · /resources/agentic-security-checklist · /resources/code-execution-sandboxing · /resources/evaluating-ai-agents\n\n## Verified sources\n\n- Spider benchmark (Yale LILY Lab): https://yale-lily.github.io/spider\n- Spider 2.0 (ICLR 2025 Oral, GitHub): https://github.com/xlang-ai/Spider2\n- BIRD-SQL benchmark: https://bird-bench.github.io/\n- LangChain SQLDatabaseToolkit docs: https://python.langchain.com/docs/modules/agents/toolkits/sql_database\n- LangChain SQL agent API reference: https://reference.langchain.com/python/langchain-community/agent_toolkits/sql/toolkit/SQLDatabaseToolkit\n- LlamaIndex text-to-SQL guide: https://developers.llamaindex.ai/python/examples/index_structs/struct_indices/sqlindexdemo/\n- Vanna (vanna-ai/vanna) GitHub: https://github.com/vanna-ai/vanna\n- MCP Postgres server (archived): https://github.com/modelcontextprotocol/servers-archived/tree/main/src/postgres\n- Datadog — SQL injection in the Postgres MCP server: https://securitylabs.datadoghq.com/articles/mcp-vulnerability-case-study-SQL-injection-in-the-postgresql-mcp-server/\n- BIRD-SQL leaderboard climb to ~80-82% EX by June 2026 (WebSearch-fallback, 3 sources — MarkTechPost, AI Navigate News, and BIRD leaderboard coverage): https://www.marktechpost.com/2026/06/12/google-releases-gemini-sql2-gemini-3-1-pro-text-to-sql-scores-80-04-on-bird-single-model-leaderboard/",
  "sources": [
    "https://yale-lily.github.io/spider",
    "https://github.com/xlang-ai/Spider2",
    "https://bird-bench.github.io/",
    "https://python.langchain.com/docs/modules/agents/toolkits/sql_database",
    "https://reference.langchain.com/python/langchain-community/agent_toolkits/sql/toolkit/SQLDatabaseToolkit",
    "https://developers.llamaindex.ai/python/examples/index_structs/struct_indices/sqlindexdemo/",
    "https://github.com/vanna-ai/vanna",
    "https://github.com/modelcontextprotocol/servers-archived/tree/main/src/postgres",
    "https://securitylabs.datadoghq.com/articles/mcp-vulnerability-case-study-SQL-injection-in-the-postgresql-mcp-server/",
    "https://www.marktechpost.com/2026/06/12/google-releases-gemini-sql2-gemini-3-1-pro-text-to-sql-scores-80-04-on-bird-single-model-leaderboard/"
  ]
}