Do you really need a separate vector database for your RAG system?

Your team just got the green light to build a Retrieval-Augmented Generation (RAG) feature — maybe a support-ticket search assistant, an internal knowledge bot, or a product recommendation engine. The first architecture diagram someone draws almost always includes a dedicated vector database: Pinecone, Weaviate, Qdrant, Milvus. Suddenly you're not just building a feature, you're onboarding a new database service, a new billing relationship, a new thing for your ops team to monitor at 2am, and a new sync pipeline to keep it consistent with your actual system of record.

Here's the question worth asking before you sign up for another vendor: if you're already running PostgreSQL — which most business systems are — do you actually need a separate vector database at all? In many real-world RAG projects, the answer is no.

What problem is a vector database actually solving?

A RAG system needs to do one core thing well: given a user's question, find the chunks of text (documents, tickets, product descriptions, contracts) that are semantically similar to that question, then hand those chunks to an LLM as context.

To do that, every chunk of text gets converted into an embedding — a list of a few hundred to a few thousand floating-point numbers that represent its meaning. Finding "similar" text becomes a math problem: find the embeddings that are closest to the query's embedding, usually using cosine similarity or nearest-neighbor search.

Dedicated vector databases exist because, at very large scale, doing this search efficiently is genuinely hard. That's a real problem — for some companies. But most companies building an internal RAG tool are not Pinecone-scale. They're searching across 50,000 support tickets, 10,000 product records, or a few thousand contracts — not billions of vectors.

How does pgvector solve this inside PostgreSQL?

pgvector is an open-source PostgreSQL extension that adds a native vector data type and similarity search operators directly into Postgres. Instead of exporting your data to a separate service, you store the embedding right next to the row it belongs to.

Concretely, imagine a support_tickets table. Today it has columns like id, subject, body, created_at. With pgvector, you add one more column:

ALTER TABLE support_tickets ADD COLUMN embedding vector(1536);

You generate the embedding once (via OpenAI, Cohere, or a local model) when a ticket is created or updated, and store it in that column. To find similar tickets to a new query, you run:

SELECT id, subject, body
FROM support_tickets
ORDER BY embedding <=> '[0.0123, -0.045, ...]'
LIMIT 5;

That <=> operator is cosine distance, built into pgvector. No API calls to a third-party vector store, no separate authentication, no data duplication. The similarity search runs in the same database, the same transaction, and can even be combined with a normal WHERE clause — for example, only searching tickets from a specific customer or created in the last 90 days. That last part is something dedicated vector databases often make awkward, because filtering and vector search live in two different systems.

What do you actually save by not adding a vector database?

• No new billing relationship. Most managed vector databases charge per index, per read/write unit, or per GB stored — on top of whatever you already pay for hosting.
• No new ops burden. Every additional service is something your team has to patch, monitor, back up, and understand during an incident. A three-person dev team maintaining Postgres, a web app, and now also a vector database is a three-person team quietly becoming a four-service team.
• No sync pipeline. Without pgvector, you typically need a job that pushes new/updated records from Postgres into the vector store and keeps them in sync. That pipeline is itself a piece of software that can break, drift, or duplicate data.
• One backup strategy. Your embeddings are backed up the same way your regular data is — with pg_dump or your existing replication setup. No separate disaster-recovery plan for the vector store.
• Combined queries. You can join vector similarity with regular relational filters, full-text search, and business logic in a single SQL statement — something that usually requires stitching together two systems otherwise.

When does a dedicated vector database still make sense?

Being fair to the other side matters here — pgvector isn't a universal answer.

• Very large scale. If you're indexing tens of millions or billions of vectors with strict low-latency requirements, purpose-built vector databases with specialized indexing (HNSW variants tuned for massive scale, sharding, distributed search) can outperform Postgres.
• Extremely high query throughput. If vector search is your product's core, high-QPS workload — not a supporting feature — a system designed only for that job may be worth the operational overhead.
• Multi-tenant SaaS with strict isolation needs at huge scale. Some vector databases offer namespace/tenant isolation features that are still maturing in the Postgres ecosystem for very large multi-tenant setups.

For the vast majority of internal tools, customer-facing chatbots, document search, and RAG features built on top of an existing business system, none of these apply. You're dealing with thousands to low millions of records, not billions.

How do you set up pgvector for a RAG project step by step?

1. Confirm your Postgres version and hosting supports extensions. pgvector works on Postgres 13+ and is supported by most managed providers (Supabase, Neon, AWS RDS, Azure Database for PostgreSQL).
2. Install the extension. CREATE EXTENSION vector; — a single SQL command, no separate service to provision.
3. Add a vector column to the table that already holds your source data (tickets, articles, product descriptions) rather than creating a brand-new table disconnected from your system of record.
4. Choose an embedding model (OpenAI's text-embedding-3-small, Cohere, or a local model like all-MiniLM) and match the vector column's dimension to that model's output size.
5. Backfill embeddings for existing records with a one-time script, then generate embeddings automatically on create/update via a trigger, background job, or application-level hook.
6. Create an index for performance once your table grows — pgvector supports both IVFFlat and HNSW indexes: CREATE INDEX ON support_tickets USING hnsw (embedding vector_cosine_ops);
7. Write your retrieval query, combining ORDER BY embedding <=> query_vector with any relational filters your business logic needs.
8. Feed the top results into your LLM prompt as context, alongside the user's original question.

What are common gotchas teams run into?

• Forgetting to re-embed on updates. If a ticket's body is edited but the embedding isn't regenerated, your search quietly becomes stale. Build re-embedding into your update path, not as an afterthought.
• Mismatched vector dimensions. Switching embedding models mid-project (e.g., from a 1536-dimension OpenAI model to a 384-dimension local model) breaks existing rows silently unless you re-embed everything.
• No index on a growing table. Without IVFFlat or HNSW, pgvector does a sequential scan — fine at a few thousand rows, painfully slow past a few hundred thousand.
• Treating similarity score as certainty. Cosine similarity tells you what's closest, not what's correct. Always sanity-check retrieved chunks before trusting them blindly in production.

FAQ

Does pgvector work with any embedding model? Yes — pgvector just stores and searches vectors; it doesn't care which model produced them, as long as you're consistent about dimensions within a column.

Is pgvector as fast as a dedicated vector database? At small-to-mid scale (up to a few million vectors) with a proper HNSW index, performance is comparable for most real-world RAG use cases. At massive scale, specialized vector databases still have an edge.

Can I migrate later if I outgrow pgvector? Yes — since your embeddings live in normal Postgres rows, exporting them to a dedicated vector database later is a straightforward migration, not a rebuild.

Do I still need a separate embedding pipeline? Yes — pgvector stores and searches vectors, but generating the embeddings still requires calling an embedding model (hosted or local). pgvector removes the storage and search service, not the embedding step itself.

Checklist: is pgvector enough for your project?

• Your dataset is in the thousands-to-low-millions of records, not billions
• You're already running PostgreSQL for your core system
• You want vector search combined with normal relational filters
• Your team wants to avoid managing an additional service
• Query latency requirements are typical for a business application, not ultra-low-latency at massive scale

If most of these are checked, start with pgvector before reaching for a dedicated vector database — you can always migrate later if you genuinely outgrow it.

Whether you're adding a RAG-based search feature to an existing FileMaker system, building it into a custom web application, or connecting it to your ERP data through an API, the architecture decision of where embeddings live matters as much as the AI model you choose. Loggix helps teams map out this kind of decision up front — from database architecture to the AI layer on top of it — so you add real capability without adding unnecessary complexity to your stack.