RAG Architecture on Azure, AWS and Google Cloud: A Practical Guide

What is Retrieval-Augmented Generation (RAG)?

RAG is a pattern for grounding a large language model in information it was not trained on — your own documents, a product catalogue, support tickets, or anything that changes faster than a model can be retrained. At query time the system retrieves relevant chunks of your data and inserts them into the prompt, so the model answers from that supplied context.

The model still supplies the language ability; the facts come from the retrieved chunks, which is why a RAG answer can be traced back to a source document.

This matters for two reasons. It reduces hallucination, because the model is answering from text it can see rather than guessing from parametric memory, and it keeps answers current, because updating the answer means updating an index, not retraining a model. RAG is not a replacement for fine-tuning; fine-tuning changes how a model behaves or writes, RAG changes what facts it has access to, and production systems often use both.

What are the stages of a RAG pipeline?

Every managed RAG service on Azure, AWS and Google Cloud implements the same six stages: ingest data from a source, transform and chunk it, embed the chunks into vectors, index those vectors into a knowledge base, retrieve the closest matches for a query, then generate an answer grounded in them.

Google's own RAG Engine documentation describes the pipeline in exactly those six steps, and the product names on the other two clouds map onto the same sequence.

Where the clouds genuinely differ is in how much of this you configure yourself. Azure AI Search's integrated vectorization and Google's RAG Engine with its default managed database can chunk, embed and index automatically from a connected data source. Amazon Bedrock Knowledge Bases offers the same hands-off path, or a customer-managed path where you pick and operate the vector store yourself.

How does Azure build RAG?

Azure builds RAG on Azure AI Search as the retrieval engine, now surfaced through Foundry IQ, the managed knowledge layer for agents in the Microsoft Foundry portal. The platform names changed again in 2026: Azure AI Studio became Azure AI Foundry, which is now Microsoft Foundry, and Azure AI Services is now Foundry Tools.

Both the old and new names still appear across current documentation. Azure AI Search itself was not renamed.

Azure AI Search documents two RAG patterns. The classic pattern runs a single hybrid query, combining keyword and vector search for recall, with optional semantic ranking, scoring profiles and minimum-score thresholds. Agentic retrieval, in preview, lets an LLM decompose a question into parallel sub-queries across one or more knowledge sources, reranking and citing the results through a knowledge base retrieve API, with an adjustable reasoning effort of minimal, low or medium.

Integrated vectorization handles ingestion: indexers pull from 10 or more data source types, chunk content with a Text Split skill or a markdown-aware Content Understanding skill, call an embedding skill and write vector and text fields into the index automatically. The Text Split skill defaults to 2,000-character pages with 500-character overlap, a 25% ratio Microsoft also recommends starting from at the token level, around 512 tokens with 25% overlap. Azure OpenAI embedding models in the catalogue range from text-embedding-ada-002, fixed at 1,536 dimensions, to text-embedding-3-large, which supports up to 3,072, and every Azure AI Search tier shares the same 4,096-dimension ceiling per vector field. Microsoft Foundry's model catalogue lists over 10,000 models, with roughly 50 added every month.

How does AWS build RAG?

Amazon Bedrock Knowledge Bases offers two architectures. In a managed knowledge base, AWS runs embedding, reranking, parsing and agentic multi-hop retrieval, with built-in connectors for S3, SharePoint, Confluence, Google Drive, OneDrive and a web crawler. In a customer-managed knowledge base, you choose and operate the vector store.

The supported stores are Amazon OpenSearch Serverless or Managed Clusters, Amazon S3 Vectors, Aurora PostgreSQL with pgvector, Neptune Analytics as a graph-based store, and third-party options including Pinecone, Redis Enterprise Cloud and MongoDB Atlas.

Chunking strategies include fixed-size, a default of roughly 300 tokens that keeps sentences intact, no chunking at all for one chunk per document, hierarchical chunking with separate parent and child token sizes, and semantic chunking, which invokes a foundation model during ingestion to place boundaries at a tunable sentence-dissimilarity threshold. AWS specifically warns against pairing hierarchical chunking with an S3 Vectors store, since combined parent and child chunks over roughly 8,000 tokens can exceed its metadata limits.

Bedrock's own embedding models include Titan Embeddings G1 at a fixed 1,536 dimensions, Titan Text Embeddings V2 selectable at 256, 512 or 1,024 dimensions in float32 or binary, and Cohere Embed v3 at 1,024 dimensions, alongside three multimodal embedding models for images, audio and video. Binary vectors, cheaper but less precise than float32, are supported only on the two OpenSearch options. At query time, Retrieve returns matching chunks only, while RetrieveAndGenerate returns a cited, generated answer.

How does Google Cloud build RAG?

Google Cloud builds RAG with RAG Engine, now described as a component of the Gemini Enterprise Agent Platform. Google renamed Vertex AI to Agent Platform at Google Cloud Next '26: Vertex AI Search became Agent Search, Vertex AI Studio became Agent Studio and Agent Engine became Agent Runtime.

The rollout is still mid-transition in Google's own documentation, which places RAG Engine under the new platform while it still lives at a vertex-ai URL path, with old bookmarked links redirecting automatically.

RAG Engine defaults to 1,024-token chunks with 256-token overlap, tunable through chunk_size and chunk_overlap parameters, and indexes them into a corpus that can run on a zero-setup managed database, on Vector Search or Vector Search 2.0, on Feature Store, or on Weaviate, Pinecone or a dedicated Cloud Spanner instance. RAG Engine runs in 22 regions in total, generally available in Frankfurt and Eemshaven and allowlist-gated in Iowa and Virginia, and supports VPC Service Controls and customer-managed encryption keys, though not data residency controls.

AlloyDB for PostgreSQL offers a different route: pgvector alongside Google's own ScaNN index, which Google states builds indexes up to 10 times faster and runs filtered vector queries up to 10 times faster than stock PostgreSQL HNSW, with embedding and generation callable directly from SQL. On the embedding side, gemini-embedding-2 reached general availability the same day as the platform rename and is Google's first natively multimodal embedding model, accepting text, images, audio up to 180 seconds and video up to two minutes in a single request, producing vectors of up to 3,072 dimensions.

How do the three clouds compare for RAG?

The managed services converge on the same pipeline but differ most in how composable the vector store is, and in how hands-off the default path can be.

How do you choose a chunking and embedding strategy?

Start from each platform's own default rather than inventing a number: roughly 500 to 1,000 tokens with about 25% overlap is the common starting point across all three clouds, and each lets you tune it per data source.

Favour a sentence- or heading-aware chunker over a hard character cut wherever the platform offers one; splitting mid-sentence is a frequent, avoidable cause of poor retrieval.

Reach for hierarchical or semantic chunking only once a simple fixed-size baseline is measurably not retrieving well enough, since both cost more: semantic chunking invokes a model during ingestion, and hierarchical chunking needs care with a store's metadata limits. Match your embedding model's output dimensions to what your index or vector field actually accepts, Azure AI Search tops out at 4,096 dimensions per field regardless of tier, and remember that most embedding models cap single-input length well under 10,000 tokens, so long documents need chunking before embedding regardless of which cloud you use.

What does a RAG implementation checklist look like?

Treat a RAG pipeline as a product with its own evaluation loop, not a one-time integration.

1. 1Define what the system must answer and from which data sources, including how often that data changes.
2. 2Pick a managed path first, Azure AI Search integrated vectorization, Bedrock's managed knowledge base, or RAG Engine's default managed database, unless you already need a specific vector store.
3. 3Chunk with a sentence- or heading-aware strategy at each platform's default size and overlap, then adjust from measured retrieval quality, not guesswork.
4. 4Match embedding model, output dimensions and index or vector field configuration exactly before the first ingestion run.
5. 5Enable hybrid or semantic ranking where the platform supports it, since vector similarity alone misses exact keyword matches.
6. 6Decide whether retrieved answers must be cited back to source documents, and choose an API or pipeline step that returns citations if so.
7. 7Set access controls on the underlying data source so retrieval cannot surface content the querying user should not see.
8. 8Evaluate end to end with real questions before launch, then monitor retrieval quality as new content is added.

How can WIEWAVE help?

WIEWAVE is a cloud, DevOps and AI company that builds retrieval-augmented generation pipelines for clients worldwide on Azure, AWS and Google Cloud, from ingestion and chunking through vector indexing, retrieval tuning and production monitoring, delivered as infrastructure-as-code in your own cloud account.

If you already have a knowledge base, support archive or product catalogue sitting unused, grounding a model in it is usually the fastest way to make that data answer questions directly.