The essential guide to building a data foundation for agentic AI

Most organizations are investing in AI to increase workplace efficiency. Many have implemented some level of generative AI and are now looking towards the next step of automating complex workflows with agentic AI.

Success in agentic AI depends on many factors, but none may be as important as your data foundation—the architectural readiness, processes, and mechanisms that allow AI agents in your organization to safely and effectively work with high-quality data.

Notably different than traditional data workflows that operate on query and response patterns, AI agents can independently reason, plan, and execute complex tasks to accomplish predetermined goals with minimal human supervision. To produce the best intended outcomes, AI agents need to discover and access high-quality data. They need to operate with the right context, memory, and latency to deliver relevant results quickly, and because they can act autonomously across large amounts of data, data governance is essential.

Preparing your data foundation for agentic AI is an evolutionary process and does not mean starting from scratch. You can begin where you are today and assess the limitations of your current capabilities as you build. Implementing AI agents based on your current state can help you prioritize which areas to invest in. 

This guide is built around the essential questions that will help you understand what AI agents need from a data foundation.

Data is very often spread across business units, legacy systems, and vendor platforms. This is commonly the result of decentralized purchasing decisions, distinct business unit needs, and rapid growth. However, it leads to data silos that can be expensive and cumbersome to overcome. And because underlying systems are built to support specific, defined access patterns, scaling to support the extra load from AI agents becomes difficult. Data foundations built on open standards allow organizations to make their data accessible to AI agents at scale while planning for future flexibility into their overall architecture.

Open table formats

Open table formats (OTFs) like Apache Iceberg bring database-like capabilities to data lakes, including ACID transactions and the ability to evolve data structures. Critically, OTFs enable interoperability, meaning any compatible tool, engine, or AI agent can safely work with the same data without reformatting or duplication. This matters especially as AI agents operate across multiple engines and tools to complete workflows. 

Open table formats also underpin lakehouse architectures, which combine OTF-based data lakes with a centralized governance layer, allowing them to act like data warehouses. Lakehouse architectures are scalable, flexible, and provide a unified system for storing and processing data. They make it easier to consolidate datasets into repeatable patterns, often called data products, with broad analytics tool compatibility and centralized access controls.

Open source software

Open source technologies like PostgreSQL for transactional data, Apache Kafka for data streaming, Apache Spark for data processing, and OpenSearch for search give organizations proven, widely adopted building blocks. Because these technologies are built on open standards with broad community support, AI agents benefit from a wider pool of publicly available code, documentation, and other information to help them make more accurate decisions.

Open protocols

Open protocols provide a standardized way for AI agents to interact with each other and the world around them. The Model Context Protocol (MCP) connects AI agents to external data sources, tools, and APIs so they can retrieve context and take action. The Agent2Agent Protocol (A2A) enables AI agents built on different frameworks to communicate with each other. Together, these protocols mean an AI agent can pull from multiple sources in a reasoning chain and hand off tasks to other AI agent.

AI agents need to be able to find the right data for their use case and often rely on a variety of data types in a single workflow. This can include structured data like customer records, transactions, and inventory levels, semi-structured data stored in HTML, JSON, or XML files, and unstructured data like documents, support tickets, or email archives. When data is spread across different systems and formats, AI agents must be able to locate and evaluate each. 

Data catalogs play an important role in helping AI agents discover what data is available to them, what it means, and where it is stored. Data catalogs include metadata, which is descriptive information about data including schemas, data types, lineage, freshness, and ownership. This is especially important in helping AI agents understand what data is contained in a source. 

When an organization maintains multiple catalogs for performance, compliance, or business reasons, catalog federation allows AI agents to discover data stored across systems. Some organizations go further by consolidating structured, semi-structured, and unstructured data into a single storage layer. This simplifies discovery by giving AI agents fewer places to look and a consistent way to find and evaluate data across types.

Before AI agents can use data effectively, it often needs to be cleaned, enriched, and transformed. Raw data may contain inconsistent formats, missing values, or redundant records that degrade an AI agent's ability to reason accurately. While AI agents can perform some data preparation at runtime, data transformation tools and pipelines handle preparation reliably at scale. Data transformation pipelines for agentic AI may also include vector embedding generation, chunking for Retrieval Augmented Generation (RAG), metadata generation and enrichment, and data quality checks. 

 A layered data architecture can be used to prepare high-quality, trusted data for AI. Often implemented in lakehouses where OTFs provide the transactional guarantees needed to maintain consistency between layers, this approach organizes data into zones based on consumption readiness: a raw zone for data as it arrives from source systems, a cleaned zone for validated and standardized data, and a curated zone with business-ready datasets optimized for downstream use. This provides a structured method for data to be cleaned and shaped for its intended use.

Many AI agents rely on access to high-quality data that reflects current conditions. Connecting directly to an operational database is one way to ensure an AI agent sees the same up-to-date information your applications use. For use cases that depend on multiple sources of data in near real-time, or when direct access to an operational database puts performance or security at risk, streaming pipelines can be used to continuously ingest events and updates as they occur. Streaming pipelines use change data capture (CDC) processes to capture changes from operational databases and publish them to an event stream where they become available for other downstream systems to consume, often within milliseconds. 

Before streaming data is passed to an AI agent, organizations should consider applying deterministic cleansing rules to incoming data events. These rules can catch large quality issues quickly and work in tandem with AI agents that can capture subtle quality issues on the data not filtered out by the deterministic rules.

Implementing data and AI governance is a critical initial step towards ensuring AI agents can operate safely and securely on high-quality data. Governance is especially important for AI agents because they dynamically discover and access data to make decisions, increasing the risk of exposing sensitive data or accessing resources beyond their intended scope if proper controls are not in place. 

Data sensitivity and access management

In general, and especially in highly regulated environments, data should be audited for sensitivity and appropriately tagged and classified, with highly sensitive information remaining inside secure environments. Access management should follow the principle of least privilege, meaning access should be granted only to the information required to complete a task. Any roles and access permissions granted should be closely monitored and revokable to ensure safety at scale.

Auditing, lineage, and observability

Effective governance practices are especially important for audit and compliance purposes, which often require keeping a record of actions taken and being able to explain why decisions were made. Lineage tracking provides accountability by recording what data was accessed and how it was transformed. This makes it possible to audit the data path an AI agent followed. Observability complements lineage by monitoring AI agent behavior in real time, tracking which actions it is taking, detecting anomalies, and flagging policy violations.

Data quality

Robust data governance practices lay the groundwork for consistently high-quality data. While governance ensures that the right processes and policies are in place to manage data access effectively, data quality refers to the accuracy, reliability, and usefulness of the data itself. For AI agents operating autonomously across multi-step reasoning chains, poor data quality can potentially produce cascading errors. Organizations assessing their data foundation for agentic AI should consider implementing a data quality framework that addresses core dimensions like completeness, consistency, accuracy, relevance, and timeliness relative to each use case. 

Security and responsibility

As AI agents take on more autonomous decision-making, the security of the underlying data foundation becomes even more critical. Organizations should ensure that AI infrastructure, data, and models are protected from unauthorized access and malicious attacks through encryption, network isolation, access controls, and threat detection. 

Equally important is ensuring AI agents operate responsibly. Trust guardrails help reduce the risk of AI agents generating harmful content or misinformation, exposing sensitive information, and returning hallucinations. These safeguards can help organizations deploy AI agents while meeting regulatory and ethical standards.

While AI agents can operate based on general knowledge alone, most organizations get more value from customizing AI agents with organization-specific context. This helps AI agents make better decisions within the business, keeps their information more current, and reduces hallucinations. 

The domain-specific data used to customize AI agents typically resides in knowledge bases. RAG is the process of retrieving information from these knowledge bases and augmenting large language model (LLM) prompts so AI agents can generate more up-to-date, relevant responses. RAG implementations typically rely on vector embeddings, or numerical representations that capture the semantic meaning of content like documents, images, and support articles. Rather than matching on exact keywords, embeddings-based retrieval finds contextually similar content, allowing AI agents to reason over vast amounts of organizational knowledge stored in sources that would otherwise be difficult to query.

Semantic layers complement RAG by helping AI agents understand business logic. They define table relationships, business metrics, and shared terminology, ensuring AI agents interpret terms like "revenue" or "active customer" consistently across teams. Semantic layers can be built from ontologies and knowledge graphs, dimensional models, table relationships, and metric definitions that map metrics, logic, and their interconnections, translating intent into business definitions.

Context helps AI agents make relevant decisions in the present, while memory helps them understand their past. LLMs are stateless, meaning every request arrives without any recollection of what came before. Without a memory architecture layered on top, AI agents cannot reason, adapt, or coordinate, and they risk wasting time and resources re-deriving the same answers repeatedly. Because AI agents operate in a loop (reason, act, remember, repeat), memory is what prevents redundant work and maintains coherent progress across iterations. Together, context and memory move AI agents from disposable, single-turn responders into systems that accumulate expertise over time.

Agentic memory exists on a continuum. Short-term memory sits on the left, accessed constantly and at high speed. Medium-term memory sits in the middle, retrieved through tools when the agent needs it. Long-term memory sits on the right, configured infrequently and treated more like infrastructure. Each type has distinct access patterns, latency requirements, and data-management responsibilities.

Short-term memory

This is the agent's working memory, analogous to RAM. It holds the current conversation turn, the latest tool result, the active step in a multi-stage workflow, and the transient state that resets once the task completes. Frameworks like Strands Agents and LangGraph typically manage this layer for you, but the underlying storage must deliver microsecond read latency because the agent accesses this memory frequently, even within a single interaction. 

Short-term memory also extends to shared state across AI agents. When multiple AI agents coordinate on a task, a shared file system allows them to exchange intermediate results and maintain checkpoints with low-latency access.

Medium-term memory

This is the layer most connected to existing data tools and systems, including data lakes, catalogs, and knowledge bases. It encompasses the facts, relationships, and experiences that an AI agent retrieves through tool calls during execution and is where most of the data preparation work concentrates. It includes the following: 

• Semantic memory holds structured knowledge of facts, concepts, and relationships, independent of when or where that knowledge was acquired. This is the timeless knowledge base that AI agents use to reason about novel situations without re-deriving answers from first principles. Databases, vector stores, knowledge graphs, and data warehouses all serve as semantic memory backends. 

• User profile memory captures the persistent identity of the person the AI agent serves, including preferences, communication styles, past choices, and accumulated attributes that personalize behavior across sessions. 

• Episodic memory holds a time-stamped record of specific events and interactions. Where semantic memory is abstract, episodic memory is concrete. It allows AI agents to recall that a particular approach failed in a previous system and adjust accordingly. This is what allows AI agents to improve through deployment rather than requiring constant retraining. Session logs, interaction histories, and event stores all feed episodic memory. 

The critical distinction from short-term memory is that medium-term memory lives outside the context window and is accessed through tools. The AI agent decides when it needs information and calls the appropriate tool, which means data sources must be discoverable and responsive enough to keep the reasoning loop moving. 

Long-term memory

This is the layer that changes least often and fundamentally shapes an AI agent’s sense of self and the decisions it makes. Long-term memory includes system prompts, guardrails, definitions for the AI agent’s persona, and procedural routines. When an AI agent is created to play the role of an insurance underwriting agent, that instruction, along with the rules, constraints, and behavioral boundaries it must follow are stored here. This layer also encompasses procedural memory, which encodes the skills and routines that give the AI agent built-in operational knowledge without deliberating at each step. Long-term memory artifacts can be managed as infrastructure as code and stored in version-controlled code repositories, configuration management systems, or key-value stores.

Traditional application workflows follow fixed, deterministic paths. When a user completes a task, the same sequence of queries runs against a database. This allows for predictability in index tuning, results caching, and capacity forecasting. An AI agent, however, may take a different path each time it completes the same task, choosing different queries based on how it reasons through intermediate results. This variability makes it more challenging to anticipate load and optimize query patterns effectively.

In some cases, this can be addressed with a semantic cache. The cache returns stored responses for semantically similar requests rather than triggering a new call. This can reduce cost, improve performance, and increase the throughput of agentic applications.

To further support the unpredictability of AI agents, underlying infrastructure should be built to scale automatically. Serverless architectures remove the need to pre-allocate resources; rather than sizing infrastructure for maximum capacity, resources scale up and down based on demand without manual intervention. This is especially important for AI agents because behavior can be bursty. Activity can spike suddenly as tasks are launched, tools are called, or workflows expand, and then drop back to zero. Because serverless capacity is available on demand, it leads to less wasted resources and organizations only pay for what they consume.

In addition to serverless provisioning to meet demand, organizations should prioritize high-availability infrastructure with automated failover and redundancy to support continuous uptime. When this is in place, a single component failure does not interrupt an AI agent's ability to complete its work. 

Preparing your data foundation for agentic AI may be the most important project you take on this decade. The questions in this guide represent the essential capabilities your data foundation needs to support AI agents effectively: interoperability, discovery, data preparation, governance, context , memory, and low-latency. Few organizations will have all of these fully in place today, and that is expected. Building a data foundation for agentic AI is an evolutionary process, and each capability you strengthen compounds the value AI agents can deliver. AWS provides purpose-built services across each of these areas to help you build, scale, and operate your data foundation for agentic AI.

For over 20 years, millions of customers have trusted AWS to deliver secure, reliable, and scalable cloud services with industry-leading price performance. Here is how AWS supports your data foundation for agentic AI.

Open data architecture

AWS embraces open standards because they provide customers with flexibility and the best value. This includes PostgreSQL through Amazon Aurora and Amazon Relational Database Service (Amazon RDS), Apache Kafka through Amazon Managed Streaming for Apache Kafka (Amazon MSK), OpenSearch through Amazon OpenSearch Service, and Apache Spark through Amazon EMR, and Trino through Amazon Athena. AWS delivers native Apache Iceberg support across the AWS Analytics stack, including fully managed tabular data storage through Amazon S3 Tables, giving AI agents interoperable access to analytical data without reformatting or duplication. AWS also supports open agent protocols, including MCP and A2A, allowing AI agents to connect to data sources, tools, and each other.

“The idea of having a company like Amazon Web Services stand behind an open source product and take care of important attributes like high availability, scaling, and overall data management is huge.”  

—Raj Cheruvu, AVP R&D, Helix ITSM at BMC

Purpose-built services for every layer of the agentic AI stack

The capabilities covered in this guide—from data preparation to memory and low latency performance—require infrastructure that consistently performs at scale. AWS delivers proven reliability, scalability, and price performance, so that you can build on foundational excellence.

To help ensure data is properly prepared for AI agents, AWS Glue provides serverless data integration and transformation at scale, making it simple to discover, prepare, move, and integrate data from multiple sources. AWS Glue Data Quality, a serverless and open-source component of AWS Glue, allows you to measure and monitor the quality of data and enforce quality rules directly within data pipelines.

For near real-time data, AWS offers a suite of managed services that enable end-to-end streaming data pipelines. Amazon Kinesis Data Streams and Amazon MSK handle high-throughput data ingestion, Amazon Managed Service for Apache Flink provides stream processing, and Amazon Data Firehose delivers data to its destinations, helping AI agents reason over high-quality, current data as conditions change.

To support context, AWS databases, including Amazon Aurora, Amazon RDS, Amazon Neptune, and Amazon ElastiCache for Valkey, provide native vector support for RAG retrieval, giving AI agents fast, relevant context at query time. Amazon OpenSearch Service provides a managed, open source retrieval engine for agentic AI, search, and analytics. It combines vector, lexical, hybrid, and agentic retrieval in a single system, delivering low-latency, highly relevant results at petabyte scale.

For a storage-first alternative, Amazon S3 Vectors can reduce the total cost of uploading, storing, and querying vectors by up to 90%, making it economically viable to create and retain the large vector datasets that improve the memory and context of AI agents. 

“AWS's infrastructure gave us the foundation to focus on what matters. We've built video AI that reasons like humans do: understanding not just what happens, but why it matters and how moments connect across time.” 
—Jae Lee, CEO and co-founder, TwelveLabs

Seeing is understanding – TwelveLabs unlocks the full potential of video for the world using S3 Vectors

The AWS Databases portfolio spans the three zones of the memory continuum. Amazon ElastiCache for Valkey delivers the microsecond latency agentic AI needs to cache tool results, preserve session context, and sustain the short-term working memory that drives coherent multi-step reasoning. Amazon DynamoDB handles the append-heavy write patterns of episodic logging and session-state tracking with single-digit millisecond reads at any throughput. Amazon Aurora and Amazon RDS are ideal for long-term memory, especially persisting multi-step task progress. Amazon Neptune powers graph-based semantic and episodic memory for relationship traversal.

To support shared state across AI agents, Amazon S3 Files provides low-latency file system access to data in Amazon S3, enabling agents to exchange intermediate results and maintain checkpoints across multi-agent pipelines. Together, these services cover every latency tier, write pattern, and durability requirement across the memory continuum, so you can build memory architectures that scale without compromise. Learn how Trend Micro built an AI security assistant with short-term and long-term memory capabilities using Amazon Neptune.

AWS database services are designed to automatically scale and adapt to real-time workload changes, making them well suited for the variable, bursty workloads that AI agents generate. Amazon DynamoDB and Amazon ElastiCache can respond to hundreds of millions of requests per second while achieving latency budgets of microseconds to single-digit milliseconds, while Amazon Aurora instantly scales with full PostgreSQL and MySQL compatibility, delivering the performance, availability, and scalability of commercial databases at one-tenth the cost.

If your organization runs commercial databases like SQL Server or Oracle on premises or self-managed on Amazon EC2, your path to an AI-ready data foundation starts with a lift-and-shift migration to Amazon RDS. Winning Agentic AI With Your Commercial Data Estates describes how Amazon RDS delivers the automated management, performance, reliability, and security that agentic AI applications need, while IDC research in The Business Value of Amazon RDS showcases the financial impact of a migration to RDS: 258% three-year ROI and measurably better quality of service across interviewed organizations.

Trusted foundation for AI

AI agents that dynamically discover and access data to make autonomous decisions require governance and security that scale with them. AWS is designed to be the most secure cloud infrastructure with data encryption by default and fine-grained access controls that extend to AI agents. AWS data governance ensures only the right users access the right data, with access controls aligned to policy and security standards. Built-in compliance features support evolving global regulations, while systematic verification processes uphold well-governed data management for trusted AI. AWS is committed to delivering trusted and context-rich data that flows securely across humans, AI agents, and apps, even for the most sensitive workloads.