How to Prepare Enterprise Data for a Production RAG System

What happens when a RAG system can retrieve information, but the information it finds is outdated, poorly structured or available to someone who should never have seen it?

That is not primarily a model problem. It is a data preparation problem.

For enterprise teams, RAG data preparation defines what information enters the knowledge base, how that information is structured and sliced for retrieval, what metadata travels with it, who can access it, and how the data remains fresh post-launch.

A solid model cannot compensate for a knowledge base filled with duplicate documents, weak permissions or stale content. Before teams can evaluate prompts, embeddings or model choices, they need assurance that the retrieval layer is working with reliable, usable and properly governed information.

This guide is on preparing enterprise data for a production RAG system. It focuses on source selection, cleaning, metadata, document chunking, permissions, update schedules, ownership and quality controls.

What Is RAG Data Preparation?

RAG data preparation is the task of selecting, cleaning, structuring, tagging, splitting, protecting and maintaining the enterprise information that is used by a retrieval-augmented generation system. The goal is to develop a knowledge base that can deliver relevant, current and properly authorized information with enough context to allow for accurate answers.

The work begins before documents are embedded or loaded into a vector database. A production RAG knowledge base may draw from PDFs, internal knowledge platforms, policies, databases, product documentation, customer records, websites or other enterprise systems.

The key question is not simply whether the system can ingest a source. Teams need to decide whether that source belongs in the retrieval environment, which version should be treated as authoritative, who should be able to retrieve it and what should happen when the source changes.

Those decisions determine whether the knowledge base can support production use rather than a controlled demonstration.

Start With the Business Questions the System Must Answer

It is not necessarily useful to start by preparing every document available. The source set needs to be defined by the workload the RAG system is intended to support.

For example, an internal policy bot will have different requirements than a technical support bot or a customer-facing knowledge bot. Each workload has its own users, source systems, permissions, freshness requirements, and tolerance for retrieval errors.

First, identify the questions that the system must answer and the decisions that the answers must support. That gives the team a working boundary for deciding which sources are in scope.

For every proposed source, establish:

• what business purpose the information supports
• who currently uses and owns it
• whether it is an authoritative source or a duplicate
• how often the content changes
• whether it contains confidential or restricted information
• which RAG users should be able to retrieve it
• how the source will be maintained after launch

This approach keeps the initial knowledge base connected to a defined use case and gives the team something meaningful to test later. Retrieval quality is much easier to evaluate when the intended questions and approved sources are already known.

Identify the Authoritative Source Before You Index Anything

Often, enterprise information is duplicated in multiple systems.

A policy can be an approved document in a records platform, a downloaded PDF in a shared folder, an older attachment in email and a copied version on an internal site. If all four are indexed without specifying the authoritative record, the RAG system might retrieve conflicting information even if the retrieval technology is working properly.

Source selection is not just the job of the team building the AI system; it requires input from the people responsible for the information. IT may know where the documents are, but the business owner needs to validate which version should dictate the answer.

A practical source inventory can record the most important decisions before ingestion begins:

Field	What to record
Source system	Where the information originates
Business owner	Who is accountable for the source
Authoritative status	Whether this is the approved record
Sensitivity	Public, internal, confidential or restricted
Intended users	Who should be able to retrieve the content
Update frequency	How often the information changes
RAG status	Include, exclude or review

The value of this inventory becomes more apparent after launch. When a source changes, access rules are modified or an answer is challenged, the team has a documented basis for tracing the information back to its owner and source of truth.

Clean the Content Without Removing Meaning

Successfully extracting text from a document does not mean the content is ready for retrieval.

Enterprise files are likely to have repeated headers and footers, navigation text, boilerplate, old revision notes, duplicated content, broken character encoding, or tables that lose their meaning when converted into plain text. These elements can produce low-value chunks that compete with useful information during retrieval.

Cleaning should remove noise without flattening useful structure.

Headings, revision dates and section labels can be important contextual elements in a policy document. Menus and footer links may not have any retrieval value in a web export. The relationships between labels and values are often required to be maintained for spreadsheets and tables so that retrieved information remains interpretable.

The original source must be traceable after cleaning. Users or reviewers might want to check a citation, follow a questionable answer, or see if retrieved information is current.

The aim is therefore not to produce cleaner-looking documents. It is to produce information that can be retrieved without losing the context required to interpret it correctly.

Treat Metadata as Part of the Retrieval Design

The text of a document is the explanation of what it means. The metadata assists the retrieval layer in understanding what the document is, who it is relevant to, and if it should be considered for a given query.

Useful metadata may include document type, department, owner, publication date, revision date, location, jurisdiction, product version, account context, confidentiality level and source identifier.

The right fields are workload dependent.

A policy assistant in Canada may require province or jurisdiction metadata. A product-support system might ask for the model number and software version. For a customer-facing application, you may need account or tenant identifiers, so that information belonging to one organization cannot be retrieved by another.

Metadata increases relevance because the system does not have to depend only on semantic similarity. It can narrow retrieval down by business rules before information is put into model context.

That connection between metadata and governance is important. The Government of Canada’s guidance on responsible use of AI addresses data use, privacy, accountability and ongoing oversight. In a RAG system, many of those controls begin with how enterprise information is labelled, filtered and governed before the model receives it.

Design Permissions Before Retrieval Goes Into Production

A RAG application should not make information any more accessible than the systems it was derived from.

If a document is restricted to finance, human resources, a project team or a specific customer account, those restrictions must still make sense once the document is indexed.

Depending on the architecture, permission-aware retrieval may depend on user identity, roles, groups, tenant boundaries, metadata filters, or document-level access rules. The exact mechanism may vary, but the requirement for production is the same: the retrieval layer should not contain information that the user requesting it is not allowed to access.

This is harder when a single RAG system merges data from multiple platforms with different permission models. One system may grant a user access to a document, while another system may deny it, and those rules can change over time.

Permission mapping therefore belongs in data preparation rather than being treated as an interface-level control added near the end of development.

The NIST AI Risk Management Framework organizes AI risk management around governance, mapping, measurement and ongoing management. For enterprise RAG, controlling which information can enter retrieval for a given user is part of that operating discipline.

Build a Document Chunking Strategy Around Meaning

In most RAG systems, rather than passing the whole files to the model for each query, sections of files are retrieved. Those pieces, often called chunks, have to have enough meaning to allow the retrieval system to find and interpret the right evidence. 

Very large chunks can embed a relevant passage in unrelated text. Very small chunks can separate a statement from the heading or condition or surrounding explanation that gives it meaning.

There is no one chunk size that solves this problem.

It’s better to begin by establishing the document structure. It is often the case that meaning is better preserved by sections, paragraphs, procedures, clauses or other natural boundaries than by splitting each file at a fixed number of characters or tokens.

The strategy must be consistent with the type of information being prepared. When procedures become the individual retrieval units, a technical manual may work well. A policy document may require that section headings remain attached to each section. Surrounding clauses may be necessary due to contract language so that a sentence is not retrieved without the conditions that limit it.

Then chunking should be tested on representative queries. The team should compare whether the retrieved content includes the expected answer, enough context around it to interpret it, and no extraneous material that makes the evidence less precise.

Plan for Freshness, Replacement and Deletion

Production knowledge bases change after launch because the systems behind them continue to change.

Policies are revised, products are updated, employees change roles, customer records are modified and old documents are replaced. If the retrieval index does not reflect those changes, a system that performed well during testing can begin returning obsolete information without any change to the model itself.

Each source therefore needs a defined update approach based on how quickly the underlying information changes.

Teams should know what occurs when a document is added, revised, superseded, reclassified or removed. They should also know how long those changes take to appear in the retrieval environment.

Deletion deserves the same attention as ingestion. Removing a source document from its original platform does not necessarily prove that every indexed representation of that document has disappeared from the RAG system.

A production readiness test should confirm that the team can update or remove indexed information predictably and verify that the retrieval layer reflects the change.

Define Ownership Before Launch

RAG data preparation crosses technical and business responsibilities, which makes unclear ownership especially expensive once the system is live.

Engineering may manage ingestion and retrieval. Security may define access requirements. A business group may own the source content. Another team may operate the application.

The organization still needs a clear answer to who can approve a source, who reviews permissions, who responds when information becomes outdated and who has authority to remove problematic data from the knowledge base.

The source owner should remain part of the operating model after launch because technical teams cannot determine whether every piece of business information is still correct or authoritative.

This is one reason a successful pilot does not automatically prove production readiness. A pilot can demonstrate retrieval performance without proving that the surrounding ownership model can maintain the system over time.

Use Evidence, Not Confidence, to Decide Whether the Data Is Ready

A useful production threshold is whether the team can produce evidence for the assumptions behind the knowledge base.

Before approving a RAG workload for production, the team should be able to show:

Production question	Evidence to collect
Are the right sources included?	Approved source inventory tied to the use case
Are the sources authoritative?	Named owners and documented source-of-truth decisions
Is retrieval properly restricted?	Permission tests using different user roles or accounts
Does chunking preserve meaning?	Representative queries showing expected supporting content
Is the information current?	Tested update, replacement and deletion process
Can retrieval failures be investigated?	Source identifiers, citations or traceable retrieval records
Does the data support the expected questions?	Evaluation set with expected source material

This is a stronger release criterion than simply asking whether the system produced convincing answers during a demonstration.

A production RAG system should be able to show where its evidence came from, why that evidence was available to the requesting user and what happens when the underlying information changes.

Validate Retrieval Before Evaluating the Final Answer

Data preparation is not completed after the successful ingestion.

Come up with a representative set of questions to mirror the real workload. Then see if the retrieval layer provides the evidence to answer those questions.

The test set should contain easy cases and more difficult conditions. Teams should check whether the system retrieves current information instead of an outdated version, respects permission boundaries, selects the correct jurisdiction or product version and avoids irrelevant documents that happen to contain similar language.

Negative tests matter, too. A protected information should not be accessible for a limited user. A query about missing information should not generate pseudo-authoritative evidence from a loosely connected source.

That separation of retrieval testing from answer testing is useful, because it makes failures easier to diagnose. Changing the prompt is unlikely to solve the actual problem if the source we expect to see never gets to the model.

Common RAG Data Preparation Mistakes

Many production problems can be reduced before launch by addressing weaknesses in the data layer:

• Indexing too many sources: Introduces duplicates and makes it harder to determine which information should govern an answer.
• Weak metadata: Limits filtering and forces retrieval to depend too heavily on semantic similarity.
• Poorly tested chunking: Separates related information or returns more context than the model needs.
• Incomplete permission mapping: Makes information retrievable by users outside its intended audience.
• Missing update and deletion processes: Allows outdated or removed information to continue influencing answers.

Some data preparation choices involve trade-offs:

• Smaller chunks can improve precision but remove useful context.
• Broader data coverage can support more questions but increase governance and maintenance work.
• Faster synchronization can improve freshness but increase infrastructure and operational demands.

Each decision should reflect the workload’s requirements, risk tolerance and operating environment, rather than a generic RAG architecture.

Questions to Ask Before Moving Into Production

Before approving the data layer, business and technical stakeholders should be able to answer a small set of direct questions with evidence rather than assumptions:

• Which business questions is this knowledge base intended to support?
• Which source systems and users are in scope?
• Which source is authoritative when duplicates exist?
• Who owns source approval, security review and ongoing maintenance?
• How are permissions applied during retrieval?
• What evidence shows that the expected source is being retrieved?
• How quickly do changes and deletions appear in the index?
• What happens when the current retrieval strategy stops meeting the workload’s requirements?

The same questions are useful when comparing external providers. A provider should be able to explain what happens to enterprise information before it reaches the model, rather than focusing only on the language model or vector database used in the architecture.

How Arcadion Supports RAG Systems

Preparing enterprise information for retrieval is one part of moving a RAG application from concept to production.

Arcadion’s RAG Systems offering encompasses work across enterprise data sources, preprocessing, retrieval design, chunking, vector search, model integration, testing and governance. This allows for the data layer and retrieval architecture to be designed according to the application that will be relying upon it.

For organizations thinking about a new RAG workload, the practical goal is to move beyond proving that a model can answer questions, to establishing whether the underlying data, control of access, and operating model can support dependable production use.

Prepare the Data Before You Judge the RAG System

A RAG implementation can’t reliably retrieve knowledge that the organization hasn’t properly selected, structured, secured, and maintained.

Strong RAG data preparation gives the system an authoritative source base, useful metadata, deliberate document boundaries, permission-aware retrieval, defined ownership and a process for keeping information current. It also provides teams with a better understanding of whether an error is originating in the source data, retrieval layer or later stages of the application.

That distinction becomes increasingly important as a RAG system moves from a controlled prototype to everyday enterprise use.

Planning a production RAG workload? Learn how Arcadion approaches enterprise RAG systems.