Think MCP, ANP, agents.json, Agora, LMOS, or AITP will “solve business context”? They won’t. These protocols are great at making data available to agents. They make it easier to pass the right chunks from this data to other agents.  They do not tell you what any given chunk actually means in your business, or when it’s appropriate to use. That’s not a transport problem—it’s a meaning and governance problem.

Business context isn’t just the metadata attached to the document your AI happened to retrieve from—or the semantic relationships you can squeeze out of a single paragraph. It’s an ecosystem of relationships, and the most important ones often live outside the immediate source.

## The Hidden Life of a Chunk

When we talk about context, we’re really talking about chunks. That’s how LLMs, RAG systems, and most vector databases see the world. A “chunk” might be a sentence, a table cell, or a paragraph—small enough to embed, big enough to convey meaning.

Here’s the catch: the exact same chunk might appear in dozens of different documents, each with its own metadata—authorship, timestamps, confidentiality flags, regulatory tags, workflow stage, and so on.

> If you only look at the document the chunk was read from, you miss the bigger story.

That’s like judging a single line of code based solely on the file it’s in—without knowing it’s been copied into six other repos, patched in three of them, and is currently being used in production by a system with a different security model.

## Why Proximate Metadata Falls Short

Most classification and governance tools still think like librarians: “This document has these attributes, so its contents must too.”
The reality?

* That “confidential” paragraph in your quarterly earnings report may be identical to one already published in last year’s public filing.
* The same compliance clause in a contract may also appear in an internal policy draft with stricter access rules.

If your governance layer only sees the local metadata, it will either over-restrict (blocking safe use) or under-restrict (risking a leak).

## The Broad-View Context Model

To really capture business context, you need to:

1. Aggregate metadata across every occurrence of a chunk, in every document, across time.
1. Resolve conflicts using precedence (source of authority) and prevalence (most common use) rules.
1. Maintain lineage so you can see exactly where that chunk has been, who touched it, and under what policy it lived at each point.

This is a fundamentally different mindset. Instead of starting with “this document says X about this chunk,” you start with “this chunk exists in 27 places, and here’s the union of everything we know about what it means and how it can be used—in other words, its business significance.”

## Why Agent Communication Protocols Can’t Fix This

Let’s be clear: protocols aren’t the villain; they’re just not the mechanism for meaning. MCP, ANP, Agora, agents.json, LMOS, and AITP define how agents talk to each other—how they exchange tasks, pass along context, and authenticate participants.

Many of them address critical security pillars for agent interactions:

* Confidentiality of communications (so only the intended parties see the data)
* Integrity (so the data isn’t altered in transit)
* Authenticity and non-repudiation (so you know who sent it, and they can’t deny it later)

That’s essential for trust between agents. But none of it answers the core questions: what does this chunk mean in your business, and under what conditions can it be used?

Knowing which agent handed you a paragraph doesn’t disambiguate its business significance. The same chunk can appear across silos under conflicting metadata. Until you resolve that conflict at the chunk level, policies will fail—regardless of how well your agents authenticate each other.

Protocols move context securely. Meaning and policy come from a cross-corpus, metadata-aggregated view of the chunk itself—the broad view we’ve been talking about.

## The Payoff

When you take the broad view:

* AI answers become explainable, because every retrieved chunk comes with a history.
* Permissions stop being brittle, because you’re applying the right policy for the chunk, not the default for the nearest file.
* Data quality improves, because duplicate and stale chunks can be identified and resolved before they pollute your retrieval set.

And perhaps most importantly, you stop treating “context” as whatever happens to be nearby—and start treating it as the complete, cross-document truth.


Stop worshiping documents; meaning lives at the chunk. Unify metadata, resolve conflicts, apply policy.

Business Context on AI Data Won’t Be Solved by Retrieval.

Recent advances in AI—semantic graphs, vector search, and attention optimization—are exciting and powerful. They move beyond keywords and help us extract deeper patterns from unstructured content. But in enterprise environments, these approaches all share a critical blind spot:

> *They rely on what data looks or smells like—rather than how it's actually used in the business.*

Business significance doesn’t live in token frequency or surface similarity. It lives in metadata, lineage, structure, and workflow. In other words, **meaning is not inferred—it’s modeled**.

Let’s examine three promising techniques that illustrate this pattern of semantic approximation. These aren't critiques of the work itself—in fact, they’re valuable building blocks. But they highlight how much more is needed for AI to operate effectively in enterprise settings.

---

## 🧱 Example 1: Chunk Overlap in Vector RAG

Many vector RAG systems assume that if two chunks share similar tokens or embeddings, they must be “related.” It’s a useful shortcut for retrieval—but in enterprise environments, it can easily mislead.

> For example: A marketing ROI report and an HR training manual may both mention “ROI” and “performance,” but their business roles are entirely different.

Chunk overlap doesn’t reveal intent, governance, or data classification. It connects content by **how it reads**, not by **what it does**.

To truly relate content, we need:
- Document type and purpose
- Departmental origin
- Workflow stage
- User roles and permissions

These aren't textual features—they're structured context.

---

## 🎯 Example 2: Context Engineering and Frequency-Driven Relevance

[LLMSTEER (arXiv:2411.13009)](https://arxiv.org/abs/2411.13009v2) introduces a smart method for improving long-context performance: boosting attention on tokens that are reused across inference steps. This can help LLMs focus on “important” information.

In narrow domains—like code repositories or customer support logs—reuse often does signal importance. But across broad enterprise corpora, it falls short.

> While a runny nose is usually just a cold, in the emergency room it could be an early sign of Churg-Strauss syndrome—a rare and potentially life-threatening autoimmune disorder.

Token frequency doesn’t equate to business meaning. Instead, meaning comes from:
- Where a term appears (e.g. header vs. appendix)
- What kind of document it’s in (template vs. signed agreement)
- Who authored it and when

LLMSTEER’s attention boost is helpful—but it needs grounding in metadata to reflect enterprise significance.

---

## 🔁 Example 3: Knowledge Graphs and the Limits of Local Semantics

The latest innovation in knowledge Graphs is [Graph-R1’s (arxiv:2507.21892v1)](https://arxiv.org/abs/2507.21892v1) use of semantic hypergraphs is a real step forward—capturing n-ary relationships like “Alice and Bob co-founded Acme in 2019.” It adds structure and depth beyond typical vector methods.

But even this approach still builds relationships from **within** chunks of text. It doesn’t look at:
- The document’s policy role
- Workflow state (draft, pending, approved)
- Business process alignment

> For instance, “Chest X-ray recommended” might appear in a discharge summary, a clinical guideline, or a training example. Graph-R1 sees one relationship. But to the business, there are three different implications.

Without metadata and process context, even rich semantic links can’t distinguish authoritative action from illustrative mention.

---

## 📌 The Pattern: Approximation Over Anchoring

Each of these innovations reflects real progress. But they share an assumption: that meaning can be **inferred from usage patterns**, rather than **anchored in structured context**.

In enterprise AI, that’s not enough.

| **Technique**            | **What It Adds**                       | **What It Misses**                               |
|--------------------------|----------------------------------------|--------------------------------------------------|
| Vector RAG chunk overlap | Semantic proximity                     | Workflow context, data lineage                   |
| LLMSTEER                 | Attention to reused tokens             | Role, author, and system-level significance      |
| Graph-R1                 | Structured semantic relationships      | Policy role, document state, metadata hierarchy  |

These techniques are useful heuristics—but they’re only **a layer**, not the full foundation.

---

## 🧠 Why Semantic Structure Is Essential

To move beyond guesswork, enterprise AI must incorporate:

- **Metadata-aware reasoning**: Understand document types, classification levels, and authorship.
- **Data lineage and versioning**: Know how a clause or chart evolved and where it was reused.
- **Process and policy integration**: Understand how content fits into formal workflows or controls.
- **Semantic layers**: Define organizational concepts explicitly and connect content to them.

This is the foundation for explainability, compliance, and trust in enterprise AI. Without it, models can retrieve related-sounding content that **misleads more than it informs**.

---

## ✅ Final Thought

Vector search, attention steering, and semantic graphs are necessary steps. But alone, they’re not sufficient. Enterprise AI must do more than pattern match—it must model business meaning explicitly.

> It’s time to shift from guesswork to grounding.  
> From what data looks like—to what it *means* in context.


AI that guesses what data looks like can’t be trusted to know what it means. Business context is not a vibe.

When What Data “Looks Like” Isn’t What it “Means”



We all know the story: you’re building out a project, tweaking code on your organically grown website. One day you prompt an AI:  
> “I need a function to do X.”

It generates something good. Then later:  
> “I need my function to do X + Y.”

Instead of improving the original function and updating all its references, the AI spins up a new function. Now you’ve got duplicated logic—some calls point to the old function, some to the new. Chaos creeps in.

In my own codebase I saw this pattern over and over. For example:

```js
// Original helper
function processOrder(order) {
   validate(order);
   submit(order);
}

// Later request to add tracking
function processOrderWithTracking(order) {
   validate(order);
   submit(order);
   track(order);
}
```

Half the code called `processOrder`, half called `processOrderWithTracking`.  
Downstream references in `invoice.js` still used the old function, while `shipment.js` used the new one.  
**The AI had no visibility into the function graph to know it should merge logic and update references.**

---

## Models Are Great, But Data Rules the Outcome

AI models are getting better fast. Switching between them is becoming less painful, and their raw capabilities feel similar. So how do you know which one to use?  
**You don’t—because the model isn’t the whole story.**

The real question is:  
👉 *What data are you feeding it?*

When you provide structure, even a little, answer quality skyrockets. For example, supplying a database schema instead of just dumping tables into a prompt improved one system’s answer quality from **72% to 87%**—a 15% jump for minutes of extra work.

---
---
_**Example:  Try #1 — Use AI to find unused and disconnected files in my codebase**_

My Prompt:
```
Identify all files that are disconnected from the marketing pages, authorization, deployment and dashboard.  I'm looking for unused files that may be referenced in other files that themselves are disconnected from the core graph.
```
AI Response:
```ai
[...]
This cleanup could remove approximately **20-25% of unused code** from your components directory and simplify the codebase significantly.
```
**Time: 3 minutes 12 seconds**

---
---

## Unstructured Dump vs. Structured Context

Dumping unstructured data into a context window often fails to deliver the right results. You need to give your AI not just data, but data **with context:**

- What each element is.
- How elements are related.
- Which relationships matter for the task at hand.

Think of it like giving your AI a map instead of a random list of street names.


## Adding Structural Hints Improves Responses

Graph-based RAG (retrieval augmented generation) is a great start, but the trick is building relationships:  
- What functions call what.  
- What database fields depend on what tables.  
- How pieces of documentation relate to each other.

Here’s a snippet from my codebase that demonstrates the value of mapping out dependencies between files:

```
index.js -> utils/logger.js
index.js -> services/api.js
services/api.js -> utils/logger.js
services/api.js -> models/user.js
```

When you represent your code this way, the AI can see that both `index.js` and `services/api.js` rely on the same logger—so a change in `logger.js` affects them both. Instead of duplicating logic, the AI could reason:  
> *Modify `logger.js` once and update its usage everywhere.*`


---
---
_**Example:  Try #2 — Was I about to delete needed code?**_

Deterministically build a dependency graph
```bash
caber% time npm run analyze:json

> caber-systems-portal@0.1.0 analyze:json
> npx madge --json src/ > dependency-graph.json

npm run analyze:json  1.41s user 0.23s system 79% cpu 2.070 total
```

My Prompt:
```prompt
Given the attached dependency graph, identify all files that are disconnected from the marketing pages, authorization, deployment and dashboard.  I'm looking for unused files that may be referenced in other files that themselves are disconnected from the core graph.
```

AI Response:
```ai
[...]
This analysis shows your codebase is relatively clean with only ~8.5% disconnected files, mostly old pages and test files.
```

 **Time: 2 minutes 3 seconds + 2.070 seconds = 2 minutes 5 seconds** ⬅️ 

---
---


## Where Does Context Come From?

Context comes from **data elements** and the **patterns in their creation and use.**  Some patterns are standardized (a function calling another). Some are complex (like how weather patterns can affect call center volume). When you surface these patterns you’re giving AI clues about your data that it often won't uncover on its own.

In the examples above providing structured context (the dependency graph), saved us about 35% of the time while getting a more accurate result (8.5% disconnected files vs the original estimate of 20-25% unused code).

## Why It Matters

Interconnectedness in data is something we take for granted but rarely exploit. By unraveling those relationships and feeding them to the AI, you elevate it from a code generator or summarizer into a trustworthy partner that understands your data.

**It’s not the model—it’s the data you give it.**


AI giving you duplicate code? The fix isn't a better model - it's structured context. See 72% → 87% jump!

Why Adding Schemas Supercharges AI Answer Quality

Every enterprise today deals with an enormous volume of PDF documents. From financial reports and contracts to regulatory filings and internal memos, PDFs have become ubiquitous due to their portability and visual consistency. Yet putting the data these PDFs contain into a format enterprise AI applications can use is a roadblock 🚧.  

Beneath their convenience lies a simple fact: PDFs have no easily parsable structures, or metadata, describing what elements are, or which text might be the title, a heading, or a page footer--and this makes recognizing these different elements painfully difficult. While AI-based parsers produce impressive results, they do so at significant cost and still struggle to attain reliability and consistency across different types of documents 📑.

So what if we could get the information those PDFs contain into a usable format without parsing PDFs at all?  The key to answering that is actually pretty easy once you consider enterprise workflows and where those PDFs came from in the first place:  It's lineage.

## 4,628 AI Models to Work Harder, not Smarter?

As I write this, Hugging Face lists 4,628 AI models for [Image-Text-to-Text](https://huggingface.co/models?pipeline_tag=image-text-to-text).  Turns out printing PDFs to images and using AI for Optical Character Recognition (OCR) works better than having AI parse the PDF directly.  

There are dozens of free and paid online converters (see our list [here](https://github.com/Caber-Systems/ComparePDFparsers/blob/master/ListOfPDF2MarkdownConverters.md)), Python libraries, and projects on GitHub dedicated to parsing PDFs. [Adobe](https://developer.adobe.com/document-services/apis/pdf-extract/) themselves have one, and so do all of the leading AI providers: 
   * [OpenAI](https://learnprompting.org/blog/openai-api-works-with-pdfs) files API
   * [Anthropic](https://docs.anthropic.com/en/docs/build-with-claude/pdf-support) via thier Converse and InvokeModel APIs
   * [Mistral](https://mistral.ai/news/mistral-ocr) OCR-based PDF conversion  
   * [LlamaIndex](https://docs.llamaindex.ai/en/stable/llama_cloud/llama_parse/) LlamaParse
   * [Microsoft](https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/?view=doc-intel-4.0.0) Azure Document Intelligence
   * [Google](https://cloud.google.com/document-ai) Document AI

While these methods can produce impressive results, the reality of scalability quickly becomes evident:

- A single PDF can take tens of seconds to minutes to process and parse accurately.
- Many enterprises have millions of PDFs stored which can quickly balloon processing into years of compute time and enormous associated costs.
- GPUs help, but not as much as you might think, by providing 3x to 4x increase in processing speed 


<div align="center">
<h3>AI-Based PDF Parsing Performance Summary</h3>
<table>
<thead>
<tr style="background-color: #f0f0f0; color: #000000">
<th>Processor</th>
<th>Total Time (s)</th>
<th>Sec/Page</th>
<th>Bytes/sec</th>
<th>Success Rate</th>
</tr>
</thead>
<tbody>
<tr style="background-color: #a8a7a9; color: #000000">
<td>Mistral</td>
<td>14.71</td>
<td>0.28</td>
<td>11493</td>
<td>100.0%</td>
</tr>
<tr style="background-color: #a8a7a9; color: #000000">
<td>PyMuPDF4LLM</td>
<td>15.00</td>
<td>0.27</td>
<td>11916</td>
<td>100.0%</td>
</tr>
<tr style="background-color: #a8a7a9; color: #000000">
<td>Docling (no GPU)</td>
<td>94.90</td>
<td>1.74</td>
<td>2415</td>
<td>100.0%</td>
</tr>
<tr style="background-color: #a8a7a9; color: #000000">
<td>Docling (GPU)</td>
<td>78.20</td>
<td>1.45</td>
<td>2938</td>
<td>100.0%</td>
</tr>
</tbody>
</table>
<strong>Total files processed</strong>: 12<br>
<strong>Total pages processed</strong>: 2,736<br>
</div><br>

These figures are consistent with those presented in the [Docling Research paper](https://arxiv.org/html/2501.17887v1) and highlight an inescapable truth: parsing PDFs using LLMs, despite being powerful, won't scale to meet the millions of documents needed to be parsed in typical production enterprise use.

## The Overlooked Power of Data Lineage

If you go to the web pages of [Apple, Inc.](https://investor.apple.com/sec-filings/default.aspx), and [Tesla, Inc.](https://ir.tesla.com/sec-filings), the sources for the SEC financial filings we used to obtain the results above, you'll see that every 10-K and 10-Q statement is provided in both HTML and PDF formats.  In fact, the EDGAR portal used to file these financial statements only accepts submissions in structured formats -- either HTML or XBRL.  

It's no surprise that the vast majority of enterprise PDFs were never original creations. They're outputs—exports from structured sources such as HTML web pages, Word documents, Excel spreadsheets, or database records. These original formats inherently encode structure, with rich metadata that describes tables, headings, and key semantic content in a clear, parse-friendly manner that conversion to the PDF format discards.

<div align="center">
<h3>HTML to Markdown Processing Performance</h3>
<table>
<thead>
<tr style="background-color: #f0f0f0; color: #000000">
<th>Processor</th>
<th>Total Time (s)</th>
<th>KB/sec</th>
<th>Elements/sec</th>
<th>Output Bytes</th>
<th>Success Rate</th>
</tr>
</thead>
<tbody>
<tr style="background-color: #a8a7a9; color: #000000">
<td>Markdownify</td>
<td>1.06</td>
<td>2627.9</td>
<td>33288</td>
<td>405516</td>
<td>100.0%</td>
</tr>
<tr style="background-color: #a8a7a9; color: #000000">
<td>html-to-markdown</td>
<td>0.49</td>
<td>4064.8</td>
<td>55248</td>
<td>403604</td>
<td>100.0%</td>
</tr>
<tr style="background-color: #a8a7a9; color: #000000">
<td>html2text</td>
<td>0.19</td>
<td>8691.6</td>
<td>126795</td>
<td>381705</td>
<td>100.0%</td>
</tr>
<tr style="background-color: #a8a7a9; color: #000000">
<td>MarkItDown</td>
<td>1.12</td>
<td>2418.0</td>
<td>30704</td>
<td>385882</td>
<td>100.0%</td>
</tr>
</tbody>
</table>
<strong>Total files processed</strong>: 12<br>
</div>
<br>

Compare the Bytes/sec and KB/sec columns in each graph. Parsing the HTML sources sources directly can be completed in milliseconds—literally ⚡️**1,000 times faster**⚡️ than parsing the PDFs.  Moreover, since all the structure and metadata exists in the HTML, the conversion is **100% accurate** because it's deterministic. 

The profound speed and accuracy advantages of parsing structured documents over PDFs become starkly evident at scale, making lineage knowledge incredibly valuable 💰💵💸.

## Trading One Hard Problem for Another?

While it's easy to "just use AI" to solve hard problems, blind recourse to brute force inherently suffers from inefficiency and unpredictability.  As in the example above, where an ounce of insight unlocks a thousand-fold increase in performance, a deep understanding of enterprise workflows, how enterprise data is created, shared, and used, can unlock better ways to use that data with AI. 

If you are thinking, "🐂💩! How am I going to find the HTML, XLS, DOCX, or whatever source the PDFs were created from?" 

We're with you...or at least we were before that question drove us to a solution.

## Data Bread Crumbs and Stepping Stones

Enterprises inherently generate massive amounts of duplicate data. Both the PDF and the HTML it was created from are stored, as are the emails that were sent with one of these in the attachments.  Formatting differences hinder straight up comparison of every PDF to every DOCX file, but while we are embedding them into Retrieval Augmented Generation (RAG) stores we break the files down into sentences, paragraphs, tables and other chunks of data.  

These chunks of data form natural, deterministic connections between documents, database tables, applications, AI agents, users, and even the APIs that move the data.  Like in the neural networks fundamental to today's AI, these connections are as important as the data itself to knowing what the data is, what business context applies to it, and how to best utilize the data.  

Knowing that the majority of PDFs exist alongside the documents and data sources they were created from, building RAGs in two-stages would give a significant performance increase:
1. Parse all non‑PDF documents first, build a RAG from them.
2. Look up PDFs, after parsing with fast, non‑AI, tools like [pypdfium2](https://pypi.org/project/pypdfium2/), and only parse with AI and add to the RAG if no good match is found. 

Like graph-based RAG or GraphRAG, knowledge graphs are key to putting these thoughts into practice. But unlike GraphRag, we need to look beyond relationships inside documents, and leverage the relationships between them. Even incomplete knowledge about the lineage of data can yeild significant benefits.  By explicitly mapping these relationships, we can uncover the origin, context, and optimal use of data, enabling the precise identification and control over data needed to optimize AI outcomes.

## It's Time to Work Smarter 🧐 With AI 🤖

The limitations of brute-force PDF parsing through LLMs illustrate an essential point: true scalability and efficiency require understanding and leveraging data relationships and lineage. PDF parsing difficulties underscore a broader enterprise truth—effective data utilization isn’t merely about computational power but rather about intelligently understanding and managing the underlying data.

If you would like to dive into the results presented in this post, the data and code are publicly available on Caber's GitHub page at the link below:

[https://github.com/Caber-Systems/ComparePDFparsers](https://github.com/Caber-Systems/ComparePDFparsers)

Dramatically faster and more accurate PDF data extraction could be achieved by leveraging existing document structures.

The PDF Parsing Secret That Outperforms AI by 1000x

Trustworthy AI is an uncomfortable subject. **It’s the most difficult question that everyone needs an answer to; but you never want an AI to answer it for you.** It’s the question that no one wants to talk about; but...everyone wants to believe we have an answer to. 

## Garbage-in, Garbage-out
The old addage "Garbage-in, Garbage-out" is a <a href="https://www.sciencedirect.com/science/article/pii/S0893395224002667" target="_blank" rel="noopener noreferrer">fundamental truth in AI</a>. If you feed an AI system bad data, it will produce bad results. With little or no stretch of the imagination you'll arrive at the corrolary: "Untrusted data in, Untrusted results out".  It's a simple concept that is often overlooked in the rush to deploy AI systems.

Unfortuntaly, lack of predictability in AI systems prevents the inverse statement, "Trusted data in, Trusted results out", from being true for all results.  Yet it does hold true that to have trusted results out, you must have trusted data in.  This is the foundation of trustworthy AI.

## The AI Trust Gap

On our journey to build a fundamentally new kind of data governance platform we’ve asked over one-hundred enterprises a simple question... **“Do you Trust your Agentic AI Systems?”</strong>**

95% of the time, the answer is, “of course not; but we use it anyway. We don’t have any real choice. Our competitors use AI, and we can’t sit idle while they gain an advantage” – FOMO is a real thing.

<div class="w-3/4 mx-auto py-4" align="left">
    <p>Today, trust in AI-driven processes is a key barrier. For AI to succeed at scale, individuals, companies and partners must trust and take responsibility for ensuring that processes powered by AI are safe, reliable and effective... A global survey found that 61% of people hesitate to rely on AI systems, often due to concerns over <strong>data security</strong> and third-party involvement.</p><span class="caption"><a href="https://reports.weforum.org/docs/WEF_AI_in_Action_Beyond_Experimentation_to_Transform_Industry_2025.pdf" target="_blank" rel="noopener noreferrer">- World Economic Forum, AI Governance Alliance</a></span>
</div>

Trustworthy AI starts and ends with your data. Data is fundamentally critical for AI models. The more data you give a model, the more information it can reason through, plan with and work with, to structure remarkable superhuman results. “Data is to AI, what oxygen is to Humans" is a universally accepted constant in the AI industry.
Data security and third party involvement get to the heart of the AI Trust Gap.  Was the data tampered with?  Where did the data come from?  

## Data Lineage
The aspirational goal of Agentic AI systems is to achieve “on-par trust” with human employees, so that the business decisions being made with AI are “Trustworthy at parity” with the employees that direct the AI (or are replaced by it). This compounded trusted relationship generates an environment that should deliver radically new remarkable superhuman results for businesses. We need both the employee and the AI system to co-operate within a **Circle of Trust** that the business can prove, believe in and govern. 

Knowing if data, and hence AI, is within this circle of trust requires knowledge of that data's lineage.  This is a modern hybrid GenAI concept that converges the business goals of data governance, security, compliance, data Lineage and trust...into a single Mission critical business objective - to achieve AI Trust.

## Data Knowledge Graph

At Caber, our core Gen AI thesis is that **AI Trust starts and ends with data**. Knowledge of data quality requires all sources of AI data and metadata consumed by Gen AI have a trusted, discoverable, lineage that is strongly connected to its security posture; at a deep granular level beyond the document container (down into the intra-document chunk, embedding and byte-segment level). We believe that trust can only be built from deeply understanding how data, metadata and segmented chunks of data flow through-out enterprise networks and Gen AI Agentic stacks (e.g. API’s, services, endpoints, databases, object stores, file systems, caches, frameworks, pipelines, etc.); and the relationships of all those data elements are graphed together in a way that trust can be deterministically computed and governed for your business.
Knowing we can't rely on such a dynamic knowledge graph of data to be pre-existing in companies is the reason why building this graph efficiently and scalably is at the heart of Caber's platform.

<div class="w-3/4 mx-auto py-4" align="center">
      <img src="/images/blog/caber_data_knowledge_graph_dark.png" alt="Caber Data Knowledge Graph" onclick="window.open(this.src, '_blank');">
</div>
<p class="caption" align="center">
      <span>Caber links common byte-sequences found in stored objects, database records, on the network, in APIs, and used inside Agentic services to trace the lineage of data through a company's environment and identify it by the relationships it has.</span>
</p>

Caber becomes your trusted data system that independently and neutrally governs AI Trust for your enterprise. As such Caber becomes your always-on system that runs as a core platform underneath and with your Gen AI Data systems, curating a complaint **circle of trust** for users as they interact and converse with your deployed Agentic AI Apps and infrastructure.

## Conclusion
Gen AI Trust starts and ends with deterministic knowledge of your data. Caber delivers data knowlege, intelligence, and governance on a platform that can redact, block and deny low quality user-centric data flows before they are injected into the LLM’s context frame; deterministically preventing low performance, error-prone, inaccurate, false and confused LLM outputs. The result is higher quality LLM performance, with guaranteed private and secure AI application data operations made compliant to existing and upcomming regulations through conversational policy inputs. 

We understand that Trustworthy AI requires an ‘Unstoppable Unyielding Always-on’ service based on intelligently knowing what constitutes useful, high-quality, trusted data – and can share that knowledge with other Agentic AI systems through powerful API services & insights. for Data that is utilized, accessed and flows throughout the entire Gen AI Agentic stack.
 
Caber is the platform that delivers this vision. We are excited to be on this journey with you.


Do trusted data inputs lead to trusted data outputs?  Lineage is fundamental to the answer.

Trust in AI Requires Trust in AI Data Use

Getting access control to work in vector databases is no easy feat. Most likely an AI agent using a service account with broad access rights, not the user the access is made on behalf of, is performing all data reads.  In this blog post by <a href="https://zilliz.com" target="_blank" rel="noopener noreferrer">Zilliz</a>, the makers of the <a href="https://milvus.io" target="_blank" rel="noopener noreferrer">Milvus</a> vector database, Caber CEO Rob Quiros discusses the challenges of data governance in AI systems like this at Zilliz's Unstructured Data Meetup.  A recording of the talk is available by clicking <a href="https://www.youtube.com/watch?v=gAqHbNrHMsM" target="_blank" rel="noopener noreferrer">HERE</a>

<div class="mx-auto py-4" align="center">
    <img src="/images/blog/zilliz_caber_blog_image.png" alt="Click this image to read the original post on Zilliz's website" onclick="window.open('https://zilliz.com/blog/beyond-rag-partitions-per-user-per-chunk-access-policy', '_blank');">
</div>
<p class="caption" align="center">
    Click the image above to read the original post on Zilliz's website
</p>


Dynamic policies and AI agent identites undermine access control

Fixing Access Control in Vector DBs

Retrieval Augmented Generation (RAG) has become essential for enterprise AI applications to make proprietary data available for responding to user queries. Optimizing RAG's accuracy and precision ensures relevant application responses while maintaining data security and privacy. However, these objectives often conflict. Current methods for enforcing permissions on RAG vector databases can degrade results or leak data to unauthorized users due to the need to divide enterprise data into chunks for GenAI applications.

## AI's Chunked View of Data
Businesses typically handle data as files, documents, videos, or database tables--units of information we can name, organize, and manage. Metadata, such as creation time, size, creator, and applicable policies, links to these data units when stored.
However, GenAI applications don't view data this way. They break down large, complex sources into smaller, semantically meaningful segments or chunks. LLMs train on chunks, vector databases store vectors created from chunks, AI applications consume data in chunks, and APIs called by AI agents move data in chunks.

<div class="w-3/4 mx-auto py-4" align="center">
    <img src="/images/blog/rag_permissions_0.png" alt="Same data chunks in different Apple 10Q statements" onclick="window.open(this.src, '_blank');">
</div>
<p class="caption" align="center">
    Common paragraphs or chunks (blue) and unique chunks (orange) mixed in two Apple 10Q statements. Denying access to all chunks from the Q2 2024 statement in a vector database would also deny users access to much of the data in the Q2 2023 statement as well.
</p>

It seems intuitive that document-level attributes should apply to their chunks. Many AI vendors reinforce this idea. However, this assumption is fundamentally flawed and can lead to inaccurate AI responses, biased results, and compromised data security.

## Data Redundancy from Chunking
Through our experience with <a href="https://www.f5.com/pdf/products/big-ip-wan-optimization-manager-ds.pdf" target="_blank" rel="noopener noreferrer">WAN optimization</a>, over 90% of data chunks flowing through enterprise networks are redundant. In storage systems holding enterprise data, 50%-90% of <a href="https://www.snia.org/sites/default/files/Understanding_Data_Deduplication_Ratios-20080718.pdf" target="_blank" rel="noopener noreferrer">storage blocks are duplicates</a> . 

As documents are shared, edited, and converted across platforms, identical paragraphs or images often appear in multiple versions. These redundancies accumulate as authors merge comments from various drafts, circulate files, and manage revisions.

<div class="w-3/4 mx-auto py-4" align="center">
      <img src="/images/blog/rag_permissions_1.png" alt="Duplicate data biases AI" onclick="window.open(this.src, '_blank');">
</div>
<p class="caption" align="center">
      <span>AI researchers at Google and University of Pennsylvania recorded the number of duplicate chunks seen across 348 million web documents. The number of chunks appearing only once have Group Size = 1. Eighty-five thousand chunks appeared 6–10 times. The length of chunks are not represented in this graph. Source: <a href="https://arxiv.org/abs/2107.06499" target="_blank" rel="noopener noreferrer">https://arxiv.org/abs/2107.06499</a></span>
</p>

## Unintended Consequences
When enterprise documents are ingested into an LLM training set or RAG vector database, identical chunks are stored only once. Identical binary sequences generate identical vectors, which vector databases use for data lookup.

However, metadata can differ significantly between documents containing the same chunks. When two such documents are ingested, the first chunk gets metadata from the first document, which the second ingestion overwrites with the second document's metadata. This can lead to problems including poor AI performance, data leakage, and biases in responses


<div class="w-3/4 mx-auto py-4" align="center">
    <img src="/images/blog/rag_permissions_2.png" alt="Graph of data chunk relationships across Apple 10Q statements" onclick="window.open(this.src, '_blank');">
</div>
<p class="caption" align="center">
    Using content-defined chunking, data in seven Apple 10Q statements (pink and blue dots) common sets of chunks (in red) can be mapped to the statements they are found in. Pink dots represent the entirety of data in each statement as would be seen by data governance systems that do not take chunking into account.
</p>

### Poor AI Performance and Data Leakage
Consider a document with metadata allowing public access and another with metadata restricting access to Amy. If a common chunk exists in both, the second document's restrictive permissions might overwrite the first's open-access metadata. Bob, who should access the first document, might be denied access, while Amy could see both documents' data.

This issue, if unchecked, diminishes RAG search precision despite improvements from graph-based RAG, cascading retrieval, and other methods. Without a monitoring system, users might receive incomplete responses or unintentionally access sensitive data like coworker salaries.

### Ethical Issues and Biases
As chunk sizes increase, sub-sequences within chunks become a concern. Paragraph-length chunks may contain common sentences appearing in multiple chunks. Research from <a href="https://arxiv.org/abs/2107.06499" target="_blank" rel="noopener noreferrer">Google and University of Pennsylvania</a> shows duplicate data in LLM training sets can introduce biases and errors in responses. Repetition in prompts has been shown to <a href="https://shelf.io/blog/10-ways-duplicate-content-can-cause-errors-in-rag-systems" target="_blank" rel="noopener noreferrer">cause errors</a> in RAG systems.

Where limits exist on the number of chunks retrieved from RAG, a user query with a high similarity score to chunks containing the same sentence, may preclude other chunks relevant to the query to be dropped thus reducing response accuracy.

## Conclusion
The rise of agentic AI--where autonomous AI agents collaborate and take intertwined actions--adds new layers of complexity to enterprise AI systems. These agents depend on dynamically retrieved and processed data, making robust data governance essential. As agents interact, risks like data mismanagement, bias, and leakage increase, elevating data governance from a best practice to a critical necessity.
Traditional data governance systems designed for documents, files, and tables--data-at-rest--are incompatible with how AI processes data. GenAI applications break data into "chunks," stripping away context and metadata needed for effective policy enforcement. To control enterprise data use in these complex AI ecosystems, data governance products must adapt by addressing the unique challenges of chunked data. Without this shift, enterprises face degraded AI precision, reduced reliability, and potential data breaches--risks amplified by agentic AI's interconnected nature.

Duplication of data across documents causes unseen biases in Retrieval Augmented Generation

How Duplicate Data Kills Your RAG

In the cybersecurity echo chamber, the relentless assault of false positives on Security Operations Center (SOC) teams is surpassed only by the cacophony of vendor marketing fluff claiming to reduce them. It's like listening to a symphony played on broken instruments. You know you're more likely to get that two or five percent reduction by turning off an existing tool than adding a new one.

False positive rates in existing tools are awful. Yet, we've accepted their inevitability to the point where we measure them to compare how good a tool is versus how bad it is. "Sucks-less" may be inevitable in political decisions, but it doesn't have to be with security tools. Deterministic detection of security incidents exists with the potential to silence the noise of those broken instruments and reveal attack signatures too faint to break through.

This blog explores the transformative potential of deterministic authorization across the three states of digital data, at rest, in transit, and in use, to eliminate false positives in incident detection.

### **The False Positive Quandary and Deterministic Solutions**

#### **Data-at-Rest: A Model of Determinism**

The foundation of data security, data-at-rest, showcases the efficacy of deterministic access controls such as Access Control Lists (ACLs). These controls set the stage for explicit, enforceable permissions that significantly mitigate the risk of false positives, demonstrating the untapped potential of deterministic methods in broader cybersecurity contexts.

#### **Data-in-Transit: Bridging the Gaps**

When data transitions into motion, it encounters a landscape where deterministic authorization becomes challenging yet increasingly necessary. The lack of embedded permissions in data packets underscores a critical vulnerability in confidentiality, despite existing measures for integrity and availability. This gap highlights the urgent need for integrating permissions within data protocols to ensure comprehensive and deterministic security measures.

#### **Data-in-Use: The Final Frontier**

Arguably the most complex state to secure, data-in-use demands a nuanced approach to deterministic incident detection. The proliferation of microservices and cloud-based architectures has obscured the clear linkage between user identities and data permissions, necessitating innovative solutions to restore determinism in authorization practices within these distributed environments.

### **Towards a Deterministic Future: Strategies and Solutions**

The journey to a more deterministic cybersecurity framework requires a multifaceted strategy. It begins with embedding permissions directly within data transactions and extends to reevaluating our architectural approaches to cloud applications. By drawing inspiration from initiatives like Caber's CA/CO, and filling in the gaps exposed by Google's BeyondProd, and enterprise Digital Rights Management we can chart a path towards achieving deterministic authorization across all states of digital data.

### **Conclusion: A Call to Action for SOCs**

The embrace of deterministic incident detection signals a pivotal moment in cybersecurity. By adopting a deterministic approach, SOCs can dramatically reduce false positives, focusing their efforts on genuine threats and enhancing overall operational efficiency. This blog has outlined the critical steps and considerations necessary for this transformation, highlighting the indispensable role of innovative solutions and collaborative industry efforts. As we move forward, the call to action is clear: the time for SOCs to pivot towards a deterministic and more secure future is now.


Directly securing data-in-transit is the key to eliminating growing false-positives

Posts