← Journal / articles

Read-only AI still moves data

An assistant can be unable to edit a source document while still moving its contents into a new system. That distinction matters when a team describes an AI pilot as safe because it is read-only.

An intact glass archive with information flowing through a reading aperture toward separate processing chambers, illustrating data movement without editing the source.

An assistant can be unable to edit a source document while still moving its contents into a new system. That distinction matters when a team describes an AI pilot as safe because it is read-only.

Read-only permissions are useful. They can prevent changes through a particular interface. They do not describe every destination that retrieved information may reach, or every person who can inspect a generated answer.

OWASP treats sensitive information disclosure and excessive agency as separate risks in its 2025 guidance. Its disclosure guidance addresses exposure of confidential information; its agency guidance includes limiting unnecessary permissions and tools. An operational review needs to examine both.

Follow one document through the request

Consider a hypothetical assistant answering questions about internal project proposals. A user asks for the commercial assumptions behind a delivery estimate. The search component finds a relevant document and includes a passage in a request to a language model.

The document has not changed. Information from it has nevertheless travelled through retrieval and inference. Depending on the deployment, some of that content could also appear in an answer, a trace or a support record. None of those destinations should be assumed to exist in every product. Each needs to be checked against the actual configuration.

A useful review follows this particular passage through the system. Record which service retrieves it, which identity authorizes the retrieval and where the next request goes. Then identify the answer's audience and any configured retention paths.

This is easier to evaluate than a broad statement that a product is local, private or enterprise-ready. Those words can describe different boundaries. The review needs a route someone can inspect.

Ask a narrower question about local deployment

A locally hosted model may form part of a larger workflow. The embedding service, search index or telemetry collector may be operated separately. Conversely, a deployment can deliberately keep all of those components inside a controlled environment. The label alone does not tell you which arrangement you have.

Ask where each component runs and what it receives. If an external service receives a representation rather than the original text, identify what that representation contains, how the service uses it and what confidentiality claims have actually been established. Do not assume that a transformed representation is anonymous or that it can always be reconstructed. Both conclusions require evidence appropriate to the representation and threat model.

The same care applies to a provider's promise that customer content is not used for model training. That promise concerns one use of the data. Retention, operational access and deletion are additional questions to resolve from the applicable terms and configuration.

Test permissions at retrieval

Suppose two users have access to different proposal folders. A test should check that the assistant cannot retrieve a restricted proposal for the other user. The source system's access rules need to survive the search path, including any indexing or caching layer used by the deployment.

Use fictional documents with distinct markers instead of real commercial information. Give each test identity only the intended permissions. Ask questions that would be easy to answer if an unauthorized document were visible, then inspect the permitted evidence about what was retrieved and supplied to the model.

An answer that happens not to repeat the restricted marker is insufficient to establish that retrieval was authorized. The information may already have entered another component. The test needs evidence at the boundary where access is enforced.

Inspect the destinations you actually operate

The review should cover the output channels configured for the pilot. A shared chat room, an exported report and a diagnostic trace can have different audiences. A sensible limit at retrieval can be undermined if the answer is subsequently delivered to a broader group.

Keep an inventory of intended destinations and the person responsible for each. Verify what content is retained, how long it remains and which administrative or support roles can inspect it. Where a setting is unavailable or a policy remains unclear, record that uncertainty before confidential data is introduced.

Avoid collecting more content merely to prove that monitoring exists. A synthetic marker can help trace a path without putting a genuine contract or personnel record into another testing archive.

Decide what a first pilot can demonstrate

A small pilot can establish a bounded claim: under a named configuration, specified identities retrieved only the permitted synthetic documents and the observed data route matched the approved design. That is useful evidence. It does not certify every future integration or guarantee that disclosure is impossible.

Change the retrieval source, telemetry settings or inference provider and the affected boundary needs another review. The decision should follow the system being operated, rather than remain attached to an earlier screenshot of its permission settings.

Read-only access belongs in that design. So does a clear account of where the information goes after a successful read.

Black Ice Labs explores these questions through applied AI research and engineering. Explore the lab’s public research notes.

Sources

OWASP — LLM02:2025 Sensitive Information Disclosure

OWASP — LLM06:2025 Excessive Agency

Research home: blackicelabs.ca/research

More articles ↗

Eva

Site guide · published material
  1. Eva

    Hello. I’m Eva, your guide to Black Ice Labs. What would you like to explore?

Explore answers from published pages. AI conversation is being connected.