Attack scenario · AI security

AI agent with access to confidential internal data

Can an AI assistant expose data a user is not authorized to see? An internal assistant retrieves from all company documents with its own service identity, so any employee can ask it for anything.

Fictional, educational scenarioData exposureLast updated · ATT&CK v20.1

Attack path · 3 stages

  1. 1

    Use the assistant as any employee

    T1078

  2. 2

    Ask for restricted information

    T1213AML.T0085

  3. 3

    Assistant discloses restricted content

    T1213AML.T0085

For leadership

Executive summary

Many internal AI assistants are connected to a large pool of company documents through a single service account. That effectively gives every user of the assistant the service account’s access — including HR files, board materials or customer contracts they could never open directly. No attacker is required: a curious employee, or one whose account is compromised, can simply ask. The safeguard is to make the assistant respect each user’s existing permissions at retrieval time, and to keep the most sensitive repositories out of general-purpose assistants unless that is proven.

For engineers

Technical summary

Retrieval runs with the assistant’s identity rather than the user’s, without document-level access filtering. Sensitive chunks are returned for any query that is semantically close (ATLAS AML.T0085 Data from AI Services; related to sensitive information disclosure in OWASP LLM guidance).

Environment and assumptions

Fictional company: a knowledge base indexes an S3 bucket that mirrors the corporate file share, including restricted HR and finance folders. The assistant is available to all employees.

  • No per-user filtering on retrieval.
  • The adversary is an employee or controls an employee account.
  • Fictional and simplified for education.

Interactive

Attack sequence

Select a stage, or use the arrow keys, to see preconditions, evidence and controls.

Stage 1

Use the assistant as any employee

A user (or compromised account) signs in to the assistant.

Preconditions
  • Valid employee access
Required access
  • Employee account.
Potential evidence
  • Assistant sign-in logs
Defensive controls
  • SSO with MFA
Text version of the attack sequence
  1. Use the assistant as any employee (T1078). A user (or compromised account) signs in to the assistant. Preconditions: Valid employee access. Evidence: Assistant sign-in logs. Controls: SSO with MFA.
  2. Ask for restricted information (T1213; AML.T0085). The user asks questions designed to surface compensation, acquisition or customer contract details. Preconditions: Restricted documents are indexed. Evidence: Model invocation and retrieval logs showing restricted sources. Controls: Document-level access filtering using the user’s identity.
  3. Assistant discloses restricted content (T1213; AML.T0085). Retrieved restricted chunks are summarized in the answer. Preconditions: No authorization check on retrieval. Evidence: Response logs. Controls: Exclude highly sensitive repositories; Sensitive-information output filters.

Technique mapping

MITRE ATT&CK and ATLAS techniques referenced in this scenario
IDTechniqueTactics (official)Stages
T1078Valid AccountsInitial Access, Persistence, Privilege Escalation, Stealth1
T1213Data from Information RepositoriesCollection2, 3
AML.T0085Data from AI Services (MITRE ATLAS)ATLAS: 1 tactic2, 3

Technique names and tactic assignments come from the imported MITRE data. Mapping a stage to a technique is Securitain’s analysis.

Why the attack can succeed

  • Authorization was delegated to the assistant’s service identity.
  • Indexing copied documents without their permissions.

What prevents or limits the attack

What may detect the attack

Detection signals and telemetry sources
SignalTelemetry sourceNotes
Responses sourced from restricted repositoriesRetrieval logs—
Users issuing many queries for restricted topicsAssistant audit logs—

Telemetry only helps if it is enabled before the incident and someone acts on it. Listing a signal here does not mean it is collected or monitored in your environment.

What Securitain may assess

  • Securitain can assess todayDetection of identities with sensitive permissions such as broad S3 access, KMS decrypt or Secrets Manager reads. Learn more
  • Securitain can assess todayPublic and cross-account exposure checks on S3, KMS, SQS, SNS and Secrets Manager resource policies. Learn more
  • Outside Securitain’s current scopeSecuritain does not perform prompt-injection testing, agent execution tracing or AI-specific scanning. It can review the AWS IAM permissions an agent’s role holds, which bounds what a manipulated agent could do.

For leadership

Executive implications

confidentiality
Internal confidential information may be exposed to employees without need-to-know.
compliance
Privacy and data-protection obligations (for example HR data) may be breached.

No financial loss estimates are given; impact depends on the data, contracts and regulation involved.

Next steps

  1. List knowledge bases and their source locations.
  2. Remove restricted sources or add per-user filtering.
  3. Narrow the knowledge-base role’s S3 access.

References

MITRE ATT&CK® and ATT&CK® are registered trademarks of The MITRE Corporation. ATT&CK data © 2026 The MITRE Corporation. This work is reproduced and distributed with the permission of The MITRE Corporation (ATT&CK v20.1). MITRE ATLAS™ data (v2026.10) © MITRE, used under the Apache License 2.0. Securitain is not affiliated with or endorsed by MITRE. Explanations, scenarios and control mappings on this page are Securitain’s own analysis. Sources and licensing

Stay in the loop

Cloud security insights, early access & compliance updates

Weekly InsightsEarly AccessExpert Content
No spam, everUnsubscribe anytimeWeekly updates
Chat with Securitain