The Copy-Paste Breach

Within 20 days of Samsung's semiconductor division granting engineers permission to use ChatGPT, three separate incidents occurred. The first engineer pasted proprietary source code into the chatbot to debug a database error. The second shared similarly sensitive code for optimisation purposes. The third transcribed an entire internal meeting using a third-party audio tool and fed the full transcript into ChatGPT to generate meeting minutes.

In all three cases, the employees were not acting maliciously. They were trying to work faster. At the time, OpenAI's default policy allowed submitted data to be used to improve future models. Samsung had issued informal warnings not to enter private information, but no technical controls existed to enforce the policy.

Samsung launched disciplinary investigations, capped all ChatGPT prompts at 1,024 bytes, and one month later banned all generative AI chatbots from company-owned devices and networks.

On 20 March 2023, a bug in an open-source Redis client library used by OpenAI created a nine-hour window in which some ChatGPT users could see the conversation titles — and in some cases billing information — of other active users. OpenAI CEO Sam Altman publicly described it as a "significant issue."

Approximately 1.2% of ChatGPT Plus subscribers active during that window had data exposed: first name, last name, email address, billing address, and the last four digits and expiry date of their payment card.

This was not caused by anything a user did. The exposure came from a third-party dependency containing a bug that corrupted cached session data, allowing one user's session to leak into another's view. Italy's data protection authority used the incident as the basis to temporarily ban ChatGPT and in December 2024 fined OpenAI €15 million for underlying GDPR violations. In March 2026, the Court of Rome annulled that fine in its entirety on appeal.

Three days after Amazon announced its enterprise AI assistant Q at re:Invent 2023, internal employees began raising urgent alarms. According to internal documents reported by Platformer, Amazon Q was "experiencing severe hallucinations and leaking confidential data" during its public preview, including the physical locations of AWS data centres, details of internal employee discount programmes, and unreleased AWS features not yet public.

Amazon disputed the severity of the reports, stating publicly: "No security issue was identified as a result of that feedback." The company accelerated tuning of Q before general availability.

This case is categorised as reported, not confirmed, because Amazon's response contested the characterisation. The underlying dynamic is nonetheless real: AI systems trained on or given access to internal knowledge can surface information through outputs in ways that were never anticipated during design.

In May 2024, security researchers discovered that Slack's revised privacy policy — quietly updated in September 2023 — allowed the company to use "Customer Data (e.g. messages, content and files) submitted to Slack" to develop AI and machine learning models. The policy was opt-out by default. Individual users could not opt out themselves — only workspace administrators could, by emailing Slack directly.

The data potentially included private direct messages, confidential internal channels, shared files, and other workspace content. Slack's AI product page simultaneously stated "Your data is yours. It won't be used to train Slack AI" — creating messaging that contradicted the underlying policy terms.

Under public pressure, Slack issued a clarification stating that direct message content was not used to train generative AI models, only aggregate platform-level data. Critics noted the original policy language had been in place for months without prominent communication.

This is not a data breach in the traditional sense. It is a data-governance warning shot: vendor terms of service can change the fate of your business data without a visible incident, without a breach notification, and without requiring anyone to do anything wrong.

This case has two components that together make GitHub Copilot one of the most documented AI data exposure vectors in development environments.

Training data memorisation. Researchers constructed 900 prompts designed to extract credentials from Copilot's training data. The result was 2,702 hard-coded credentials surfaced through Copilot's code suggestions, of which at least 200 were confirmed functional secrets still active on GitHub. GitGuardian further found that repositories where Copilot is actively used show a 40% higher rate of secret leaks compared to the average public repository.

CamoLeak vulnerability. Security researchers disclosed a vulnerability in GitHub Copilot Chat that allowed attackers to embed malicious prompt injections inside pull request descriptions. When a developer asked Copilot about the pull request, the hidden instructions activated, causing Copilot to encode stolen API keys and source code into image URLs and exfiltrate them to an attacker-controlled server. GitHub addressed the vulnerability by disabling image rendering in Copilot Chat. Because primary-source CVE confirmation is still pending, this article avoids naming a CVE ID.

Between 8 and 18 August 2025, a threat actor compromised one of Salesloft's internal GitHub repositories, discovered a sensitive OAuth token, and used it to authenticate into Salesloft's account within the Drift AI chatbot — a sales conversation tool integrated with customers' Salesforce and Google Workspace environments.

Using Drift's trusted integration access, the attackers systematically exfiltrated data from the Salesforce instances of over 700 organisations globally. Confirmed victims included Cloudflare, Palo Alto Networks, Zscaler, Tenable, and Proofpoint. FINRA issued an emergency alert to all member financial firms. Google's Threat Intelligence Group published a detailed advisory.

Cloudflare published a public postmortem confirming their Salesforce instance was accessed between 12 and 17 August. The attacker exfiltrated support case content including, in some cases, customer-submitted API tokens, passwords, and configuration details. Cloudflare rotated 104 API tokens and notified all affected customers directly.

One company was not affected: Okta, which had implemented IP allow-listing on its Salesforce integration. The stolen OAuth token was useless against Okta's IP restrictions.

Microsoft 365 Copilot does not have a data leakage bug. It is working exactly as designed. The problem is what it is designed to do: index and search across SharePoint, OneDrive, Teams, and Exchange — respecting whatever permissions exist across those systems.

In most organisations, those permissions are years of accumulated mismanagement. Widely shared links, "everyone except external users" settings, sites that were never locked down after projects ended. When Copilot arrived, it became capable of surfacing documents that employees were never meant to see — not because Copilot had an exploit, but because the permissions had been wrong for years and no one noticed until AI made them easy to query.

Research found that approximately 16% of business-critical data is overshared across the average Microsoft 365 tenant, with around 802,000 files at risk per organisation. A Gartner survey found that 40% of organisations delayed their Copilot rollout by three months or more due to concerns about oversharing. Security researchers also documented prompt injection attacks where malicious payloads embedded in emails caused Copilot to exfiltrate sensitive SharePoint documents via Teams or SharePoint links.