Data Privacy When Using Generative AI at Work: A Practical Policy Guide

Employees are already using generative AI tools to summarize documents, draft emails and debug code, often without telling anyone. Each prompt can carry company data to an external service. This guide explains where privacy risks come from and how to set rules that reduce them without banning a useful technology. It is general information, not legal advice; your obligations depend on your industry and location.

Where your data goes

When someone pastes text into an AI tool or sends it through an API, several things may happen:

1. The text is transmitted to the provider and processed on their systems.
2. It may be stored for some period, for abuse monitoring, debugging or history features.
3. It may be reviewed by provider staff in limited circumstances.
4. Depending on the product and plan, it may be used to improve models, unless you opt out or use a business tier that excludes it.
5. It may pass through sub-processors, such as cloud hosts, in other countries.

The terms differ widely between consumer apps, business plans and API access. The only reliable way to know is to read the provider's data processing terms and privacy documentation for the specific product and plan you use.

The main risks

Confidential data leakage. Source code, contracts, customer lists, unreleased financials and credentials pasted into a prompt leave your control. Secrets in particular should never be pasted (secrets management).

Personal data exposure. Sending customer or employee personal information to a tool that has no agreement with you may breach privacy law (for example the GDPR or CCPA) or contractual promises.

Training and retention. Data used to improve a model may influence later outputs, and data kept in history can be exposed by account compromise.

Shadow AI. Staff using personal accounts or unapproved tools beyond IT's visibility. Bans often increase this rather than stop it.

Output risks. Generated content may contain inaccurate claims, copyrighted material or biased statements (hallucinations).

Integration risks. Connecting AI assistants to email, drive and chat gives them broad access, and prompt injection can turn that access against you (OWASP LLM Top 10).

Classify your data first

Rules are easier when data has simple tiers:

What a good acceptable-use policy covers

1. Approved tools. List which tools and plans are allowed and for which data tiers. Provide a good approved option, since people use the unapproved one when the approved one is worse.
2. Prohibited inputs. Credentials and keys, regulated personal data, and anything covered by a confidentiality agreement unless explicitly permitted.
3. Account rules. Use company accounts with single sign-on; no personal accounts for work data.
4. Output responsibility. The employee is accountable for anything they use. Review for accuracy, and do not present generated content as verified.
5. Disclosure. When AI-generated content goes to customers or the public, state when required by policy or law.
6. Intellectual property. Guidance on using generated code or text, and on avoiding pasting third-party licensed material.
7. Incident reporting. What to do if sensitive data was pasted by mistake: report quickly so access can be revoked and secrets rotated.
8. Review cadence. The landscape changes quickly, so review the policy at least twice a year.

Technical controls that support the policy

• Provision approved tools through single sign-on with logging and retention controls.
• Use data loss prevention (DLP) rules to detect secrets and personal data in outbound traffic to AI sites.
• Offer an internal gateway that redacts sensitive patterns before forwarding to a model provider.
• Apply least privilege to AI assistants connected to company systems (agent identity and authorization).
• Log usage for security review, with access limited and a defined retention period.
• Where data must stay in your control, consider models hosted in your own cloud account or on your premises.

Vendor questions to ask

• Is our data used to train or improve models? Can that be disabled by contract?
• How long is it retained, and can we set retention to zero or delete it on request?
• Who can access it, and under what circumstances?
• Where is it processed and stored, and which sub-processors are involved?
• What security certifications and audit reports exist (for example SOC 2 or ISO 27001)?
• What is the breach notification commitment?

Start simple

If you have no policy today, begin with three rules everyone can remember: use approved tools only, never paste credentials or customer personal data, and verify before you publish. Then build out classification and technical controls as your usage grows. See also governing generative AI.