POPIA compliance for AI systems: what section 19 actually asks of you

POPIA does not have an AI section. It was passed in 2013 and commenced in 2020, before the current generation of LLM applications existed. That has not stopped the Information Regulator from applying it to AI systems. If your AI processes personal information of South Africans, you are subject to POPIA, full stop. This post is about how to actually meet the standard.

What POPIA section 19 actually says

Section 19(1) of POPIA: "A responsible party must secure the integrity and confidentiality of personal information in its possession or under its control by taking appropriate, reasonable technical and organisational measures to prevent (a) loss of, damage to or unauthorised destruction of personal information; and (b) unlawful access to or processing of personal information."

That single paragraph has done a lot of work in regulatory practice. Three words to focus on. Appropriate: the measures must fit the sensitivity of the data and the size of the operation. A scanned ID document deserves more protection than a marketing email address. Reasonable: the measures must be commercially feasible and aligned with industry norms. Measures: plural, and both technical (encryption, access control, monitoring) and organisational (policies, training, incident response).

"Appropriate" and "reasonable" are deliberately open-ended. The Information Regulator gets to decide after the fact whether what you did was enough. The way you de-risk that conversation is by aligning to recognised industry frameworks, documenting your choices, and being able to show evidence after an incident.

The eight POPIA conditions, briefly

Section 19 is the security condition. POPIA has eight. They all apply to your AI system. Accountability, processing limitation, purpose specification, further processing limitation, information quality, openness, security safeguards (section 19), and data subject participation. If you've never read them in full, do.

For AI systems specifically, three of the eight cause the most trouble. Purpose specification (you have to know and disclose what the data is for, before you process it). Further processing limitation (using customer data to train or fine-tune a model is "further processing" and needs a basis). Data subject participation (the customer can ask what you have on them and ask you to delete it; "it's in a model weight, we can't delete it" is not an acceptable answer).

Where AI systems break POPIA

From real engagements, the recurring failure patterns:

Sending personal data to third-party LLM providers without a lawful basis or contract. You pipe customer messages to OpenAI or Anthropic or Cohere for inference. That's a cross-border transfer of personal information to an operator. You need: a written operator agreement (their standard terms usually meet this), a section 72 basis for the cross-border transfer (consent, contract necessity, or the recipient being in a jurisdiction with similar laws), and disclosure to the customer in your privacy notice.

Training or fine-tuning on customer data without notice or consent. Customers gave you their data for the stated purpose. Using it to train a model is a new purpose. You need a fresh basis. Most often that means updating your privacy policy and either an opt-in, an opt-out depending on context, or a contractual basis that explicitly contemplates training.

Logging that captures more than you can defend. AI systems generate large volumes of inference logs (prompts in, completions out). Those logs frequently contain personal information. They are subject to the same retention limits, access controls, and deletion rights as your primary database. Most teams treat them as ephemeral debug data and lose track of them.

No deletion mechanism for the model. A customer asks you to delete their data. You delete the row in the database. The model fine-tuned on their data still has weights influenced by them. POPIA's right to deletion is not absolute (section 24 has exceptions for legal obligations and the like) but you cannot just ignore it. If you trained on customer data, you need a plan for what "delete" means.

Vendor sprawl with no audit trail. Modern AI stacks touch five to fifteen vendors per request (the LLM provider, the vector database, the observability tool, the eval platform, the prompt-management tool, the orchestration layer, the embedding provider). Each one that sees personal data is an operator under POPIA. The Information Regulator can ask you to enumerate them. Most teams cannot.

Recent enforcement notices and what they tell us

The Information Regulator has become noticeably more active. Confirmed enforcement actions include the Department of Justice (R5 million administrative fine), the Department of Basic Education (R5 million), Lancet Laboratories (R100,000), Blouberg Municipality (R500,000), and FT Rams Consulting (R100,000). The Regulator has also issued enforcement notices against several private and public bodies in subsequent months. Notices are published on inforegulator.org.za and are now a real reputational and financial risk, not a theoretical one.

The pattern in those decisions is informative. The Regulator looks for: evidence of pre-incident security measures (audits, pentests, monitoring), evidence of an incident response plan that was actually followed, timely notification of the Regulator and affected data subjects, and demonstrable remediation. The absence of any of those is what turns an investigation into a fine.

A working POPIA checklist for AI builders

If your AI system processes personal information of South Africans, you should be able to produce evidence of the following:

1. A current data flow diagram showing every place personal data enters, transforms, leaves, and is stored.
2. An operator register listing every third party that processes the data, what they process, where, and under what contract.
3. Operator agreements with each (LLM vendor standard terms usually qualify, but check).
4. A lawful basis for cross-border transfers (most LLM inference goes to US-hosted endpoints, so section 72 applies).
5. A privacy notice that disclosees AI use, the categories of data the AI processes, and any training or fine-tuning use.
6. A retention policy for inference logs and training data, with automated deletion at the retention horizon.
7. A documented deletion-request workflow that addresses model retraining or weight-impact where applicable.
8. Technical security measures (encryption in transit and at rest, access controls, monitoring) appropriate to the sensitivity of the data.
9. Independent technical testing (vulnerability assessment plus penetration test) evidencing your security measures.
10. An incident response plan with the Information Regulator notification path, and a tabletop exercise log to show you've rehearsed it.

The penalty math

POPIA section 109 caps administrative fines at R10 million per infringement notice. Section 107 sets out criminal offences (R10 million or 10 years' imprisonment for the most serious). The Regulator can also issue enforcement notices requiring specific remediation. The reputational cost of being on the public enforcement notice page typically dwarfs the fine itself, particularly for fintechs that depend on enterprise diligence to close deals.

The takeaway is not "POPIA is scary." It is: if you build AI on personal information, you can defend yourself or you cannot, and that question is decided by what you documented and tested before the incident, not by what you say afterwards.

Building an AI product without a POPIA story is building a product you cannot sell to any serious enterprise customer in South Africa and cannot defend to the Regulator if anything goes wrong. The remediation is straightforward, but it has to happen before you ship at scale, not after.