Blood in the water: FDA just cited AI in a warning letter

Howdy friends,

Well, it finally happened.

If you work at the intersection of life sciences and AI, you were probably holding your breath for this one too — the first FDA warning letter citing inappropriate use of generative AI. It went to Pureolea Cosmetics Lab out of Livonia, MI, and boy, did it blow up my feed. Blood in the water for every "expert" to dive in with a hot take (sigh…yes, including me).

As much as I hate to say it, a lot of us saw this coming. But today I want to talk about what else this letter means — because the AI angle is getting all the oxygen, and I think there are more important signals buried in it.

First, no shade to Ms. Maria at Pureolea. People get warning letters all the time for exactly the things that made up the bulk of this one — unapproved drug, insanitary conditions, cGMP violations, yadda yadda. These are the very reasons CMC consultants like me (and probably many of you) have made a tidy living helping companies navigate regulatory minefields. AI was one part of this letter, under the heading "Inappropriate Use of Artificial Intelligence in Pharmaceutical Manufacturing." Three paragraphs out of the whole thing.

There are a lot of reasons small businesses end up in regulatory trouble, but the main one — in my opinion — is not knowing when to ask for help.

What's new is this weird, poorly-defined layer on top of everything, where individuals genuinely believe AI will solve all their problems, act as the consultant they don't want to pay for, and (maybe worst of all) "knows everything."

So what actually got flagged? The AI section is short, but it's worth reading carefully because it tells you exactly how FDA is thinking about this. Three things:

1. Using AI to create compliance documents without human review. Ms. Mattina told investigators she used AI agents to generate drug product specifications, procedures, and master production records to meet FDA requirements. FDA's response: if you use AI as an aid in document creation, you must review the output to make sure it's actually accurate and actually compliant. Failure to do that is a straight-up violation of 21 CFR 211.22(c). The AI isn't the problem. The missing human in the loop is.

2. Overreliance on AI as a substitute for regulatory knowledge. The investigators found the firm hadn't performed process validation before distributing drug products (required under 21 CFR 211.100). Her reply, quoted in the letter: she didn't know it was required because the AI agent never told her it was. I cannot stress enough how much this is the sentence in the letter. "The AI didn't tell me" is now on the record as a failed regulatory defense.

3. No QU oversight of AI output. FDA closed the section by spelling out the expectation going forward: any output or recommendation from an AI agent used in CGMP activities must be reviewed and cleared by an authorized human representative of the firm's Quality Unit. In other words — AI can draft, but your QU signs.

Now, before you read that and think "well, that could never happen to me" — let me stop you right there.

I personally know GMP professionals — working at real companies, holding real quality titles — who are using ChatGPT like it's Google. They type in a work question, get an answer, and pass it around their team as if it were vetted. No verification. No second source. No "let me check this against the reg." Just an, “oh man, this looks great!”

If that's you, or anyone on your team: stop. That is the exact failure mode in the Pureolea letter. The only difference is that Ms. Mattina got caught because FDA was already in the building for other reasons. Most of the people doing this won't get a warning letter. They'll just quietly put out bad work until something breaks — a client catches it, an audit flags it, or a regulator shows up for something else and starts pulling threads.

None of this is new thinking if you already work in a regulated quality system. It's just the first time it's been written down in a warning letter.

But aside from all that, there’s a different point that should get your attention.

AI is enabling FDA to widen the scope of who they have the resources to go after.

Only 64 of 807 CDER warning letters since January 2021 even mentioned cosmetics. On my vibe-only analysis, that's because FDA had bigger, more threatening fish to fry and not enough hours in the day. That's changing. We're going to see the hammer come down on a lot of companies that have been flying under the radar — and the agency’s own use of AI gives FDA the jet fuel they needed.

So if you're using gen AI in regulated workflows without thinking about a wider program to contain it, consider this your wakeup call.

Think of it like data integrity. We all know what DI is. None of us would dare operate without a program in place for it. This is the same thing. I think because AI is so easy to access and so easy to get started with, there's a creeping assumption that it can't possibly undermine a quality program. Hopefully by now you know it can.

Which brings me to something I've been building for exactly this moment.

I'm running a 3-hour live workshop called AI-Proof Your Practice, where we'll discuss, develop, and lay out a complete, customized AI Use Policy for your practice or team. It maps directly onto the ALCOA+ principles you already live by — scope and applicability, model selection, verification and QA, disclosure and traceability, and incident response. The next time a client, auditor, or your leadership team asks "what's your AI policy?", you'll have a real answer instead of winging it.

The Pureolea letter is the first of these, and it won't be the last. Let's make sure none of them have your name on them!

Thanks for reading!

Alexa