Part 3 of our series on Privacy-Enhancing Technologies.
Of everything covered in this series so far, synthetic data is the topic that generates the most genuine excitement. Fake patient files that behave like real ones. Fake transaction histories. Fake everything—statistically real enough to build on, legally fake enough to use freely. With enterprise large language models (LLMs) now accessible to most organizations, generating synthetic data at scale has moved from a research project to a routine engineering task.
Which is precisely why we need to address the part that rarely makes it into a vendor's pitch deck: synthetic does not automatically mean private.
Some of the most enthusiastic engineering teams we speak with have also become the most skeptical, and for reasons that are worth taking seriously before you build a compliance strategy on this technology.
What is Synthetic Data? The Core Promise
The appeal of synthetic data is best illustrated with a concrete example. Consider a hospital that builds a synthetic patient population derived from real records. The synthetic dataset preserves the same statistical relationships between age, diagnosis, treatment outcome, and length of stay—but no synthetic record maps back to an actual patient.
Once that dataset exists, it can be used indefinitely: for software testing, machine learning model training, external research partnerships, and product demonstrations, all without ever touching a real patient's file again.
The core value proposition is straightforward: build the effort once, reuse it indefinitely, and eliminate the usual data-handling compliance overhead.
How This Differs from De-Identification
De-identification works on the real records themselves - stripping or masking direct identifiers (names, medical record numbers, exact dates) from the original dataset, typically under a standard like HIPAA's Safe Harbor or Expert Determination method. The real patients' records still exist; they've just been altered. Synthetic data takes a different approach: it generates new records that never belonged to a real person at all, learned from the statistical patterns of the originals.
That distinction is also why synthetic data invites more scrutiny, not less. HIPAA de-identification has decades of regulatory guidance behind it, defined methods, and a known error rate. "Synthetic" doesn't have an equivalent standard yet - which is exactly why the testing described below isn't optional.
The Problem Nobody's Pitch Deck Mentions
The critical limitation of synthetic data is one that is rarely disclosed upfront. Generative models do not always generate novel data- sometimes they memorize and reproduce actual training records nearly verbatim. This is not an edge case. Researchers have extracted real training data from a model marketed as "privacy-safe" in under 30 minutes. A widely cited 2022 Royal Society report identified the same phenomenon across multiple synthetic data systems.
Beyond direct memorization, two additional failure modes are worth understanding:
- Structural Leakage. Even without a single direct identifier present, synthetic records can preserve enough statistical correlation and pattern that an adversary with access to external data can re-identify an individual by matching structure rather than values. The person's name is absent; their unique combination of attributes is not.
- Membership Inference. An attacker does not need to recover your exact record. In many cases, simply confirming that someone matching a given profile was present in the original training dataset constitutes a privacy violation in its own right. The question "was anyone with this rare combination of conditions in the training set?" can be enough.
Underlying both of these risks is a fundamental tension: fidelity and privacy pull in opposite directions. Synthetic data that is too close to the original leaks. Synthetic data that is too far from the original loses the statistical utility that made it valuable in the first place. There is no configuration that delivers maximum realism and maximum privacy simultaneously.
This is likely why your engineering team is less enthusiastic about synthetic data than your business stakeholders. The concern is not that "synthetic data is bad." It is that "synthetic data marketed as inherently private is bad." Those are materially different claims, and the distinction matters for compliance purposes.
Is Synthetic Data Worth the Investment?
The answer is yes, but only with a critical caveat that transforms it from a marketing claim into a rigorous engineering practice.
| Stage | What It Involves | Why It Cannot Be Skipped |
|---|---|---|
| Step 1: Generation | Create a synthetic dataset from real records using a generative model. | Produces statistically representative data that can be reused, analyzed, and shared more safely. |
| Step 2: Testing | Assess the synthetic dataset for memorization, structural leakage, and membership-inference vulnerabilities before any external use. | Without testing, synthetic data may still expose information from the original records, undermining its claimed privacy protections. |
Where Differential Privacy Fits In
Testing catches leakage after the fact. Differential privacy is the tool for reducing it during generation in the first place - adding calibrated statistical noise to the process so that no single training record has an outsized effect on the output. It doesn't replace testing (a differentially private generator can still be tuned badly), but the two together are a stronger position than either alone: DP bounds the theoretical worst case, testing confirms what actually happened.
We cover how differential privacy works, and where organizations like the U.S. Census Bureau, Apple, and Google use it, in The One Anonymization Method With Actual Math Behind It: Differential Privacy Explained.
Where does synthetic data earn the investment?
Recurring reuse cases where the same dataset will be used for years—such as a hospital patient population used for ongoing model training—rather than a one-off internal report.
Where is it probably not worth it?
High-stakes accuracy contexts, such as diagnostic models or risk-scoring systems, where fidelity must be near-perfect. This is precisely where leakage risk is also at its highest.
One additional point worth noting: synthetic data generation does not eliminate the need for data cleaning. Garbage in still means garbage out. Synthetic generation does not skip that step—it simply moves it earlier in the pipeline.
Coming Up
Next in our series: Federated Learning.
It is a narrower use case, but it represents a fundamentally different architectural approach to privacy: training machine learning models without ever moving the underlying data at all.
---
This post is for general informational purposes and does not constitute legal advice. Whether a specific synthetic dataset satisfies a given anonymization or de-identification standard depends on how it was generated and tested. If you want a second opinion before relying on "synthetic" as a compliance answer, our attorneys specializing in data privacy are happy to review it with you.
---
P.S. The gap between what vendors promise and what technology actually delivers is one of the most important things a skilled professional can learn to navigate. Synthetic data is a useful case study in exactly that. As you evaluate new technologies for your organization, the ability to ask the right skeptical questions, and to distinguish a marketing claim from an engineering reality, is a skill that compounds in value over time. It is worth cultivating deliberately.
- Synthetic data is often marketed as inherently private, but generative models can memorize and reproduce real training records nearly verbatim, so synthetic doesn't automatically mean anonymous.
- Beyond direct memorization, synthetic data carries two subtler risks: structural leakage, where statistical patterns alone allow re-identification, and membership inference, confirming someone matching a profile was in the training data.
- There's an inherent tradeoff between fidelity and privacy: synthetic data realistic enough to be useful tends to leak more, while data private enough to be safe loses the statistical value that made it worth generating.
- Synthetic data is worth investing in for recurring, long-term reuse cases, but probably not for high-stakes accuracy contexts like diagnostic or risk-scoring models, precisely where leakage risk is highest.
- Testing a synthetic dataset for memorization, structural leakage, and membership-inference vulnerabilities is a required step, not optional, before treating it as a compliant substitute for real data.
- Generating synthetic data doesn't remove the need for data cleaning; it just moves that step earlier in the pipeline.
| The core promise | Synthetic data preserves the statistical relationships of real records without any single synthetic record mapping back to a real person, in theory allowing indefinite reuse. |
|---|---|
| The problem vendors don't mention | Generative models sometimes memorize and reproduce actual training records nearly verbatim; researchers have extracted real data from a model marketed as privacy-safe in under 30 minutes. |
| Structural leakage | Even without direct identifiers, a unique combination of attributes in a synthetic record can let someone with outside data re-identify a real individual. |
| Membership inference | An attacker doesn't need your exact record, just confirmation someone with a rare profile was present in the original training data, which can itself be a privacy violation. |
| Fidelity vs. privacy tradeoff | Synthetic data close enough to the original to be useful tends to leak; data private enough to be safe loses the realism that made it valuable, so no setting maximizes both at once. |
| When it's worth it | Best suited to recurring, long-lived reuse cases like ongoing model training; probably not worth it for high-stakes accuracy contexts like diagnostics or risk scoring. |
| The real requirement | Synthetic data must be tested for memorization, structural leakage, and membership-inference risk before external use, treating privacy as engineering practice, not a marketing claim. |
Does synthetic data automatically satisfy privacy or anonymization requirements?
No. Whether it does depends on how it was generated and tested; the article stresses that synthetic is a marketing claim, not an automatic guarantee of privacy, and testing for leakage is required.
How can a generative model leak real data if it's producing synthetic records?
Generative models sometimes memorize and reproduce actual training records nearly verbatim, and researchers have extracted real data from a model marketed as privacy-safe in under 30 minutes.
What is membership inference, and why does it matter?
It's confirming that a person matching a certain profile was present in the original training dataset, without needing to recover their exact record; in some contexts, that confirmation alone is a privacy violation.
When is synthetic data actually worth the investment?
In recurring reuse cases where the same dataset will be used for years, like an ongoing model-training population; it's probably not worth it for high-stakes accuracy contexts like diagnostic or risk-scoring models.
Does synthetic data generation eliminate the need for data cleaning?
No, garbage in still means garbage out; synthetic generation just moves the data-cleaning step earlier in the pipeline rather than removing it.
- So Which Privacy-Enhancing Technology Should You Use?No single PET wins. A practical guide to picking the right one for your team.
- Federated Learning: Great Idea and Expensive RealityMove the model, not the data. Great for privacy, expensive for infrastructure.
- A GDPR-Compliant Vendor Doesn't Automatically Mean Your Data Transfer Is LegalYour vendor says they're GDPR compliant. That doesn't mean your data transfer to the US is legal—those are two separate questions.