Is Your Data Safe with an AI Coach? Privacy, Ethics and the Questions to Ask

After three months with an AI coach, it holds a record almost nobody else has: the doubts you had before accepting that promotion, the tension with your co-founder you have never named in public, how you actually felt walking out of the board meeting. Add a self-assessment profile and a list of goals, and the picture is more candid than anything in your HR file.

So the question deserves a straight answer, not a reassuring paragraph buried in a FAQ: where does that data go, who can read it, and what happens to it when you leave?

We build an AI coach ourselves, so we think about this every day — and we notice that almost nobody writes about it. Comparison articles rank features and pricing; very few ask what happens to some of the most sensitive professional data you will ever type. This article covers what an AI coach actually knows about you, the questions that matter more than any privacy badge, the ethical framework emerging from coaching research, and a checklist of eight questions to put to any provider — including us.

What does an AI coach actually know about you?

Think of it as four layers, roughly in order of sensitivity.

Account data. Name, email, employer if the company pays. The boring layer, and the least revealing.

Assessment data. Most serious AI coaches start with some form of self-assessment. In our case, the Octagon maps your leadership across eight pillars. By design, this layer says where you are strong and where you are not.

Conversation logs. The full text of every session: hesitations, half-formed complaints, the names of the colleagues involved, the version of events you would never put in an email.

Derived data. The inferences the system builds on top: recurring patterns, how you respond under pressure, which themes you avoid, how your goals have drifted.

Here is the uncomfortable part: that fourth layer is exactly what makes an AI coach useful. Graßmann and Schermuly (2021), in their review of AI coaching capabilities, are clear that the value of these systems rests on collecting and analyzing coachee data over time — memory across sessions and pattern recognition are the product, not a by-product. If you are new to how these systems work, we unpack the mechanics in what an AI coach actually is.

Which means "just share less" is bad advice. A coach you censor yourself with is barely coaching you. The goal is not to starve the system of data; it is to control what happens to the data you give it.

Is your data used to train the model?

Most AI coaches are built on top of large language models from a handful of providers. That creates two separate questions, and vendors often answer only the easier one:

1. Does the coaching provider itself train or fine-tune models on your transcripts?
2. Do its subprocessors — the LLM provider, the analytics stack — use your data for training?

Commercial API agreements from the major model providers generally allow businesses to exclude their data from training. So "our model provider trains on everything" is a choice, not a fatality. Ask for the answer to both questions in writing. A provider who cannot answer in two sentences either does not know or does not want to say — and both of those are answers.

Can your employer read your sessions?

This is the hardest question, because in corporate deployments the employer pays.

Human coaching settled it decades ago: the coach may discuss themes with the sponsor — "we are working on delegation" — but never hands over transcripts. That boundary survives because reading a coach's notes at scale was never practical. AI quietly removes the friction: it is technically trivial to build sentiment dashboards, per-person "engagement" scores, or keyword alerts on top of coaching logs.

Terblanche (2024) names this tension directly. Organizations fund AI coaching to improve performance, but coaching only works if the coachee keeps genuine autonomy over the process. His position, which we share: participation should be voluntary — opt-in, not auto-enrolled — and the monitoring temptation is precisely where the ethics of AI coaching break first. What an organization gains in dashboard visibility, it loses in the one thing that makes coaching work.

The research on psychological safety explains why this is more than principle. Edmondson (1999) showed that people only take the interpersonal risks that produce learning when they believe those risks will not be used against them. Nobody examines their real weaknesses inside a tool they suspect their manager reads. A monitored coach is not a coach; it is a compliance system with a friendly tone.

The defensible standard is simple: individual sessions inaccessible to the employer, full stop. Aggregate, anonymized adoption data at most.

Can you actually delete everything?

"Delete my account" can mean at least three different things: deactivation (the data stays), a deletion flag (the data stays for a while), or true erasure across production systems, backups and subprocessors.

Under GDPR, Article 17 gives you a right to erasure and Article 20 a right to take your data with you in a usable format — which is why export should come before deletion, not instead of it. The questions worth asking: what exactly does deletion cover (conversations, profile, derived data)? How long do backups persist after deletion? Are subprocessors contractually required to delete too? And what happens to your data after long inactivity — silent retention forever is a policy, just an unstated one.

Is GDPR compliance enough?

No — it is a floor. GDPR gives European users real leverage: access, portability, erasure, purpose limitation, transparency about automated processing. But it does not by itself forbid employer reporting or training use. Both can be lawful with the right clauses, consented to in terms nobody reads. "GDPR-compliant" on a landing page is a starting point, not a verdict. The checklist below asks what the badge does not.

Does privacy actually change coaching outcomes?

This would matter less if privacy were mere hygiene. It is not — it is a performance variable.

The evidence that AI coaching works is real but specific. Terblanche and colleagues (2022) ran two longitudinal randomized controlled trials and found that an AI coach matched human coaches on goal attainment, with both beating the control group — we review the full evidence in does AI coaching actually work. A 2024 study in Frontiers in Psychology found that a working alliance — the collaborative bond that predicts coaching outcomes — can begin forming with an AI coach within a single session.

That alliance runs on disclosure. One documented advantage of AI coaches is that many people disclose more freely to a machine: no social judgment, no image to manage, no fear of disappointing someone — we compare the trade-offs in AI coach vs human coach. That advantage evaporates the moment users suspect a human might read the log. Weak privacy does not just create legal risk; it quietly degrades the coaching itself.

The eight questions to ask any AI coaching provider

1. Are my conversations used to train models — by you or by your subprocessors? Get the answer in writing, covering both levels.
2. Who at my company can see my data, and at what granularity? The only good answer for individuals: nobody. Aggregate and anonymized at most.
3. Is participation opt-in? An enrolled user is not a committed one — and forced coaching rarely coaches.
4. Can I export all my data in a usable format? Your reflections are yours; you should be able to walk away with them.
5. What exactly does deletion cover, and how long until backups are purged? "We delete your account" is not the same as "we erase your data."
6. Which third parties receive my data, and under what terms? LLM providers, analytics, hosting — the list should be public.
7. How long do you retain my data after I stop using the service? Indefinite silent retention is a red flag.
8. What happens to my data if you are acquired or shut down? Data is an asset in an acquisition — unless the terms say otherwise.

A provider who answers all eight without flinching is a provider taking the problem seriously.

Where we stand

It is fair to turn the checklist on us. On unfollowtheleader, you can export your data, and you can delete your account and everything attached to it — your conversations with Lumia, your Octagon profile, your history — completely. We will not wave certifications at you here; we would rather you judge us on the same eight questions as everyone else. And since what you share shapes what you get back, it is worth deciding deliberately — some thoughts on that in how to use an AI coach well.

Stop reacting. Start seeing — it applies to your data too. Read the answers before typing your doubts into any system, ours included. And once the answers hold, give the process what it needs to work: practice it with Lumia, your AI coach.