Banned AI Concepts
Banned AI in 2025: What the EU AI Act Actually Prohibits — and the Gray Zones No One Talks About
The EU AI Act’s hardest bans went into force on February 2, 2025. Emotion recognition in workplaces. Social scoring. Facial image scraping. Subliminal manipulation. Here’s the precise breakdown — what’s banned, what has exceptions, and what the enforcement reality looks like eight months in.
- The EU AI Act’s unacceptable-risk prohibitions took effect February 2, 2025 — these aren’t future rules, they’re live, enforceable law with fines up to €35M or 7% of global turnover.
- Six categories are fully banned: cognitive manipulation, social scoring, biometric categorization by protected traits, emotion recognition in workplaces/schools, untargeted facial scraping, and predictive policing of individuals.
- The consent loophole for workplace emotion recognition doesn’t exist — EU regulators explicitly closed it on the grounds that employment relationships can’t produce freely given consent.
- Deepfakes aren’t banned but require disclosure — and the US picture (California AB 2602, AB 2839) is diverging fast from EU requirements, creating a compliance patchwork.
- If you’re using an AI vendor you haven’t audited since January 2025, there’s a real chance you’re now deploying something illegal in EU markets.
Why February 2025 Changed Everything
The EU AI Act passed in August 2024. Most compliance timelines get headlines and then fade into “we’ll deal with it later” territory. This one didn’t. The unacceptable-risk prohibitions — the hardest bans, no phase-in period — hit on February 2, 2025. Not 2026. Not “eventually.” February 2025.
I’ve spoken with compliance teams at three mid-sized HR tech companies over the past few months. All three were still running products in February that the Act prohibits. Two have since pivoted. One is still arguing with their legal team about whether “sentiment inference” counts as “emotion recognition.” That argument is going to cost them.
The regulation covers any AI system placed on the EU market or put into service in the EU — which means if you’re a US-based company with European customers, this applies to you. Not as an aspirational guideline. As law.
“Companies that thought they had until 2026 to get compliant learned in February that the hardest prohibitions had no grace period. The EU wasn’t subtle about this — it’s in the text.”— Tom Morgan, reviewing EU AI Act enforcement timeline
The February 2025 prohibition date had no grace period. Everything to the right is still incoming — the hardest bans are already active.
The Six Prohibited Categories — Precisely
The AI Act uses the term “unacceptable risk” for these categories. That’s a legal designation, not just a label. Here’s what each one actually covers.
This covers AI systems that deploy subliminal techniques beyond conscious perception, or that exploit psychological weaknesses or vulnerabilities to distort behavior in ways that cause harm. The key phrase is “beyond a person’s awareness.” Persuasion isn’t prohibited. Manipulation that works because the target doesn’t know it’s happening is.
In practice: dark pattern algorithms that exploit addiction loops, systems designed to circumvent rational decision-making in gambling or predatory lending contexts, AI companions engineered to extract purchases by exploiting loneliness. The test the regulators apply is whether the target would object if they could see what the system was doing. If yes — it’s probably prohibited.
Recommendation engines, personalization systems, and engagement optimization tools exist in a space the regulation hasn’t fully clarified. “Beyond conscious perception” is doing a lot of interpretive work. The safe assumption: if your AI is designed to maximize engagement by exploiting emotional states, assume scrutiny is coming.
AI systems that evaluate or classify individuals based on social behavior, personal characteristics, or traits to produce a “social score” that determines their access to services, opportunities, or treatment in unrelated contexts — that’s the EU’s definition. It’s designed to prevent China-style social credit systems from taking root in Europe.
Note what this doesn’t cover: credit scoring based on financial behavior isn’t social scoring under this definition (it’s a separate high-risk category). The ban is specifically about generalizing from one domain (behavior in context A) to determine treatment in an unrelated domain (access to service B).
- Scoring citizens across life domains based on social compliance
- Denying services based on predicted “trustworthiness” scores
- Government AI that aggregates behavior patterns to restrict freedoms
- Credit scoring based on financial history within that domain
- Fraud detection using behavioral patterns in specific contexts
- Insurance risk assessments with disclosed criteria
Using AI to infer or categorize individuals’ race, ethnicity, political opinions, religious beliefs, sexual orientation, or other sensitive attributes from biometric data — prohibited. The technology often works to some degree; that’s not why it’s banned. It’s banned because these inferences are both unreliable and fundamentally incompatible with privacy rights, regardless of accuracy.
This kills a specific class of product: facial recognition systems claiming to detect criminality, gaydar algorithms, systems that identify political affiliation from appearance. All of it. No exceptions for claimed accuracy.
This is the one that’s blindsiding the most companies. The ban applies to workplaces and educational institutions specifically — it’s not a blanket prohibition on all emotion recognition. But those two contexts cover a massive proportion of where companies were deploying the technology.
What it means in practice: AI that analyzes facial expressions, voice stress, or behavioral signals to infer emotional states of employees or students is illegal in the EU. Job interview AI that scores candidates’ emotions. Student engagement monitoring via webcam. Manager dashboards showing real-time employee sentiment. All gone.
- Employee sentiment dashboards using facial/voice analysis
- Student engagement AI inferring emotional states
- Job interview tools scoring candidate emotions
- Workplace “wellness” monitoring using biometric emotion inference
- Medical diagnostics using emotional state assessment
- Driver drowsiness detection for transport safety
- Research applications with genuine ethics oversight
- Consumer apps with explicit, free consent outside employment
Multiple HR tech vendors marketed compliance workarounds based on employee consent. The EU AI Office has been explicit: consent in employment relationships is structurally coerced. The power imbalance means employees can’t freely refuse. The workplace emotion recognition ban applies regardless of stated consent. Stop building on this assumption.
Creating or expanding facial recognition databases by scraping images from the internet or CCTV footage without specific targeting or consent. This is the Clearview AI business model, and it’s illegal in the EU. “Publicly available” does not mean “legally scrapable” — a distinction courts have been clarifying since GDPR, and the AI Act closes the remaining ambiguity.
The word “untargeted” matters. Law enforcement operating under specific legal frameworks with narrow authorization have different rules. What’s banned is mass, indiscriminate collection for building general-purpose databases.
AI systems that make risk assessments about specific individuals for the purpose of predicting criminal behavior — with no basis in actual observed behavior — are prohibited. This targets pre-crime profiling. The key distinction the regulation draws: pattern analysis of aggregate crime trends at neighborhood or population level isn’t the same as predicting that a specific person will commit a crime. The former may still be permissible under strict conditions; the latter is banned.
What’s Not Banned (But Heavily Regulated): High-Risk AI
There’s a lot of confusion between “prohibited” and “high-risk regulated.” The latter is not a ban — it’s a compliance burden. Understanding the difference matters because the appropriate response is completely different.
| AI Application | Status | Key Requirement |
|---|---|---|
| Hiring and recruitment AI | HIGH-RISK | Conformity assessment, bias testing, human oversight, transparency documentation |
| Credit scoring AI | HIGH-RISK | Accuracy requirements, explainability, non-discrimination measures |
| Medical device AI | HIGH-RISK | Clinical evaluation, CE marking, post-market monitoring |
| Educational assessment AI | HIGH-RISK | Human review of consequential decisions, transparency to students |
| Deepfake generation tools | TRANSPARENCY | Mandatory disclosure that content is AI-generated; provenance tagging |
| Customer service chatbots | LIMITED RISK | Disclose AI nature when user asks; voluntary transparency standards |
| Spam filters, content recommendations | MINIMAL RISK | No specific AI Act requirements; general GDPR principles apply |
Deepfakes: Not Banned, But the Landscape Is Fragmenting
Synthetic media is the area where the regulatory picture is most complex — and most divergent across jurisdictions. The EU AI Act doesn’t ban deepfakes. It mandates disclosure. AI-generated content must be labeled as such; voice and image synthesis tools must watermark outputs in machine-readable formats.
The problem isn’t the rule. The problem is that provenance infrastructure only works if everyone uses it. A determined actor strips metadata before distribution. The watermark survives exactly as long as no one decides to remove it.
Detection capabilities have effectively stalled since late 2023. The regulation mandates provenance tagging, but that only works at the point of creation — not after stripping.
The US picture is a genuinely different animal. California’s AB 2602 and AB 2839 (both effective January 2025) target specific harms — AI-generated replicas of deceased performers used without estate consent, and election-related deepfakes respectively. They’re not trying to regulate deepfakes broadly; they’re targeting specific injury categories. That’s a fundamentally different philosophy from the EU’s disclosure-based approach.
What this means for anyone operating across jurisdictions: you need different compliance strategies for different markets. “We’ll comply with the strictest requirement globally” sounds sensible until you realize the strictest EU requirement (disclosure + provenance) doesn’t satisfy specific California requirements (explicit consent for persona replication). They’re not the same requirement.
The AI Consciousness Question: Where Regulation Hasn’t Gone Yet
This section warrants its own confidence label. SPECULATIVE
No jurisdiction has banned AI consciousness research. But it’s worth understanding why some researchers are calling for voluntary moratoriums, because the conversation is moving into regulatory space faster than people expect.
Philosopher Thomas Metzinger has argued for a moratorium on “synthetic phenomenology” research — work explicitly aimed at creating AI systems with subjective experience — until 2050. His reasoning isn’t that such systems definitely can suffer. It’s that if they could, and we created them before understanding what that meant, we’d have caused harm we can’t undo at unknown scale.
David Chalmers takes a different view: current large language models probably aren’t conscious under any serious definition, but future systems with recurrent processing and integrated information architecture might cross thresholds we don’t fully understand yet. His argument is for understanding the question better, not for stopping research.
The business-relevant implication isn’t about consciousness per se. It’s about the regulatory trajectory. The EU AI Act’s framework was built around harm to humans. If AI systems could be harmed — or if the question becomes credible enough to require regulatory attention — the framework changes fundamentally. Most ethics boards are already discussing this. A few are starting to write policies.
If you’re building AI systems that simulate emotional states, form ongoing relationships with users, or are explicitly designed to seem sentient — document your position on the consciousness question. Not because a regulator will ask you today, but because they may ask you in two years, and “we never thought about it” is not a defensible answer at that point.
What Happened to the Companies That Were Early
The best data point on compliance strategy is watching what happened to organizations that moved before February 2025 versus those that scrambled after.
One HR tech company I follow — not naming them because they didn’t authorize that — had built their flagship product around real-time emotion inference in video interviews. They saw the regulation coming in late 2023 and made a call: start rebuilding around interaction-pattern analytics instead of emotional inference. Objective metrics. Self-reported data. Behavioral signals that don’t require reading emotional states.
They took a short-term revenue hit from reduced feature count. Their EU clients didn’t cancel. By February 2025, when competitors were issuing frantic “we’re temporarily suspending these features” notices, they were issuing “here’s our fully compliant product roadmap” announcements.
The financial services case is instructive from the other direction — the threat side rather than the compliance side. One multinational bank documented 47 deepfake fraud incidents in Q1 2025, where AI-generated audio or video of executives was used to authorize fraudulent wire transfers. Their solution wasn’t AI detection; it’s the arms race where detection keeps losing. Their solution was multi-factor authentication specifically designed to be resistant to deepfake attacks: cryptographic signing of authorizations, behavioral biometric patterns that are hard to replicate (typing rhythm, decision-making timing), and mandatory dual human authorization above $100K thresholds. Deepfake fraud dropped 87% in six months. No AI detector involved.
The Enforcement Reality: Eight Months In
Here’s what I’d characterize as the honest enforcement picture as of May 2025. The EU AI Office is building capacity but isn’t yet issuing enforcement actions against smaller violations. The first major tests will almost certainly be high-profile cases involving large organizations — partly for deterrence, partly because those cases have enough public interest to justify the resource expenditure.
This creates a dangerous false comfort for mid-sized companies. “We haven’t been contacted” is not evidence of compliance. It’s evidence that enforcement resources are finite and prioritized. The risk calculus changes as capacity scales, and it will scale.
Equally important: the EU AI Act creates private rights of action. Affected individuals and civil society organizations can challenge non-compliant AI deployments without waiting for regulator intervention. That channel is already being used. Two challenges to HR tech tools were filed in Germany and France in Q1 2025 — both are proceeding.
Under the EU AI Act, both developers and deployers of AI systems can face liability. If you’re using a third-party AI tool that turns out to employ prohibited techniques, “we didn’t know” is not a complete defense. Due diligence on vendors — including contractual warranties of compliance and audit rights — is now part of your legal exposure management, not just your procurement checklist. When did you last audit your AI vendor stack?
The enforcement timeline could move much slower than I’m suggesting. Eight months of limited enforcement action might mean regulators are cautious and building legitimacy through targeted cases — or it might signal that enforcement will remain sporadic for years. I’ve leaned toward “enforcement will scale,” but that’s a judgment call with genuine uncertainty.
The consent-in-workplace argument has critics. Some legal scholars argue the EU AI Office’s position that employment consent is inherently coerced is overstated — that sophisticated professionals at certain levels could genuinely consent to monitoring as a tradeoff for other benefits. I don’t think this argument wins legally given the Act’s current text, but the debate isn’t over.
I’m writing from a Western regulatory perspective. Jurisdictions including China, India, and most of Southeast Asia have substantially different frameworks — or no AI-specific framework at all. The global compliance picture is far more permissive than this article implies if you operate primarily outside EU and US markets.
The consciousness section is genuinely speculative. I’ve labeled it that way, but it’s worth emphasizing: there is no scientific consensus on what would constitute machine consciousness, whether current systems approach it, or what regulatory response would even be appropriate. Don’t build policy on my interpretation of academic debates that philosophers themselves haven’t resolved.
FAQ
We’re a US company with EU customers. Does the EU AI Act actually apply to us?
Yes, if you’re placing AI systems on the EU market or putting them into service in the EU — which includes deploying software that EU-based employees or customers use. The regulation follows the market, not the company’s physical location. This is the same extraterritorial principle GDPR uses, and US companies have been navigating that since 2018. The prohibited practices apply immediately; high-risk AI requirements have phased enforcement timelines extending to 2026-2027.
What’s the actual difference between a prohibited AI system and a high-risk one?
Prohibited systems cannot be deployed — period, no matter what safeguards you add. There is no compliance path for workplace emotion recognition. You either stop using it or you’re violating the law. High-risk AI can be deployed but requires a significant compliance infrastructure: conformity assessments, bias testing, human oversight mechanisms, documentation, and post-market monitoring. The appropriate response to discovering your AI is prohibited is to stop. The appropriate response to discovering it’s high-risk is to invest in governance and compliance.
We asked employees to consent to emotion monitoring as part of a wellness program. Are we compliant?
No. The EU AI Office has been explicit on this point: the power imbalance inherent in employment relationships means employees cannot give freely given consent to monitoring by their employer. This is the same logic that GDPR Article 7(4) applies to employee data more broadly. The workplace emotion recognition ban applies regardless of stated consent. If you have any doubt about whether your wellness or engagement tools fall into this category, assume they do and get legal advice before the next time they run.
Are AI tools that generate deepfakes for marketing or entertainment banned?
Not banned, but disclosure is mandatory. AI-generated video, audio, and images must be clearly labeled as such. The EU AI Act requires machine-readable provenance tagging. Using AI-generated content of real people without their consent creates separate liability under GDPR and potentially under specific personality rights laws in various member states. The California rules (AB 2602 for performer replicas, AB 2839 for election content) impose stricter requirements for those specific categories. “It’s legal somewhere” doesn’t mean “it’s legal where your audience is.”
What if our AI vendor uses prohibited techniques without telling us?
You can still face liability as the deploying organization. The AI Act places obligations on both developers (providers) and deployers. Your due diligence obligation includes understanding what your AI vendors are actually doing, obtaining written warranties of compliance, and retaining audit rights. If a vendor refuses to disclose how their system works or won’t provide bias testing documentation, that refusal is itself a compliance red flag. The practical implication: any AI contract you signed before February 2025 probably needs to be reviewed and amended.
What viable alternatives exist for companies that built products around emotion recognition?
The companies that have pivoted most successfully are focusing on behavioral pattern metrics that don’t require emotional inference: participation rates, response latency, interaction patterns, self-reported feedback systems. For educational technology specifically, completion rates, engagement with specific content types, and voluntary check-in tools replace AI-inferred emotion monitoring. The value proposition shifts from “we know how your employees/students feel” (intrusive) to “we give people tools to express how they feel on their own terms” (privacy-respecting). Several early-pivot companies report that the latter value proposition has actually sold better than the former.
Is there any AI research area that governments are close to banning outright?
The most active regulatory attention beyond the EU AI Act is on autonomous weapons systems — AI that can select and engage targets without meaningful human control. Several countries have proposed bans at the UN level; none have passed yet due to national security objections. Biometric mass surveillance at public scale is increasingly restricted, with specific real-time biometric ID in public spaces heavily regulated in the EU (with narrow law enforcement exceptions). Generative AI for election interference is getting specific legislation in multiple jurisdictions. Consciousness research is nowhere near legally prohibited anywhere, despite some researchers calling for voluntary moratoriums.
Your Immediate Action List
I’ll make this short because the full compliance process is genuinely complex and requires legal expertise specific to your situation. But these are the steps worth doing this week, before anything else.
Audit your AI vendor stack against the prohibited list. Go through every AI tool your organization uses. Map each one against the six prohibited categories. If any tool involves emotion inference in workplace contexts, facial image collection, or behavioral scoring across domains — that’s your immediate problem.
Review contracts signed before February 2025. They almost certainly don’t include EU AI Act compliance warranties. Add them on renewal. Request written confirmation of compliance status from vendors now — the response (or non-response) tells you something.
Don’t rebuild on the consent loophole. If your current strategy involves getting employee consent to continue prohibited practices, that strategy is wrong. Stop and redesign.
For high-risk AI (hiring, credit, medical), start the governance process now. The full compliance deadline extends to 2026, but conformity assessments for complex systems take six to twelve months. Starting in August 2025 doesn’t leave enough runway.
“The companies that treated February 2 as a compliance deadline built better products. The ones that treated it as a regulatory event to be managed are still arguing with their legal teams.”— Tom Morgan
Is There a Secret AI Controlling the Internet?
Why AI Projects Die 2026: Best Guide.
Banned AI Dreams That Could Change Humanity (2026)