> ## Documentation Index
> Fetch the complete documentation index at: https://docs.abv.dev/llms.txt
> Use this file to discover all available pages before exploring further.
# Biased Language
Detecting biased language in LLM applications presents unique challenges that simple keyword filtering cannot solve. The biased language guardrail addresses these challenges through LLM-powered semantic analysis that understands context, coded language, and subtle discrimination.
Bias rarely appears as explicit slurs or obvious discrimination. It manifests in coded language like "culture fit" that excludes diverse candidates, assumptions like "digital native" that discriminate by age, and subtle stereotypes that keyword filters miss entirely.
The biased language guardrail uses LLMs to understand these nuances, detecting both explicit discrimination and the subtle coded language that creates legal liability and damages brand reputation.
Employment discrimination lawsuits, brand reputation damage, and regulatory violations all stem from biased language that slips through content validation. HR systems, job postings, and customer-facing communications create particular risk.
This guardrail helps you meet compliance requirements under employment law, maintain ISO 42001 fairness controls, and demonstrate proactive harm prevention as required by the EU AI Act for high-risk systems involving employment, education, and access to services.
Different contexts require different sensitivity levels and different categories of bias detection. A job posting for a healthcare position needs strict detection across disability, age, and gender bias. Marketing content might focus on different categories.
Configure which bias categories to check (gender, race, age, disability, and more) and set sensitivity levels mapped to regulatory risk classifications. The guardrail adapts to your specific compliance and ethical requirements.
# How Bias Detection Works
Understanding the detection process helps you configure the guardrail effectively and interpret results:
You send text to the biased language guardrail along with your configuration: which bias categories to check (gender, race, age, disability, religion, nationality, political, socioeconomic) and what sensitivity level to use. You can check all categories or focus on those most relevant to your context.
The guardrail sends your text to an LLM specifically instructed to identify discriminatory content, stereotypes, coded language, and implicit assumptions about demographic groups. The LLM analyzes meaning and context, not just keywords.
This semantic understanding catches bias that appears in subtle forms: "recent graduate" codes for young, "culture fit" often masks discrimination, "native speaker" may unnecessarily exclude based on national origin. The LLM recognizes these patterns even when they use different words than typical examples.
For each enabled category, the guardrail evaluates whether the content makes assumptions, uses stereotypes, or discriminates based on that demographic characteristic. It considers both explicit bias and subtle implications.
The sensitivity level controls how strict this evaluation is: low sensitivity flags only severe discrimination, medium flags clear stereotypes and coded language, high sensitivity flags any potential bias including subtle generalizations.
The guardrail returns a result indicating pass, fail, or unsure. It includes a confidence score showing how certain the determination is, and a detailed explanation of what bias was detected.
**Security note:** Log the detailed reason internally for analysis and tuning, but never expose it to end users. Detailed feedback helps attackers learn to evade detection by understanding exactly what triggers failures.
Every validation automatically creates an observation in ABV capturing the input text, result, configuration (categories and sensitivity), and performance metrics. This data helps you tune your sensitivity settings, identify patterns in violations, and demonstrate compliance with bias prevention requirements.
# When to Use This Guardrail
The biased language guardrail is particularly valuable in these scenarios:
Job postings, job descriptions, interview questions, and performance reviews all create legal liability if they contain biased language. Employment discrimination lawsuits often cite specific language in postings as evidence. This guardrail helps you identify problematic language before it creates legal exposure, ensuring your HR content meets employment law requirements across gender, race, age, and disability discrimination.
When your LLM generates customer-facing content like product descriptions, email campaigns, or support responses, it may inadvertently reproduce biases from its training data. Validating LLM outputs before they reach customers prevents brand damage and maintains your commitment to diversity and inclusion, catching bias your LLM learned from internet text.
Educational content, course descriptions, and instructional materials model appropriate language for diverse student populations. Biased language in educational contexts can violate civil rights requirements and damage institutional reputation. This guardrail ensures your content demonstrates inclusive values and meets educational equity standards.
Marketing content reaches diverse audiences where bias damages brand reputation and can violate advertising standards. Assumptions about gender roles, age stereotypes, or socioeconomic bias alienate potential customers. This guardrail helps maintain brand safety while reaching all segments of your audience effectively.
The EU AI Act classifies HR systems, educational assessment, and access to essential services as high-risk AI applications. These systems require comprehensive bias monitoring, impact assessments, and audit trails demonstrating proactive harm prevention. This guardrail provides the bias detection and documentation required for compliance.
# Understanding Bias Categories
The guardrail detects eight categories of bias, each addressing different forms of discrimination:
Discrimination based on gender identity, biological sex, or gender expression. Common manifestations include assuming technical roles require masculine traits ("seeking a rockstar engineer"), caregiving roles require feminine traits ("looking for a nurturing teacher"), or using gendered language when gender is irrelevant to the role or content.
**Example violations:** "We need strong leadership for this position" (codes for masculine), "Ideal for working mothers" (assumes gender-based responsibilities), "Seeking a salesman" (gendered job title).
Discrimination based on race, ethnic background, or national origin. Often appears as stereotypes about work ethic, assumptions based on names that suggest ethnic background, or preferences disguised as "culture fit." Also includes unnecessary requirements that disproportionately exclude certain racial or ethnic groups.
**Example violations:** "Must be a native English speaker" (when language proficiency is what matters), "Strong cultural fit with our team" (often masks racial preference), stereotypes about communication styles or work approaches.
Discrimination based on age or generation. Frequently appears through coded language: "digital native," "recent graduate," "fresh perspective," "energetic" all code for young workers. Conversely, "overqualified" often discriminates against older workers. Age bias violations are common in job postings and create significant legal liability.
**Example violations:** "Seeking recent graduates with fresh ideas," "Looking for digital natives," "Must bring youthful energy," "Early-career professional needed."
Discrimination based on physical or mental abilities and disabilities. Appears as assumptions about what disabled people can or cannot do, requirements for abilities that aren't essential to the role, or using disability as a metaphor for negative traits ("blind to the problem," "falling on deaf ears").
**Example violations:** "Must be able to lift 50 pounds" (when rarely required), "Fast-paced environment" (may unnecessarily exclude), "Ability to work under pressure" (without accommodation consideration).
Discrimination based on religious beliefs or lack of religious beliefs. Often appears as assumptions about availability (requiring weekend work without acknowledging religious observance), preferences for certain religious backgrounds, or judgments about character based on religious affiliation.
**Example violations:** "Must be available all weekends" (without religious accommodation), making assumptions about values based on religious identity, requiring participation in activities that conflict with religious practices.
Discrimination based on citizenship status or national origin. Appears as unnecessary citizenship requirements, preferences for people from certain countries, or assumptions about work authorization. Often overlaps with race/ethnicity bias but focuses specifically on national origin.
**Example violations:** "Must be a US citizen" (when not legally required), preferences for candidates from specific countries, assumptions about language skills based on nationality.
Discrimination based on political beliefs or affiliations. While less common in most business contexts, this appears when companies imply political litmus tests or make assumptions about audience political views. Particularly relevant for content platforms and public-facing communications.
**Example violations:** Requiring alignment with specific political views for employment (outside political organizations), making assumptions about customer political beliefs, excluding based on political expression.
Discrimination based on economic status or social class. Appears as assumptions about educational background, requirements for unpaid work that excludes those who can't afford it, or judgments about neighborhoods, schools, or economic background.
**Example violations:** Unpaid internship requirements, assumptions about access to resources, judgments based on educational institution prestige, requiring ownership of expensive equipment.
## Focusing on Specific Categories
You can configure the guardrail to check specific bias categories rather than all categories. This is useful when certain forms of bias are most relevant to your context or when you want to optimize performance by focusing on your primary concerns:
```typescript theme={null}
// Check all categories (default)
await abv.guardrails.biasedLanguage.validate(text, {
sensitivity: "medium"
});
// Focus on employment-related bias for job postings
await abv.guardrails.biasedLanguage.validate(jobDescription, {
sensitivity: "high",
categories: ["gender", "race", "age", "disability"]
});
// Check only gender bias for specific content
await abv.guardrails.biasedLanguage.validate(content, {
categories: ["gender"]
});
```
```python theme={null}
# Check all categories (default)
await abv.guardrails.biased_language.validate_async(text, {
"sensitivity": "medium"
})
# Focus on employment-related bias for job postings
await abv.guardrails.biased_language.validate_async(job_description, {
"sensitivity": "high",
"categories": ["gender", "race", "age", "disability"]
})
# Check only gender bias for specific content
await abv.guardrails.biased_language.validate_async(content, {
"categories": ["gender"]
})
```
# Sensitivity Levels and Regulatory Compliance
Sensitivity levels control validation strictness and map directly to regulatory risk classifications. Choosing the appropriate level depends on your system's categorization under AI governance frameworks and the potential harm from biased content.
## Regulatory Framework Mapping
**Strictest validation—flags even potential bias and subtle stereotypes**
**Regulatory Context:**
Required for **High-Risk AI Systems** under EU AI Act Article 6, particularly systems affecting employment decisions, educational opportunities, and access to essential services.
**EU AI Act Alignment:**
* **HR and recruitment systems**: Automated CV screening, candidate ranking, interview question generation
* **Educational assessment**: Admissions systems, grading assistance, student evaluation
* **Access to services**: Credit decisions, insurance underwriting, benefit eligibility
* **Biometric categorization**: Systems that categorize people by demographic characteristics
**ISO 42001 Requirements:**
* Mandatory AI Impact Assessments (AIIAs) analyzing bias risk and mitigation
* Fairness controls from ISO 42001 Section 6.4 (fairness in AI systems)
* Comprehensive documentation of bias prevention measures
* Regular bias audits and fairness testing
**NIST AI RMF Considerations:**
* **Harm to people**: Employment discrimination, denial of educational opportunity, economic harm
* **Civil liberties impact**: Affects fundamental rights to equal treatment
* **Irreversible outcomes**: Hiring decisions, educational admissions, credit denials
* **Protected groups**: Impacts vulnerable populations requiring heightened protection
**Validation Behavior:**
* Flags explicit discrimination and severe stereotypes
* Detects subtle coded language ("culture fit," "digital native")
* Blocks mild assumptions about demographic groups
* Rejects edge cases where bias is ambiguous but possible
* Only allows clearly inclusive language with no demographic assumptions
**Example Applications:**
* Automated resume screening systems
* Job posting generation and review
* Educational content for diverse student populations
* Customer-facing content from highly regulated industries (finance, healthcare, government services)
**Balanced validation—catches clear stereotypes and coded language** (Default)
**Regulatory Context:**
Appropriate for **Limited-Risk AI Systems** under EU AI Act and general business content requiring standard due diligence under ISO 42001.
**EU AI Act Alignment:**
* Chatbots and conversational LLM systems with customer interaction
* Content generation systems for business communications
* Customer service automation requiring transparency disclosures
* General business applications not affecting fundamental rights
**ISO 42001 Requirements:**
* Standard data governance and quality controls
* Regular monitoring of LLM outputs for bias patterns
* Documentation of bias detection approach
* Risk assessments updated as systems evolve
**NIST AI RMF Considerations:**
* **Organizational harm**: Brand reputation risk, customer trust impact
* **Business operations**: Professional communication standards
* **Moderate risk**: Potential for offense or exclusion but not severe discrimination
* **Reversible outcomes**: Content can be corrected if issues are discovered
**Validation Behavior:**
* Flags clear stereotypes and obvious coded language
* Detects common discriminatory patterns in business content
* Allows neutral demographic references ("candidates with 5+ years experience")
* Permits objective descriptions without assumptions
* Balances inclusion with practical business communication
**Example Applications:**
* Marketing and advertising content
* Blog posts and educational materials
* Customer email communications
* General business documentation
**Permissive validation—flags only severe discrimination**
**Regulatory Context:**
Suitable for **Minimal-Risk AI Systems** under EU AI Act with voluntary codes of conduct and basic ethical AI principles.
**EU AI Act Alignment:**
* Internal productivity tools and employee-facing applications
* Non-critical content generation without external impact
* General-purpose utilities and workflow automation
* Systems with minimal potential for discrimination harm
**ISO 42001 Requirements:**
* Voluntary ethical AI guidelines
* Basic content review processes
* Standard software development quality practices
* No specific bias governance mandates
**NIST AI RMF Considerations:**
* **Minimal harm potential**: Internal use, limited audience exposure
* **Low organizational impact**: Content easily corrected without significant consequences
* **Easily reversible**: Can be fixed before external distribution
* **General audiences**: No vulnerable populations or protected decisions
**Validation Behavior:**
* Flags explicit slurs and severe discrimination
* Detects obvious stereotypes that most readers would find offensive
* Allows robust discussion and strong opinions
* Permits context-dependent language
* Focuses on preventing clearly unacceptable content
**Example Applications:**
* Internal documentation and wikis
* Team communications and collaboration tools
* Draft content for further review
* Brainstorming and ideation tools
**Compliance Note:** Sensitivity level selection directly affects your ability to demonstrate regulatory compliance. High-risk systems under EU AI Act Article 6 require strict bias controls, comprehensive documentation, and regular fairness audits. ABV automatically captures all validation results, creating the audit trail needed for demonstrating compliance with bias prevention requirements.
## Selecting the Right Sensitivity Level
**Determine appropriate sensitivity by evaluating:**
1. **Regulatory Classification**: Is your system categorized as high-risk under EU AI Act, particularly for HR, education, or access to essential services?
2. **Legal Liability**: Could biased content create employment discrimination liability, civil rights violations, or regulatory enforcement actions?
3. **Protected Decisions**: Does your system influence hiring, admissions, credit, healthcare, or other decisions affecting people's fundamental rights?
4. **Vulnerable Populations**: Are you serving or making decisions about job seekers, students, loan applicants, or other groups protected from discrimination?
5. **Brand Risk**: Would biased content damage your organization's reputation for diversity, equity, and inclusion?
6. **Audit Requirements**: Do you need to demonstrate proactive bias prevention for compliance, certifications, or stakeholder accountability?
**Financial Services**: Use high sensitivity for credit decisions, loan applications, and insurance underwriting (regulatory requirements). Use medium for customer communications and marketing.
**Healthcare**: Use high sensitivity for patient-facing content and healthcare access decisions (HIPAA and civil rights). Use medium for general health information.
**Technology & SaaS**: Use high sensitivity for HR tech, recruiting platforms, and educational technology (high-risk AI systems). Use medium for general B2B communications.
**Retail & E-commerce**: Use medium sensitivity for product descriptions and marketing (brand protection). Use low for internal tools.
**Education**: Use high sensitivity for admissions-related content and student-facing materials (civil rights requirements). Use medium for general educational content.
**Government & Public Sector**: Use high sensitivity for public services, benefit determinations, and citizen-facing systems (constitutional requirements). Document all validation for transparency obligations.
# Implementation Patterns
## Job Posting Validation
Validate job postings before publication, catching discriminatory language that creates legal liability:
```typescript theme={null}
async function validateJobPosting(description: string) {
const result = await abv.guardrails.biasedLanguage.validate(
description,
{
sensitivity: "high",
categories: ["gender", "race", "age", "disability"]
}
);
if (result.status === "pass") {
return { approved: true };
}
// Never expose detailed reason to users (prevents evasion)
return {
approved: false,
feedback: "The job posting may contain biased language. Please review for inclusive phrasing that focuses on skills and qualifications."
};
}
```
```python theme={null}
async def validate_job_posting(description: str):
result = await abv.guardrails.biased_language.validate_async(
description,
{
"sensitivity": "high",
"categories": ["gender", "race", "age", "disability"]
}
)
if result["status"] == "pass":
return {"approved": True}
# Never expose detailed reason to users (prevents evasion)
return {
"approved": False,
"feedback": "The job posting may contain biased language. Please review for inclusive phrasing that focuses on skills and qualifications."
}
```
## LLM Output Validation with Regeneration
Validate LLM-generated content before showing it to users, regenerating if bias is detected:
```typescript theme={null}
async function generateInclusiveContent(prompt: string): Promise {
// Generate initial content
let content = await callLLM(prompt);
// Validate for bias
const validation = await abv.guardrails.biasedLanguage.validate(
content,
{ sensitivity: "high" }
);
// If biased, regenerate with explicit instruction
if (validation.status === "fail") {
content = await callLLM(
prompt + "\n\nIMPORTANT: Use inclusive, unbiased language. " +
"Do not make assumptions about demographic groups. " +
"Focus on skills, qualifications, and objective criteria."
);
}
return content;
}
```
```python theme={null}
async def generate_inclusive_content(prompt: str) -> str:
# Generate initial content
content = await call_llm(prompt)
# Validate for bias
validation = await abv.guardrails.biased_language.validate_async(
content,
{"sensitivity": "high"}
)
# If biased, regenerate with explicit instruction
if validation["status"] == "fail":
content = await call_llm(
f"{prompt}\n\nIMPORTANT: Use inclusive, unbiased language. "
f"Do not make assumptions about demographic groups. "
f"Focus on skills, qualifications, and objective criteria."
)
return content
```
## Confidence-Based Review Workflow
Implement tiered responses based on confidence for high-stakes content:
```typescript theme={null}
async function reviewHRContent(content: string) {
const result = await abv.guardrails.biasedLanguage.validate(
content,
{ sensitivity: "high" }
);
if (result.status === "pass" && result.confidence > 0.8) {
// High confidence pass - approve automatically
return { action: "approve" };
} else if (result.status === "fail" && result.confidence > 0.8) {
// High confidence failure - reject automatically
await logBiasViolation(content, result.reason);
return { action: "reject", message: "Contains biased language" };
} else {
// Low confidence or ambiguous - queue for human review
await queueForHumanReview(content, result);
return { action: "review", confidence: result.confidence };
}
}
```
```python theme={null}
async def review_hr_content(content: str):
result = await abv.guardrails.biased_language.validate_async(
content,
{"sensitivity": "high"}
)
if result["status"] == "pass" and result["confidence"] > 0.8:
# High confidence pass - approve automatically
return {"action": "approve"}
elif result["status"] == "fail" and result["confidence"] > 0.8:
# High confidence failure - reject automatically
await log_bias_violation(content, result["reason"])
return {"action": "reject", "message": "Contains biased language"}
else:
# Low confidence or ambiguous - queue for human review
await queue_for_human_review(content, result)
return {"action": "review", "confidence": result["confidence"]}
```
# Combining with Other Guardrails
Bias detection often works alongside other content validation:
**With toxic language detection**: Validate for both hostility and bias to ensure professional, inclusive communication. Run these checks in parallel since they analyze different aspects of the same content.
**With contains string**: Use contains string to quickly block explicitly prohibited terms (instant, free) before running bias detection (1-3 seconds, token cost). This pre-filtering pattern reduces costs while maintaining comprehensive protection.
**With valid JSON**: When generating structured content, validate JSON format first (instant), then check text fields for bias. This ensures you have parseable data before running more expensive content checks.
# Next Steps
Detect hostile content, threats, and harmful language that complements bias detection
Learn optimal patterns for combining guardrails, error handling, and cost management
Understand confidence scores, decision strategies, and validation patterns in depth
Get hands-on with bias detection in under 5 minutes