📖 How to use this Pack: Stop “training” and start “designing”

<aside> 💡

This is not a traditional audit kit. Based on Design Science Research methodology (Hevner et al.), these tools assume your algorithm is an ‘artifact’ that intervenes in reality.

</aside>

Recommended workflow:

1. Start with the Canvas: Before touching the data. Define your problem and values. If you can’t fill the ‘Appeal Mechanism’ box, don’t build the model.
2. Use the Calculator: AUC lies. Use the matrix to decide which errors you’re willing to tolerate and who will pay for them.
3. Respect the Traffic Light: Fairness metrics are often incompatible with each other. This dashboard forces you to choose a priority metric and stop the system if it crosses the red safety line.

<aside> 💡

⚖️ CORE STANDARD: The Variance Funnel (5% vs 8%)

You will notice two different thresholds across our tools. This is intentional:

• INPUTS (Data): Strict 5% Limit.

◦ Where: In the Label Integrity Gate.

◦ Why: Bias in training data is systemic and invisible. We tolerate maximum 5% disparity (Hardt et al.) because "garbage in, garbage out."

• OUTPUTS (Results): Flexible 8% Limit.

◦ Where: In this Outcome Fairness Pack.

◦ Why: Real-world deployment allows for mitigation. We tolerate up to 8% disparity ONLY IF you have a "Human-in-the-Loop" mechanism (like Option B in the Trade-offs Matrix) to correct errors.

◦ Stop Rule: Gaps >8% are unmanageable and require a full stop.

</aside>

🎨 TOOL 1: Socio-Technical Design Canvas

The Fairness Design Canvas

⚠️ Central piece that legitimizes the model before writing a single line of code.

1️⃣ Definition of “Risk” and “Fraud”

Don’t assume the data defines it. Explicitly write what behavior counts as fraud and what doesn’t.

What behavior counts as fraud?

[Specifically define what actions, patterns, or
behaviors are considered fraud in your context]

Examples:
- Intentional false declarations
- Unauthorized use of credentials
- Behavioral patterns X, Y, Z

What behavior is NOT fraud?

[Explicitly define what legitimate behaviors
may APPEAR suspicious but aren't]

Examples:
- Honest errors in forms
- Atypical behavior due to special circumstances
- Irregular but authorized use

Prohibited variables (proxies)