The Indian Financial Automated Advice Bias (IFAAB) Dataset: Auditing Socioeconomic and Demographic Disparities in LLM-Generated Financial Advice in India (arXiv (Cornell University))

Summary

• IFAAB is a counterfactual audit dataset of 31,734 GPT-4o-generated financial-advice responses for synthetic Indian personas.p.1
• In the gender-only audit, male-labelled profiles receive 60.1% higher initial total resources and 46.8% higher post-advice total resources than comparable female-labelled profiles.p.1
• In the caste-augmented audit the post-advice total-resource gap remains 49.4%, while adding district information reduces the aggregate gender gap to 7.9%.p.1
• Paired gendered responses show an average TF–IDF cosine similarity score of 0.44.p.1
• Human review of 200 random cases agreed with 97% of LLM-as-judge assessments across 12 financial-planning domains.p.4
• Exact matches between paired responses were 0% across all gender and caste pairs.p.4

Key insights

1. Women are more often steered toward cautious saving, community support and basic financial literacy, while men receive more growth-oriented advice on investing and entrepreneurship.p.5
2. Unspecified caste profiles are estimated closer to General than to OBC, SC or ST profiles, so missing caste information is treated as relatively advantaged.p.5
3. All financial variables are reported in units of 1,000 rupees, and total resources are the sum of wealth, income and savings.p.3
4. Relative to General-category profiles, initial total resources coefficients are -57.0 for OBC, -87.5 for SC and -86.8 for ST.p.13
5. Male coefficients in OLS regressions are positive and significant in the gender-only and gender-by-caste specifications but much smaller when district is included.p.5