Gender bias in dependency parsing

Abstract

Recent high-profile advances in natural language processing (NLP) have spurred interest into identifying and rectifying socially harmful problems common in NLP systems such as gender bias. Unfortunately, many works which attempt to tackle the issue of gender bias suffer from methodological deficiencies such as the assumption of a binary and immutable concept of gender. We scrutinize one such work which found gender bias in dependency parsing and evaluate if the claims have merit. Our results were inconsistent with the gender bias findings of that paper, and further investigations through error analysis and treebank analysis revealed methodological flaws which artificially introduced differences between their female and male data sets. Mistakes made during preprocessing compromised the outcome; therefore, their results do not prove the existence of gender bias in dependency parsing. Through our findings, we suggest a different methodology for identifying and alleviating syntactic bias that is more inclusive for everyone-no matter their gender.