UK Financial Conduct Authority publishes research note on bias in natural language processing

The Financial Conduct Authority has published a research note on a pilot study into bias in natural language processing (NLP). The research note presents the results of a technical investigation into biases in word embeddings. There are three main findings from the investigation, namely that:

• No individual measurement technique can fully capture bias in embeddings, but bias can be seen more clearly when a mix of techniques is used.
• Even when multiple methods are used, tackling bias is still complicated. The research note highlights that existing tools have limits, and bias is often shaped by context, language, and social factors.
• Techniques that try to reduce bias, like Hard Debiasing, do not always work as well as anticipated. While they can lower bias in some areas, they often reduce the overall quality of the model. The research note explains that future research could involve testing applications that utilize embeddings—for instance, studying the impact of biased embeddings on downstream outcomes for consumers. The authors of the research note also believe that mitigation of bias in contextual and sentence embeddings would also be a worthy avenue of enquiry.

Return to main website.