Challenges for a Fair AI

In this document, we elaborated a list of technical recommendations for the development of Artificial Intelligence (AI) projects, specifically Machine Learning based systems. These recommendations are the result of structured interviews with people who work in practical applications of data-based systems in various roles and organizations within the Argentine technological ecosystem, and have been elaborated through the lens of our AI ethics team, composed of activists, social scientists and computer science researchers.

The main recommendations we propose are:
1. Before proceeding with any development, analyze the need and social value of it. Consider the potential benefits and drawbacks of implementing the project.
2. Define the problem to be solved in non-technical terms, based on the joint reflection of the parties interested in developing an AI application. Consider the possible secondary uses of the application of the project and its possible harmful effects.
3. Analyze the data sources with which the model may be trained, taking into consideration the protection of personal data and intellectual property. Pay close attention to potential biases to ensure that all groups are adequately and non-discriminatorily represented, with a special focus on minorities.
4. When training and evaluating predictive models, incorporate a methodology for checking well-known limitations of models that lead to pernicious results: underrepresented classes, majority classes, a tendency to overfit, etc. Incorporate also more general error analysis to detect previously unknown limitations.
5. Carry out deployment in phases, accompanied by the corresponding monitoring mechanisms for early detection of pernicious effects of the implementation of the system in actual contexts, incorporating different perspectives into the analysis of the behavior of the system.

Equity analysis must be transversal, i.e., a reflective resource present throughout the entire system development and construction process, rather than being applied only when evaluating the discriminatory impacts of a system after it has been deployed. This document intends to contribute to the understanding of the development processes of AI systems and the problems associated with them. Besides, it serves as a guide for those interested in developing more human rights-conscious technological products.

Document available here.