When Machine Learning Goes Off the Rails

This article does an excellent job of defining of how machine learning is a new paradigm for risk management in organizations. Although the background and examples are from healthcare and medical devices, authors I. Glenn Cohen and collaborators abstract those learnings into sound advice for all business practitioners. They anticipate the pace of technological change and broader use across industries to increase risk, observing that "accidents or unlawful decisions can occur even without negligence on anyone’s part" simply because of the nature of ML. Given that reality, the appropriate responses companies should take include:

• Conceive of ML systems as living entities and manage them accordingly.
• Create policies and controls around how much and which models are allowed to evolve.
• Act ahead of emerging regulation to ensure systems are fair, safe, and efficacious.
• Develop internal standards and processes for certification of ML before putting it into production.

Share   Tweet   Email

Regulators catching up on use of analytics; compliance better take notice

As we've noted in previous issues, there is a rising awareness across business functions of the power and dangers of predictive analytics. In addition to this article facing compliance teams, this one facing Chief Information Security Officers (CISOs) expects more AI/ML regulations at state and local levels and encourages designing for privacy and auditability from project initiation.

The title of this piece for compliance leaders does not capture the full scope of the article, which not only discusses how data will be used by regulators but also the need to ensure compliance of data analytics assets. Rather than getting their hands on the data, regulators will be more interested in program design and ensuring compliance teams are aware of data assets and usage, which often operates in a blind spot for compliance today. Once included, compliance will also need to learn to ask the right questions about how the company uses data so that it can prevent investigations, penalties and reputational harm.

Share   Tweet   Email

An overview of the CDEI's review into bias in algorithmic decision-making

Building on the content of our last issue, Europe and the United Kingdom have seen a groundswell of thought, studies, and discussion about the regulation of ML and AI, particularly as it relates to bias. The UK's Centre for Data Ethics and Innovation published its report on bias in algorithmic decision-making. The authors contrast the potential that algorithms hold to identify and combat bias with their deployment in the real world today, which has engendered a notable absence of public faith in the fairness of algorithmic decisions. In Europe, the EU Agency for Fundamental Rights (FRA) issued its own report on how AI should support human rights. Frustrated by the focus on principles and ethics, NGOs like Access Now are also driving the discussion and pushing for more stringent standards and regulatory requirements, proposing a layered approach even with bans on certain types of AI deployments that are at greatest risk for violating people's rights.

Share   Tweet   Email

AI research survey finds machine learning needs a culture change

The abrupt and opaque firing of AI bias researcher Timnit Gebru last month generated enormous amounts of press and attention. The reverberations were felt at the top AI research conference, NeurIPS, where a schism is emerging in the field of AI research because of the heavy influence that the major tech vendors like Google, Microsoft, and Amazon hold, including comparisons between Big Tech and Big Tobacco's tactics at a Resistance AI workshop.

This survey revealed a "wild west" culture in which well-known and documented biases inherent in large language models are ignored as a result of breakneck development. It highlights the broader need for a more comprehensive approach to documentation and data management. We concur, while thinking the problem of bias in ML extends well beyond the data. We see those same discussions happening in the private sector as technologists and business owners grapple with the potential harm that intelligent systems can inflict on companies' bottom lines, risk profiles, and public brand.

Share   Tweet   Email

When AI Reads Medical Images: Regulating to Get It Right

Stanford researchers at the institute of Human-Centered Artificial Intelligence propose a framework for regulating diagnostic algorithms that will ensure world-class clinical performance and build trust among clinicians and patients. After summarizing the current state of technology and the risks posed by it, the article identifies the most immediate priority as creating diagnostic tests that are independent of the various algorithms so that their real-world performance can be compared objectively. That capability will require a master dataset shared across developers that incorporates different pathologies, demographics, and images from prominent manufacturers. With a shared dataset and independent tests, algorithm builders could submit confidently to a regulatory agency charged with measuring and reporting on its performance. Having a shared regulatory framework and increased transparency would benefit the industry as a whole by overcoming resistance to augmented decisioning in the medical community. As of now, clinicians are extremely reluctant to trust algorithms, especially when they cannot "see" the dynamics of AI decisions or understand the broader outcomes and success of those decisions. This panel discussion and this excellent essay on AI in medicine explore related territory and are well worth your time.

Share   Tweet   Email

Technology Can't Fix Algorithmic Injustice

A Princeton team of political philosophers wrote this expansive and thorough essay on how democratic societies should engage with the wide-scale deployment of AI and ML systems. They use numerous real world examples to illustrate some of the cognitive dissonance that exists in the public, academic, and industry discussions of bias and fairness in the U.S. today, as well as questioning the viability of what they call "quality control" approaches to solving for algorithmic bias. Ultimately, they determine that only the democratic process with a deeply engaged public interface is capable of addressing the problem: "Rather than allowing tech practitioners to navigate the ethics of AI by themselves, we the public should be included in decisions about whether and how AI will be deployed and to what ends." This perspective falls in line with a growing chorus for contestability over transparency and recourse over explainability. If developers and system owners do not take more aggressive leadership positions, they may find themselves facing a more aggressive regulatory regime and public concern than they would like.

Share   Tweet   Email

A “Light Touch” Regulatory Framework for AI – Transparency at the Heart of AI Regulation

This process-focused piece is a great companion piece to the above article. Authors Roger Bickerstaff and Aditya Mohan continue their exploration of a regulatory framework for artificial intelligence in this third installment that focuses on transparency. They divide the world of AI applications into those higher stakes deployments that will require regulatory approval and lower stakes deployments that should have a simple public registry. They draw a parallel to patent protection, stating that "Sufficient information should be made available to enable meaningful scrutiny without requiring important confidential information to be disclosed." However, they distinguish this requirement from explainability, noting that the end-goal should be the assessment of outcomes from the machine-driven decisions based on an index of the technical quality of the system and the human impact of the decisions made. They lay out a grading system and corresponding tranches of actionability, from none at the low end to requirements for notification, impact assessment, and prior approval at the high end. Ungoverned, some intelligent systems may have the effect of creating policy, around which developers, executives, and compliance professionals need to develop controls and better risk management practices.

Share   Tweet   Email