Black Box vs. Glass House
Achieving Sustainable AI through Transparency
The last five years have brought forward tremendous innovations in AI accessibility. Tooling historically only available to the largest organizations can now be sipped or slurped from cloud service providers (CSPs) that make its use accessible to all. Simultaneously, data systems have also become faster and easier to integrate, creating a Cambrian event for deployable AI systems.
In the last decade, the broad adoption of deep learning algorithms, so called because of their layered structure (not, as some have thought, because of their ability to learn more “deeply” than other algorithms) also brought complexity that often made such systems hard or impossible to comprehend. This created a sort of tradeoff between model performance and model explainability.
More recently, approaches have been created to do both: create solutions which are both accurate and explainable.
What then is the relationship between Explainable AI (XAI) and Sustainability? Here are a few reflections of what we’ve seen in the trenches with Global 1000 clients who are wrestling with these ideas to create extensible AI solutions that survive the test of time, and continue to benefit humanity.
The Ethical AI Value Chain
As a publicly traded registered B-Corp, my employer, Kin and Carta is continuously evaluated by ourselves and others, against our commitment to a triple bottom line: people, planet and profit. In the realm of data-driven digital transformation, a very clear relationship has emerged which binds together the concepts of transparency, explainability, ethics and sustainability: transparent AI enables explainable AI. Explainable AI enables ethical AI. And Ethical AI has a much higher probability of being sustainable.
Transparent AI → Explainable AI → Ethical AI → Sustainable AI.
Not all transparent AI is sustainable, nor is all sustainable AI transparent. However, in using this mental model, we can greatly improve our chances of building AI that not only works better for everyone, but also has a direct impact on long term total value achieved by way of its sustainability.
Model explainability is often really DATA explainability
One of the first questions faced by those on this journey is what exactly do we need to explain? Various powerful open source tools are available to help make sense of what complex models are doing with data to generate predictions. Python packages such as SHAP can help practitioners to understand the marginal effects of each variable in a predictive model.
Built-in explainers like variable importance plots do the same, and recursive partition trees (RPART, aka Decision Trees) can make multi-variate contexts accessible for everyone.
However, in many cases, the insights generated by such tooling is less about explaining a model, as it is about explaining the relationships that exist within the data itself– independent of the modeling technique. In this way, models derived from this data, even if generated by vastly different algorithmic approaches, will often identify the same relationships and attempt to exploit them. In this way, it’s not about accuracy OR explainability, it’s about accuracy AND explainability– achieving both.
Transparency vs. Explainability
If such techniques as these can enable transparency into those structures within the data, what then is the difference between transparency and explainability? Anyone who’s worked in either artificial intelligence or business intelligence can confess to how much of the “sexiest job in the 21st century” is actually data janitorial work– data cleansing and normalizing. But it is also creative data enrichment and feature engineering.
Here we get to use our imaginations and inquisitiveness to consider “what if” certain new data or new derivatives of our data could help us unlock new levels of opportunity? If transparency is about seeing the good and the bad, as well as the opportunity, then explainability is about improving that transparent state into a better, more equitable solution. And typically it is one that can outperform human understanding or design alone.
Example: Business Rules vs. Individualized Recommendation
Often in the process of building explanatory models, business-savvy clients are looking for “truths” about their products and customers that they can they exploit to better their outcomes. This follows the precedent of business intelligence — where we would pore over one-off analyses to define strategy for the next season of our business.
In creating these rules-of-thumb, or best practices, we look for shortcuts or refinements to a generalized practice to improve results. For example “always launch marketing campaigns on Tuesdays” or “people over 65 prefer physical mail over email”. Developing protocol is deeply engrained in certain industries such as healthcare, but sometimes protocol does not satisfy the needs of some, or even anyone perfectly.
However, what to do when there are hundreds or even thousands of such dimensions to each customer, product, patient or genome? Rules of thumb no longer do the job, and a new approach must be taken to be informed skeptically by the AI built atop our historical data, in generating fully individualized recommendations based on the way each new case is presented.
The Right Job for the Right Player
At this point, it’s important to point out the primary limitation of AI– it is limited (and often stymied) by the history from which it’s built. Even as AI seeks to explore contexts and find mathematically optimal states for exploitation, it is limited to the contexts of prior observations. Predicting or recommending outside of these boundaries are where you’ll often hear data scientists squeezing the brakes. “If two emails per week doubles purchases, then why not send ten?” The world is full of non-linear or “sweet spot” effects and to extend projections into the unknown risks finding those cliffs, often abruptly.
The good news is that human intuition can often flag these danger zones ahead of AI, if given the tooling of transparency and explainability. Humans can also identify creative workarounds, or even whole new approaches to improve AI through danger avoidance through creatively enhancing models with new data sources. In doing so we have the opportunity to break the patterns of our history that do not serve us in a way that AI alone cannot.
Example: Correlation vs. Causation
No discussion on the matter would be complete without a brief pause on the correlation vs. causation concept. New research has uncovered ways to tease out causality in models. And in some cases, i.e. where a strict event timeline is in place, we can definitively say causality is impossible. However, in the quest for “truths” and “rules of thumb” clever business insight consumers (and data scientists as well) can be fooled by otherwise spurious correlations. It turns out Nicolas Cage movies do not actually cause drownings.
Explainable vs. Ethical
What then is the leap from explainable to ethical? We can think of transparency as the ability to look into models and data, warts and all. We can think of explainability as the means to communicate those truths (hopefully after iterative refinement) to any and all stakeholders once we have a workable approach.
Then, we can consider Ethical AI as achieved only when transparency and explainability are achieved. The key difference between explainable and ethical is the subject of the interpretation– while explainable AI can be understood by an individual (and in different ways), ethical AI must be understood and accepted by all. Critically, by people with different perspectives, backgrounds and world views. In this way, ethical AI is something that is approached and continuously improved, not a one time validation or certification.
Ethical vs. Sustainable
By putting this process on a timeline of continuous iteration and improvement, we achieve also a silver lining: that such systems achieve a level of sustainability not possible by fixed code solutions. AI systems by definition both interpret AND affect systems where they operate. They consume data to make a recommendation or decision, then enact that decision or recommendation, which in turn requires another iteration of modeling built upon its effect.
Building systems that are continuously improving and adapting makes them both resilient and less susceptible to decay. Accuracy drift, dampening performance, and lack of adaptability to ground realities can all spell the end of a model’s lifespan. However by bringing together the best of automation through AI and the best of human judgment and creativity, we can create systems that achieve accuracy/performance today, and adapt to realities of tomorrow.
