Teach AI simulations what they don’t know | MIT News

Artificial intelligence systems like chatgpt provide reasonable sounding answers to any questions you may ask. However, they do not always reveal gaps in their knowledge or areas of uncertainty. As AI systems are increasingly used to develop drugs, synthesize information and drive self-driving cars, this problem can have huge consequences.

Now, MIT’s rotating body AI is helping to quantify model uncertainty and correct output before causing bigger problems. The company’s CAPSA platform can be used with any machine learning model to detect and correct unreliable output in seconds. It enables it to detect patterns in data processing by modifying the AI ​​model to indicate ambiguity, incompleteness, or bias.

“The idea is to take a model, wrap it in CAPSA, determine the uncertainty and failure mode of the model, and then enhance the model,” said Daniela Rus, co-founder of Themis AI and professor at MIT. “We are excited to provide solutions that can improve the model and provide a proper functioning of the model.”

Rus built Themis AI in 2021 with Alexander Amini ’17, SM ’18, PhD ’22 and Elaheh Ahmadi ’20, Meng ’21. Since then, they have helped telecom companies to plan and automate networks, help oil and gas companies understand earthquake images using AI, and published papers on developing more reliable and trustworthy chatbots.

“We want to enable AI in the highest sales apps in every industry,” Amini said. “We all see examples of AI hallucinations or making mistakes. As AI is deployed more widely, these mistakes can have devastating consequences. Our software can make these systems more transparent.”

Help the model know what they don’t know

Rus’ lab has been studying the uncertainty of the model for years. In 2018, she received funding from Toyota to study the reliability of machine learning-based autonomous driving solutions.

“This is a safety-critical environment and it is very important to understand the reliability of the model,” RUS said.

In separate work, Rus, Amini and his collaborators built an algorithm that detects racial and gender bias in facial recognition systems and automatically re-grants the model’s training data, indicating that it eliminates bias. The algorithm rebalances it by identifying non-representative parts of the basic training data and generating new, similar data samples.

In 2021, the final co-founder showed that similar approaches could be used to help pharmaceutical companies use AI models to predict the properties of drug candidates. They built Themis AI later that year.

“Guiding drug discovery can save a lot of money,” RUS said. “That’s a use case that makes us realize how powerful the tool is.”

Today, Themis is working with companies in a variety of industries, many of which are building large language models. By using CAPSA, the model can quantify the uncertainty of each output.

Stewart Jamieson SM ’20, PhD ’24, the technology leader at Themis AI, many companies are interested in using data-based LLM, but they are worried about reliability. “We help LLMS self-report their confidence and uncertainty, which makes more reliable questions answered and marks unreliable outputs.” ”

Themis AI is also in discussions with semiconductor companies to build AI solutions on their chips that can work outside of cloud environments.

“Usually, these smaller models used on phones or embedded systems are not very accurate compared to the models you can run on a server, but we can get the best of both worlds: low latency, efficient edge computing without sacrificing quality,” Jamieson explained. “We see the future of edge devices doing most of the work, but whenever they are unsure of the output, they can forward these tasks to a central server.”

Pharmaceutical companies can also use CAPSA to improve AI models used to identify drug candidates and predict their performance in clinical trials.

“The predictions and outputs of these models are very complex and difficult to explain – experts spend a lot of time and effort understanding them,” Amini said. “CAPSA can provide insights directly to see if the predictions are supported by evidence in the training center, or just guessing without a lot of foundations. This can accelerate the strongest predictions, and we think that there is a huge potential for social benefit.”

Impact Research

Themis AI’s team believes the company is well positioned to improve the forefront of evolving AI technologies. For example, the company is exploring CAPSA’s ability to improve accuracy in AI technology, called “through thinking chain” reasoning, and LLMS explains the steps they took for the answer.

“We have seen signs of CAPSA that can help guide these inference processes to identify the highest confidence chains of reasoning,” Amini said. “We believe this has a huge impact in improving the LLM experience, reducing latency and reducing computational requirements. This is a huge opportunity for us.”

For Rus, who has co-founded several companies since she arrived at MIT, Themis AI is an opportunity to secure her MIT research impact.

“My students and I are becoming increasingly keen to take the extra steps to make our work relevant to the world,” Ross said. “Artificial intelligence has great potential to transform the industry, but AI has also attracted attention. What excites me is the opportunity to develop technological solutions to these challenges and build trust and understanding between people and technologies that are becoming part of their daily lives.”