Nvidia helps enterprises guide and control AI responses with NeMo Guardrails

A major challenge for generative AI and large language models (LLM) overall, there is the risk that a user will get an inappropriate or inaccurate answer.

The need to protect organizations and their users is well understood by Nvidiawho published today the new NeMo Railing open-source framework to help solve the challenge. The NeMo Guardrails project enables organizations building and deploying LLMs for different use cases, including chatbots, to ensure responses stay on track. Guardrails provide a set of defined controls with new policy language to help define and enforce boundaries to ensure that AI responses are topical, safe, and do not introduce any security risk.

>>Follow the news of VentureBeat Generative AI cover<

“We believe that every company will be able to leverage generative AI to support their business,” Jonathan Cohen, vice president of applied research at Nvidia, said during a press and analyst briefing. “But to use these models in production, it’s important that they are deployed in a safe and secure way.”

Why Guardrails are Important for LLMs

Cohen explained that a guardrail is a guide that helps keep the conversation between a human and an AI on track.

The way Nvidia looks at AI guardrails, there are three main categories where there is a specific need. The first category are topical safeguards, which aim to ensure that an AI response literally stays on topic. The themed railings are also meant to make sure the answer stays in the correct tone.

Safety guardrails are the second main category and are designed to ensure that answers are correct and verified. Responses should also be vetted to ensure that they are ethical and do not include any type of toxic content or misinformation. Cohen acknowledged the general concept of AI “hallucinations” to explain why there is a need for security guardrails. With an AI hallucination, an LLM generates an incorrect answer if they do not have the correct information in their knowledge base.

The third category of guardrails where Nvidia sees a need is security. Cohen commented that since LLMs are allowed to connect to third-party APIs and applications, they can become an attractive solution. attack surface for cybersecurity threats.

“Whenever you allow a language model to actually perform an action in the world, you want to monitor the requests sent to that language model,” Cohen said.

How NeMo Guardrails works

With NeMo Guardrails, what Nvidia is doing is adding another layer to the stack of tools and templates that organizations need to consider when deploying AI-powered applications.

The Guardrails framework is code deployed between the user and an LLM-compliant application. NeMo Guardrails can work directly with an LLM or with LangChain. Cohen noted that many modern AI applications use open-source LangChain framework to help build applications that chain together different LLM components.

Cohen explained that NeMo Guardrails monitors conversations to and from the LLM-powered application with a sophisticated pop-up dialog engine. The engine tracks conversation state and provides developers with a programmable way to implement guardrails.

The programmable nature of NeMo Guardrails is enabled with the new Colang policy language that Nvidia has also created. Cohen said Colang is a domain-specific language for describing conversational flows.

“Colang’s source code reads like natural language,” Cohen said. “It’s a very easy to use tool, it’s very powerful, and it basically lets you script the language model into something almost like English.”

At launch, Nvidia provides a set of predefined common policy templates to implement thematic, security, and safety guardrails. The technology is freely available as open source, and Nvidia will also provide commercial support to enterprises as part of the Nvidia AI Company suite of software tools.

“Our goal is really to enable the ecosystem of large language models to evolve in a safe, efficient, and useful way,” Cohen said. “It’s hard to use language models if you’re afraid of what they might say, so I think the guardrail solves an important problem.”