Nvidia’s new Guardrails tool will make AI chatbots less crazy
Nvidia is introducing NeMo Guardrails for AI developers, which is an open-source framework that promises to keep AI chatbots from going off the rails.
Hollif
Digital Trends may earn a commission when you buy through links on our site. Why trust us?
Nvidia is introducing its new NeMo Guardrails tool for AI developers, and it promises to make AI chatbots like ChatGPT just a little less insane. The open-source software is available to developers now, and it focuses on three areas to make AI chatbots more useful and less unsettling.
The tool sits between the user and the Large Language Model (LLM) they’re interacting with. It’s a safety for chatbots, intercepting responses before they ever reach the language model to either stop the model from responding or to give it specific instructions about how to respond.
Jacob Roach / Digital TrendsNvidia says NeMo Guardrails is focused on topical, safety, and security boundaries. The topical focus seems to be the most useful, as it forces the LLM to stay in a particular range of responses. Nvidia demoed Guardrails by showing a chatbot trained on the company’s HR database. When asked a question about Nvidia’s finances, it gave a canned response that was programmed with NeMo Guardrails.
This is important due to the many so-called hallucinations we’ve seen out of AI chatbots. Microsoft’s Bing Chat, for example, provided us with several bizarre and factually incorrect responses in our first demo. When faced with a question the LLM doesn’t understand, it will often make up a response in an attempt to satisfy the query. NeMo Guardrails aims to put a stop to those made-up responses.
The safety and security tenets focus on filtering out unwanted responses from the LLM and preventing it from being toyed with by users. As we’ve already seen, you can jailbreak ChatGPT and other AI chatbots. NeMo Guardrails will take those queries and block them from ever reaching the LLM.
Although NeMo Guardrails to built to keep chatbots on-topic and accurate, it isn’t a catch-all solution. Nvidia says it works best as a second line of defense, and that companies developing and deploying chatbots should still train the model on a set of safeguards.
Developers need to customize the tool to fit their applications, too. This allows NeoMo Guardrails to sit on top of middleware that AI models already use, such as LangChain, which already provides a framework for how AI chatbots are supposed to interact with users.
In addition to being open-source, Nvidia is also offering NeMo Guardrails as part of its AI Foundations service. This package provides several pre-trained models and frameworks for companies that don’t have the time or resources to train and maintain their own models.
Editors' Recommendations
Even Microsoft thinks ChatGPT needs to be regulated — here’s why What is ChatGPT Plus? Everything we know about the premium tier Google’s ChatGPT rival is an ethical mess, say Google’s own workers Auto-GPT: 5 amazing things people have already done with it What is ChatGPT? How to use the AI chatbot everyone’s talking about
Jacob Roach is a writer covering computing and gaming at Digital Trends. After realizing Crysis wouldn't run on a laptop, he started building his own gaming PCs and hasn't looked back since. Before Digital Trends, he contributed content to Forbes Advisor, Business Insider, and PC Invasion, covering PC components, monitors, and peripherals.
Outside of tinkering with his PC and tracking down every achievement in the latest games, Jacob spends his time playing and recording music. Before switching to writing full time, he worked as a recording engineer in St. Louis, Missouri.
The dark side of ChatGPT: things it can do, even though it shouldn’t
Have you used OpenAI's ChatGPT for anything fun lately? You can ask it to write you a song, a poem, or a joke. Unfortunately, you can also ask it to do things that tend toward being unethical.
ChatGPT is not all sunshine and rainbows -- some of the things it can do are downright nefarious. It's all too easy to weaponize it and use it for all the wrong reasons. What are some of the things that ChatGPT has done, and can do, but definitely shouldn't?
A jack-of-all-trades
GPT-5: release date, claims of AGI, pushback, and more
Based on reports earlier this year, GPT-5 is the expected next major LLM (Large Language Model) as released by OpenAI. Given the massive success of ChatGPT, OpenAI is continuing the progress of development on future models powering its AI chatbot.
GPT-5, in theory, would aim to be a major improvement over GPT-4, and even though very little is known about it. Here's everything that's been rumored so far.
Release date
OpenAI has continued a rapid rate of progress on its LLMs. GPT-4 debuted on March 14, 2023, which came just four months after GPT-3.5 launched alongside ChatGPT.
It’s happening: people are recreating deceased loved ones with AI
It might sound like science fiction, but undertakers and tech-savvy people in China have already started using AI tools to create realistic avatars of people who have passed away.
Using a blend of tools such as the ChatGPT chatbot and the image generator Midjourney -- in addition to photos and voice recordings -- funeral companies are starting to fashion a rendition of the deceased loved one that grieving families and friends can “communicate” with, according to Guangzhou Daily via the Straits Times.
