NVIDIA’s “Chat With RTX” Is A Localized AI Chatbot For Windows PCs Powered By TensorRT-LLM & Available For Free Across All RTX 30 & 40 GPUs – Wccftech
Expanding its AI ecosystem, NVIDIA has introduced “Chat with RTX”, a chatbot for Windows PCs that is powered by TensorRT-LLM & available for free on the latest RTX GPUs.
The utility of the “Chat with RTX” chatbot is very simple, it is designed as a localized system which means that you will have a personalized GPT chatbot available to you all the time on your PC without the need to go online. Chat with RTX can be fully personalized by utilizing a dataset that is available locally on your PC and the best part is that it runs across almost all RTX 40 & RTX 30 GPUs.
Starting with the details, Chat with RTX leverages NVIDIA’s TensorRT-LLM & Retrieval Augmented Generated (RAG) software which was announced for Windows PCs last year & takes full advantage of the RTX acceleration available on RTX hardware to deliver the best possible experience to users. Once again, the application is supported across all GeForce RTX 30 & 40 GPUs with at least 8 GB of video memory.
After downloading “Chat with RTX” for free, users can connect it to a local dataset available on the PC (.txt, .pdf, .doc, .docx, .xml) and connect it to a large language model such as Mistral and Llama 2. You can also add specific URLs for example for YouTube videos or entire playlists to further enhance the dataset search results. After connecting, users can then use Chat With RTX the same way as they would use ChatGPT by running different queries but the results generated will be based entirely on the specific dataset, giving you better responses compared to online methods.
Having an NVIDIA RTX GPU that supports TensorRT-LLM means that you will have all your data and projects available locally rather than saving them in the cloud. This would save time & deliver more precise results. RAG or Retrieval Augamanted Generation is one of the techniques used in making AI results faster by using a localized library that can be filled with the dataset you want the LLM to go through & then leverage the language understating capabilities of that LLM to provide you with accurate results.
NVIDIA states a 5x performance boost with TensorRT-LLM v0.6.0 which will be available later this month. Furthermore, it will also enable support for additional LLMs such as Mistral 7B & Nemotron 3 8B.
You can download NVIDIA’s “Chat with RTX” application here. It is supported by both Windows 11 & Windows 10 PCs and requires the latest NVIDIA GPU drivers for optimal performance.
Subscribe to get an everyday digest of the latest technology news in your inbox
Some posts on wccftech.com may contain affiliate links. We are a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to amazon.com
© 2024 WCCF TECH INC. 700 – 401 West Georgia Street, Vancouver, BC, Canada
