Custom chatbots and model gardens: putting Private AI to work – ComputerWeekly.com
The expert team at Freeform Dynamics offer news and views on the latest hot topics, and pragmatic advice based on real-world experience and meaningful research studies.
It’s essential to decide where to run your Private AI – or who you will get to run it for you, in some cases – but that is just the start. Assuming that you also want custom chatbots and AI assistants – and that is the primary reason for having your own AI, trained on your business data – there is even more to think about.
The choice of model – by which I mean the large language models (LLMs) that underpin GenAI – will be a key one for many organisations. Notable LLMs include OpenAI’s GPT versions (as in ChatGPT and Bing Chat), Google’s PaLM (used in Bard), Meta’s LLaMa family, and Anthropic’s Claude 2, but there are many, many more.
Each model is typically better suited to some tasks than others, some are proprietary but many are open source, and there are models tailored to specific industries, professions or tasks, such as medicine, finance or cybersecurity.
This is why we see a growing number of vendors offering model choice, such as Google Cloud with its ‘model garden’, and Box which, as well as planning to offer a curated set of models, will also allow users to BYOM or ‘bring your own model’. In fact, any vendor or service provider worth its salt will have some options here, whether it’s a short menu of LLMs or a link-up with someone like Hugging Face, which describes itself as an AI/ML community and collaboration platform, and has over 300,000 models available.
One other thing worth paying attention to here is model size. While it’s LLMs that have gotten most of the attention, these are costly and time-consuming to train. For instance, ‘foundational’ general-purpose LLMs can have hundreds of billions of parameters, and training GenAI on that scale requires many thousands of GPU hours and costs millions of dollars.
In addition, while inferencing, which is when you apply the resulting trained model to data, needs significantly less compute power than training does, it still isn’t free. That’s especially true if you want your AI to carry on training itself on new data as it goes along, a process called reinforcement learning.
So there’s quite a bit of research going into smaller ‘nimble’ models, with just a few billion parameters (under 15bn is a useful rule of thumb). Evidence thus far suggests that models such as IBM’s 13bn parameter Granite, or the 1.3bn parameter open-source Phi 1.5 from Microsoft Research, can be just as effective as an LLM on specific tasks or in constrained settings, and a whole lot cheaper to run.
If you’ve not started planning for your organisation’s Private AI future, hopefully this blog and part 1 have given enough of an outline to do so. Whether you prefer to use private cloud, the ‘GPT in a box’ concept favoured by the likes of Dell and VMware, or the SaaS approach offered by the public cloud and collaboration specialists, the concepts behind Private AI – at the very least – need to become the default for enterprise use, for all the usual reasons of security, governance, privacy, etc.
U.S. senators and industry experts are pushing for a new agency to regulate digital industries and tackle competition, privacy …
CIOs face higher SaaS prices, with a new report pegging the inflation rate at 8.7%. The wider cloud segment saw a 3.2% inflation …
The concept of an autonomous enterprise is becoming more real as enterprises combine technologies to automate all kinds of human …
The government advisory follows several high-profile attacks attributed to Scattered Spider, which uses advanced social …
MeridianLink said it recently identified a “cybersecurity incident,” but the Alphv ransomware gang claims it breached the company…
In times of economic uncertainty, cybersecurity budget cuts can make the security team’s job even more challenging. Here are …
A private 5G network could provide organizations with greater control over their wireless environments. But cost and complexity …
Network challenges include visibility, scalability and security. NetOps teams should follow management best practices and …
Enterprises have three main options when it comes to cloud network architecture: Use built-in CSP tools, buy virtual networking …
Rocky Linux and AlmaLinux are new distributions created after Red Hat announced the discontinuation of CentOS. These …
Red Hat’s discontinuation of CentOS is causing individual users and companies to look elsewhere for an RHEL-compatible OS. Look …
With CentOS 7 and RHEL 7 reaching end of maintenance, RHEL admins and developers must look into other OS options. Follow this …
The data management and BI specialist has released Data Intelligence, an AI-driven suite that includes data quality and data …
The database vendor is making industry verticalization a significant part of its product roadmap to help customers in various …
The vendor’s new integration with vector database specialist Pinecone is designed to enable users to ensure quality of the data …
All Rights Reserved, Copyright 2000 – 2023, TechTarget
Privacy Policy
Cookie Preferences
Do Not Sell or Share My Personal Information
