Advantech has announced a new software offering – GenAI Studio for the Advantech Edge AI SDK, addressing the growing demand for cost-effective, on-premises large language model (LLM) solutions.
This initiative aims to make LLM adoption more accessible for developers and small- to medium-sized businesses (SMBs) by significantly reducing the reliance on GPUs, thereby lowering costs and enabling broader utilisation of generative AI technologies.
Accelerating AI development to address industry challenges
As one of the Advantech Edge AI SDK software offerings, GenAI Studio addresses industry pain points, such as reducing factory operator wait times for critical information and easing the documentation workload of healthcare professionals. Its no-code, cost-effective platform streamlines LLM adoption, enabling businesses to deploy AI solutions quickly and efficiently, improving productivity and operational efficiency. Built for the Advantech Edge AI SDK, GenAI Studio leverages a versatile LLM platform with exceptional integration capabilities for both local and cloud-based LLMs, including OpenAI, Gemini, Anthropic, and Ollama. It also introduces full-parameter fine-tuning functionality, optimised for environments with limited GPU resources, enabling broader accessibility and performance enhancements.
The GenAI Studio offers a comprehensive suite of tools for LLM development:
- Integrated fine-tuning and inference services: GenAI Studio combines fine-tuning and inference capabilities to maximise hardware utilisation, enabling more flexible and efficient allocation of resources.
- Advanced GPU resource management and task scheduling: These features enable users to optimise AI hardware performance, improving the cost-effectiveness of high-value equipment.
The rapid rise of AI has underscored the need for accessible LLM solutions, but many companies are hindered by limited resources. The Advantech Edge AI SDK addresses this challenge by offering a toolset that enables efficient evaluation, development, and deployment of edge AI applications. For instance, fine-tuning a 70-billion-parameter LLM, which traditionally requires over 30 GPUs with 48GB memory each, can be achieved with just 4 GPUs using the Advantech Edge AI SDK. This represents an 87% reduction in resource requirements, dramatically lowering costs and making LLM solutions more attainable.
Empowering AI innovation with Advantech’s GenAI Studio and Edge AI server
To complement GenAI Studio, Advantech’s AIR-520 edge AI server provides a robust hardware platform equipped with NVIDIA RTX GPUs and Phison AI SSDs. This integration delivers reliable, high-efficiency computing capabilities designed to meet the demands of AI applications across industries such as manufacturing, healthcare, and retail.
Advantech Edge AI SDK: streamlining workflows from model fine-tuning to deployment
The Advantech Edge AI SDK is a fully integrated platform designed for seamless edge AI development. With pre-configured hardware and optimally tuned software, it delivers a plug-and-play experience enables cost-effective LLM customisation, seamless toolkit compatibility, and effortless management of large-scale edge deployments. Designed with reliability, scalability, and user-friendliness in mind, the Advantech Edge AI SDK simplifies the path to AI innovation. It now includes three core components:
- GenAI Studio: Facilitates cost-effective creation, evaluation, and integration of custom LLMs on-premises.
- Inference kit: Enables rapid optimisation and assessment of efficient AI runtimes compatible with embedded operating systems.
- Orchestration platform: Provides efficient management of AI model and application updates across large-scale edge deployments, integrating MLOps for streamlined operations.
Advantech is enhancing businesses’ ability to develop and deploy AI solutions more efficiently and cost-effectively through its Edge AI SDK and the newly added GenAI Studio software module.
