AMD launches Gaia open source project to run LLM locally on any PC

Running large language models (LLMs) on personal computers is becoming increasingly popular. In response to this trend, AMD has introduced Gaia, an open source project that allows you to run LLMs on any Windows computer.

Gaia is designed to run a variety of LLMs on Windows PCs and is optimized for Ryzen AI processors, including Ryzen AI Max 395+. The application uses the open source Lemonade SDK from ONNX TurnkeyML for LLM output. Models running with Gaia can be adapted for a variety of tasks such as summarizing information and solving complex instructions.

Models running with Gaia can be adapted for a variety of tasks such as summarizing information and solving complex instructions.

Gaia functions through a Retrieval-Augmented Generation (RAG) agent that integrates the LLM with the knowledge base. This allows the LLM to provide more accurate and context-dependent answers, as well as create an interactive experience for users. Currently, RAG includes four agents:

1. Simple Prompt Completion – for model testing and evaluation.
2. Chaty – a chatbot that interacts with the user.
3. Clip – an agent that searches YouTube for information and answers questions.
4. Joker – a joke generator that adds humor to chatbot communication.

Gaia works by providing specific tasks for LLM through the Lemonade SDK and supporting them in a variety of runtime environments. It opens an LLM web service that interacts with the Gaia application through an OpenAI-compatible REST API. Gaia also extracts and processes data, vectorizes external content (e.g., from GitHub, YouTube, and text files), and stores it in a local vector index.

This means Gaia can improve user queries before they are processed by LLM, improving the accuracy and relevance of responses.

The new AI chatbot has two installers: a basic installer that runs on any Windows PC, and a “hybrid” installer optimized for Ryzen AI PCs. The hybrid installer allows Gaia to utilize Neural Processing Unit (NPU) computing and Ryzen AI’s integrated graphics to improve performance.