Deploying the Gemma 3 (4B) AI Model on the DC-ROMA RISC-V AI PC Using Ollama

Gemma 3 is Google’s next-generation open-source large language model. The 4B parameter version strikes a good balance between performance and resource consumption, making it well-suited for running on local devices. This article demonstrates how to deploy and run the Gemma 3 (4B) model on the DC-ROMA RISC-V AI PC using the Ollama platform, enabling an offline AI inference experience.

DC-ROMA RISC-V AI PC Configuration：

1、Download and Compile Ollama

Ollama is a lightweight local LLM deployment tool that supports quick loading and running of various open-source models. Since we are running on the RISC-V architecture, we need to build from source:

$ git clone https://github.com/ollama/ollama.git  #Clone source code

$ cd ollama 

$ sudo apt-get install golang
## Build with go (method-1): go version must >=1.24.0

$ go generate ./...   # A Go command used to automatically run code generation tasks before compilation

$ go build .  #BuildCode language: Shell Session (shell)

2、Start the Ollama Service and Run the Model

Ollama requires starting the service process first, then running the model in a new terminal:

$ ./ollama serve  #Start serviceCode language: Shell Session (shell)

In another terminal:

# After starting the service, open a new terminal. The first run will download the model.
$ ./ollama run gemma3   #By default downloads the 4B modelCode language: PHP (php)

3、Model Automatically Runs After Download

Once the download is complete, the model will load automatically and enter interactive mode.

After deployment, you can directly type questions or commands into the terminal, and Gemma 3 will respond in natural language.

Performance on the DC-ROMA RISC-V AI PC

On the DC-ROMA RISC-V AI PC, Gemma 3 (4B) runs smoothly. The collaborative computing between the CPU and NPU significantly enhances inference performance. At the same time, it operates entirely offline, ensuring data privacy without reliance on the cloud.

Conclusion

Using the Ollama platform, we successfully deployed the Gemma 3 (4B) model on the DC-ROMA RISC-V AI PC, realizing local deployment of an open-source AI model. Combined with the openness of the RISC-V architecture and the modular design of Framework, this not only empowers developers to freely explore AI applications but also enables them to experience a repairable, upgradeable, and sustainable hardware platform.

Follow us on LinkedIn, X, and Facebook for more updates, and visit our website to learn more about the DC-ROMA RISC-V AI PC.