# Developing an LLM chat-client with Spring AI
This tutorial aims to provide a quick, hands-on introduction to Spring AI. Starting with [devpack-for-spring](https://www.youtube.com/watch?v=aHYwD8yTUZA), and using a locally installed large language model, the tutorial demonstrates how easy it is to create an LLM chat-client using Spring AI.
## Introduction to Spring AI
The [Spring AI](https://docs.spring.io/spring-ai/reference/index.html) project aims to streamline development of applications that incorporate artificial intelligence functionality. It is an extension to the Spring and Spring Boot frameworks. It includes abstractions for various artificial intelligence interactions and protocols that hide complexity and provide an easy programming interface for Java applications. Spring AI offers a consistent interface across AI providers and models, making your code portable.
Spring AI’s core abstractions include:
- [ChatClient API](https://docs.spring.io/spring-ai/reference/api/chatclient.html)
- [Structured output converters](https://docs.spring.io/spring-ai/reference/api/structured-output-converter.html)
- [Prompt templates](https://docs.spring.io/spring-ai/reference/api/prompt.html)
The advanced features include support for:
- [Retrieval-Augmented Generation (RAG)](https://docs.spring.io/spring-ai/reference/api/chatclient.html#_retrieval_augmented_generation)
- [Vector databases](https://docs.spring.io/spring-ai/reference/api/vectordbs.html)
- [Tool/Function calling](https://docs.spring.io/spring-ai/reference/api/tools.html)
- [Model Context Protocol (MCP)](https://docs.spring.io/spring-ai/reference/api/mcp/mcp-overview.html)
- [Observability](https://docs.spring.io/spring-ai/reference/observability/index.html)
The above feature-set makes LLM interactions effortless and other complex interactions relatively easier.
## An LLM chat-client using Spring AI
In this tutorial, Spring AI’s ChatClient API is used to create a simple LLM chat-client. The application offers a REST API to send questions to a configured LLM, and relay the answers as responses. We use Qwen 2.5 VL model installed locally using an inference snap. The `devpack-for-spring` CLI is used to initialize this application.
### Installing Qwen 2.5 VL locally
Install `qwen-vl` for the 2.5 track:
`user@host:~$ ``sudo snap install qwen-vl --channel 2.5/beta`
To check the status, use the `status` command. We revisit this in a later section.
`user@host:~$ ``qwen-vl status`
When the `status` command reports an openai endpoint, Qwen 2.5 VL is installed and the server used to perform inference with it is up and running. To learn more, refer to the [documentation](https://documentation.ubuntu.com/inference-snaps/) on inference snaps.
### Initializing the chat-client project
Install `devpack-for-spring` (skip this step when already installed):
`user@host:~$ ``sudo snap install devpack-for-spring --classic --channel latest/edge`
Install the content snap for Spring AI:
`user@host:~$ ``devpack-for-spring add-library content-for-spring-ai-20`
#### IMPORTANT
The `content-for-spring-ai-20` snap installs Spring AI 2.X, which is compatible with Spring Boot 4.0.x.
To initialize a Spring Boot and Spring AI project, use `devpack-for-spring`’s CLI wizard.
Alternatively, initialize it using this command:
`user@host:~$ ``devpack-for-spring boot start \
> --path $PWD/chat-client \
> --project gradle-project \
> --language java \
> --boot-version 4.0.8 \
> --version 0.0.1-SNAPSHOT \
> --group demo.chatclient \
> --artifact demo \
> --name chat-client \
> --description "An LLM chat-client using Spring AI" \
> --package-name demo.chatclient \
> --dependencies spring-ai-openai,web \
> --packaging jar \
> --java-version 21`
#### IMPORTANT
Note the chosen dependencies. The `spring-ai-openai` dependency supports interaction with OpenAI-compatible models. The `web` dependency supports the creation of an HTTP endpoint.
### Implementing the chat-client backend
After successful project initialization, you find the file `src/main/java/demo/chatclient/ChatClientApplication.java`. The code in this file marks the primary entry point of your application. We now add code related to the LLM chat-client application.
Let us first define the basic abstractions.
File `src/main/java/demo/chatclient/Question.java` defines the Question type.
```java
package demo.chatclient;
public record Question(String question) {}
```
File `src/main/java/demo/chatclient/Answer.java` defines the Answer type.
```java
package demo.chatclient;
public record Answer(String answer) {}
```
File `src/main/java/demo/chatclient/DemoChatClient.java` defines the basic chat-client interface.
```java
package demo.chatclient;
public interface DemoChatClient {
Answer askQuestion(Question question);
}
```
We then add a service implementing the above interface and a controller that defines the REST endpoint.
File `src/main/java/demo/chatclient/DemoChatService.java` implements the DemoChatClient interface.
```java
package demo.chatclient;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;
@Service
public class DemoChatService implements DemoChatClient {
private final ChatClient chatClient;
public DemoChatService(ChatClient.Builder chatClientBuilder) {
this.chatClient = chatClientBuilder.build();
}
@SuppressWarnings("null")
@Override
public Answer askQuestion(Question question) {
var response = chatClient.prompt()
.user(question.question())
.call()
.content();
return new Answer(response);
}
}
```
File `src/main/java/demo/chatclient/DemoChatController.java` adds the RestController to create an endpoint named `/ask`
```java
package demo.chatclient;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class DemoChatController {
private final DemoChatClient chatClient;
public DemoChatController(DemoChatClient chatClient) {
this.chatClient = chatClient;
}
@PostMapping(path = "/ask", produces = "application/json")
public Answer askQuestion(@RequestBody Question question) {
return chatClient.askQuestion(question);
}
}
```
### Adding a simple HTML frontend
Paste the following code into `src/main/resources/static/index.html`:
```html
AI Chat
Ask a Question
```
### Configuring the LLM to be used
We installed the Qwen 2.5 VL model through the `qwen-vl` inference snap. To configure Spring AI to use this model, we need to find the model-id and its openai endpoint.
`user@host:~$ ``qwen-vl status`
```text
engine: intel-gpu
endpoints:
openai: http://localhost:8326/v3
```
The openai endpoint is `http://localhost:8326/v3`.
`user@host:~$ ``curl http://localhost:8326/v3/models 2>/dev/null | jq | grep "id"`
```text
"id": "Qwen2.5-VL-3B-Instruct-ov-int4",
```
The model-id is `Qwen2.5-VL-3B-Instruct-ov-int4`.
Use the above values to define properties in `src/main/resources/application.properties`:
```properties
spring.application.name=chat-client
spring.ai.openai.base-url=http://localhost:8326/v3
spring.ai.openai.api-key=ignored
spring.ai.openai.chat.options.model=Qwen2.5-VL-3B-Instruct-ov-int4
```
#### ATTENTION
Because we are using a local inference model, we do not need to provide an API key. The `spring.ai.openai.api-key` property is mandatory to access online models.
### Launching and using the chat-client
We are now set to launch the chat-client application and run simple prompts.
Before launching the application, ensure you have `openjdk-21-jdk` installed:
`user@host:~$ ``sudo apt install openjdk-21-jdk`
Launch the application using this command:
`user@host:~$ ``./gradlew bootRun`
A successful launch produces a log message indicating the application started. No exceptions are logged.
```text
2026-06-18T21:17:05.427+05:30 INFO 977098 --- [chat-client] [ main] demo.chatclient.ChatClientApplication : Started ChatClientApplication in 1.364 seconds (process running for 1.584)
```
Open `http://localhost:8080` in a browser to use the chat-client. A sample interaction is captured below:
