Getting Started with Quarkus and Langchain4j

Quarkus is a lightweight, cloud-native Java framework optimized for GraalVM and Kubernetes. When combined with Langchain4j, it allows developers to integrate AI capabilities into Java applications efficiently. In this tutorial, we’ll explore how to use Langchain4j with Ollama’s Llama model, creating a simple Command-line application.

Understanding the AI Model

We will use the following models in our application:

• Llama3.2 (Chat Model): A highly capable Large Language Model (LLM) designed for natural language conversations and reasoning.
• Embedding Model 3b: A model specialized in generating embeddings, useful for semantic search and text similarity tasks.

Quarkus’ configuration for these models is defined in application.properties:

quarkus.langchain4j.ollama.chat-model.model-id=llama3.2
quarkus.langchain4j.ollama.embedding-model.model-id=3b

To run this tutorial on your laptop, simply download and install Ollama Chat Model. On the other hand, if you prefer using an OpenAPI key and another model like ChatGPT, just update your application.properties accordingly:

quarkus.langchain4j.openai.api-key=your-openai-api-key
quarkus.langchain4j.openai.chat-model.model-id=gpt-4
quarkus.langchain4j.openai.embedding-model.model-id=text-embedding-ada-002

Now, let’s build our Quarkus project and integrate these models.

---

Step 1: Setting Up the Quarkus Project

First, generate a new Quarkus project using any tool like the Online initializer. If you are new to Quarkus, check this article: Getting started with Quarkus 3

Then, include the following dependencies:

<dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-ollama</artifactId>
    <version>0.23.3</version>
</dependency>

<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-arc</artifactId>
</dependency>

Finally, navigate into the project directory and open the project with the editor of your choice:

cd quarkus-langchain4j-demo

---

Step 2: Creating a Command-Line AI Chatbot

We’ll now implement a Quarkus CLI chatbot that interacts with the AI model.

Define the AI Service

Create a new class OllamaAiService.java:

import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.V;
import io.quarkiverse.langchain4j.RegisterAiService;
import jakarta.enterprise.context.ApplicationScoped;

@ApplicationScoped
@RegisterAiService
public interface OllamaAi {
    
    @SystemMessage("Answer concisely.")
    @UserMessage("{{question}}")
    String answerQuestion(@V("question") String question);
}

Here is a description of the key components of this interface:

1. @RegisterAiService (Quarkus-specific)

• This tells Quarkus to automatically register this interface as an AI service.
• Quarkus will generate an implementation that interacts with a configured AI model (e.g., Ollama).

1. @SystemMessage("Answer concisely.") (dev.langchain4j.service.SystemMessage)

• Represents an instruction to the AI model that sets the response behavior.
• The AI will be instructed to answer concisely.
• Think of this as a system-level prompt that affects the conversation globally.

1. @UserMessage("{{question}}") (dev.langchain4j.service.UserMessage)

• Defines the user input format.
• The placeholder {{question}} will be replaced with the actual input.
• LangChain4j will construct a request where this message represents the user’s question.

1. @V("question") String question (dev.langchain4j.service.V)

• This annotation binds the method parameter (question) to the placeholder {{question}} in @UserMessage.
• It tells LangChain4j to inject the method argument into the prompt dynamically.

Create the Quarkus CLI Application

Finally, create the command-line application in OllamaCliApp.java:

import io.quarkus.runtime.QuarkusApplication;
import io.quarkus.runtime.annotations.QuarkusMain;
import jakarta.enterprise.context.control.ActivateRequestContext;
import jakarta.inject.Inject;
import java.util.Scanner;

@QuarkusMain
public class OllamaCliApp implements QuarkusApplication {
    @Inject
    OllamaAiService ollamaAiService;

    @Override
    @ActivateRequestContext
    public int run(String... args) {
        Scanner scanner = new Scanner(System.in);
        System.out.println("🔮 Ask Ollama AI anything (type 'exit' to quit):");

        while (true) {
            System.out.print("> ");
            String question = scanner.nextLine();

            if ("exit".equalsIgnoreCase(question)) {
                System.out.println("👋 Exiting...");
                break;
            }

            String response = ollamaAiService.askQuestion(question);
            System.out.println("🤖 Ollama AI: " + response);
        }

        scanner.close();
        return 0;
    }
}

Finally, build and run the CLI chatbot with:

java -jar target/quarkus-app/quarkus-run.jar 

Try asking a question, and the AI model will respond!

Conclusion

This article showed how to use a Language Model with Quarkus through a simple Command Line Application.

You can find the full source code for this article here: https://github.com/fmarchioni/mastertheboss/tree/master/quarkus/quarkusai