European Sovereign AI in Two Lines of Code

The OpenAI-compatible API format means you are not locked in to any single inference provider. Switching is a configuration change, not an engineering project. So the question becomes: where should your requests go?

If your workload has data residency requirements under EU law, or if you are building for organizations that need their inference stack under European jurisdiction, the migration cost to get there is two lines of code by switching to evroc Think.

Introducing evroc Think

For teams deploying AI workloads, the same requirements that drive cloud decisions, sovereignty, security, control, and European jurisdiction, now apply to inference and models.

evroc Think is built to meet those requirements, running on infrastructure that is entirely European-owned and operated.

Two hosting options are available:

• Shared models - You pick a model from the catalog and start sending requests, with no deployment or scaling on your side.

• Dedicated model instances - Models deployed by you onto evroc infrastructure where you control the model, instance size, and compute allocation.

Both options expose the same OpenAI-compatible API. Your code does not change between them and switching between them is done by changing the model argument.

Through evroc Think you have access to some of the latest open source models. To get the up-to-date model listings, you can use the CLI command: evroc think sharedmodel list.

API keys are managed through the evroc CLI. The full API spec is published at the evroc Think docs.

Why the two-line switch works

The /v1/chat/completions endpoint format is widely adopted across LLM inference providers. The request shape (messages array with role/content), the response shape (choices with message content), streaming via SSE, tool calling schemas, and structured output via response_format are all part of this shared protocol.

evroc Think implements this format natively. The API exposes:

• POST /v1/chat/completions: chat, streaming, tool calling, vision input
• POST /v1/embeddings: text embeddings
• POST /v1/audio/transcriptions: speech-to-text
• GET /v1/models: list available models

The base URL is https://models.think.evroc.com/v1. Auth is a Bearer token.

Your existing client library works unchanged. You point it at a different URL and pick a different model name.

evroc Think and LangChain example

If you have a LangChain app using ChatOpenAI today, these are the only changes:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://models.think.evroc.com/v1",   # line 1: point to evroc
    model="zai-org/GLM-5.2",                         # line 2: pick a Think Model
    api_key=os.environ["EVROC_API_KEY"],
)

Everything else in your application, prompts, chains, output parsers, message history, tool definitions, stays identical.

What you need before you start

An evroc account and a Think API key. Create one with:

evroc think apikey create my-app-key

The key is shown once. Save it to an environment variable:

export EVROC_API_KEY="<your-key>"

To see the full list of available shared models (managed by evroc, no deployment needed):

evroc think sharedmodel list

Use any model from the catalog as the model argument. The model namespace follows the provider/model-name convention, not a flat string.

Full LangChain walkthrough

The patterns most LangChain apps use: basic chat, streaming, prompt chains, conversational memory, structured output, and tool calling. Every example uses the same ChatOpenAI instance with the two-line switch applied. If your app uses any of these patterns, it already works with Think Models.

Install dependencies

pip install langchain langchain-openai langchain-community

Configure the model

import os
from langchain_openai import ChatOpenAI

os.environ.setdefault("EVROC_API_KEY", "<your-key>")

llm = ChatOpenAI(
    base_url="https://models.think.evroc.com/v1",
    model="zai-org/GLM-5.2",
    api_key=os.environ["EVROC_API_KEY"],
)

Basic chat

from langchain_core.messages import HumanMessage, SystemMessage

response = llm.invoke([
    SystemMessage(content="You are a concise assistant."),
    HumanMessage(content="Explain sovereign AI in one sentence."),
])
print(response.content)

Streaming

evroc Think Models supports SSE streaming. Tokens arrive as they are generated.

for chunk in llm.stream("Write a haiku about European cloud infrastructure."):
    print(chunk.content, end="", flush=True)
print()

Chain with prompt template

The canonical LangChain pattern: prompt template, model, output parser.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a senior engineer. Answer in under 50 words."),
    ("user", "{input}"),
])

chain = prompt | llm | StrOutputParser()

result = chain.invoke({"input": "What are the benefits of running inference on sovereign infrastructure?"})
print(result)

Conversational memory

Multi-turn chat with retained context using RunnableWithMessageHistory.

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_community.chat_message_histories import ChatMessageHistory

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    MessagesPlaceholder(variable_name="history"),
    ("user", "{input}"),
])

chain = prompt | llm

store = {}

def get_history(session_id: str):
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]

chat = RunnableWithMessageHistory(
    chain, get_history,
    input_messages_key="input",
    history_messages_key="history",
)

print(chat.invoke({"input": "My name is Korey."}, {"configurable": {"session_id": "s1"}}).content)
print()
print(chat.invoke({"input": "What's my name?"}, {"configurable": {"session_id": "s1"}}).content)

Structured output

Ask the model to return JSON, then parse it into a Pydantic model. This approach works with any OpenAI-compatible model, regardless of native structured-output support.

One thing to watch: ChatOpenAI.with_structured_output(ModelClass) can produce unparseable output with some models on the evroc Think API. The reliable approach is manual JSON prompting. Instruct the model to respond only with a JSON object in the system prompt, pipe through StrOutputParser, then json.loads() and validate with Pydantic.

import json
from pydantic import BaseModel
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

class CloudService(BaseModel):
    name: str
    description: str
    sovereign: bool

structured_prompt = ChatPromptTemplate.from_messages([
    ("system", "Respond ONLY with a JSON object matching this schema: "
               '{"name": str, "description": str, "sovereign": bool}. '
               "No markdown, no explanation, just the JSON object."),
    ("user", "{input}"),
])

structured_chain = structured_prompt | llm | StrOutputParser()

raw = structured_chain.invoke({"input": "Describe a sovereign object storage service."})
data = json.loads(raw)
result = CloudService(**data)

print(f"Name:        {result.name}")
print(f"Description: {result.description}")
print(f"Sovereign:   {result.sovereign}")

Tool calling

evroc Think Models supports OpenAI-style tool calling. Define a tool and let the model invoke it.

from langchain_core.tools import tool

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return f"The weather in {location} is 18°C and cloudy."

llm_with_tools = llm.bind_tools([get_weather])

response = llm_with_tools.invoke("What's the weather in Stockholm?")

if response.tool_calls:
    for call in response.tool_calls:
        print(f"Tool:   {call['name']}")
        print(f"Args:   {call['args']}")
        result = get_weather.invoke(call["args"])
        print(f"Result: {result}")
else:
    print(response.content)

What doesn't change

Here is what stays identical when you switch to Think Models:

• Prompts: your system prompts, user prompts, and template variables
• Chains: LCEL chains (prompt | llm | parser)
• Output parsers: StrOutputParser, JsonOutputParser, PydanticOutputParser
• Message history: RunnableWithMessageHistory, ChatMessageHistory, in-memory or persistent stores
• Tool calling: bind_tools(), tool schemas, tool execution loops
• Streaming: stream() and astream() return chunks via SSE
• Structured output: response_format and manual JSON prompting both work
• Embeddings: OpenAIEmbeddings with base_url pointed at Think Models

The only thing that changes is where the request goes and which model answers it.

Get started

For the first 100 signups, we are offering €300 credits to help you get a started building on evroc and using evroc Think.

To claim your credits, sign up here.

For more information about evroc Think, see the evroc Think docs.