Scribd
Upload
13 views2 pages

Integrate OpenRouter with FastAPI

The document outlines the process of replacing OpenAI's paid API with OpenRouter for accessing open-source LLMs. It provides step-by-step instructions for obtaining an API key, integrating the model into a FastAPI application, and testing the setup using Postman. Additionally, it lists free model options available on OpenRouter, including Mistral, LLaMA 3, and Gemini 2 Flash.

Uploaded by

Neelam Pawar
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
13 views2 pages

Integrate OpenRouter with FastAPI

The document outlines the process of replacing OpenAI's paid API with OpenRouter for accessing open-source LLMs. It provides step-by-step instructions for obtaining an API key, integrating the model into a FastAPI application, and testing the setup using Postman. Additionally, it lists free model options available on OpenRouter, including Mistral, LLaMA 3, and Gemini 2 Flash.

Uploaded by

Neelam Pawar
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Context

You’ll replace OpenAI’s paid API with OpenRouter, which acts as a universal gateway
for open-source LLMs like Mistral, LLaMA 3, or Gemini Pro Free. This means you can
call:

[Link]

and just change the model name.

1. Get a Free OpenRouter API Key


1. Go to [Link]
2. Sign in with your Google/GitHub account
3. Go to Settings → API Keys → Create Key
4. Copy the key (it begins with sk-or-... )

Then create a .env file in your project:

OPENROUTER_API_KEY=sk-or-your-key-here

2. Replace the Model Integration


Instead of using:

from openai import OpenAI


client = OpenAI(api_key=...)

you’ll make a POST request manually to OpenRouter’s endpoint using requests or


httpx — both are free and simple.

Endpoint

POST [Link]

Headers

Authorization: Bearer <OPENROUTER_API_KEY>


Content-Type: application/json
HTTP-Referer: [Link] (or your domain)
X-Title: AI Knowledge Assistant

Body

{
"model": "mistralai/mistral-7b-instruct",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain FastAPI in simple words"}
]
}
3. FastAPI Integration Flow (Concept)
Your /ask endpoint will:

1. Accept JSON input ( topic , question )


2. Combine them into a single prompt
3. Send it to OpenRouter using [Link]()
4. Parse the model’s choices[0].[Link]
5. Return the answer to the

client So your logic flow becomes:

FastAPI receives request


↓
Build payload for OpenRouter
↓
POST → [Link]
↓
Return model response as JSON

4. Testing via Postman


You can call your FastAPI /ask route exactly as before:

{
"topic": "Mistral",
"question": "How is Mistral different from GPT?"
}

And under the hood, your backend will reach out to OpenRouter’s free Mistral endpoint.

5. Free Model Options on OpenRouter


Model Use Case Identifier

Mistral 7B Instruct Fast, light Q&A mistralai/mistral-7b-instruct

meta-llama/llama-3-8b-
LLaMA 3 8B Instruct Balanced reasoning
instruct

Gemini 2 Flash (via Conversational, free


google/gemini-flash-1.5
Google) tier

All of these work with the same /chat/completions API.

You might also like