The power of local AI: Experiment with every LLM via Open WebUI & OpenRouter.ai

Imagine: you run advanced AI chat software locally or on your own VPS, have full control over your data and access to virtually every existing LLM (Large Language Model) through one powerful API key. That is exactly what the combination of Open WebUI and OpenRouter.ai makes possible. And the best part? It is completely open source and freely available to anyone who dares to build and experiment.

🔓 Open source AI – the new normal?

Open WebUI is an intuitive chat interface for local AI models such as LLaMA, Mistral or Phi-2. Thanks to the integration with OpenRouter.ai, you open the door in one go to commercial and open source models from OpenAI, Anthropic, Mistral, Meta, Google and dozens of smaller labs, among others. A simple API key is enough to switch between models. That means:

• Experimenting with LLMs to suit your needs: from ultra-light models for devices to top models such as GPT-4o / 4.1 and 4.5 or Claude 3.5 / 3.7.
• Cost-conscious testing before you commit to a model or vendor.
• Developing practical use cases in your own secure environment.

🔍 Explanation

• Open WebUI: A self-hosted, extensible AI interface that runs locally or on your own VPS. It supports multiple LLM runners and can be extended with pipelines for extra functionality.
• OpenRouter.ai: Offers a unified API that gives you access to various LLMs with a single API key. It optimises model routing and pricing.
• Various LLMs: Depending on your needs, you can choose from different models, each with their own specialisms and advantages.

🧠 Feedback Companion – one of many possibilities

I developed the Feedback Companion myself (www.feedbackcompanion.eu), an AI tool for education that gives students formative feedback on work they upload. Thanks to Open WebUI and OpenRouter.ai, I can now easily test with different models how they generate, structure or deepen this feedback – without vendor lock-in. That way I quickly see which model best suits a specific context or target group.

And that is just one example. Think also of:

• AI tools for career advice or intake interviews
• Local knowledge assistants with your own documents
• AI chatbots within an intranet without cloud dependency

💼 For education & business: take control!

I strongly believe that both educational institutions and businesses should take control of their own AI infrastructure. By combining open source software with powerful APIs, you can become self-sufficient with limited resources and at the same time use the latest technologies.

That is why – on behalf of Mariany Design, Mariany Hosting and Education Oracle – I support institutions and organisations with:

• Hosting their own AI environment (e.g. via a VPS or local server)
• Designing and building AI tools, dashboards or web apps
• Training and coaching teams in using AI within their own workflow

🎥 Knowledge clips on Open WebUI and connecting OpenRoute.AI

Before the summer, two or three knowledge clips will be released covering the installation and configuration of Open WebUI. They will also show how to connect OpenRouter.AI and how to access all the LLMs within Open WebUI. Stay tuned! 😎👌

🚀 Ready to build yourself?

Would you like to know more about these tools or get started with your own AI environment? Then visit www.onderwijsorakel.nl or get in touch with me via LinkedIn.

Together we'll explore how you can use AI smartly and safely for your goals.