Best AI LLMOps & Frameworks AI Tools

NPM for LLMs

Full-stack LLMOps platform to monitor, manage, and improve LLM-based apps.

Load and run large LLMs locally to use in your terminal or build your apps.

Evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.

Route, track, and debug all LLM traffic.

Multilingual, multimodal, scalable AI tool; open-source.

An open-source framework for building production-grade LLM applications. It unifies an LLM gateway, observability, optimization, evaluations, and experimentation.

Build, deploy AI apps easily; no-code, multi-model integration.

European frontier AI models with open and closed options

Automates and monitors LLMs for quality, compliance, and performance.

The open-source LLM observability for developers.

Open-source drag-and-drop LLM app and chatbot builder

APIPark is an open-source enterprise API developer portal designed to streamline the management of large language models (LLMs) in production environments...

Scoopika is a leading open-source platform revolutionizing the development of multimodal Large Language Model (LLM) applications. It simplifies the creation of AI agents capable of interacting with te

A local Word Add-in for you to use local LLM servers in Microsoft Word. Alternative to "Copilot in Word" and completely local.

Open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications. [#opensource](https://github.com/langfuse/langfuse)

An AI platform providing a range of tools for language model fine-tuning, deployment, and usage.

Developed by Stanford, Alpaca is an instruction-following model based on LLaMA, fine-tuned for improved performance on specific tasks. Alpaca can be deployed locally, allowing researchers to work with large language models offline for greater privacy and control.

An open-weight model that competes with LLaMA, offering a balance between performance and efficiency. Mistral can run on local servers or personal hardware for high-performance, private deployment of language models.

Local LLM Red-Teaming Tool

Developer platform

API / platform

Enterprise-grade LLM for business applications

The open-source LLM observability for developers.

An open-weight model that competes with LLaMA, offering a balance between performance and efficiency. Mistral can run on local servers or personal hardware for high-performance, private deployment of language models.

The open-source LLM observability for developers.

Open Source LLM Native Mock API Test Engine

Agents-Flex is a powerful Java framework designed for large language model (LLM) applications. This lightweight and elegantly simple framework provides de...

Open Interpreter is an innovative platform that allows Large Language Models (LLMs) to execute code directly on your computer, enabling the automation and completion of various tasks. With 49,000 star

Highly optimized LLM inference engine in pure C++

Dify.ai is an open-source platform for developing large language model (LLM) applications, capable of building agents, orchestrating AI workflows, managing models, and utilizing a Retrieval-Augmented

Snowflake's new Cortex platform offers a streamlined solution for developing Large Language Model (LLM) applications with ease, speed, and security. Leveraging Snowflake's robust architecture, Cortex

Flowise is a powerful open-source low-code tool designed for developers to build customized Large Language Model (LLM) orchestration flows and AI agents. With Flowise, developers can easily extend and

Klu is an all-in-one LLM App Platform that allows users to experiment, version, and fine-tune GPT-4 Apps. It supports collaborative prompt engineering, enabling teams to explore, save, and prototype c

A framework for developing applications powered by language models.

Open-source LLMOps platform for prompt management, LLM evaluation, and observability. Build, evaluate, and monitor production-grade LLM applications. [#opensource](https://github.com/agenta-ai/agenta)