A high-throughput and memory-efficient inference and serving engine for LLMs
-
Updated
Jun 11, 2024 - Python
A high-throughput and memory-efficient inference and serving engine for LLMs
AI Observability & Evaluation
🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
Open-Source Evaluation for GenAI Application Pipelines
A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap
🤖 𝗟𝗲𝗮𝗿𝗻 for 𝗳𝗿𝗲𝗲 how to 𝗯𝘂𝗶𝗹𝗱 an end-to-end 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻-𝗿𝗲𝗮𝗱𝘆 𝗟𝗟𝗠 & 𝗥𝗔𝗚 𝘀𝘆𝘀𝘁𝗲𝗺 using 𝗟𝗟𝗠𝗢𝗽𝘀 best practices: ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 11 𝘩𝘢𝘯𝘥𝘴-𝘰𝘯 𝘭𝘦𝘴𝘴𝘰𝘯𝘴
Python SDK for running evaluations on LLM generated responses
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
ZenML 🙏: Build portable, production-ready MLOps pipelines. https://zenml.io.
No-code multi-agent framework to build LLM Agents, workflows and applications with your data
Run any open-source LLMs, such as Llama 2, Mistral, as OpenAI compatible API endpoint in the cloud.
🪢 Langfuse JS/TS SDKs - Instrument your LLM app and get detailed tracing/observability. Works with any LLM or framework
RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry
The open source Tines alternative. Automate security workflows at scale with code and no-code.
Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
A python library to enable GenAI and LLMOps within Google Cloud Platform
Add a description, image, and links to the llmops topic page so that developers can more easily learn about it.
To associate your repository with the llmops topic, visit your repo's landing page and select "manage topics."