Langfuse

Overview

Langfuse is an open-source platform for engineering teams building with large language models (LLMs). Its primary function is to provide observability, evaluation, and prompt management capabilities for LLM-powered applications. The platform enables developers to monitor the behavior of their LLM integrations in production, offering insights into latency, token usage, and error rates across complex chains and individual model calls. This visibility is designed to assist in debugging and performance optimization.

For debugging, Langfuse allows users to trace the execution flow of LLM applications, from user input through various model calls, tool uses, and retrieval steps. This tracing functionality helps identify bottlenecks or unexpected behaviors within multi-step LLM workflows. The platform also supports both human and automated evaluation of LLM outputs, which is critical for measuring model quality and tracking improvements over time. Developers can define evaluation criteria and run tests against different prompt versions or model configurations to determine which perform best for specific use cases.

Langfuse is suited for developers and technical buyers who are building and deploying LLM applications and require tools to ensure reliability, performance, and quality. It is particularly relevant for scenarios involving agents, RAG (Retrieval Augmented Generation) systems, and multi-turn conversational interfaces where understanding the internal state and outputs of the LLM is complex. The platform's prompt management features facilitate versioning and A/B testing of prompts, allowing teams to iterate on their LLM interactions more effectively. Langfuse offers SDKs for Python and TypeScript/JavaScript, integrating with common LLM frameworks such as LlamaIndex and LangChain.

The platform's focus on end-to-end visibility aims to bridge the gap between development and production for LLM applications. By providing structured data on LLM interactions, Langfuse assists in identifying regressions, optimizing costs associated with token usage, and systematically improving the user experience of AI-driven features. Its open-source nature allows for self-hosting, while a managed cloud offering provides a hosted solution with compliance certifications like SOC 2 Type II.

Key features

• LLM Observability: Provides real-time monitoring of LLM application performance, including latency, token usage, and error rates across individual calls and complex chains.
• Production Tracing and Debugging: Offers detailed traces of LLM application execution, allowing developers to inspect inputs, outputs, and intermediate steps for debugging and performance analysis.
• Human and Automated Evaluations: Supports defining evaluation criteria and running tests to assess the quality of LLM outputs, facilitating both human-in-the-loop and programmatic evaluations.
• Prompt and Model Iteration: Enables version control and experimentation with different prompts and models, including A/B testing capabilities, to optimize LLM performance.
• Prompt Management: Centralized management of prompts and configurations, allowing teams to store, version, and deploy prompts consistently across applications.
• SDKs for Python and TypeScript: Provides client libraries for integration with common programming languages and LLM frameworks.
• Open-Source Core: The core platform is open-source, allowing for self-hosting and community contributions, while a managed cloud service is also available.
• Compliance: Adheres to data security and privacy standards, including SOC 2 Type II and GDPR compliance.

Pricing

Langfuse offers a tiered pricing model, including a free developer plan and paid options for higher usage and advanced features. Pricing is primarily based on the number of observations (traces, generations, spans, scores, events) per month.

Pricing as of 2026-05-06. For detailed and up-to-date pricing information, refer to the Langfuse pricing page.

Common integrations

• LangChain: Integration for tracing and evaluating LangChain applications. LangChain Integration Guide
• LlamaIndex: Support for observing and evaluating LlamaIndex applications. LlamaIndex Integration Guide
• OpenAI: Direct integration for tracing calls to OpenAI models. OpenAI Integration Guide
• Anthropic: Compatibility for tracing interactions with Anthropic models. Anthropic Integration Guide
• Hugging Face: Integrates with Hugging Face models for observability. Hugging Face Integration Guide
• FastAPI: Middleware for integrating Langfuse tracing into FastAPI applications. FastAPI Integration Guide

Alternatives

• Helicone: An open-source observability platform focused on caching, logging, and monitoring LLM API calls.
• Vellum: Offers a platform for prompt engineering, deployment, and monitoring of LLM applications, including evaluation tools.
• Arize AI: An ML observability platform that provides monitoring, root cause analysis, and explainability for ML models, including LLMs.
• Cohere Evaluations: Cohere provides tools for evaluating the quality and performance of its own models, offering a more integrated approach for users of their API.
• Google Cloud Vertex AI: Offers MLOps tools, including model monitoring and evaluation capabilities for custom and generative AI models deployed on Google Cloud.

Getting started

To begin using Langfuse, you typically install the SDK and configure it with your project keys. The following Python example demonstrates how to initialize Langfuse and trace a simple LLM call.

from langfuse import Langfuse
from openai import OpenAI

# Initialize Langfuse with your public and secret keys
# Replace with your actual keys or set them as environment variables
langfuse = Langfuse(
    public_key="pk-lf-...", 
    secret_key="sk-lf-...",
    host="https://cloud.langfuse.com" # Optional, defaults to cloud.langfuse.com
)

# Initialize OpenAI client
client = OpenAI()

# Create a Langfuse trace
trace = langfuse.trace(name="my-first-trace", user_id="user-abc")

# Use the trace context for an LLM call
# The 'trace_id' and 'span_id' are automatically propagated
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."}, 
        {"role": "user", "content": "What is the capital of France?"}
    ],
    langfuse_options={"trace_id": trace.id, "span_id": trace.id}
)

print(response.choices[0].message.content)

# Ensure all data is sent to Langfuse before exiting
langfuse.flush()

This example demonstrates the basic setup for tracing an OpenAI API call. The langfuse_options parameter is used to link the OpenAI call to the active Langfuse trace. For more detailed setup and advanced usage, including evaluation and prompt management, consult the Langfuse documentation.