How-To: Track metrics with Ragbits#
Similar to traces, Ragbits also collects metrics. These metrics offer insight into system performance, resource usage, and operational health, allowing users to monitor and optimize their workflows effectively.
Default metrics#
By default, the SDK tracks only histogram metrics for LLMs:
input_tokens
: the number of input tokens sent to the modelprompt_throughput
the time taken to process a prompt and receive a responsetoken_throughput
: the number of tokens processed per secondtime_to_first_token
: the time taken (in seconds) to receive the first token in a streaming response
Info
For now Ragbits support only histogram metrics, in the future we plan to extend the API for counter and gauge metrics.
Collecting custom metrics#
The Histogram metric is particularly useful when you want to measure the distribution of a set of values.
You can use this metric for measuring things like:
- The duration of a request.
- The size of a file.
- The number of items in a list.
To create a histogram metric, use the create_histogram
function.
from ragbits.core.audit import create_histogram, record
request_duration = create_histogram(
name="request_duration",
unit="ms",
description="Duration of requests",
)
for duration in [10, 20, 30, 40, 50]:
record(request_duration, duration)
Using OpenTelemetry meter#
To export metrics to the OpenTelemetry collector, configure the provider and exporter, and set up the OtelMetricHandler
using the set_metric_handlers
method.
from opentelemetry import metrics
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from ragbits.core.audit import set_metric_handlers
resource = Resource(attributes={SERVICE_NAME: "ragbits-example"})
metric_exporter = OTLPMetricExporter(endpoint="http://localhost:4317", insecure=True)
metric_reader = PeriodicExportingMetricReader(metric_exporter, export_interval_millis=1000)
metrics.set_meter_provider(MeterProvider(metric_readers=[metric_reader], resource=resource))
set_metric_handlers("otel")
Info
This code snippet exports metrics to the local OpenTelemetry collector running at http://localhost:4317. To visualize metrics from Ragbits, open a browser and navigate to the Grafana dashboard at http://localhost:3000.
A full example along with a detailed installation guide is available here
.