Model service quality monitoring calls the specified model service API from monitoring Probes, collects response status and performance data during the call, and aggregates and analyzes the data. Common metrics include Success Rate, First Token Time, Full Response Time, Average Time per Token, HTTP Error Rate, Timeout Rate, Interruption Rate, repeated-call stability, and Cache Hit Rate.This monitoring type is suitable for continuously observing the service performance of model service providers, model vendors, and model versions across different times and different monitoring Probes. Users can use model service quality monitoring to understand whether the model API is accessible, whether responses are timely, whether the call process is stable, and whether issues such as timeouts, interruptions, or error responses occur.
Model output quality monitoring continuously monitors the quality of content returned by a model. The system calls the specified model API according to the task configuration, collects the model's returned content and call result, and analyzes whether the model output is normal, complete, and meets expectations based on the quality evaluation rules configured on the platform.Compared with model service quality monitoring, model output quality monitoring focuses not only on whether the API call succeeds and how long the response takes, but also on whether the returned model content itself is usable. This monitoring type helps users identify issues such as whether the model answers as expected, whether the format is abnormal, whether the output is unstable, or whether the result does not meet business requirements. It is suitable for business scenarios that require continuous validation of model output stability, such as Q&A, summarization, classification, and content generation.
