Model Quality Monitoring Metrics

Model Service Quality

Monitoring Metric	Metric Definition
Reasoning Token Count	The number of Tokens consumed by the model for internal thinking, reasoning, and intermediate computation before it formally outputs an answer.
Content Token Count	The number of Tokens consumed by the final body content actually output by the model to the user. Content Token Count = Output Token Count - Reasoning Token Count.
Input Token Count	The number of Tokens consumed by the user input and the system-added context in a single request.
Output Token Count	The total number of Tokens consumed by the result returned by the model in a single request. Output Token Count = Reasoning Token Count + Content Token Count.
Total Token Count	The total number of Tokens consumed during the complete processing of a single request. Total Token Count = Input Token Count + Output Token Count.
First Token Time	The time from when the request is sent completely to when the client receives the first valid Token returned by the model. First Token Time = First valid Token arrival time - Request sent completion time.
Reasoning Time	The time from when the model starts returning the first Token to when the reasoning stage ends. It reflects the duration of the model's "thinking" stage. Reasoning Time = Last reasoning content arrival time - First Token Time. If there is only one reasoning content token, the reasoning time is 0.
Content Generation Time	The time from when the model starts outputting body content to when all body content is generated. Content Generation Time = Last body content arrival time - First body content arrival time. If there is only one body content token, the content generation time is 0.
Full Response Time	The total time from when the request is sent completely to when the model finishes returning all content. Full Response Time = Last valid Token arrival time - Request sent completion time.
Average Time per Token	After the first Token is returned, the average time consumed by the model to generate each subsequent output Token. Average Time per Token = (Full Response Time - First Token Time) / (Output Token Count - 1).
Reasoning Speed	The average number of reasoning Tokens generated per second during the reasoning stage. Reasoning Speed = Reasoning Token Count / Reasoning Time, rounded up.
Content Speed	The average number of content Tokens generated per second during the body content output stage. Content Speed = Content Token Count / Content Generation Time, rounded up.
Average Speed	The average number of Tokens generated per second during the entire response. Average Speed = Total Token Count / Full Response Time, rounded up.
Success Rate	Within the statistical time range, the proportion of model requests that successfully return valid results among all requests. Success Rate = Successful Requests / Total Requests x 100%.
HTTP Error Rate	Within the statistical time range, the proportion of requests that return HTTP error status codes among all requests. HTTP Error Rate = Requests with HTTP status code > 400 / Total Requests x 100%.
Timeout Rate	Within the statistical time range, the proportion of requests that do not receive a complete response within the specified time among all requests. Timeout Rate = Timed-out Requests / Total Requests x 100%.
Interruption Rate	Within the statistical time range, the proportion of requests that have started responding but are abnormally interrupted during the return process among all requests. Interruption Rate = Interrupted Requests / Total Requests x 100%. The following cases are also considered interruptions: 1. the first packet has been returned, but no `usage` data is received; 2. there is no `content`, only `usage` data.
Cache Hit Rate	The proportion of requests that reuse existing cached results or cached context. It is used to measure the effectiveness of the caching mechanism. Cache Hit Rate = Cache-hit Requests / Total Requests x 100%.

Model Output Quality Monitoring

Monitoring Metric	Metric Definition
Correctness	Whether the model output is consistent with the expected answer. It is used to measure whether the model answer is correct. Evaluation method: match and verify the model output against the preset standard answer or standard rule. If the output satisfies the preset answer rule, it is determined as correct; otherwise, it is determined as incorrect.
Consistency	Whether the model keeps stable and consistent results when answering multiple times under the same question and the same input conditions. It is used to measure output fluctuation. Evaluation method: ask the same question multiple times, and verify each answer against the preset answer rule. If the multiple answers are consistent and all meet expectations, they are determined as consistent; otherwise, they are determined as inconsistent.
Structured Output Accuracy	The accuracy with which the model outputs results according to the specified structure, format, or field requirements. It is used to measure the model's ability to generate structured results. Evaluation method: obtain the model's first-round structured answer, then use the first-round answer as the next-round question input for further verification, and check whether the output structure, fields, and values meet the requirements according to the preset answer rule. If both structure and content meet expectations, it is determined as passed; otherwise, it is determined as failed.

Private Protocol Metrics Overview