Analytics
LILT Analytics Overview
LILT Analytics provides comprehensive insights into the performance and accuracy of AI translations over time. This article will guide you through the various metrics and visualizations available in LILT Analytics.
Overview Page
AI Accuracy
AI Accuracy provides insights into the accuracy of AI suggestions across 3 models and workflows: Unadapted AI (pure MT) and Adapted AI (LILT Contextual AI model with Data Sources).
Unadapted AI (Green): Represents the raw AI translation accuracy without any modifications.
Adapted AI (Orange): Shows the AI translation accuracy after adjustments and improvements from Fine Tuned Data Sources.
AI Accuracy Per Language Pair
The "AI Accuracy Per Language Pair" table provides detailed accuracy metrics for different language pairs over several months. This table helps identify which language pairs have higher or lower accuracy and track their performance trends.
Quality
Note: Updated 9/5
There are two versions of Quality metrics that you will see under Analytics. One is for customers with a Customer Review or LQA step, and one without. Without customer review, we are not able to provide more granular error breakdown and quality metrics. Please contact your LILT representative if you’d like access to Customer Review or LQA and hence the more granular view.
Metrics without LQA or Customer Review
Errors Resolved / 1k words
This metric shows the number of errors per 1000 words that were found and resolved by a LILT reviewer after an initial LILT AI suggestion and translator pass through.
Errors Resolved / 1k change
This metric shows the numerical difference in number of errors per 1000 words that were resolved between the last month and the current month.
Most Common Error Type Resolved
This metric shows the most common error type that was found and resolved during the LILT review step based on the MQM framework.
Top 5 Error Types Resolved
This chart shows the 5 most common error types that were found and resolved during the LILT review step along with their corresponding percentage of appearances - the denominator of this percentage composes of only the errors found.
View with LQA or Customer Review
Average Quality Score
The average quality score is based on the MQM quality framework and showcases the resulting scores here for both overall and MoM for individual language pairs.
Errors Resolved / 1k Change
This metric shows the numerical difference in number of errors per 1000 words that were resolved between the last month and the current month.
Most Common Error Type Resolved
This metric shows the most common error type that was found and resolved during the LILT review step based on the MQM framework.
Errors per 1k words + Model Size
Errors Per 1k Words
The "Errors Per 1k Words Over Time" table categorizes errors into major, neutral, and minor types, helping to understand the common issues in translations. This breakdown is based on the MQM framework.
Model Size Over Time
The "Model Size Over Time" chart tracks the growth of the LILT AI model in terms of the number of words from January 2024 to June 2024. This metric shows how the AI model expands its knowledge base to improve translation accuracy.
Connector Jobs
The "Connector Jobs" table provides insights into the performance of various connectors, including their AI accuracy, volume, and submission errors. This information helps identify which connectors perform well and which may need improvements.
On-Time Delivery (OTD) and Words Per Hour (WPH)
OTD Over Time: Tracks the on-time delivery performance of translations over several months.
WPH Over Time: Monitors the words processed per hour, indicating the efficiency of the translation process.
OTD and WPH metrics
OTD Per Language Pair
The "OTD Per Language Pair" table provides a breakdown of on-time delivery performance for different language pairs.
Project Stats
The Project Stats Report allows you to see active projects. You will have an overview of the number of jobs that are in each state (i.e. Ready to Start, In Progress, In Review, Customer Review, and Completed). Under the Job Stats tab, you will be able to see further details regarding individual projects that you’ve sent to LILT for Translation. Stats such as State, Created date, Due date and Delivery Date help you and your localization teams manage your day-to-day tracking of projects directly from our platform. You will be able to filter by time (“From” and “To”), Source language, or Target language.
Conclusion
LILT Analytics offers a detailed view of translation accuracy, quality, and efficiency. By monitoring these metrics, users can identify areas for improvement, track progress over time, and ensure high-quality translations across various language pairs and connectors.