ISO25010 Sub-characteristic | Metrics |
Functionality | · Report Coverage: Percentage of business requirements covered by available reports. · Analytics algorithm accuracy: Accuracy of predictions or recommendations generated by BI tools. · Data source compatibility: Number of different data sources (like SQL, NoSQL, APIs) BI can natively connect to. · Advanced analytics support: Availability of features like predictive analytics, anomaly detection, and trend analysis. · Ad-hoc query support: Ability for users to create and run their own queries without relying on predefined templates or reports. OLAP capabilities: Support for Online Analytical Processing operations like slice-and-dice, drill down/up, etc. |
Reliability | · Data refresh rate: Frequency of data updates in BI dashboards. · Report generation success rate: Percentage of reports generated without errors. · Historical data accuracy: Consistency of historical data representations over time. · Scheduled report reliability: Percentage of scheduled reports that run and deliver as expected. · Data transformation accuracy: The integrity of data when transformed from its raw form to a more structured or aggregated form for BI purposes. · Alert accuracy: Accuracy of automated alerts based on certain business conditions or thresholds. |
Usability | · Dashboard intuitiveness: User feedback or rating on the clarity and usefulness of BI dashboards. · Custom report creation ease: Time and steps required for users to create custom reports. · Visualization variety: Number of different visualization types (bar charts, pie charts, heatmaps, etc.) supported. · Interactive capabilities: Ability of end-users to drill down, slice, or interact with reports dynamically. · Template availability: Number of predefined report and dashboard templates available for different business scenarios. · Guided analytics: Availability of guided or suggested analytical paths for users based on their objectives. |
Efficiency | · Query response time: Average time taken to execute standard complex queries within the BI tool. · Load scalability: How well the BI system performs as data volume or user count increases. · Data caching efficiency: Reduction in report generation time due to caching mechanisms. · Compression efficiency: Storage savings from data compression without sacrificing query performance. · Data ingestion speed: Rate at which new data is ingested into the BI system. · Load balancing efficiency: Effective distribution of computational tasks across servers or nodes for optimized performance. |
Security | · Data masking and anonymization: Percentage of sensitive data fields that are masked or anonymized in reports. · Access control granularity: Level of detail at which access rights can be specified (e.g., by report, by data field). · Audit trail quality: Completeness and clarity of logs capturing report access, data modifications, and user activities in the BI system. · Row-level security: Ability to restrict data access at a granular row level based on user roles or attributes. · Field-level security: Ability to restrict access to specific fields within a dataset, not just rows or entire datasets. · Data encryption standards: Level and methods of encryption for data at rest and in transit. |