ISO25010

Sub-characteristic

Metrics

Functionality

· Report Coverage: Percentage of business requirements covered by available reports.

· Analytics algorithm accuracy: Accuracy of predictions or recommendations generated by BI tools.

· Data source compatibility: Number of different data sources (like SQL, NoSQL, APIs) BI can natively connect to.

· Advanced analytics support: Availability of features like predictive analytics, anomaly detection, and trend analysis.

· Ad-hoc query support: Ability for users to create and run their own queries without relying on predefined templates or reports.

OLAP capabilities: Support for Online Analytical Processing operations like slice-and-dice, drill down/up, etc.

Reliability

· Data refresh rate: Frequency of data updates in BI dashboards.

· Report generation success rate: Percentage of reports generated without errors.

· Historical data accuracy: Consistency of historical data representations over time.

· Scheduled report reliability: Percentage of scheduled reports that run and deliver as expected.

· Data transformation accuracy: The integrity of data when transformed from its raw form to a more structured or aggregated form for BI purposes.

· Alert accuracy: Accuracy of automated alerts based on certain business conditions or thresholds.

Usability

· Dashboard intuitiveness: User feedback or rating on the clarity and usefulness of BI dashboards.

· Custom report creation ease: Time and steps required for users to create custom reports.

· Visualization variety: Number of different visualization types (bar charts, pie charts, heatmaps, etc.) supported.

· Interactive capabilities: Ability of end-users to drill down, slice, or interact with reports dynamically.

· Template availability: Number of predefined report and dashboard templates available for different business scenarios.

· Guided analytics: Availability of guided or suggested analytical paths for users based on their objectives.

Efficiency

· Query response time: Average time taken to execute standard complex queries within the BI tool.

· Load scalability: How well the BI system performs as data volume or user count increases.

· Data caching efficiency: Reduction in report generation time due to caching mechanisms.

· Compression efficiency: Storage savings from data compression without sacrificing query performance.

· Data ingestion speed: Rate at which new data is ingested into the BI system.

· Load balancing efficiency: Effective distribution of computational tasks across servers or nodes for optimized performance.

Security

· Data masking and anonymization: Percentage of sensitive data fields that are masked or anonymized in reports.

· Access control granularity: Level of detail at which access rights can be specified (e.g., by report, by data field).

· Audit trail quality: Completeness and clarity of logs capturing report access, data modifications, and user activities in the BI system.

· Row-level security: Ability to restrict data access at a granular row level based on user roles or attributes.

· Field-level security: Ability to restrict access to specific fields within a dataset, not just rows or entire datasets.

· Data encryption standards: Level and methods of encryption for data at rest and in transit.