Design and Validation of a Quality Assurance Framework for AI Educational Technology in Low-Resource Settings

Abstract

The diffusion of artificial intelligence-enabled educational technology in low- and middle-income countries has outpaced the development of context-appropriate quality assurance mechanisms, leaving procurement agencies without evidence-based instruments to evaluate the pedagogical, technical, governance, and equity dimensions of such products. This study develops, validates, and pilot-tests the Quality Assurance Framework for Artificial Intelligence Enabled Educational Technology (hereafter, AEQAF) through Design Science Research grounded in responsible artificial intelligence theory, critical educational technology scholarship, and decolonial perspectives on technology in the Global South. The framework was constructed across three iterative cycles over twenty-four months through four phases: landscape analysis of 147 products across 23 markets; co-design workshops with 68 stakeholders in five venues; a three-round modified Delphi with 24 experts across 14 countries; and pilot application on 18 products in Nigeria, Ghana, and Kenya using three trained evaluators. The AEQAF comprises six dimensions operationalised through 42 indicators. Delphi consensus at the 75% threshold was achieved for 39 of 42 indicators. The pilot application demonstrated substantial interrater reliability (Fleiss kappa = 0.81). A non-circular discriminant validity test using five dimensions yielded a Cohen's d of 1.42, supporting the framework's capacity to differentiate product quality. The framework reveals systematically low scores for data governance, equity, and evidence dimensions. The AEQAF is ready for multi-country validation and provides an openly licensed instrument bridging technical and educational evaluation for resource-constrained systems.