A META-ANALYSIS OF ARTIFICIAL INTELLIGENCE-DRIVEN DATA ENGINEERING: EVALUATING THE EFFECTIVENESS OF CLOUD-BASED INTEGRATION MODELS

Abstract

This study conducts a comprehensive meta-analysis to evaluate the effectiveness of artificial intelligence (AI)-driven data engineering approaches within cloud-based integration models. Drawing from 122 peer-reviewed studies published between 2015 and 2025—with a combined citation count exceeding 25,000—this research synthesizes empirical findings on how AI techniques such as machine learning, deep learning, reinforcement learning, and natural language processing are transforming core data engineering functions. The analysis focuses on performance outcomes related to data ingestion, transformation, orchestration, and quality assurance across leading cloud platforms including AWS Glue, Azure Data Factory, and Google Cloud Dataflow. Findings reveal that AI integration significantly improves ingestion latency, schema adaptability, and throughput by automating real-time stream handling and multi-source harmonization. In data transformation workflows, AI models enhance feature extraction, reduce redundancy, and facilitate semantic alignment in high-dimensional and unstructured data. AI-enabled orchestration further supports adaptive scheduling, failure recovery, and self-healing pipelines, resulting in increased operational resilience and resource efficiency. Azure Data Factory offering robust hybrid integration and compliance support, and AWS Glue leading in data lake environments. The results affirm that AI is no longer a supplemental feature but a foundational element in building scalable, intelligent, and autonomous data engineering infrastructures. This study contributes to the growing body of literature by offering evidence-based insights and platform-level comparisons that inform strategic decisions for enterprises adopting AI-driven cloud data solutions.