AI test automation and auto healing

In our previous blog, we explored how AI-powered quality engineering enhances DevOps and decision-making by enabling speed, automation, robustness, security, and accountability throughout the release lifecycle.

We now delve deeper into the practical side of AI integration, specifically on augmenting test automation frameworks with AI-powered auto-healing capabilities.

The automation maintenance bottleneck

Test automation is a cornerstone of modern quality engineering, helping teams keep pace with frequent application updates. However, as applications evolve, so do their object properties. Even minor UI changes can cause automation scripts to fail, leading to costly maintenance cycles and slowed development processes.

The most common culprit? Changes in object identifiers. When these properties shift, automation testers must manually update locators, often leading to delays and inconsistencies.

Auto-healing: the first step toward intelligent automation

Auto-healing in test automation addresses the leading cause of test script failure—object changes.

Auto-healing detects changes in object properties and automatically updates them, allowing scripts to run uninterrupted. Logic-based auto-healing frameworks reduce manual intervention and improve execution reliability. However, implementing this logic manually is complex and not easily scalable.

This is where artificial intelligence comes in.

Evolving from code-based to AI-driven frameworks

Enhancing auto-healing in test automation can be achieved by harnessing AI to proactively detect changes and adapt, turning errors into opportunities for continuous improvement.

At Celsior, we used a traditional code-based auto-healing solution which, although it yielded solid results, was too rigid to scale effectively. Thus, we explored AI to bolster our test automation framework for improved reliability and adaptability.

We identified three potential AI-based approaches to auto-healing:

• Using DOM object attributes
• Using object images
• Using a hybrid approach combining both attributes and images

Building the AI model using DOM properties

We began with a proof of concept (POC) using DOM properties. The first task was to establish seamless integration between the AI model and our test automation framework, ensuring that the model could be invoked dynamically during test execution.

We then focused on data collection and processing by gathering object attributes from applications across various domains and organizing them in Excel for consistency. Initially, we segmented the data by element type and labeled it for training, but the model’s low accuracy prompted a strategic shift in our approach.

To improve performance, we:

• Restructured the data, grouping by object attributes instead of element types
• Removed null-value attributes to focus on fields with meaningful data
• Re-trained the model, resulting in improved confidence scores and better precision

To enhance accuracy, we:

• Applied hyperparameter tuning techniques like grid search and random search
• Used feature selection methods to prioritize impactful attributes and eliminate noise

To integrate the AI model into our test automation framework:

• We deployed the trained model to a server and exposed it via APIs
• If an object was missing, an API call was made to the AI engine
• The engine returned the closest match and performed the operation
• It also returned updated object properties to refresh the object repository automatically

These refinements led to a model accuracy of nearly 60%, building a solid foundation that confirmed the feasibility of AI-driven auto-healing. However, we were still targeting 90% accuracy, so we turned our attention to the more promising image-based approach.

Enhancing accuracy with image-based AI

In this method, object identification is driven by analyzing images captured directly from the application under test. We introduced a wide range of image variations, such as zoom in / out, contrast adjustments, color filters, shearing, translation, and horizontal flips to simulate real-world UI changes. The model was then trained on this enriched dataset and integrated into our test automation framework.

Developing a new utility: Object Manager

We also developed an Object Manager utility to streamline object handling. This tool captures and crops object images, associates them with relevant properties, and feeds them into the AI engine for training, retraining, and continuous feedback.

Compared to the DOM-based model, this image-based approach has delivered significantly better accuracy and consistency across test executions.

Toward a unified, intelligent QE framework

Our next step is to bring together the strengths of both DOM-based and image-based AI models into a single, hybrid auto-healing framework. This unified approach will improve object identification accuracy, support a wider range of applications, and ensure compatibility with various automation tools, making the framework more robust and scalable.

Celsior goes beyond automation by delivering intelligent, AI-powered solutions that adapt to change, reduce maintenance effort, and accelerate release cycles. Our advanced framework improves test resilience and scales seamlessly across evolving tech environments, empowering you to deliver quality at speed.