Expert Insights

As Robotic Process Automation (RPA)/Artificial Intelligence (AI) technology continues to progress both in capability and scope, businesses are expanding their use beyond main frame/legacy/desktop/web applications into the realm of Citrix/virtualized environments in an effort to offset costs while increasing productivity.

However, every RPA Developer who has created an automation in a virtualized environment knows the pain that comes with “surface automation”. Before delving into the details, let’s give a brief overview of how RPA works and why integrating with virtual applications can be so time-consuming. By providing a real-world example of how we overcame these challenges to provide a robust and sophisticated automation, we hope to give a clearer picture of how to approach these types of automations.

How Does RPA Really Work?

RPA Platforms automate repetitive work by uniquely identifying each part of a task or action a user performs at the Graphical User Interface (or simply GUI) stage. Many of the leading RPA platforms in the market today are based on the .net framework which allows for flexible platform/application support.

To put it simply, RPA platforms use object IDs to identify the elements inside the target applications and then perform various actions (keystrokes, hotkeys, HTML extraction) to get the desired end result. Most RPA tools utilize the operating system’s accessibility API or HTML document models to achieve this.

Okay…So What’s so Hard About Citrix?

Again, striving for simplicity, all the controls that RPA is able to use to manipulate the UI via Object IDs are not available in Citrix or any type of virtualized environment. Essentially, the bot is looking at a “picture”. Thus, every interaction the bot has will have to be based off of image recognition. While this may sound simplistic to a human, many RPA platforms are unable to accommodate this type of functionality. This is where Artificial Intelligence comes in.

Coordinated use of anchors, X/Y coordinates and OCR are required in order to correctly locate and interact with the image. Even with these safeguards in place, there are additional control criteria that need to be implemented for system latency. However, as these platforms begin to mature and “computer vision” technology becomes more sophisticated the error rates of these automations continue to decrease.

Realistically, if you wanted to avoid surface automation you can install the RPA software on the server where the thin-client application is housed, however, this may not always be an option. Computer vision is concerned with the automation extraction, analysis and understanding of information from images. The interpretation of these images is where the complication lies with RPA and Citrix automations, only a few RPA platforms support this feature and are sophisticated enough to be a reliable option. UiPath, one of the leading RPA platforms in the industry, has consistently been ranked by Forrester/Gartner reports as the leader in Citrix/RDP automations, attributed mostly to their industry-leading computer vision capabilities.

How About a Proper Use Case?

No article can be complete without a proper use case, and this article is no different, one of our clients’ processes was to monitor a client’s subscription for “Job Updates” and to update accordingly based upon a set criteria of statuses (successful, failed, etc…) and their correlated Start/Stop times. Seems simple enough? However, the challenge was in location of this end clients’ application. It was a thin-client, thus all interaction had to be done via “Surface Automation”, increasing the difficulty substantially.

Our first challenge was to ensure that all interactions (input dialog box for search string, copying of results and extraction for data collation) had some type of fixed “anchor” that we could use as a reference point. Secondly, extensive use of while loops based on “Image Recognition” as the condition was required on top of the first safeguard. Finally, error handling criteria such as “Try/Catch” methodology was critical to ensure this automation operated without much human interaction.

For this automation, “computer vision” would be the main driver for platform selection. UiPath’s reputation as industry leader in Citrix automations made it the ideal choice for this project. Addressing the first challenge in terms of UiPath commands, Recording Wizard has specific settings suited specifically for Citrix. For every interaction made (keystroke, mouse click, etc…) we made sure to use an anchor to avoid exceptions, the “Click Image” command passes in an “Accuracy” argument that allows you to adjust on 0-1 scale and a Boolean output. The best practice is to start at a higher range (.8+) and work your way down if exceptions are encountered at higher ratios.

The End Result?

After 4 weeks of development, plus an additional few weeks for parallel testing, we created a robust automation that allowed the resources onsite to be better utilized. Additionally, due to the extensive mapping and process outlining in the initial part of our engagement, our bot was able to detect certain job statuses that their human counterparts were unable to detect. By providing this level of accuracy and effectiveness in such a short time span, our client was able to better utilize their resources and we, even now, continue to develop their COE for future automations.