RPA and AI – The Coming Governance Nightmare

Last Updated: August 26, 2019By


[glossary_exclude]As a business or technology person, if you have not been inundated by discussions of Robotic Process Automation (RPA), Machine Learning (ML), and/or Artificial Intelligence (AI) over the last few years, then you might be hiding under the proverbial rock. While AI has undergone multiple boom and bust cycles over the last century, RPA came roaring onto the enterprise software stage approximately 5-6 years ago and has experienced explosive growth in interest and adoption ever since. RPA promises to deliver fast, cheap and effective automation of business processes while reducing risk and enhancing control. But, can RPA live up to this gargantuan hype, and what are the governance implications of this new class of software?

As someone who has worked in and around the foundations of Information Governance (IG) for almost thirty years, I can say with certainty that IG is frequently undervalued. Most return on investment (ROI) calculations supporting IG initiatives don’t support the initiative; that is, at least not until there is some data breach, ransomware attack, or public disclosure of critical information. Once this occurs, boards of directors and chief executives suddenly become intensely interested in IG; regrettably a day late and a few million dollars short. It is my observation that IG is typically treated a bit like toilet paper in a public toilet: no one wants to pay for it, but it better be there when you need it.

As many early adopters of RPA have discovered, this technology brings with it a host of IG issues, many of which were neither anticipated nor addressed. These issues are both foreseeable and manageable, because RPA is, at its core, an IG-enabling technology.

Architecturally, RPA platforms are a mixture of old and new technologies, applying a modern, robust IG platform to an aggregation of established, and perhaps even obsolescent technologies such as macros, screen-scrapers and optical character recognition (OCR). RPA uses these technologies that manipulate information at the user interface, and allows them to act on interface data in a controlled, documented and governable manner.


I believe that part of RPA’s dramatic appeal stems from a degree of desperation. Organizations continue their relentless march for incremental process improvement and cost savings long after such savings have been largely tapped-out. After decades of small, lean, Kaizen-driven process improvements, the vast majority of legacy business processes have little further room for improvement; there is no more blood to squeeze from these stones.

This loss of capacity for incremental improvement collides with the era of digital transformation, where new organizations are being born digital and operate digital—they are real-time, context-aware, and data-enabled. These digital-native-processes are dramatically more efficient and effective than 20th Century, paper-based, analog processes that have been digitized, rationalized, outsourced, off-shored, re-shored, de-shored, and de-capitalized for thirty or forty years.

RPA promises to allow companies to perform these same, tired, analog business processes at twice the speed, and half the cost, without fundamentally changing them. RPA promises to further automate these old processes in a way that is simple and inexpensive to design, build, deploy, and support. For business leaders who need to cut their costs by 3% every year to keep earning their bonuses, RPA feels like a miracle diet pill sold on late night cable TV.

Despite the promise of RPA, the reality falls somewhat short, at least thus far. By 2019, according to Gartner, only 5% of organizations are succeeding with RPA at-scale, while a quick Google search of the phrase “RPA implementation failure” yields over 5.7 million results. Organizations are rapidly discovering that RPA is not the quick fix that many of its proponents have promised, and the dominant factor driving success or failure of RPA implementations happens to be IG.


After decades of enterprise automation, nearly all organizations are still heavily reliant on Excel spreadsheets to run their business. In its course of business, a typical Finance department in a Fortune 500 company likely uses hundreds, if not thousands, of Excel spreadsheets with macros to perform a myriad of calculations and processing tasks. These tasks are performed outside of the enterprise’s core financial system either for convenience or for cost considerations. Nonetheless, they all occur outside of well-regulated finance systems and processes.

This vast flock of free-range Excel macros represent a significant risk to organizations and executives that depend upon them. They are rarely documented, sparsely maintained, and invisible to most oversight mechanisms. Consequently, they are critical to the ongoing operations of the business and yet represent enormous unquantifiable risk.

One of the greatest benefits of RPA is that it allows all of these “in-the-wild” macros to come under some semblance of management and control. With RPA, such automations can be controlled from a centralized system with complete transparency. I would argue that the real value of RPA is not that it may allow you to run your Finance department 5% cheaper than last quarter. Rather, RPA allows a CFO or Controller to have full visibility of their business, and hence keep their job through next quarter.


Despite this emphasis on governance capabilities, few organizations capitalize on, or even recognize, this value. In their effort to reduce costs and achieve positive ROIs, most organizations heavily discount the need for and value of IG. Everyone buys into the need for creating an RPA Center of Excellence (CoE), but few organizations show any interest in actually funding a CoE to the extent necessary for success. Replacing a $20/hour data entry clerk with a $10/hour robot does not leave a lot of room for weekly meetings of an oversight committee, staffed by people earning six figures. Hence, most CoEs are mere shadows of what they really need to be: Information Governance Boards.

Governance is extremely undervalued in early adoption of bots. It’s viewed as unnecessary overhead, structure, and process and runs completely counter to the quick-and-dirty approach that RPA encourages. While I have seen many companies define and adopt a CoE as part of their initial rollout of RPA, frequently that CoE is nothing more than some process maps in a PowerPoint which are loosely followed and rarely referenced.

There is a perception that bot governance is another example of IT overcomplicating things. The business wants quick financial wins through bots, rather than more meetings discussing budgets, architectures, risks, and objectives. The more time spent in a governance meeting, the more the ROI of automation is eroded.


This is true up to a point. When there are only a few bots running in an organization, there is little need for coordination or oversight. Bot errors are relatively simple to correct, resources are plentiful, and there are few interdependencies. However, increase the numbers of bots (processes) and robots (instances performing the work) to beyond twenty or thirty, and you will likely find your bots, and their masters, stepping on each other’s toes with growing frequency.

Most RPA business case implementations fail because of poor utilization and poor throughput. For a range of reasons, bots end up not working 24/7, and when they are working, they fail to complete their assigned tasks. The product of these two factors is Bot Effectiveness, and it is not unusual to find bots running at single-digit effectiveness early in their lifecycle. Marry this poor performance with the perceived high-cost and low value of governance, and it’s easy to see why many organizations under-invest in RPA governance.

But, these very shortcomings in throughput and utilization are the kinds of things that an effective IG infrastructure would prevent. In a chicken-or-the-egg scenario, the lack of effective RPA IG leads to ineffective RPA, and so on. In my experience, the majority of the 95% of companies who fail to deploy RPA at-scale do so because they did not create, deploy, support, and grow an effective IG framework within their CoE. Their maniacal focus on ROI made them completely miss the real value of RPA: governance.


The 5% of companies that are succeeding at-scale with RPA share several common characteristics. First, they deployed RPA at-scale, rather than piecemeal. Instead of deploying automations one at a time (with each having to justify its own incremental ROI), successful organizations recognize that bots use shared infrastructure. Their costs, and hence ROIs, are interdependent. The more bots you deploy simultaneously, the greater the collective ROI.

Second, these companies recognized that most ROI analyses do not value IG, which is the dominant value proposition of RPA. The point of RPA is not to create task automations, since in most organizations such macros already abound. Rather, RPA puts all of these automations under centralized control with transparency, so that the hidden costs of operating these automations free-range can be captured and eliminated. The significant reduction of risk that this process delivers is an added bonus. Third, these organizations have recognized that RPA isn’t a new technology, it’s a new workforce. Through effective governance, they harmonize the work and outputs of both human and digital workers in a way that increases speed, accuracy, flexibility and efficiency. Only through such harmonization of these workforces, through RPA’s IG, can RPA’s true benefits be realized.


How do leading companies use IG to succeed with RPA? There are a few simple principles to follow:

  1. Have IG before you need it, because the more we rely upon automation working, the worse the consequences will be when it inevitably doesn’t.
  2. Define your IG to support hundreds or thousands of bots, but only deploy as much as your in-production bots demand. Retrofitting IG is dramatically more painful than designing for scale from the beginning.
  3. Engage all parts of your organization from the beginning. Participation leads to buy-in and support and reduces the likelihood that people will go rogue and build their own bots off the reservation. Free-range bots are seductive to business owners looking for seemingly fast, cheap and good automation, but they rarely realize the expected returns laid out before deployment.


RPA, and eventually AI, are here to stay. Ultimately, both will lead to dramatic changes in how organizations operate. Choosing to not utilize these technologies would be akin to choosing to not have a website in the 1990s, a business-ending decision.

But, in order to succeed with RPA, it is critical to recognize that these applications aren’t innovations in functionality (macros, screen-scraping, and OCR); they’re innovations in orchestration (resource management, governance and auditability). When measuring the value of these applications, an ROI analysis that depends upon labor arbitrage, and discounts the value of governance and control, will almost always lead to disappointment and disillusionment. To be part of the 5% of companies that succeed with digital labor, it is critical to understand exactly what benefits it offers.

With RPA, organizations will reap significant rewards from digital labor, as long as they are willing to cover the cost of making sure the paper roll never runs out.[/glossary_exclude]


recent posts

About the Author: Christopher Surdak

Christopher Surdak, J.D., is an industry-recognized expert in mobility, social media and analytics, big data, information security, regulatory compliance, artificial intelligence and cloud computing with over 25 years of experience. He is currently an Executive Partner at Gartner. He is the author of several books, including the upcoming, Care and Feeding of Bots which is a guide to the use of AI, Machine Learning and Robotics in the business world. He can be reached at [email protected].