Key Points
- Only 32 percent of compliance officers test or monitor ethics and compliance controls on an ongoing basis, according to an exclusive 2024 study by Rethink Compliance and Radical Compliance
- Just 22 percent of respondents said they have sufficient budget and resources to test or monitor important ethics and compliance controls
- Only 19 percent said most high-priority ethics and compliance controls are tested using data from enterprise systems
- Only 42 percent of compliance departments receive access to data when requested, and only 28 percent have the same data access level as other business functions
- Merely 31 percent of organizations are confident they can capture and preserve all business communications necessary for investigations or litigation
- Only 17 percent of firms conduct audits of their disciplinary processes
- 79 percent of organizations conduct some compliance testing, but the majority fail regulatory expectations for effectiveness
- Harvard Business Review authors Hui Chen and Eugene Soltes report firms treat compliance as “box-checking” without assessing program effectiveness
- The Justice Department’s updated guidelines expressly require testing, monitoring, and data access as prosecutor priorities
- Compliance officers with data access test and monitor controls 88 percent of the time versus only 74 percent without easy access
The compliance test most firms are failing to run is ongoing testing and monitoring of ethics and compliance controls, according to an exclusive new report that should give compliance officers significant pause. An exclusive new report finds that most corporate compliance teams struggle to perform adequate testing and monitoring of their compliance programs, and also struggle to get access to the enterprise data they need to address their organizations’ compliance risks effectively.
Those are just some of the findings of a study released in October 2024 by Rethink Compliance and Radical Compliance. The organisations surveyed more than 200 compliance officers to ask them about the testing and monitoring they perform on their programs and about the access to data they do or don’t have at their organisations. The findings weren’t all bad, but several of them should give compliance officers pause.
Why Are Most Companies Not Testing Compliance on an Ongoing Basis?
Only 32 percent of respondents indicated they test or monitor ethics and compliance controls on an ongoing basis. When you dig into the group that conducts some testing, another 33 percent indicated they test or monitor controls on a “planned” basis, while 10 percent said they test or monitor controls on an “ad hoc” basis.
The good news is that 79 percent of respondents said their organisations conduct some sort of testing and monitoring of the compliance program. When you dig into that group, however, only 32 percent say they test on an “ongoing” basis.
What Reason Does Jamie McKillop Give for Testing Shortfalls?
“Our biggest takeaway from the study is that most programs’ testing and monitoring efforts are falling below regulatory expectations,” said Jamie McKillop, vice president of advisory services at Rethink. “The results indicate that the biggest reason behind this shortfall is compliance teams not having access to the right internal data or systems… If you don’t have access to the correct data, you can’t get the complete picture of compliance program performance that you need. If you don’t have that complete picture, then your ability to understand where the program is or isn’t effective — that gets a lot harder, if not impossible”.
How Much Budget Do Compliance Teams Have for Testing?
Only 22 percent said that they have sufficient budget and resources to test or monitor important E&C controls. This budget shortfall directly impacts the ability of compliance teams to conduct thorough, ongoing testing of their ethics and compliance programs.
Why Is Only 19 Percent Using System Data for Controls Testing?
Only 19 percent said that most of their high-priority E&C controls are tested or monitored using data from systems. That is, data pulled directly from other business operating teams in the enterprise. This represents a critical gap, as compliance testing without integration into engineering and business systems is “just a quarterly fire drill”.
What Percentage of Firms Can Capture Off-Channel Communications?
We also asked about companies’ efforts to preserve business communications, since enforcement of off-channel messaging has been a hot issue this year with regulators. Those findings weren’t great either: only 31 percent said they were confident that their organisation has the ability to capture and preserve all business communications as necessary for investigations or litigation.
This is particularly concerning given that enforcement of off-channel messaging has been a hot issue with regulators in 2024.
How Many Firms Audit Their Disciplinary Processes?
Only 17 percent conduct audits of their disciplinary processes. That’s all rather yucky, because it suggests that companies are not meeting expectations that the Justice Department has for effective compliance programs.
What Does the Justice Department Require for Effective Compliance Programs?
The department’s guidelines — updated as recently as last week — expressly talk about testing, monitoring, and access to data as important issues that prosecutors are likely to ask about if your business is ever under investigation. The updated guidance directs prosecutors to consider whether the company is “appropriately leveraging data analytics tools to create efficiencies in compliance operations and measure the effectiveness of components of compliance programs”.
Those factors include whether the company is “managing the quality of its data sources”; “measuring the accuracy, precision, or recall of any data analytics models it is using”; and demonstrating that “it is proactively identifying either misconduct or issues with its compliance program at the earliest stage possible”.
What Data Access Do Compliance Departments Actually Have?
Our findings suggest that most compliance functions need more access to a broader set of data. Respondents said they are generally doing well leveraging the data and systems that compliance teams typically “own,” such as internal reporting or compliance training systems; but considerably fewer are leveraging data and systems outside of their direct control.
For example, only 42 percent of respondents said the compliance department is given access to data when requested, and only 28 percent said the compliance function has the same level of access to necessary data, systems, and tools as other business functions at their organisations do.
Well, consider how that would look to prosecutors investigating your company for an issue. The compliance team might be doing great analysing data it owns — but if that’s all the data you get, then your compliance program is just dancing in a corner by itself, while the rest of the enterprise is grooving to a tune you can’t hear. Does that sound to you like a company that takes ethics and compliance seriously?
How Does Data Access Correlate with Testing Frequency?
One final point for today: we also found that compliance officers who do get necessary access to data also tend to test and monitor their controls more thoroughly. That is, among those who said they typically get access to data when requested, 88 percent of them also said they test and monitor controls. Among those who didn’t have easy access to data, the figure was only 74 percent.
What Types of Testing Do Compliance Officers Conduct Most?
And what types of testing do compliance officers conduct? Most common were tests to see whether your policies and procedures are consistently followed. Compliance-focused surveys were in second place, independent assessments in third.
Who Performs Compliance Testing at organisations?
Fifty-five percent of respondents said they work with internal audit, finance, or some other Second Line control function to perform testing, while 35 percent said they use external resources such as an audit or consulting firm. A brave 33 percent said they do testing themselves (although I immediately wonder how many compliance teams have sufficient manpower and skill to do that alone).
What KPIs Do Compliance Officers Use to Monitor Programs?
Roughly two-thirds of respondents also said they rely on various key performance metrics to monitor their compliance program. Delighted to see that most compliance officers are using KPIs to monitor the success of their program, although I do have two notes of caution here.
First, it’s important to think about which KPIs measure how busy your program is, which might not be the same KPIs that tell you how effective it is. You’ll likely need to track multiple KPIs over time, to see whether changes to your program (new training modules, new policies, new procedures) lead to changes in behavior (more complaints about certain issues, higher named reports versus anonymous, and so forth).
At What Level Do Most Companies Track Compliance KPIs?
Second, our report found that most compliance officers who establish KPIs only track them at the enterprise level (cited by 67 percent of respondents). Far fewer track compliance KPIs at the business unit level (28 percent) or department level (27 percent); and only 10 percent track KPIs at the individual level.
Enterprise-level KPIs do have their place; you can’t monitor the success of your compliance program without them. But the further down within the organisation that you can establish KPIs, the more you’ll be able to identify new or evolving risks. For example, KPIs at business unit level might help you uncover bribery issues in far-flung business units; KPIs at the department or individual level might uncover managers engaging in personal misconduct or blocking employees from raising issues.
Why Does Hui Chen Say Compliance Programs Fail?
As reported by Hui Chen and Eugene Soltes of Harvard Business Review, too many firms treat compliance as a box-checking exercise, making employees sit through training and attest that they understand the rules, but failing to assess the effectiveness of their compliance programs, or doing so with faulty metrics.
Firms spend millions of dollars annually on whistle-blower hotlines, training, and other efforts to ensure adherence to laws, regulations, and company policies. Yet malfeasance remains entrenched in the corporate world.
What Statistics Show Fraud Remains Entrenched?
According to the Association of Certified Fraud Examiners, almost half of all fraud cases are never reported publicly, and a typical organisation loses close to $3 million in annual revenue to fraud. Furthermore, of the nearly 3,000 executives interviewed for EY’s 2016 Global Fraud Survey, 42 percent said they could justify unethical behavior to meet financial targets. Clearly, malfeasance remains deeply entrenched in private enterprises today.
Millions of fraudulent accounts at Wells Fargo. Systemic deception by Volkswagen about its vehicles’ emission levels. Widespread bribery at Petrobras that damaged both the government and the economy of Brazil. While those corporate scandals made headlines in recent years, countless others failed to penetrate the global consciousness.
How Should Firms Fix Their Compliance Programs?
As reported by Hui Chen and Eugene Soltes of Harvard Business Review, firms should start by linking compliance initiatives more closely to specific objectives: preventing misconduct, detecting it, or aligning policies with laws and regulations. Then, using careful model design and some creativity, firms can develop better metrics to measure what’s working and what isn’t.
Firms cannot design effective compliance programs without effective measurement tools. For many firms, appropriate measurement can spur the creation of leaner compliance programs.
What Three Categories Do Compliance Testing Failures Fall Into?
When compliance testing fails, failures fall into three categories: missing evidence, broken controls, and out-of-scope drift.
Missing evidence: The control exists but the test does not produce an auditable artifact. Fix: add logging or screenshot capture.
Broken control: The system stopped doing what the standard requires. Fix: actual code change, then re-test.
Out-of-scope drift: A new feature was shipped without compliance review. Fix: process change so every PR reviews compliance impact. The third category is the most common in fast-moving teams.
What Are Common Challenges in Compliance Testing?
Many teams run into roadblocks that slow them down, muddy the results, or even lead to failed audits. Companies that fail to conduct regular audits and proactive testing and monitoring are less likely to identify shortcomings in the practical application of their program.
What Risks Come From Failing Compliance Requirements?
Failing to uphold the requirements of regulatory bodies can result in heavy fines and sanctions that directly impact your ability to operate. Standards and regulations change, so organisations must put a compliance management plan in place to regularly audit any new or current regulations that impact their business. This ensures you don’t fall into noncompliance or miss updated regulatory requirements.
How Can organisations Improve Their Compliance Testing?
First, document all the required laws, standards, and regulations that impact your company. Once that information is documented, you can start your team testing those compliance standards directly. The process includes creating compliance test cases, executing compliance tests, analysing results, reporting issues, verifying fixes, automating easy resolutions, and performing regular compliance audits.
Are you or your organisation struggling to meet regulatory compliance testing requirements? Our Regulatory Compliance training courses provide the practical skills and knowledge needed to design, implement, and monitor effective compliance programs that meet Justice Department standards and avoid costly fines.
What Does the Report Say About Technology Control?
Some important compliance processes (internal reporting of possible violations, for example) were run by dedicated tools that the compliance team itself could control (58 percent, in this case). Other processes highly important to the compliance program, however, were typically run by larger systems beyond the compliance officer’s control. For example, 55 percent of respondents said invoice and payment requests were managed by larger systems (presumably ERP software such as Oracle or SAP) beyond the CCO’s control.
Obviously CCOs can’t expect to have stand-alone technology tools under their direct control for all relevant issues; that would cost a fortune and be a data management nightmare. Hence the importance of senior management supporting the CCO’s access to data in a broader way, either by directing business leaders to share necessary data or by designing IT systems to allow easy transparency into the data that compliance officers need.