Jeremy Birch, Hannah Cassidy and Kyle Wombolt, Herbert Smith Freehills, warn that the private sector is at risk of failing to prioritise modernising compliance and investigations functions just as regulators become increasingly technologically sophisticated.

Businesses are lagging behind regulators and law enforcement in their use of big data when it comes to monitoring for misconduct, investigations and enforcement. Just 9% of all businesses in Herbert Smith Freehills’ recent survey said that they ‘relied extensively’ on data analytics to monitor employee conduct. Only another 25% said that they were ‘in the early stages’ of implementing employee supervision through data analytics.

With regulators and law enforcement increasingly relying on advanced data analytics for monitoring and surveillance, the risk is growing that they detect misconduct by employees or customers that has gone undetected by a company’s own internal systems.

This exposes businesses to two significant risks. First, the loss of an opportunity to self-report and secure cooperation credit. Secondly, regulators may, with the benefit of hindsight, consider that a firm with adequate internal controls would have detected such misconduct, bringing a firm’s systems and controls themselves under scrutiny. These risks are particularly problematic in an era in which businesses have encountered a significant increase in the volume of data and data sources requested by regulators and law enforcement.

Despite this, nearly 65% of respondents to our recent survey said that their budgets for collecting data to satisfy regulators or law enforcement had stayed the same despite massive data volume increases.

Such data-heavy investigations are no longer restricted to the financial services sector. Hong Kong’s Securities and Futures Commission (SFC) has focused enforcement resources on listed companies for many years now. Initially focused on IPOs, investigation of individual misconduct by company directors has grown steadily.

This raises the key question of what businesses should learn from regulators’ increasingly sophisticated use of big data in monitoring – and how can businesses that are grappling with the use of big data in monitoring best adapt to this new reality.

Data use focused on commercial not compliance concerns

In February 2020, Herbert Smith Freehills surveyed a number of clients in general counsel, senior legal, compliance and risk roles in organisations across multiple sectors and jurisdictions. Organisations clearly recognised the commercial benefits of data. Almost 80% of respondents said that data featured in their current commercial strategy and over 66% had a data governance strategy in place. According to survey respondents, data analytics or artificial intelligence was not only used to improve efficiency and operations, but also featured heavily in research and reporting, and for data collection, analysis and business strategy. 

Increased commercialisation of big data also has upsides for compliance. Its use will ultimately improve the data quality within an organisation and, over time, allow that data to be used for compliance purposes. However, data analytics or AI does not appear to be used much currently for investigations or for surveillance purposes. This may indicate that the private sector is at risk of failing to prioritise modernising compliance and investigations functions just as regulators become increasingly technologically sophisticated.



Quality over quantity?

Data quality, for some organisations, may be preventing greater adoption of data as a monitoring and compliance tool.

One in five of respondents reported concerns over the quality of their organisation’s data. While respondents didn’t note this as their main concern, it still demonstrates that not all organisations have the right data sources to implement monitoring for misconduct in a meaningful way.

The sophistication of monitoring systems and issues around how data is collected and used are part of the problem. While corporate goliaths, big tech and some governments have the resources to collect and analyse massive amounts of data, that level of sophistication in employee monitoring is still relatively rare in the private sector.

Many private organisations do not have access to the right data, or the quality of data they can access isn’t sufficient to conduct sophisticated analytics with useful output. An increase in resources may help to manage this, but businesses need to agree their data gathering aims before implementing an appropriate IT infrastructure to create the data pool they need and will find valuable.

If a business needs to monitor a certain type of misconduct or a specific product area, it is better to focus their efforts around those goals. This has the added advantage of allowing them to better manage the risks of complying with privacy, employment and discrimination law.

Investigations: ‘collect everything’ is no longer an option

The approach to digital forensics in investigations is constantly evolving to keep pace with the increasing volume, velocity and variety of data within organisations. Almost every action we take leaves a digital trail and the type of information businesses are collecting, both internal and external, is expanding. It is vital that businesses are alert to the challenges arising in an investigations context of how they acquire, hold and manage their data. With more data comes the need for more careful planning, both before and after the commencement of an investigation.

In our recent survey, over 60% of respondents said that the volume of data and the number of data sources used for internal investigations (or requests from regulators or law enforcement) had grown noticeably over the past two years, requiring more time and resources. Managing the challenge of big data in digital forensics is made easier by having in place an effective data governance framework at the start. A robust framework has a number of broad-based benefits, including strengthening the usability and reliability of its data assets.

A failure to maintain a sound approach to data governance creates problems when the business is facing the prospect of an investigation. First, it may expose businesses to risks, where keeping irrelevant, outdated or erroneous data may potentially hinder the analysis process of investigations and increase unnecessary costs. Second, businesses often run into problems with data integration, where the data needed for investigations comes from diverse sources, meaning that the need to remove duplicate documents and contradictory data may frequently arise.

Regulators’ use of big data targets detection and prosecution

Two prominent areas in which regulators have historically had great success in using big data to detect and prosecute misconduct are insider dealing and tax evasion. Historically, it has been easy to predict the catalysts for insider trading investigations – namely, unusual spikes in the prices of securities shortly prior to the disclosure of material non-public information such as the announcement of a takeover bid or unexpected profit results.

These types of ‘security-based’ investigations have traditionally been reactive, in that they rely on large movements of the market being reported publicly, or matters being reported to regulators (for example, brokers reporting potentially unusual trades by their clients).

However, in recent years, the US Securities and Exchange Commission (SEC)’s Market Abuse Unit has pioneered a ‘trader-based’ approach to insider dealing enforcement which has been quickly emulated by other regulators, including the Hong Kong SFC and the Australian Securities and Investment Commission.

The use of big data has also allowed tax authorities globally to adopt an increasingly sophisticated approach to the detection of tax evasion, generally through the use of data matching protocols. Tax authorities compel the production of data from third parties or request data from other government agencies, and then match that data against records held by tax authorities. In Australia, for example, the Australian Taxation Office (ATO) uses a wide variety of data matching protocols, including:

  • matching of credit and debit card records against income reported to the ATO to identify businesses trading as ‘cash only’, and
  • analysis of insurance records to identify owners of lifestyle assets, such as luxury boats and racehorses, whose assets are inconsistent with the income they have reported to the ATO.


Incorporating big data into monitoring processes

As businesses start to grapple with the challenges of incorporating big data and data analytics into their monitoring and surveillance processes, they should consider how they can use the data sources already available to them, alongside data analytics, in their own internal surveillance and monitoring capabilities. This approach is outlined below.

For example, it may not be possible for a bank to detect an employee trading on inside information if the trades are executed through accounts held with other institutions that have not been reported by the employee. However, the use of predictive analytics should increasingly make it possible for organisations to predict how frequently employees in particular parts of their business need to access confidential information – and identify employees who may be accessing such information more frequently than would be expected.

However, the regulation of data use and collection may slow broader adoption of AI-based compliance monitoring. 50% of survey respondents cited regulation as a challenge around the use of big data and AI, though 75% believed the development and use of AI should be subject to regulation.


Certain monitoring activities may raise issues under data protection laws, surveillance laws, telecommunications laws, cybercrime laws, industry-specific requirements or a combination of these. Organisations should understand what they are aiming to achieve and consider the types of monitoring undertaken of the risks the business faces. There is also an increasing focus on the ethics of AI and other sophisticated data analytics when applied to monitoring people or processing personal data. This makes a thorough analysis of use cases from all angles critical before implementation.

Jeremy Birch, Hannah Cassidy and Kyle Wombolt

Herbert Smith Freehills


SIDEBAR: Trader-based approach –a case study

In May 2019, the Securities and Exchange Commission (SEC) in the US announced that its Market Abuse Unit’s work in detecting patterns of suspicious trades had led to the filing of insider dealing charges against an investment banker and his plumber, who the SEC alleged had made US$76,000 in illicit profits by trading on tips passed to him by the banker (

Similarly, the SEC has credited the Market Abuse Unit’s work as helping identify repeated trades by a Silicon Valley executive prior to his employer’s announcements of missed profit forecasts, through which the executive realised profits of US$120,000 and avoided losses of US$76,000 (

SIDEBAR: Data matching –a case study

While this sort of data matching – particularly when combined with predicative analytics to identify the types of taxpayers most likely to commit tax evasion – has been used to identify potential cases for investigation, tax authorities have also sought to use big data to build their cases. Most notably, in December 2018 the US Internal Revenue Service (IRS) released a Request for Information for a social media scraping tool that would be used to both advance their ongoing investigations as well as identify potential areas for audit.