Interpreting AI-based Cybercrime Prediction
Crime detection using Artificial Intelligence (AI) approaches, especially Machine Learning (ML) techniques have attracted a lot of attention in both academic research and industry practices. However, the inability of humans to understand how some of these models arrive at their decision is raising a lot of ethical issues. In addition, data protection laws such as the General Data Protection Regulation (GDPR) has empowered data subjects to be entitled to an explanation of automated decisions impacting them and the right to challenge such decisions. Thanks to Explainable AI, an active area of research aimed at ensuring transparency in the use of AI and ML. This article explores the potentials and biases of AI and ML and highlights interpretability methods and libraries that can be leveraged in rooting out inequality in the application of automated decision-making in risk profiling and behavioral analytics as it relates to criminal justice and financial crime prediction.
With the evolution and advancement of internet technologies and services such as online banking and e-commerce, many traditional crimes have taken a new leap into the digital and cyberspace. For example, between March 6, 2020, and March 31, 2021, the Canadian Anti-Fraud Centre which collects information on fraud and identity theft reports more than seventeen thousand COVID-19 related fraud targeted at 15,198 Canadian victims leading to a loss of more than seven million dollars. This shows the complexity of detecting and preventing crimes in cyberspace considering that this report only covers known cases of fraud and identity theft related to the COVID-19 pandemic. This is where the superior ability of Artificial Intelligence (AI) in processing vast quantities of data to detect patterns and relationships comes into play. In recent years, interesting research advancements in the areas of cybercrime prediction have emerged from Machine Learning (ML), a sub-discipline of AI which focuses on teaching machines to learn by applying algorithms to data. Currently, there are many organizations including financial institutions and services using ML technology to combat cybercrime. However, a major setback to the application of ML and AI models in combating cybercrime is the inability of humans to understand the decision-making process of the models. A key mechanism for delivering ethical ML and AI is being open and transparent. This article is aimed at informing both technical and non-technical audiences on how interpretable ML is applied to enable humans to understand the decision-making mechanisms of complex BlackBox models designed for cybercrime detection and prevention.
Cybercrime
Cybercrime refers to any illegal activity carried out by rogue individuals, organized crime groups, or state-sponsored factions using computer technology. The computer can either be the tool for committing the crime or the target of the crime. Cybercrime encompasses a broad range of activities such as identity theft and fraud, credit card theft, cryptojacking, data leakage, drug sale, phishing, cyberstalking, cyber terrorism, cyber laundering, cyber hacking, cyberstalking, sexually explicit content, and child pornography. The following are two real-world examples of fraudulent activities with a specific focus on financial and payment services.
- Payments Fraud. Payment services are one of the most digitalized parts of the financial industry involving several stakeholders. This makes it particularly vulnerable to fraudulent activities such as identity theft by malicious hackers, merchant scams by rogue companies operating through marketplaces, and credit card payment fraud. Payment cards and other related sensitive information may be obtained through social engineering attacks and can be used to perform card-not-present transactions or account takeover. The implication of this is that legitimate cardholders and payment service providers whose credentials were compromised and used for illicit transactions and money laundering can involuntarily become part of a criminal scheme. This is why the development of ML and AI technologies that can effectively detect suspicious activities or predict payment frauds is a strategic goal for the banking and payments industries.
- Insurance Fraud. The insurance industry has grappled with the challenge of identity theft and insurance claim fraud from the onset. These include staging incidents, exaggerating damages, and false claims. Delayed claim payout, the prolonged investigation when fraud is suspected, and the resulting losses due to fraud can impact all involved stakeholders through increased premium costs and trust deficit. The insurance industry must increase its capacity in developing capabilities that can help identify potential frauds with a high degree of accuracy. This will ensure that legitimate claims are cleared rapidly while identified cases are further scrutinized.
For further information about cybercrime and cybercrime activities, we will refer interested readers to websites of crime investigations such as Interpol, US Federal Bureau of Investigation (FBI), UK National Crime Agency (NCA), and Royal Canadian Mounted Police (RCMP).
Cybercrime Detection & Prediction
One of the key objectives of combating cybercrime as it relates to financial and payment services is to automatically identify suspicious events in a transaction and report them to those responsible for further investigation while legitimate transactions are being processed. Financial institutions have over the years relied on static rule-based systems or heuristics developed around fraud indicators. However, with the rate at which cybercrime activities are evolving, the rule-based systems on their own are unable to detect new and emergent cybercrime events. This is where ML comes to the foreground to bring significant improvements to the process. Compared to rule-based solutions, ML approaches have higher precision and can identify new suspicious patterns as they consider multiple parameters and data points from many sources. This is why the trend to incorporate ML capabilities into new and existing security products, including anti-fraud systems, will continue apace. The following are examples of ML-based cybersecurity solutions specific to the financial and payment services described in the previous section.
Payments Fraud Solutions. Banking and payment services are switching to dynamic AI and ML-driven methods. This is because traditional rule-based systems are unable to distinguish between common errors or unusual legitimate transactions from real fraudulent activities. Some financial institutions and services have already embraced ML technology to combat fraudsters. For example,
- IBM’s AutoAI was developed and open-sourced to predict fraudulent transactions. It leverages Apache Spark, IBM’s Object storage, Watson, and Watson ML to enable a faster decision-making process in fraud prevention.
- Feedzai, a fraud and money laundering risk management platform, also developed and open-sourced their OpenML Engine for fighting new and evolving financial crimes. The company claims that a fine-tuned ML solution can detect up to 95% of all fraud.
- PayPal recently launched a new solution for their larger merchants called Fraud Protection Advanced. The technology leverages ML and analytics capabilities integrated with threat intelligence sources to identify, investigate, resolve, and mitigate fraud.
- Visa Advanced Authorization: leverages ML to provide issuers across the globe with sophisticated in-flight risk scoring, for more targeted and better-informed authorization decisions.
- MasterCard’s Decision Intelligence leverages ML to increase approvals for genuine transactions. The system integrated ML and AI to track and process variables such as transaction size, location, time, device, and purchase data to assess account behavior in each operation and provides real-time judgment on whether a transaction is fraudulent.
- Other solutions in the e-commerce sector include Amazon Fraud Detector and eBay Credit Card Fraud.
Insurance Fraud Solutions. There are many hidden clues buried in unstructured insurance datasets curated from investigation files written by insurance clients, agents, and enforcement agencies. ML algorithms and semantic analysis techniques are well suited for the analysis of such files in search of inconsistencies and interesting patterns. Analyzing claims data with ML techniques may uncover hidden correlations in the behavior of insurance agents, repair services, and clients. For example, Wipro, an AI-based research company conducted a comparative analysis of vehicle insurance claims. The initial analysis result using various ML algorithms revealed several interesting patterns. Comprehensive Review of Cybercrime Detection Techniques and Credit Card Fraud Detection: How Machine Learning Can Protect Your Business From Scams.
Ethical Issues
Many fraud detection solutions in payments and other financial services are focused on analyzing user-profiles and behavior during transactions. In some cases, transactions and users are scored and then categorized into predefined prediction groups ranging from legitimate to suspicious and fraudulent. In such scenarios, predictions alone and metrics calculated on these predictions do not suffice to characterize the model. Many stakeholders now additionally demand an explanation at the level of human understanding as to why a high future crime score was automatically assigned to an individual or a group of people. Let us consider the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) used by some courts to assess the likelihood of a defendant becoming a recidivist or a career criminal. This study published in ProPublica investigated COMPAS and found that blacks are almost twice as likely as whites to be labeled a higher risk, however, the proportion of the whites that usually re-offend are higher than their black counterparts. A black female who had never been arrested before but completed probation and got a felony burglary charge reduced to misdemeanor trespassing was rated a medium risk. According to the report, the score made her struggled to find work. The company, a team of researchers at Microsoft, and another team at the Community Resources for Justice (CRJ) published rejoinders challenging the results of the analysis stating in their separate rebuttals that the COMPAS claims are correct, free of racial bias, and accurately reflect the outcomes from the application of the model. Another recent study by a team at the Sudikoff Lab has shown that the COMPAS software is more accurate than individuals with little or no criminal justice expertise and less accurate than groups of individuals. There was also a court ruling stating that COMPAS risk scores can be considered by judges during sentencing, but there must be warnings given to the scores to represent the tool's limitations and cautions. According to Credera, there are many other use cases where AI and ML technology can be racist, a few examples include:
- The research revelation about autonomous vehicles being able to easily recognize light-skinned pedestrians but have difficulty in recognizing people of color. This is a major concern for darker-skinned individuals as the technology becomes more widely adopted.
- The issue of racial bias in health decision algorithm which uses health costs as a proxy for health needs. Because less money is spent on Black patients who have the same level of need as their White counterparts, the algorithm thus falsely concludes that Black patients are healthier than equally sick White patients.
- The issue of impermissible discrimination in mortgage loan pricing where lenders and financial technology companies were found to charge Latinos- and African-American borrowers higher interest rates for mortgage refinancing. The study reveals that the algorithms operate on rent information extracted in weaker competitive environments and the profile of borrowers on low-shopping behavior.
The goal of model interpretations, therefore, is to serve those important real-world objectives that are difficult to model and optimize. This study elucidates real-world objectives of interpretable ML systems as follows:
- Trust, which simply is the confidence in the system predictions.
- Causality, which helps to infer properties of the natural world
- Generalization, which looks into model deployment in a non-stationary environment
- Informativeness, which is the additional information about the model's decision process
- Fair and ethical decision-making which focuses on preventing discriminatory outcomes
Model interpretations could be enabled with different methods in several ways, an excellent reference is the Interpretable Machine Learning book. Many open-source and commercial libraries have implemented a variety of interpretability methods. A few among these include Captum, ELI5, LIME, SHAP, SKATER, and tf-explain.
Use Cases
This section presents two use cases of how interpretability methods and libraries are leveraged in two cybersecurity application areas both relevant to cybercrime detection and prediction.
Credit Dard Default Model
This feasibility study by a team of researchers at Riskcare, a financial services and technology consultancy, is aimed at overcoming the interpretability challenge around the application of DL models in business risk assessment in the financial industry. The study leverages several methods to analyze the mechanisms of a credit risk model, developed with deep neural networks, in the context of a credit card portfolio. The methods include:
- Relevance analysis to understand or interpret how much of the output (e.g. a probability of default) is directly due to a given input variable.
- Sensitivity analysis to understand or interpret how much does the output change subject to a (small) change in a given input variable.
- Neural activity analysis to understand or interpret which neural paths are most activated by a given input variable.
The study reported that because there are many flavors of interpretability, no single technique can produce all possible desired insights. For example, the Layer-wise Relevance Propagation (LRP) methods can produce scorecard-like equivalents for the model which is useful for understanding the model from a traditional risk management perspective. In addition, the neural activation analysis was found to be better suited for the identification of non-trivial biases in the data or the model.
Intrusion Detection Model
Detecting and identifying malicious activity occurring on a network of host machines can be extremely difficult in scenarios where attackers move laterally from host to host. ML-based intrusion detection systems are leveraged to detect known threats and predict intrusion attempts at the cloud, network, and host levels. The following are two recent preliminary research studies geared toward interpreting intrusion detection and prediction models.
- The research study by Mane, S., & Rao, D. (2021) aimed at understanding the mechanism behind a deep neural network model developed to classify and flag certain network traffic. The study leverages interpretability methods in (i) the SHAP library to provide a global understanding of the model’s behavior to data scientists, (ii) Protodash to show data analysts samples from training dataset which are similar or different to a given sample, and (iii) the LIME library to help end-users understand which features in the input instance are contributing to the model’s final decision and how the model’s decision can be changed by tweaking which features values by what amount.
- The research study by Sarhan et. al (2021) aimed at understanding the generalizability of network traffic analysis models learned using datasets from various generators including the CICFlowMeter developed by the Canadian Institute for Cybersecurity (CIC). The SHAP library was leveraged to explain and interpret the classification decisions of two ML models. The Shapley values of the features were analyzed across the multiple datasets to determine the influence contributed by each feature towards the final ML prediction.
These studies highlight important insight about adding transparency at every stage of an ML pipeline deployed in cybersecurity applications. The interpretations also provide measurable factors as to the degree of influence of model features used in cyberattack prediction.
Conclusion
AI and ML have become strategically important in securing applications in critical areas against cyberattacks. Financial and payments services such as MasterCard, PayPal, and Visa have all embraced ML in their transaction fraud solutions. However, even though DL models have shown remarkable improvements in prediction capabilities, they are hard to interpret. The complexity of DL models and the inability of humans to understand or trust their decision-making mechanism also hinders their deployment in production systems. This article provides an overview of interpretability methods and libraries as well as their application in enabling humans to understand the decision-making mechanisms of complex DL models utilized in cybersecurity applications such as intrusion and payments fraud detection and prevention. The importance of developing unbiased AI systems cannot be overstated. ML model interpretability is becoming a regulatory requirement and is aimed at enabling fairness, accountability, and transparency in automated decision-making. The Government of Canada through its Algorithmic Impact Assessment (AIA) and Directive on Automated Decision-Making is ensuring that the use of AI in government programs and services is governed by clear values, ethics, and laws. It is also important for ML practitioners and engineers to pursue interpretability as a means to build better models. Finally, the Directive on Automated Decision-Making will have an automatic review process planned every six months. We, therefore, recommend that more cybersecurity researchers and practitioners be involved.
