Cowbell Factors: A Defining Milestone in Cyber Risk Quantification

Enterprises wrestle with cybersecurity challenges on a daily basis – emerging cyber threats, the implementation of nascent technologies, or software vulnerabilities. The surge in digitization, artificial intelligence, and the proliferation of interconnected systems, coupled with remote work, has expanded the attack surface for malicious actors. While rapid technological innovation brings numerous advantages, it also elevates the risk of exploitation, irrespective of size or sector. This further compounds the persistent challenges associated with risk quantification and underwriting cyber insurance.

In 2019, we pioneered an innovative approach to cyber risk assessment. With four years of (re)learning and processing trillions of data points, we reached a significant milestone last month. This reinforced our confidence and provided further validation for our approach and the effectiveness of the Cowbell Factors, the risk ratings system that we developed to address the complexity of underwriting cyber risks.

Breaking Boundaries: Rethinking Cyber Risk

Traditional actuarial science relies on mathematical and statistical methods to analyze historical data and predict frequency and severity of future events. However, predicting future events without historical data poses a challenge, which is the case with cyber insurance.

Leveraging domain expertise, qualitative insights from experts, proxy data, feature engineering, and scenario modeling, we approach this problem with creativity, adapting as more information becomes available. AI, with its ability to handle complex patterns and large datasets, helps us in making predictions even in data-scarce scenarios. To evaluate the cyber risk of an enterprise, we are able to draw insights from its industry peers—companies of comparable size and operating in the same sector. By applying AI and data imputation models, we leverage the data network effect, maximizing the collective knowledge to enhance our risk assessment capabilities.

The foundational premise of this approach is that companies of comparable sizes within the same industry often adopt similar systems and processes. Even the human element, known to be the weakest link in the security chain, tends to migrate within the same industry. Over time, every industry organically develops its unique security posture within its size category. This is how we construct a relative benchmarking model to calculate the cyber risk ratings of small or medium-sized organizations.

Cowbell Factors: A novel approach

In 2019, Cowbell emerged as a groundbreaking cyber insurance provider, introducing its unique “Cowbell Factors” — a proprietary multivariate risk rating system to the market. This relative risk rating method explicitly built for cyber insurance positioned Cowbell at the forefront of modern, data-driven underwriting for cyber risk in the small and medium-sized enterprise (SME) market.

Cowbell Factors compare an organization’s cyber risk profile against its industry peers within the Cowbell Risk Pool of 38 million U.S. and U.K. small and medium-sized enterprises. This benchmarking enhances our understanding of an organization’s risk landscape and insurability.

Cowbell Factors recognize that the complexity of cyber risk transcends the scope of cybersecurity alone. For example, two organizations boasting identical security scores and revenue – one might assume their risk levels are comparable, the multidimensional nature of cyber risk requires a more comprehensive approach. The Cowbell Factors represent risk ratings that surpass cybersecurity scores, encompassing contextual intricacies of business operations, network complexity, regulatory aspects, geopolitical factors, industry dynamics, threat intelligence, historical claims, dark web presence and more. Presenting a more personalized and accurate risk profile.

Continuous Learning: Four years and counting

Over the past four years, we have amassed trillions of data points from over 38 million small and medium-size enterprises across the U.S. and the U.K. In addition to the quantity, the quality of our data has also improved over time and the effectiveness of AI models is directly tied to the quality of the data utilized.

Fine-grained precision in collected signals

Data comes from various sources in different shapes and sizes, including both raw and pre-modeled data. There are advantages of using pre-modeled data as it reduces the workload on internal teams, however, the reliance on the black box has its own implications. As part of the routine auditing and human oversight of our AI models, we identified opportunities of improvement in the pre-modeled data and took proactive steps to swap proxied information with the raw data. This strategic decision not only eradicated uncertainties tied to shifting modeled “scores” but also facilitated more direct oversight. This has resulted in an enhanced level of AI transparency and increased effectiveness of the models.

With unprocessed raw data at our fingertips, we not only maintain complete control over our modeling but also leverage the capability to intricately understand data sources. This is achieved through our comprehensive Cyber Risk Framework (CRF), encompassing 1,600 controls.

Tightening the Close-Loop

Cowbell Factors now also incorporates detailed insights from our in-house claims and risk engineering teams. This integration significantly enhances the predictive accuracy of Cowbell Factors. Claims frequency and severity, segmented by business category, are intricately connected with a spectrum of incident types—from ransomware and extortion occurrences to cybercrimes, data breaches, invoice falsifications, third-party events, and more.

Enhanced granularity using NAICS codes

We are also transitioning from a broad-level comparison between companies in the same industry to a more refined approach, focusing on the 3 to 6 digit levels of the NAICS (North American Industry Classification) codes. This refined approach allows for more precise risk comparisons. For instance, it facilitates contrasting the risk between dentist offices (621210), rather than using a broad category like healthcare services (62).

Figure 1: # of codes at each level of the NAICS classification

Leveraging the granularity of the NAICS Code classification, we achieve up to 42 times more precision when utilizing the 6-digit NAICS codes compared to broader classifications. This heightened precision allows for a more nuanced and detailed assessment of business risk profiles.

Actuarial Science meets Data Science

Our data scientists use data, advanced analytics, and machine learning to extract meaningful insights from vast and complex datasets. Our actuaries leverage data science tools to enhance their risk assessments, allowing for more accurate predictions and better decision-making. This integration enables actuaries to handle larger and more diverse datasets, identify patterns and trends, and refine risk models in real-time. The marriage of traditional actuarial principles with data science empower our actuaries to adapt to the rapidly evolving landscape of risk management and financial forecasting.

Risk Pool Quality (RPQ) Index

The efficacy of AI models is contingent on the quantity of data available for training. A larger and more diverse dataset allows AI models to learn and generalize patterns effectively, mitigate overfitting, and enable complex model training. Larger datasets enhance feature representation, address imbalances, and generally improve accuracy. Our Risk pool has grown to 38 million small and mid-sized enterprise businesses, however, a vast amount of low-quality or irrelevant data doesn’t enhance model performance.

To address this very concern, we use Risk Pool Quality Index to ensure that we are feeding quality data into the modeling process. The RPQ index is designed to

• Verify the adequacy of data we collect for our evaluation needs
• Ensure data quality aligns with our standards
• Confirm that the timeliness of the data meets the standard for continuous risk assessment

Milestone Unlocked: A Breakthrough in Cyber Risk Quantification

The substantial volume and improved quality of data, coupled with a more robust closed-loop system and refined labeling and classification, have significantly influenced the effectiveness of our Cowbell Factors. This improvement has resulted in a noteworthy 436% enhancement in predicting claims frequency and a 254% increase in predicting claims severity.

Cowbell Factors Efficacy

T-statistic analysis is a primary method used by statisticians to gauge efficacy, employed in comparing means between groups. In hypothesis testing, the t-statistic is computed to evaluate differences in relation to within-group variability. T-statistic analysis aids in discerning whether observed differences in groups are genuine, providing a robust statistical foundation to interpret the efficacy in a concise and meaningful manner.

We crafted the Cowbell Factors with a singular focus: to quantify an organization’s cyber risk and predict its probability of facing a cyber incident leading to a claim. Now, with a solid foundation of four years’ worth of data, we can measure the efficacy of Cowbell Factors with statistically meaningful precision.

To assess the predictive effectiveness of Cowbell Factors, we categorized our portfolio into two groups: policyholders with claims and policyholders without claims. Using a series of t-tests, we validated the hypothesis that the distinction between these groups is by design and not a result of chance.

T-Test: The null hypothesis in our t-test suggests that the two groups are identical. To ensure a focused examination of enterprise Cowbell Factors in relation to industry peers, we utilized the distance between the Company Cowbell Factor (CCF) and Industry Cowbell Factor (ICF). Of course, a successful test would mean rejecting the null hypothesis with strong confidence.

Result: With p-value below 0.001, the likelihood of these results occurring by random chance is very low. The high t-value (18.21) signals a substantial difference between groups. In simpler terms, this test provides strong evidence that the observed differences are real and not just a fluke.

Claims Frequency: This chart demonstrates that as the difference (delta) increases between the Cowbell Company Factor (CCF) and the Industry Cowbell Factor (ICF), the probability of claims decreases. An organization with CCF seven points lower than ICF is 11 times more likely to file a claim compared to an organization with a CCF eight points higher than its industry.

Claims Severity: This chart demonstrates that as the delta increases between the Cowbell Company Factor (CCF) and the Industry Cowbell Factor (ICF), the severity of claims decreases.

Profitable Underwriting

Leveraging the power of Cowbell Factors, our underwriting teams integrate security signals, risk data, automation, and artificial intelligence—all converging towards our mission: to propel sustainable growth in the cyber insurance market and provide compelling risk transfer solutions for SMEs. The objectification of cyber risk through Cowbell Factors plays a pivotal role in the underwriting process, influencing risk selection, pricing and automation.

Following heatmap shows all submissions across various industries (2-digit NAICS code). Dark green indicates lower risk compared to regular green, followed by yellow and red. Dark red is the worst.

If we filter the entire set of submissions for active policyholders, we see a lot more green and no reds, indicating responsible underwriting and active risk-engineering

The Road Ahead

At Cowbell, we understand that innovation is a continuous process. Our commitment to refining risk modeling remains steadfast. As part of this journey, we’re broadening our horizons to incorporate more diverse data sources, with a current focus on how to enhance our ability to measure third-party cyber risk, new risk related to AI, insider threats, and a range of other risks.

Third-Party Risk Data

Each new piece of data undergoes rigorous normalization and categorization. A notable enhancement in our risk assessment is the recent integration of the Supply Chain Cowbell Factor, introduced in December 2021. The forthcoming addition of third-party risk data promises to further improve the precision and comprehensiveness of this factor.

Generative AI and Risk Assessment

We launched mooGPT in April 2023 to provide on-demand guidance to various support teams in risk assessment, security awareness, incident response planning, risk improvement recommendations, and assistance with the claims process. While MooGPT can provide responses to general questions from various customers, agents, and support teams, it is not designed to cater to multiple users with personalized content, and it does not ensure isolation and customization for each user’s specific needs. To address this challenge, we must either develop a distinct model for each customer or design AI systems with multi-tenancy capabilities.

Multi-tenancy challenge in AI involves a combination of architectural design, security measures, and operational considerations. We have been working on underwriter co-pilot, which ensures strict data isolation and privacy to prevent unauthorized access and maintain compliance with data protection regulations.

At Cowbell, our sights are set on the future, ensuring our policyholders always have an ever more accurate and up-to-date understanding of their cyber risks.