AI Accomplice

How have cybercriminals been using AI to support their attacks?

I’ll give a nuanced answer to this across multiple dimensions. AI has become a force multiplier for cybercriminals, making their attacks more effective and scalable. We’ve seen evidence of this in the real world that I’ll walk through.

No. 1, AI-enhanced phishing attacks: AI enables the generation of highly convincing phishing emails, eliminating traditional red flags such as grammatical errors and poor formatting. We’ve seen this being used to increase the effectiveness of a phishing campaign.

Based on sources that we’ve reviewed, AI-generated phishing emails have a demonstrated 78% higher click-through rate compared to manually written scams.

What does this mean to organizations? Organizations must invest in AI-driven threat detection and continuous employee security awareness. So, if you’re aware of whether an email is AI-generated or not, you can make better decisions on what you do with it. Example No. 2 is around deepfakes. There has been evidence of deepfake-based financial fraud. Executives have been impersonated. Even nation-state leaders have been impersonated recently. All of this says that we need to authenticate the identity of the individual that is presenting some information to you. So, robust multifactor authentication and verification protocols become very important for financial transactions as we think about the future.

The last example is on AI-powered social engineering attacks. Criminals use AI to analyze publicly available data for individuals. And they can craft highly successful and targeted spear-phishing campaigns. You can be offered fake jobs, and you can be provided misleading information about family members. All of these are examples of social engineering. If you look at all of these different themes—phishing attacks, financial fraud, social engineering—the common theme is AI basically provides a boost to the threat actors in making these easier to carry out and increases the probability of success.

You’ve cited autonomous AI agents as a growing threat. What is an autonomous AI agent?

Autonomous AI agents are self-learning cyber tools that can conduct reconnaissance, identify vulnerabilities, and deploy malware without human intervention. All of the things that a trained cybercriminal can do, these autonomous agents can do. Unlike traditional cyber threats, these agents adapt in real time, lowering the cost and effort required for attackers to execute complex operations. What does this mean to the industry at large? Security strategies must shift from being static defenses to dynamic and AI-powered threat detection and response systems.

How do the autonomous AI agents support scaling of cyberattacks?

With no human in the loop, they can execute attacks at unprecedented speed and scale. It’s really the scale element that is quite concerning for the insurance industry, as it cares more about one campaign that can impact multiple companies in their portfolio rather than one large breach. The ability for these autonomous AI agents to scale cyberattacks is a particularly worrisome development for the insurance market, and threat actors can automate and optimize these attacks, from identifying targets to breaching defenses and exfiltrating data using these autonomous agents.

I picture the cyber singularity with AI agents assembling AI gangs, but perhaps that’s not in the future yet.

It could be. AI controlling AI is certainly one of the concerns around AI security at large. So, you do want some controls in place on where AI gets utilized. So far, I’ve really talked about human threat actors deploying AI to make their attacks more effective. But you could envision a future state of the world where there are AI agents controlling AI agents.

Similar to large language models, can autonomous AI agents be trained on prior exploits?

Just like large language models can improve through training on large data sets of relevant value, autonomous AI agents can refine their attack strategies by learning from past cyber incidents in terms of what has worked and what has not worked. In the simplest possible terms, you can think of an AI agent as a human brain. As a human being, you try out certain things and you utilize things that work based on past experience. That’s where experience matters.

AI agents are no different except that they start with a large corpus of useful data that helps them optimize their attack strategies. This allows them to evade traditional security controls and exploit vulnerabilities faster than ever before. As a result, organizations must continue to update security postures and leverage AI-driven threat intelligence to stay ahead of these evolving attack methodologies. This idea of being adaptive is what I would call out as one of the characteristic traits of AI agents.

Recently DeepSeek roiled the AI industry. Does the availability of cheaper, easier AI help cybercriminals?

AI acts as an aid to an enterprising cybercriminal, so as AI becomes more accessible and cost-effective, even lower-skilled cybercriminals can launch highly sophisticated attacks and scale these attacks. The business of cybercrime is no different from the business of technology: it is to basically make money with the lowest expense possible. With the availability of AI-generated malware, automated phishing campaigns, and deepfake technology tools, this can lead to a democratization of cybercrime. While we believe that the proportion of cybercriminals relative to the broader population will stay somewhat uniform, we believe that the cybercriminals themselves will be more sophisticated and empowered through the use of these AI technologies.

How is CyberCube using AI to strengthen security for its clients?

CyberCube utilizes AI to enhance cyber risk assessment and mitigation strategies for businesses via insurers. The customers that we serve are the broader insurance market—brokers, carriers, reinsurers. We are using AI-driven risk quantification to provide predictive analytics to the insurance market where we identify emerging threats and flag the potential financial impact associated with these threats. We also provide portfolio risk modeling for insurers where we help carriers assess accumulation risk, and we help them optimize their underwriting strategies.

Finally, we provide automated cyber risk assessments, which are AI-powered evaluations that provide insights into a company’s security posture. The implication of all of this is that insurers benefit from understanding the drivers of cyber risk, and they utilize this information for underwriting decisions, thereby improving the cyber hygiene of businesses that they provide coverage for. So that’s really the value chain that we work around.

Have new defense tools improved cyber hygiene?

The short answer is yes. We have seen insurers prioritize certain questions in their underwriting questionnaires, such as asking whether multifactor authentication is deployed at scale, and the reason why insurers are doing this is because some of these factors are directly correlated to the frequency of a cyber event. If you have multifactor authentication in place, you’re less likely to have a crippling cyberattack as a business. This in turn has improved the security standard for an average business.

If you think about a small business, in the prior world without cyber insurance, they looked at cybersecurity as an afterthought. But now businesses are forced to consider cybersecurity as a business priority rather than an afterthought. Insurers have been instrumental in driving the understanding of cyber risk and making those factors important in how businesses invest in cybersecurity tools.

What role do brokers play in advising on cyber threats, particularly to small- and medium-size businesses?

Cyber brokers play a pivotal role in guiding SMEs [small and medium enterprises] through the evolving cyber threat landscape in a variety of different ways. No. 1, providing cyber risk education and dispelling the misconception that SMEs are not prime cyber targets. Cybercriminals are opportunistic. They will go after any vulnerable target that they come across.

No. 2, offering tailored risk assessments using AI-powered tools, such as CyberCube, to assess the security hygiene of a company and recommend appropriate coverage.

No. 3, simplifying complex cyber policies, like translating technical jargon into actionable business insights for the buyer of insurance and advocacy during the claims process. Whenever SMEs are attacked, they receive benefits from policies, post-incident, and brokers make sure that SMEs are supported when they work with them as clients.

Finally, facilitating access to security resources. Brokers are increasingly aware of cybersecurity solutions that can help solve business problems. Unlike large corporations that have a CISO [chief information security officer] and a security team, SMEs often lack these capabilities, and as a result, brokers can act as advisors to recommend cybersecurity solutions, to improve cyber hygiene to get better terms and better rates with carriers. The implications for this from a business perspective is that SMEs should proactively engage with cyber insurance brokers to build resilience, to secure comprehensive, cost-effective cyber insurance coverage.