Artificial intelligence and cybersecurity¶

Artificial intelligence (AI) and cybersecurity are becoming ever more tightly intertwined, as AI can be used effectively both for defence and for attack. Complex legal, political, and ethical perspectives, as well as concerns about data validity, are central challenges for AI.

AI systems can analyse large volumes of data, identifying anomalies and potential threats that traditional methods would not be able to detect. Machine learning models can predict and prevent phishing and denial-of-service attacks by analysing behavioural patterns and network traffic. In addition, AI helps to automate security tasks such as vulnerability scanning and the management of security updates. This enables the processing of large amounts of data and frees people to focus on more strategic work.

Cybersecurity of artificial intelligence solutions¶

AI systems themselves are also vulnerable to attacks, for example:

Attacks against learning processes. Feeding incorrect data to an AI system is known as poisoning.

Data manipulation. Data contained in or produced by an AI system is modified. The system is made to produce plausible-looking but incorrect answers. This phenomenon is known as hallucination or fabrication, and it can also occur without external manipulation.

Model inversion attacks. A technically skilled or aggressive user extracts information from an AI system that was not intended to be disclosed. This may include personal data, material related to the preparation of crimes, or illegal content. Model inversion can also involve attempts to reconstruct the system’s training data or internal structures.

An AI Trojan is similar to a traditional Trojan horse: malicious functionality built into an AI system that remains hidden. The system may otherwise function normally, but with a specific input it begins to produce harmful content or disclose confidential information.

It is important to protect AI systems throughout their entire lifecycle. This includes, among other things, strong access control, secure data handling, continuous monitoring, and regular security audits.

Artificial intelligence and copyright¶

AI can both create new content and modify existing material in ways that challenge traditional notions of copyright. This raises questions about who owns the copyright: the developer of the algorithm, the user, or someone else?

Content generated by AI blurs the line between original and derivative works. AI algorithms can produce text, images, music, and videos that may closely resemble human-created art and communication. In addition, AI may incorporate protected material into its outputs, making copyright enforcement and the detection of infringements even more difficult. Moreover, infringement may already have occurred during the training phase if material has been used without permission.

On the other hand, AI can help to protect copyrights and combat copyright infringement. AI–based tools can scan vast amounts of online content quickly and accurately, identifying copyright violations such as unauthorised copies and plagiarism. AI tools can automatically monitor and supervise content usage, making it easier for rights holders to detect and respond to infringements. In addition, AI can help generate watermarks and other protective measures that make unauthorised use and distribution of content more difficult.

Cyber attacks¶

AI has a growing role in cyber attacks, as it enables the planning and execution of increasingly sophisticated and targeted attacks. Attackers can use AI to analyse large amounts of data in order to identify vulnerabilities and optimise their attack strategies. For example, AI-assisted malware can learn and adapt in real time, bypassing traditional security systems. In addition, AI can automate complex attack processes such as phishing, increasing their effectiveness and making them harder to detect.

In combating cyber attacks and maintaining cybersecurity, AI is an important tool. It can analyse network traffic in real time, identifying and responding to anomalies much faster than traditional methods. For example, machine learning can be used to develop models that detect unusual behaviour, enabling rapid intervention before damage occurs. Furthermore, AI can predict potential threats by analysing historical data and current trends, helping organisations to prepare and strengthen their protective measures in advance.

AI can be used, for example, to detect and prevent phishing. Compared to other technologies, AI is more effective at identifying untrustworthy emails or websites. When detection is carried out in real time, an attack can be stopped at the very beginning.

Influencing people¶

Text, images, audio, or video generated by AI enable highly effective social engineering attacks. On the other hand, AI can be used to analyse behaviour online or in applications, allowing suspicious messages and actions to be stopped before they affect people. See more details in module 5.7 here and here. Here the discussion continues towards broader influence, propaganda.

With the help of AI, highly targeted communication can be created and disseminated efficiently. Algorithms can analyse large data sets and identify user profiles, allowing propaganda messages to be precisely targeted at different groups based on their interests and beliefs.

AI can also be used to produce convincing fake images, videos, and news articles, making it harder to distinguish false information from truth. This development highlights the need for critical media literacy and for recognising reliable sources of information.

On the other hand, AI provides powerful tools for countering propaganda. AI-based systems can automatically analyse and filter vast amounts of online content to identify suspicious or harmful material. They can detect fake images, videos, and news using algorithms that identify inconsistencies and signs of manipulation. In addition, AI can help monitor and analyse the spread of propaganda campaigns in real time, enabling rapid response and countermeasures.

Research on the misuse of artificial intelligence¶

The research article Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data (Marchal & al. 2024) presents a comprehensive classification of ways in which generative AI (GenAI) is misused. It is based on academic literature and a qualitative analysis of 200 media reports addressing the misuse of GenAI systems, covering examples from the period January 2023 to March 2024.

The study found that:

Manipulation of human likeness and falsification of evidence are the most common misuse tactics. Most of these were carried out with a clear intent to influence public opinion, enable scams or fraud, or generate profit.

The majority of reported misuse did not involve technologically advanced use of or attacks on GenAI systems. Instead, it mainly involved the exploitation of readily available GenAI features, requiring only limited technical expertise.

The development, proliferation, and increased availability of GenAI tools appear to bring with them new, lower-level forms of misuse. These are not overtly malicious and do not clearly violate terms of use, yet they still have concerning ethical implications. These include the emergence of new forms of communication for political influence, self-promotion, and defence. Such forms blur the boundaries between authenticity and deception.

The classification consists of the goals of misuse and the forms of GenAI use applied to them. A concise overview can be found in the tables in the article’s appendices. However, the article does not cover the following perspectives: denial-of-service, data mining to discover attack methods, Trojan horses (in the AI system itself or in its software environment), or nested structures of critical systems in which an advanced persistent threat attack may succeed by operating from an internal component assumed to be harmless, such as AI itself.