Network security tools

The previous sections examined attacks and defences at the protocol level. This section presents established tools for defence. Their deployment has already been discussed in the module on security operations.

Firewalls

Firewalls can be placed alongside routers or implemented as separate servers, depending on the network structure and implementation. Their role is to act as gatekeepers of the network by inspecting all incoming and outgoing traffic.

Firewalls are typically systems operating at the edge of the network, designed to block malicious traffic and restrict unauthorised access. They monitor traffic and filter data packets according to predefined rules. The rules specify how packets are handled: they can be allowed, dropped, or dropped with a notification sent to the sender, for example via an ICMP message. Filtering is based on factors such as source and destination addresses, protocols used (TCP, UDP, ICMP), port numbers, and TCP flags (such as SYN and ACK).

Traditionally, firewalls operated as simple packet filters that examined only packet header information. Nowadays, stateful firewalls are commonly used; they maintain information about active connections and can associate individual packets with the same data stream. This allows them to monitor connection states and ensure that only expected traffic is allowed through the network.

In modern organisational networks, multiple firewalls are typically used in different parts of the network and for different purposes, for example for network segmentation and monitoring internal traffic.

According to the rules in the figure, packet traffic is managed as follows:

• All hosts in the internal network (172.16.0.0/24) can establish connections to external services using HTTP and HTTPS protocols (ports 80 and 443) (rule #1).
• External devices can connect to the internal SSH server on TCP port 22 (rule #2).
• Return traffic associated with these connections is allowed (rule #3).
• All other traffic is blocked (the so-called default deny rule) (rule #4).

In real environments, firewall rule sets are often considerably more complex than in this example. Defining consistent and comprehensive rules is typically challenging. A common principle is the so-called default deny approach: all traffic is blocked by default, after which only explicitly defined necessary traffic is permitted. Tools such as Firewall Builder and Capirca can assist in managing and maintaining rules.

Application gateway (AG), also known as an application-level proxy, provides access control and enables additional authentication before a connection is accepted. It can also inspect traffic at the application layer, provided that the traffic is not end-to-end encrypted. In a typical operating model, the client establishes a connection to the AG, which handles authentication and then creates a separate connection to the target server. The AG acts as an intermediary and creates two separate sessions. This technically resembles a MITM situation, but in a controlled and intentional manner. A common use case is handling TLS traffic: the encrypted connection can be terminated at the AG, allowing the computationally heavy encryption and decryption to be performed centrally, after which the traffic is forwarded to backend servers. In practice, the AG can also inspect encrypted outbound traffic if the necessary certificates are installed on it.

Circuit gateway (CG) acts as a proxy for TCP connections, enabling internal network devices to connect to external networks. It is typically located alongside a firewall. The most common implementation is SOCKS, which operates almost transparently for applications when client systems are configured to use it. CG is lighter to implement than AG because it does not need to understand application-layer protocols.

Demilitarised zone (DMZ): Secure network design requires segmentation and careful firewall placement. For this purpose, a DMZ area (a so-called perimeter network) is often used, where publicly accessible services such as web servers and DNS servers are placed. The internal network is further divided into multiple security zones according to the organisation’s security architecture. This limits the visibility and accessibility of services so that only authorised users and devices can access them.

Question 1

What is a benefit of using a firewall when implementing network security?

A firewall blocks all insecure and unprotected data transmission in the network.

A firewall prevents the spread of viruses into the protected network.

A firewall is of no benefit, because a skilled attacker can bypass it.

A firewall can allow or block data transmission according to its rule set.

A firewall blocks all attacks on the network if the rule set is sufficiently good.

A firewall detects if someone attempts to eavesdrop on network traffic.

Intrusion detection and prevention systems (IDS/IPS)

Intrusion detection systems (IDS) provide information about abnormal network behaviour. They examine packet payload and higher-layer data, as well as other session-related characteristics that traditional firewalls do not process. IDS systems monitor traffic using sensors and generate alerts when suspicious activity is detected.

Detection is based on comparing traffic with behaviour defined as normal. Alerts can be generated based solely on header information, but Deep Packet Inspection enables analysis of transport- and application-layer payload to identify known malicious patterns. Widely used IDS systems include Snort, Zeek, and Suricata. IDS is used in many contexts, such as detecting malware, denial-of-service attacks, and various network attacks.

A key challenge for IDS systems is accuracy. False positives (legitimate traffic interpreted as malicious) and false negatives (malicious traffic going undetected) still place a significant burden on monitoring.

Signature-based IDS systems compare traffic against a database of known attack patterns, which must be continuously updated. Signatures can be simple (e.g. IP addresses, ports) or more complex payload structures. For example, a rule may trigger an alert if traffic on TCP port 80 contains a specific string (such as “GET”). Such systems require substantial computational power, as traffic is compared against a large number of known patterns. For this reason, parallel and distributed solutions as well as hardware acceleration are used to improve performance.

In anomaly-based IDS systems, traffic is compared with previously learned normal behaviour. During the learning phase, the system builds a model of “normal” traffic, and alerts are generated based on deviations from this model. The challenge is to create a sufficiently representative and clean baseline model. For example, unusual port scanning can be detected as a deviation in the statistical properties of traffic. Despite machine learning methods, the accuracy of anomaly-based detection remains limited in practice.

IDS systems can also be classified based on their placement. A host-based intrusion detection system (HIDS) operates on individual devices and monitors their activity and traffic. Many antivirus programs include such features. A network-based IDS (NIDS), on the other hand, is placed at key points in the network to monitor traffic between segments.

Intrusion prevention system (IPS) extends the functionality of IDS by not only detecting threats but also responding to them automatically. An IPS can block traffic by dropping packets, terminating connections, or updating network filtering rules. The system typically operates in real time and most often also includes IDS functionality.

Question 1

IDS/IPS system in network protection.

The advantage of IDS/IPS is that it reduces the need for network maintenance and thus saves costs.

An intrusion detection system requires a large amount of data transfer in the network to function.

An IDS system works only in small networks because it does not include automated protection.

An IDS/IPS system provides good protection in a network when used together with other security systems.

An intrusion detection system is now an outdated way to secure a network.

Network security monitoring (advanced)

Network monitoring tools help to form an overall picture of the network security situation. Flow-based solutions, such as NetFlow and IPFIX, collect information on data flows and provide an overview of network traffic. They are lightweight in terms of both computation and storage, enabling long-term monitoring. This data can be utilised both for anomaly detection and for forensic analysis.

If necessary, more detailed content can be extracted from traffic for further analysis. For example, NetworkMiner and Xplico can reconstruct files, emails, and HTTP sessions from traffic, as well as identify network devices and their characteristics.

Network mapping and scanning tools, such as Nmap and Zmap, enable the identification of network devices and services using, for example, ICMP and SYN probes. When using such tools, it is important to understand their impact on both network operation and security. Correct interpretation of the information produced by the tools is essential in assessing network security.

IP telescopes are network address ranges in which no services are provided but which are still routable. Traffic arriving at these addresses can be analysed to detect activities such as network scanning or other malicious behaviour. They also enable the observation of so-called backscatter, where traffic related to IP address spoofing can be detected.

A honeypot is a system deliberately made vulnerable with the aim of attracting attackers. It can simulate servers, applications, or devices and collect information on attack methods and tools. The information obtained in this way helps to understand threats and develop more effective protection measures. Honeypots are continuously monitored, and the data they produce is particularly valuable for research and development use.

Security assessment services can map network vulnerabilities by analysing, for example, IP addresses and domain names. Such services are offered both by commercial providers and by internet service providers. However, the reliability of the results may be limited, especially if the assessment is short-term or if the addressing of the target network changes (e.g. due to NAT). Therefore, comprehensive assessment requires both appropriate tools and deep expertise.

SIEM systems aggregate event data from multiple sources, such as IDS systems, firewalls, host-based agents, and log files. They analyse this data centrally and highlight events relevant to security. Combining multiple data sources enables a broader situational picture and helps identify, for example, brute-force attacks, the spread of malware, and network scans.

Access control (advanced)

Networks do not inherently restrict which devices can connect to or use them. The previously described IEEE 802.1X introduces access control by requiring devices to authenticate before gaining access to the network. However, this alone does not guarantee that devices operating within the network remain trustworthy.

For example, a device initially deemed trustworthy may become untrustworthy as a result of changes made to the system. This issue is addressed through network access control, which can be implemented using the TNC architecture. In this case, a device joining the network must meet predefined security requirements before access is granted. In addition, this approach enables more detailed monitoring of traffic, for example in firewalls, allowing specific network traffic to be associated with the client software that generated it.

Ensuring the trustworthiness of the device performing the inspection is itself required. This is covered in the next module.

A significant limitation of TNC and similar mechanisms is that they describe only the state of the device at the time of inspection. Changes during system operation cannot be reliably detected or prevented. The security state of a device may deteriorate later, for example due to malicious software or changes made via remote administration. In such cases, the current state of the device no longer corresponds to the trusted state identified in the initial inspection.

The principle of zero trust in networks (advanced)

In zero trust networks, the traditional assumption that devices within the network can be inherently trusted is abandoned. The idea is that all devices are untrusted unless their trustworthiness has been explicitly verified. This represents a significant paradigm shift compared to traditional network models, where the network is divided into trusted and untrusted parts, for example using firewalls.

The shift is a result of the fact that in traditional networks it is easy to lose visibility and control over which devices are connected to a seemingly protected internal network. Mobile devices in particular constantly move between different networks, such as home networks and organisational networks, which reduces the significance of the traditional network boundary.

Transitioning to a zero trust model is not straightforward. It requires a comprehensive understanding of network resources, such as users, devices, services, and data. In addition, continuous situational awareness of the security level of these resources is needed. All service requests must be approved through strong authentication, preferably using multi-factor authentication combined with single sign-on to maintain a smooth user experience.

A challenge is that not all traditional services can be easily adapted to the zero trust model. This requires modifications to services so that they support common authentication mechanisms (e.g. OpenID Connect, OAuth or SAML).

One of the best-known zero trust architectures is BeyondCorp. In this model, network access control is used to identify devices, and users are authenticated using a centralised single sign-on system. Previously internal services are treated as external services, protected through an access proxy. This enforces strong encryption and precisely defined access control for all connections.

Countermeasures against denial-of-service attacks (advanced)

Denial-of-service (DoS) attacks can be broadly divided into two categories based on the resources they target. Large-scale attacks aim to exhaust the target’s network bandwidth. Typical examples include amplification attacks and distributed denial-of-service (DDoS) attacks, which utilise large botnets to generate massive traffic volumes. These attacks often target individual services or networks but can, in the worst case, affect the entire network infrastructure.

The effects of attacks can be mitigated most effectively by stopping malicious traffic as early as possible, before it reaches the target network. For this purpose, commercial filtering services (“scrubbing services”) are used, which are positioned between the Internet and the customer network and remove malicious traffic before it reaches the target.

Alternatively, traffic can be redirected or filtered at the routing level. For example, BGP announcements can be used to steer traffic away from the target or into dedicated filtering zones, and the BGP FlowSpec mechanism can define more detailed traffic filtering rules for routers.

In application-layer DoS attacks, the target is the service’s software resources, such as memory and CPU processing power. In these attacks, the attacker attempts to overload the service, for example with a large number of requests. Defensive measures are often application-specific. In TCP-based services, SYN cookies and rate limiting can protect against attacks targeting connection establishment.

In addition, CAPTCHA mechanisms can be used to distinguish between traffic generated by humans and automated systems. This is particularly useful in web services where it is necessary to prevent load and abuse caused by bots.

Posting submission...