Computer Networking Overview

May 06, 2025

Great. I’ll compile a modular, section-by-section overview of computer networking tailored for senior-level software engineers and system design interviews—especially within the AWS ecosystem. The sections will be delivered in markdown format here in the chat for easy reference and study.

I’ll get started and share the first section with you shortly.

Computer Networking Overview

This comprehensive overview covers core computer networking models, protocols, components, architectures, and AWS cloud networking services. It is tailored for a senior software engineer preparing for system design interviews and working with AWS. Each section below is modular and self-contained, providing high-level concepts, practical examples, and best practices relevant to real-world systems design and cloud infrastructure.

Fundamental Networking Models

Networking models provide a layered framework for understanding how data travels across networks. The two primary models are the OSI seven-layer model and the TCP/IP model. These models break down networking functions into layers, each building on the layer below. In practice, the models guide design and troubleshooting by isolating issues to specific layers.

OSI Model (7 Layers)

Figure: The 7 Layers of the OSI Model, from Physical (Layer 1) to Application (Layer 7). Each layer has specific responsibilities and interfaces with the layers above and below.

The Open Systems Interconnection (OSI) model defines seven abstraction layers that describe how data moves from an application to the physical network medium and back. The layers, from 1 (lowest) to 7 (highest), are: Physical, Data Link, Network, Transport, Session, Presentation, and Application. Each layer encapsulates a set of functions (for example, the Network layer handles routing of packets, while the Transport layer ensures reliable delivery). This layered approach provides a “universal language” for different systems to communicate. In practice, the OSI model is mostly used as a teaching and troubleshooting tool – real-world protocols often span multiple layers or skip some. For example, an issue with an unreachable web service might be diagnosed by checking connectivity at Layer 3 (IP routing) and Layer 4 (TCP handshake) before looking at an application-layer (Layer 7) problem. The OSI model is rarely implemented exactly in modern networks, but it remains valuable for understanding and explaining network behavior.

Layers and their roles: At Layer 1 (Physical), raw bits are transmitted over a medium (cables, Wi-Fi, etc.). Layer 2 (Data Link) handles framing and local node-to-node delivery (e.g., Ethernet MAC addressing). Layer 3 (Network) manages addressing and routing between networks – the Internet Protocol (IP) operates here. Layer 4 (Transport) provides end-to-end communication and reliability; e.g. TCP (connection-oriented, reliable) and UDP (connectionless, best-effort). Layer 5 (Session) governs the establishment and teardown of sessions (e.g., managing multiple logical connections, as with an RPC session). Layer 6 (Presentation) deals with data format and syntax, ensuring that data from the application layer can be understood by the receiving system (examples: encryption/decryption, serialization like JSON or XML). Layer 7 (Application) is the closest to the end-user; it includes protocols for specific networking applications – e.g. HTTP for web, SMTP for email. These layers interact in a stack: each layer on the sender adds its header (encapsulation) and the corresponding layer on the receiver reads and strips it (decapsulation). Understanding the OSI layers is helpful in interviews for explaining terms like “L4 load balancer” or debugging (e.g., identifying if a problem is at the network layer vs. the application layer). Not every system uses all layers distinctly (some layers might be empty or combined), but the model’s separation of concerns aids in clarity.

TCP/IP Model

The TCP/IP model is the pragmatic model on which the modern Internet is built. It condenses the OSI layers into four or five layers: typically Link (Network Interface), Internet, Transport, and Application (some versions separate Link into Physical and Data Link, totaling five layers). While the OSI model is a theoretical reference, the TCP/IP model maps more directly to real protocols in use. For example, in the TCP/IP model, the Internet layer corresponds to IP (IPv4/IPv6) for routing, the Transport layer includes TCP and UDP, and the Application layer encompasses everything from HTTP to DNS. The TCP/IP model’s simplicity reflects the design of the Internet, where protocols are defined in these four layers and interoperability is key. In practice, when designing systems we often refer to TCP/IP layers; e.g., designing a solution “at the transport layer” likely implies working with TCP/UDP rather than inventing a new layer. The OSI model remains useful for conceptual understanding, but the TCP/IP model is more commonly used in practice today, especially when discussing real-world networking (for instance, engineers often speak of “layer 4 vs layer 7 load balancing” in terms of TCP/IP and OSI equivalently). A senior engineer should understand both models: OSI for its vocabulary and thoroughness, and TCP/IP for its direct mapping to actual protocols and the Internet’s architecture.

Essential Networking Protocols

Modern networks rely on a suite of core protocols that operate at different layers to enable communication. This section covers key protocols and concepts: HTTP/HTTPS at the application layer, TCP vs UDP at the transport layer, IP (v4 & v6) at the network layer (with addressing/subnetting), and DNS for name resolution. Understanding these is crucial for system design (e.g., choosing TCP or UDP for a service, or designing a domain name scheme for microservices).

HTTP and HTTPS (Web Protocols)

HTTP (HyperText Transfer Protocol) is the fundamental application-layer protocol of the World Wide Web. It defines how clients (typically web browsers) request resources from servers and how servers respond. HTTP is a stateless, request-response protocol, meaning each request from a client is independent – the server does not retain session information between requests by default. A client (like a browser) sends an HTTP request (e.g., a GET request for a webpage), and the server returns an HTTP response (e.g., an HTML page). Common HTTP methods include GET (retrieve data), POST (submit data to be processed), PUT, DELETE, etc., often corresponding to CRUD operations in RESTful APIs. HTTP responses come with status codes indicating the result of the request: for example, 200 OK (success), 404 Not Found, 500 Internal Server Error, etc. These codes are grouped into classes – 1xx informational, 2xx success, 3xx redirection, 4xx client error, 5xx server error. For instance, a 200 means success, 404 means the requested resource was not found, 503 means the server is unavailable. Understanding status code classes is useful in debugging and designing REST APIs (e.g., returning 404 vs 400 for different error conditions).

HTTPS is the secure version of HTTP. It stands for HTTP Secure and essentially means HTTP over TLS/SSL encryption. When using HTTPS (e.g., https:// URLs), the client and server perform a TLS handshake to establish an encrypted connection before exchanging HTTP data, thereby providing confidentiality and integrity. TLS (Transport Layer Security) is the modern version of SSL and is a security protocol that encrypts communication between a client and server. A primary use case of TLS is securing web traffic (HTTPS) to prevent eavesdropping or tampering. In practice, when a browser connects to an HTTPS site, it verifies the server’s identity via an X.509 certificate and then negotiates encryption keys (this is the TLS handshake). After that, HTTP requests and responses are encrypted. TLS provides assurances that the client is talking to the genuine server (authentication via certificates) and that no one can read or alter the data in transit. Modern best practices require HTTPS for virtually all web traffic (e.g., browsers flag non-HTTPS sites as insecure). In system design, one should note that HTTPS adds a bit of overhead (CPU for encryption, and latency for the handshake), but it’s necessary for security. Also, load balancers or proxies often terminate TLS (performing decryption) and forward traffic internally over HTTP – this is a common architecture for handling HTTPS at scale. In summary, HTTP/HTTPS knowledge includes understanding the stateless nature of HTTP (and the need for mechanisms like cookies or tokens to maintain sessions), knowing the common status codes, and recognizing how TLS secures communications.

TCP vs UDP (Transport Layer Protocols)

At the transport layer, TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) are the two fundamental protocols. Both use IP underneath but differ significantly in behavior and use cases:

TCP is a connection-oriented, reliable protocol. It establishes a connection via a three-way handshake (SYN, SYN-ACK, ACK) before data transfer, and ensures all data arrives intact and in order. It achieves reliability with sequence numbers and acknowledgments, retransmitting lost packets and controlling flow. This means that if you send data via TCP, you either get it at the other end or get an error – TCP will automatically handle packet loss, retransmissions, and reorder out-of-sequence packets. It also provides congestion control to avoid overwhelming networks. The cost of these features is overhead (extra packets like ACKs) and latency (waiting for ACKs, etc.). Use cases for TCP include applications where accuracy is critical – e.g. web traffic (HTTP), file transfers, database connections – basically most application protocols on the Internet use TCP by default for its reliability.
UDP is a connectionless, “best-effort” protocol. It sends packets (datagrams) without establishing a prior connection and without acknowledgments. UDP does not guarantee delivery, ordering, or duplicate protection – packets may arrive out of order, or not at all, and UDP itself won’t inform the sender. Because of this simplicity, UDP has much lower overhead and latency. Use cases for UDP involve scenarios where speed is prioritized over reliability, or the application layer handles its own error correction. Examples: real-time media streaming (video calls, online gaming) often use UDP because a dropped packet is better than delaying the stream (and protocols like RTP add their own minor recovery or just live with occasional loss). Another use is DNS queries – DNS is typically over UDP for quick request/response (with the application retrying if needed). Also, IoT devices or any custom protocol that doesn’t need TCP’s guarantees may use UDP. In interviews, a classic question is when you’d use UDP over TCP – for real-time systems (where old data is irrelevant by the time it’d be retransmitted) or high-throughput systems on reliable networks.

In summary, TCP vs UDP trade-off comes down to reliability vs. latency. TCP gives you heavy guarantees (ordering, no duplicates, reliable delivery) at the cost of extra network chatter and complexity, whereas UDP is essentially just “send and forget,” suitable for cases where the application can tolerate or handle some loss. As a senior engineer, it’s important to know that TCP can struggle with high-latency links or multicast (UDP supports multicast, TCP doesn’t), and that UDP requires you to consider packet loss at the application level. Many protocols build on UDP for speed and add their own reliability only if needed (e.g., QUIC is a modern protocol used in HTTP/3 that runs over UDP to get the benefits of UDP with built-in congestion control and reliability at the application layer). In AWS, most services (like HTTP-based services) use TCP, but things like AWS CloudWatch Logs or metrics might use HTTP (thus TCP) for reliability. If designing a custom service, say a telemetry ingestion service that can lose the occasional packet but needs low latency, you might design it over UDP.

Reliability and ordering: TCP ensures in-order delivery or the connection breaks – it’s “near-lossless”. UDP is “lossy”; the application might receive packets out of order or not at all, and it must deal with that. For example, a video call app might see a missing UDP packet and decide to not display a few pixels for a moment rather than pause the whole video. Knowing which transport to use is crucial: e.g., a payment system should not use UDP (you don’t want lost data in transactions), whereas a live metrics dashboard could use UDP for real-time updates where occasional drops are acceptable.

IP Protocol (IPv4, IPv6 and Addressing/Subnetting)

IP (Internet Protocol) operates at the network layer (Layer 3) and is the core protocol that delivers packets from source to destination across networks. It provides addressing (each device has an IP address) and routing (forwarding through intermediate routers). There are two versions in use:

IPv4: Uses 32-bit addresses (e.g., 192.168.10.150) which allow about 4.3 billion unique addresses. IPv4 addresses are typically written in dot-decimal notation (four octets). Due to historical allocation and inefficiencies, IPv4 addresses effectively ran out, leading to private addressing (RFC1918 private networks like 10.0.0.0/8) and widespread use of NAT (Network Address Translation) to allow multiple devices to share one public IP. IPv4 has been the workhorse of the Internet since 1983.
IPv6: Uses 128-bit addresses (written in hexadecimal colon-separated form, e.g. 2001:0db8:85a3::8a2e:0370:7334). This provides an astronomically large address space (approximately 3.4×10^38 addresses), essentially solving the address exhaustion problem by offering 1,028 times more addresses than IPv4. Besides address size, IPv6 incorporates improvements such as simplified header format, built-in IPsec security, and no need for NAT because there are enough addresses for end-to-end addressing. IPv6 addresses are longer (eight groups of four hex digits). For example, an IPv6 address might look like 2406:da1c::1. Transition to IPv6 has been gradual – many systems and cloud providers (like AWS) support dual-stack (both v4 and v6). For a system design, being aware of IPv6 is important (AWS allows creating IPv6-enabled VPCs, for instance), especially for future-proofing. One key interview point is that IPv6 does not require NAT due to abundant addresses; instead, routing and firewall rules control traffic. Another is compatibility: IPv4 and IPv6 are not directly interoperable, so many systems run both.

Addressing and Subnetting: An IP address has two parts: network prefix and host identifier. Subnetting is the practice of dividing a network into smaller networks (subnets) by extending the network prefix. Each subnet is identified by a network address and a subnet mask (or prefix length, e.g., /24). For example, 192.168.1.0/24 represents a subnet where the first 24 bits are network (192.168.1.0) and the remaining 8 bits are for hosts (256 addresses, of which a few are reserved). Why subnet? It improves routing efficiency, security, and management. A subnet (subnetwork) is a network inside a larger network – it allows localizing traffic so it doesn’t have to traverse the entire network. By subnetting, network traffic can travel shorter distances without passing through unnecessary routers. For instance, in a data center, you might subnet by rack or by department so that most traffic stays local. From a design perspective, subnetting helps segment networks (e.g., separating a database subnet from a web subnet for security). In cloud environments like AWS, you must subnet your VPC IP range into subnets (often at least one public and one private subnet per availability zone). Each subnet has a CIDR block (range of IPs) and belongs to one availability zone.

A concept related to subnetting is the subnet mask (e.g., 255.255.255.0 for /24) which delineates network vs host bits. Another concept is CIDR (Classless Inter-Domain Routing) notation (like /16, /24) which replaced classful networks to allow more flexible allocation. For an interview, one should know how to calculate the number of addresses in a subnet (e.g., a /24 has 2^(32-24)=256 addresses) and understand broadcast vs network addresses, etc., though cloud platforms abstract some of these details. Key takeaway: IP addressing enables global unique identification of endpoints, and subnetting is used to logically partition networks to suit organizational or topological needs.

In practice, when designing an AWS VPC, you might choose an IPv4 CIDR like 10.0.0.0/16 (gives 65k addresses) and then subnet it into /24s for different zones or services. IPv6 can also be enabled (AWS gives a /56 or /64 block) to allow IPv6 addressing. Many system design considerations (like load balancer addressing, NAT gateways, etc.) tie back to IP addresses and subnets.

DNS (Domain Name System)

DNS (Domain Name System) is the phonebook of the Internet, translating human-friendly domain names (like example.com) into IP addresses that computers use to route traffic. When you enter a URL or use an API endpoint, a DNS resolution occurs to find the server’s address. DNS is an application-layer protocol, but it underpins practically all internet communications.

How DNS resolution works: DNS is a distributed, hierarchical system. There are different types of DNS servers that work together in a lookup:

Recursive resolver: This is the DNS client’s agent (often provided by your ISP or a public DNS like Google’s 8.8.8.8 or Cloudflare’s 1.1.1.1). When your computer needs to resolve a name, it asks a recursive resolver. The resolver will do the legwork of querying other DNS servers if the answer isn’t cached.
Root servers: There are 13 root server addresses (server clusters) worldwide. They know where to direct queries for top-level domains (TLDs). If a resolver doesn’t know the answer, it asks a root server which returns a referral to the appropriate TLD name server (e.g., for .com domains).
TLD name servers: These handle top-level domains like .com, .org, .net, country TLDs, etc. A .com TLD server will direct the query to the authoritative name server for the specific domain.
Authoritative name servers: These are the servers that actually host the DNS records for a domain (often managed by your DNS provider or registrar). For example, if you are resolving api.example.com, after the resolver queries the root and the .com TLD, it will reach the authoritative server for example.com, which will provide the IP address (A/AAAA record) for api.example.com.

This process is iterative: the recursive resolver goes step by step, and it caches responses along the way to speed up future lookups. Caching is critical to DNS’s performance – once a name is resolved, the resolver will remember it for the TTL (time-to-live) specified in the DNS record, so subsequent requests are answered quickly from cache rather than hitting the root/TLD servers again.

DNS records: DNS records map names to data. Common record types include A (IPv4 address for a host), AAAA (IPv6 address), CNAME (alias one name to another), MX (mail exchange servers for email), TXT (arbitrary text, often for verification or SPF/DKIM email security), NS (delegates to a name server), and SOA (start of authority, domain metadata). For instance, an A record for www.example.com might point to 93.184.216.34. A CNAME for photos.example.com might alias to images.example.com (so that the latter’s A record is used). In system design, you use DNS to provide friendly endpoints for your services and to do things like load distribution (by returning multiple A records or using weighted DNS). DNS caching can sometimes cause stale data issues (e.g., if you change an IP but the old one is cached), which is why TTLs should be chosen carefully – short TTLs for highly dynamic services, longer for stable mappings.

Server roles: A single machine can be configured as different DNS roles:

Recursive resolver (often at ISP or local network – e.g., your Wi-Fi router often does simple DNS recursion for you).
Authoritative server for a domain (managed by the domain owner or DNS host). In AWS, Route 53 is an authoritative DNS service (more in AWS section). Also, operating systems have a stub resolver (the part that talks to the recursive resolver, usually via /etc/resolv.conf config or OS settings pointing to a DNS server).

From a security standpoint, DNS has weaknesses (like spoofing), which led to DNSSEC (DNS Security Extensions) where responses are signed to ensure authenticity. For performance, many large services use CDNs and Anycast DNS to make sure DNS queries are answered by a nearby server.

For a senior engineer, understanding DNS is key: e.g., how a CDN like CloudFront uses CNAMEs, how to design domain naming for microservices (perhaps using subdomains), how to handle DNS-based load balancing or failover (Route 53 can do health-check based failover). Also, knowing that DNS resolution adds latency to the first request (DNS lookup time) – typically a few tens of milliseconds – and that clients cache results (browsers, OS cache) which is why a bad DNS record can linger. In summary, DNS translates names to IPs and is structured in a hierarchy of servers (root → TLD → authoritative), with caching at resolvers to improve performance. It’s a critical piece often brought up in system design (for example, how do services discover each other? Possibly via DNS names).

Common Networking Components

Networking involves hardware and software components each playing distinct roles. Key components include switches, routers, and gateways (for moving data through networks), load balancers and proxies (for distributing and intermediating traffic), and firewalls, VPNs, and security groups (for securing network boundaries). Understanding these is important both for on-premise architecture and cloud (where many of these exist virtually).

Routers, Switches, and Gateways

In a nutshell: switches connect devices within a network (LAN), routers connect different networks (LAN to LAN or LAN to WAN), and gateways connect fundamentally different networks or protocols.

A network switch operates mainly at Layer 2 (Data Link). It has multiple ports and forwards Ethernet frames between devices based on MAC addresses. Switches effectively create a network where all connected devices can communicate directly. They learn MAC addresses by examining incoming frames and only send frames out the port destined for the target MAC, which is more efficient than an old-fashioned hub (which would broadcast to all ports). Switches are fundamental for building local networks (e.g., all devices in an office floor might plug into switches). They isolate collision domains, meaning each link is its own segment, allowing concurrent communications. Some advanced switches (layer-3 switches) can perform routing functions too, but generally, a switch is about connecting devices on the same network. For example, in a data center, you might have a top-of-rack switch connecting all servers in that rack.
A router operates at Layer 3 (Network). Routers connect different IP networks and make forwarding decisions based on IP addresses. In simple terms, switches create networks, routers connect networks. Your home Wi-Fi router, for instance, routes between your home network (e.g., 192.168.0.0/24) and the internet (via your ISP). Routers use routing tables to decide which interface to send an outgoing packet to, based on the destination IP and network prefixes. They enable inter-network communication – without routers, networks would be isolated islands. On the Internet, routers owned by ISPs exchange routes using protocols like BGP. In system design, routers might not be directly discussed unless designing network architecture, but one should know that in AWS, for example, a VPC has an implicit router and you manage routes via route tables. If you have multiple subnets, the VPC router handles traffic between them (if allowed by NACLs etc.). A router can also do NAT (network address translation) as often done in home routers or cloud NAT gateways. A key point is routers separate broadcast domains – a broadcast in one subnet won’t go through a router to another. Gateway is a term often used interchangeably with router in the context of “default gateway,” which is simply the router that connects a local network to other networks (often the internet).
A gateway in networking is a broader term. It generally means a device that acts as an entry/exit point between two networks. Often a router serving as the “edge” of a network (connecting to an external network) is called a gateway. More specifically, a gateway joins two dissimilar networks, potentially translating between different protocols. For example, early networks had email gateways between different email systems, or a gateway could connect a TCP/IP network with an older protocol network. In IP terms, your default gateway is the IP address of the router your host sends traffic to when the destination is outside your local subnet. In AWS, an Internet Gateway (IGW) is what allows VPC traffic to reach the internet – effectively acting as the gateway between your private cloud network and the public internet. Another example: a API gateway (in application terms) acts as an entry point between external clients and internal services (though that’s higher-level than network layer). But at the network level, think of gateway as either a synonym for a router at the edge or a device that bridges different network systems. In summary, routers vs gateways: “While a router is used to join two similar networks, a gateway is used to join two dissimilar networks.” – in practice, many devices (including home routers) perform both roles, so the terms can blur. In exam or interview contexts, it’s good to mention a gateway might perform protocol conversion if needed and is the “last stop” before traffic leaves a network or the first point of entry.

In cloud design, you don’t manually handle switches and routers – AWS handles those, but they expose abstractions: Subnets and route tables (router behavior) and Gateways (internet gateway, NAT gateway, etc.). On-premise, one designs which subnets go to which routers, uses switches to connect servers, etc. A strong understanding ensures you can reason about things like why two instances in different subnets can’t communicate – maybe a route is missing (router issue) or a NACL blocking (firewall issue) rather than a switch issue, etc.

Load Balancers & Reverse Proxies

Load balancers and reverse proxies are mechanisms to distribute and manage incoming traffic to servers. They often overlap in functionality (and a single software or device can be both). Both typically sit between clients and back-end servers, but there are subtle differences and use cases:

A load balancer (LB) is designed to distribute incoming load across multiple servers (targets) to improve scalability and reliability. The load balancer presents a single address (IP or DNS name) to clients; behind that, it maintains a pool of servers. When requests come in, the LB chooses a server (based on a balancing algorithm: round-robin, least connections, etc.) and forwards the request. The client is unaware of which server handled it. Load balancers can operate at Layer 4 or Layer 7. A Layer 4 load balancer (e.g., AWS’s Network Load Balancer, or a TCP LB in a hardware device) looks at networking info (IP and port) and forwards packets without understanding the application protocol. It’s very fast and efficient, suitable for raw TCP/UDP traffic. A Layer 7 load balancer (e.g., AWS Application Load Balancer or an Nginx/HAProxy configured for HTTP) actually looks at the application protocol (HTTP, for instance) and can make smarter decisions – like routing based on URL path or HTTP headers, terminating HTTPS, or applying policies. Layer 7 LBs are essentially also reverse proxies, because they fully parse client requests and then issue their own requests to the servers. Load balancers often have health checks: they periodically ping the back-end servers (e.g., via HTTP) and if a server is down, they stop sending traffic to it. This provides high availability. In an interview, if you discuss scaling a web service, you’d likely introduce a load balancer so you can add more servers transparently. Also mention session stickiness if needed (some LBs can ensure the same client goes to the same server for session affinity).
A reverse proxy is a server that sits in front of one or more web servers, intercepting requests. It typically operates at Layer 7. Clients send requests to the proxy, which then forwards them to the appropriate server (possibly modifying them in the process) and then returns the server’s response to the client. The term “reverse” proxy distinguishes it from a “forward” proxy (which proxies outbound traffic for clients, like a corporate web proxy). A reverse proxy can perform load balancing, but it can also do caching, request/response rewriting, authentication, etc. For example, Nginx and HAProxy can act as reverse proxies. If you have an application running on multiple servers, you might put an Nginx reverse proxy in front to distribute requests (i.e., doing the job of a load balancer) and to serve static files or cache common responses. Reverse proxies are often used to terminate SSL/TLS – the proxy decrypts HTTPS and passes plain HTTP to internal servers – this offloads the CPU work of encryption from each server (this is common with Nginx or Apache as a TLS terminator). They can also add headers (like X-Forwarded-For to pass client IP to the server) or enforce policies (deny certain requests, etc.).

In cloud environments, the distinction blurs: for instance, AWS’s Application Load Balancer is effectively a reverse proxy that does HTTP(S) load balancing (it looks at headers, can do path-based routing, etc.) – it operates at Layer 7. AWS’s Network Load Balancer is purely packet-based (Layer 4), not modifying traffic, just forwarding at network level with ultra-high throughput. In system design, mention Layer 4 vs Layer 7 load balancing. Layer 7 (application load balancing) allows smarter routing (like sending all /api/users/* requests to a specific set of servers, or implementing blue/green deployments by routing traffic based on cookie or host), but it introduces a bit more latency due to deeper packet inspection. Layer 4 load balancing is useful for non-HTTP protocols or when you need extreme performance (millions of connections) and just want to distribute by IP/port (it’s also typically what is used for UDP-based services, since Layer 7 LB for arbitrary UDP is not feasible).

A real-world example: Suppose you have a microservices architecture with many services. You might use a reverse proxy (like an API Gateway or Nginx) to route requests to different services based on the URL. That reverse proxy might also handle authentication, caching, or compressing responses. Meanwhile, each service might have multiple instances, so there’s load balancing happening for each service cluster as well. In AWS, you could achieve this with an Application Load Balancer that has multiple target groups (one for each microservice set) and listener rules to route to them by path or host. Alternatively, you could chain a Network Load Balancer in front of an ALB (AWS allows NLB -> ALB) to combine benefits (though that’s an edge case).

In summary, load balancers focus on distributing traffic to ensure no single server is overwhelmed and to provide redundancy, and they come in L4 and L7 flavors. Reverse proxies can do load balancing but also provide a single point to implement cross-cutting concerns (logging, caching, TLS, etc.) at the application protocol level. Most modern load balancers (like ALB, Nginx, F5 BIG-IP LTM) are effectively reverse proxies working at L7. Knowing how to use them is crucial: e.g., in a system design, a load balancer can allow horizontal scaling and zero-downtime deployments (by draining connections on one server while others take traffic). Also, from a security view, a reverse proxy can shield the backend servers (clients only ever see the proxy’s IP/name).

Firewalls, VPNs, and Security Groups

These components are all about security and controlled access in networking:

A firewall is a network security device (or software) that monitors and filters incoming and outgoing network traffic based on a set of rules. It’s like a gatekeeper that decides which traffic is allowed through and which is blocked, typically based on criteria like IP addresses, port numbers, and protocols. Traditional firewalls operate at network and transport layers (filtering IP packets), though more advanced ones (Next-Gen Firewalls) can inspect application-layer data. At its core, a firewall sits at the boundary of a network (or host) and blocks or permits traffic according to rules. For example, a firewall can allow web traffic (port 80/443) but block telnet (port 23) from outside. Firewalls can be hardware appliances or built into OS (like iptables in Linux or Windows Defender Firewall). In an enterprise, you might have a firewall at the network perimeter to block unauthorized access from the internet, and internal firewalls between sensitive network segments. In cloud (AWS), the concept of a firewall is implemented by Security Groups and Network ACLs at the VPC level (more on those soon). The key point: firewalls are usually stateful (they keep track of connections; if traffic is allowed one way, the response is auto-allowed) and default to deny everything not explicitly allowed – providing a secure default stance. A senior engineer should recall that firewalls protect by IP/port filtering and are crucial for defense-in-depth (e.g., even if an attacker gets a foothold in one server, a firewall can limit what else that server can talk to).
A VPN (Virtual Private Network) creates a secure, encrypted tunnel over a public network (like the internet) to connect remote computers or networks as if they were directly connected to a private network. It’s often used by remote employees to access corporate internal resources securely, or to link two offices over the internet. Essentially, a VPN encapsulates private network traffic inside encrypted packets so that it can traverse untrusted networks without exposing data. A common scenario: you have an AWS VPC and you set up a Site-to-Site VPN to your on-premise network – this encrypts traffic between your data center and AWS, and the two networks can communicate securely over the internet. Another: using a VPN client on a laptop to connect to the company network (through a VPN gateway appliance). VPNs are described as creating a “secure tunnel” between endpoints. For example, when connected, your computer might get an IP address from the company’s network and all your traffic to company IPs goes through the encrypted tunnel to the VPN server in the company network. VPN protocols include IPsec (at layer 3), OpenVPN or WireGuard (layer 4 over UDP), etc. In cloud services, AWS offers a managed VPN endpoint and also Client VPN service for end-user VPN access. For system design, if you have a scenario where services in different networks need to talk securely, you might mention using a VPN vs. exposing them to the internet. Also, VPNs add overhead (encryption) and latency, but significantly improve security by preventing eavesdropping. A simple way to think: “A VPN is a secure 'tunnel' between two or more devices used to protect private traffic from snooping or interference.”. In an interview context, you might consider VPNs when talking about hybrid cloud or accessing private services.
Security Groups (SGs) are an AWS cloud concept acting as virtual firewalls for your instances (at the instance or ENI level). If you run EC2 instances, each instance is associated with one or more security groups. A security group contains rules that allow inbound traffic (by default, everything not allowed is denied) and similarly outbound rules. Notably, security groups in AWS are stateful – if you allow inbound on port 443 from IP X, the response traffic to IP X is automatically allowed out, even if no outbound rule explicitly allows it. Security groups are attached to the network interface; thus, they filter traffic to/from that specific instance (regardless of which subnet it’s in). You might define a security group for web servers that allows inbound TCP 443 from anywhere (so the public can reach your HTTPS service) and perhaps allows inbound TCP 22 from only your IP (for SSH). Another security group for a database might allow inbound TCP 3306 only from the web servers’ security group (using SG reference, a powerful AWS feature) and no public access. By default, if not configured, a security group denies all inbound traffic and allows all outbound. Think of a security group as a firewall at the instance level: you decide what traffic can talk to that instance. In system design with AWS, you’d mention security groups when discussing locking down access (e.g., “the database is in a private subnet and its security group only allows the app servers to connect to it”). Security group rules can reference other groups (e.g., allow inbound from SG “web-servers”), which is great for dynamic cloud environments (doesn’t require specifying IPs that might change). They work within the VPC, so if something outside AWS needs access, you’d use the public IP and the SG would see that IP. One common pitfall is forgetting that security groups default deny inbound – if your service is not reachable, check the SG rules. Another is not understanding that SGs are stateful (so you don’t need to add matching outbound rules for replies).

To sum up this section: Firewalls and security groups protect resources by permitting or blocking connections. VPNs securely connect networks or clients over untrusted networks. In a layered security model, you might have a network firewall at your VPC edge (AWS offers Network Firewall service, or you use NACLs), plus security groups on each instance for a second layer, plus maybe host-based firewalls on the OS. A best practice is the principle of least privilege – only open the ports and sources needed. For instance, open database port only to app servers, not to the world. AWS Security Groups make this easier by allowing who can connect in terms of other groups or IP ranges. And finally, always consider encryption (VPN or TLS) when sending sensitive data across networks you don’t fully control.

Networking Architectures & Patterns

Beyond individual protocols and components, how we organize and pattern our network interactions is critical in system design. Two fundamental paradigms are client-server and peer-to-peer. Additionally, content delivery via CDNs (Content Delivery Networks) has become a staple pattern for improving performance for global users. We examine each and their use cases, pros/cons.

Client-Server Architecture

Client-server is the classic architecture where a server provides a service or resources, and clients consume them over a network. The server is often a centralized (or a distributed cluster acting as central) authority that hosts data or functionality, and clients (which could be end-user applications, browsers, mobile apps, or other services) initiate requests to the server and receive responses. This pattern underlies the majority of web and enterprise systems. For example, when you use a web application, your browser (client) sends an HTTP request to a web server (perhaps running on AWS EC2 behind a load balancer) which then responds with the webpage or data.

Characteristics:

The server is typically always-on, listening on a well-known address/port for requests. Clients connect to the server’s address.
Servers can handle multiple clients concurrently (using multi-threading, async I/O, etc.), and clients can be served either one-to-one or one-to-many (one server serving many clients).
Clients usually do not share resources with each other directly; the coordination happens via the server.

Pros:

Centralized control: The server can manage data (e.g., a single source of truth in a database) and enforce security and access control. It’s easier to maintain and update one central service than many peers.
Easier to secure: You can put your server behind a firewall, add authentication at one point, etc., rather than trust arbitrary nodes.
Simpler clients: Clients can be lightweight (e.g., a thin client that just presents data), while heavy processing or state is managed by the server (e.g., web browser vs. web server + database doing the heavy lifting).
**Well-suited for most web services: Browsers (client) always talk to web servers (server). In mobile apps, the app (client) calls a REST API (server). In a corporate network, user PCs (clients) query an Active Directory server (server). The model is conceptually straightforward.

Cons:

Scalability bottleneck: The server can become a single bottleneck. If you have 1 million clients and one server (or one cluster), that server must scale up (vertically or horizontally). This is where load balancing and clustering come in. But inherently, client-server concentrates load on the server side.
Single point of failure: If the server (or all servers in cluster) goes down, clients cannot function (no service). High-availability techniques (multiple servers, failover, etc.) are needed to mitigate this.
Cost of maintenance: The server infrastructure and upkeep is on the service provider side. In peer-to-peer, by contrast, each peer contributes resources.
Latency for distant clients: If the server is located in one region, clients far away might have higher latency. This can be mitigated by CDNs or having multiple server replicas globally (which then introduces complexity of syncing data).

Example patterns:

Two-tier: a client directly communicates with a server (e.g., a thick client app talking to a database server – though direct DB access by clients is rare in modern setups).
Three-tier (or n-tier): a common extension where you have a client, a middle-tier (application server), and a back-end (database). The client communicates with the app server, which in turn communicates with the database. This is still logically client-server between each layer (client → app server is one client-server relationship; app server → database is another).
Microservices: each microservice can be thought of as a server providing some API, and other services or clients call it. That’s effectively client-server on a smaller scale among services.

In interviews, when asked to design a system (like Twitter, etc.), the default assumption is a client-server model (users use the service via a client that calls the centralized service). One might mention using load balancers to handle more clients, caching on server side to reduce load, etc., but not converting it to a peer-to-peer system (unless it’s something explicitly P2P like a file sharing service).

Peer-to-Peer Networks

Peer-to-peer (P2P) architecture decentralizes the client-server model by making each node both a client and a server. In a pure P2P network, there is no central coordinator; every node (peer) can initiate or service requests. Peers directly exchange data with each other. This model gained fame with file-sharing systems (Napster, Kazaa, BitTorrent) and is also used in blockchain networks and some communication apps.

Characteristics:

Decentralization: There’s often no single point of control. Resources (files, processing power) are distributed among peers.
Scalability: As more peers join, they also contribute resources, so capacity can scale with demand in an ideal scenario.
Peers find each other either through a discovery mechanism or overlay network. For example, BitTorrent uses tracker servers or DHT (distributed hash table) to let peers find others who have the desired file chunks – after that, peers download from each other rather than a central server.
Resilience: If some peers leave or fail, the network can still function (as long as there are enough other peers with the data). There’s no single server whose failure brings down service (though some P2P networks have semi-central components for coordination that could be points of failure).
Each peer often has equal privilege/responsibility, but in practice capabilities differ (some peers act as super-nodes, etc., to improve efficiency).

Pros:

Robustness: No central server to attack or fail – network can be highly fault-tolerant if data is well replicated on peers.
Resource utilization: It leverages computing resources of all participants. For example, in torrenting, each downloader also uploads to others, so the load of distributing a file is shared (unlike client-server where the server bears all load).
Scalability: In theory, P2P can scale better for certain applications (every new client also a server). E.g., distributing a large software update via P2P can be faster and cheaper because peers share the upload burden.
Reduced cost for the originator: The service provider doesn’t need to maintain huge server infrastructure; the users collectively provide it.

Cons:

Management and Security: Without central control, ensuring data integrity and security can be challenging. Peers might be untrusted. How do you ensure a peer isn’t distributing corrupted data? Many P2P systems use reputation or verification (e.g., BitTorrent pieces have hashes to verify integrity).
Complexity: Writing P2P systems is more complex (need to handle peers joining/leaving, discovery, incentives for sharing, etc.). Also handling things like firewalls and NAT (peers behind NAT routers might not accept incoming connections easily) adds complexity.
Inconsistent performance: Since peers are volunteers, some may have slow networks or go offline, causing variability. Also, without central servers, reaching data might take longer if the peers holding it are on slow links or far away.
Use cases limited: P2P is great for sharing static data (like files, or blockchain ledger where data replicates to everyone). For something like a search engine or real-time stock trading system, P2P is not a typical approach – those rely on central authority or database. So P2P is not a solution to all problems; it shines in specific scenarios (decentralized file storage, content distribution, certain collaborative networks).

Use cases and examples:

File sharing: BitTorrent protocol – when you download a Linux ISO via torrent, you get pieces from dozens of peers. There’s no single official server sending the file (though initially someone seeds it).
Blockchain networks (cryptocurrencies): Bitcoin or Ethereum network is P2P – each node communicates transactions and blocks to others. There’s no central bitcoin server – consensus is achieved by all peers verifying blocks.
VoIP/Video chat (partially P2P): Skype (in its original design) used a hybrid P2P where calls were direct P2P, but it also had some super-nodes for directory. Modern WebRTC allows browsers to establish P2P connections for video streaming (often still needs a signaling server to help them find each other, but actual media can flow peer-to-peer to reduce latency and server load).
Distributed computing: projects like BOINC (SETI@home) use P2P-ish models to harness computing power of many peers for a task.

In system design, P2P might come up if discussing, say, a content distribution system or how to avoid central bottlenecks. Often, though, interview designs lean on client-server due to simplicity and control. But mentioning P2P could show awareness of alternatives. For instance, for a video streaming platform, one might mention a peer-assisted delivery (like clients also upload popular chunks to nearby clients) to reduce bandwidth on origin – some commercial systems (and WebRTC based CDNs) do that.

One should also understand hybrid models: Many systems use a central index but P2P for data. Napster had a central server for search, but file transfers were P2P between users. BitTorrent uses trackers (or DHT) to coordinate but then peers directly exchange. This often yields a balance: a bit of centralization for efficiency where acceptable, combined with decentralization for scale.

CDNs (Content Delivery Networks) and Caching Strategies

A Content Delivery Network (CDN) is a distributed network of servers strategically placed around the globe to cache and deliver content to users with high availability and performance. Instead of every user hitting your origin server (which might be in one region), users fetch content (especially static content like images, scripts, videos) from a CDN node (edge server) that is geographically closer to them. This reduces latency and offloads work from the origin. AWS CloudFront is an example of a CDN service.

How CDNs work: When using a CDN, your DNS for, say, assets.myapp.com is configured to point to the CDN’s network. A user in Europe might resolve that to an edge server in Frankfurt, while a user in Asia gets an edge in Singapore. The CDN edge will check if it has the requested content cached; if yes, it serves it immediately. If not, it will fetch it from the origin (your server), cache it, and then serve the user. Subsequent requests from nearby users get the cached copy. CDNs thus operate as reverse proxies with distributed caching. CDNs like CloudFront also handle HTTPS, can compress content, and even edge compute (like AWS Lambda@Edge).

Benefits:

Reduced latency: Because data travels a shorter distance from edge server to user, response is faster. The globally distributed nature of a CDN means the distance between users and the content is reduced, yielding faster load times.
Offloading origin traffic: Many user requests are served by edge cache, so the origin sees a fraction of the traffic. This helps scale – your origin needs capacity mainly for cache misses and dynamic content.
Bandwidth savings and cost: CDNs often have cheaper bandwidth at scale than your servers, and by caching, they reduce total data your servers send. They also can reduce data transferred by compressing files and optimizing content (minification, etc.).
Reliability: Good CDNs have many edge nodes; if one goes down or is overloaded, requests can be directed to another. Also, if your origin goes down temporarily, a CDN can often serve stale cached content (if configured to do so) to mask brief outages.
Security and DDoS protection: CDNs can act as a shield for your origin. They often include DDoS mitigation (absorbing flood traffic at the edge) and web application firewall features at the edge. CloudFront, for example, integrates with AWS Shield and WAF to protect against attacks.

AWS CloudFront specifics: It’s a managed CDN where you define “distributions” that pull from an origin (could be an S3 bucket, an HTTP server, etc.). CloudFront has edge locations across the world. It supports dynamic content as well, including WebSockets and can even proxy through to a custom origin for API calls (though dynamic responses may have Cache-Control: no-cache so it just forwards them). You can configure behaviors for different URL path patterns (cache some things longer, some not at all, etc.). CloudFront also allows invalidation – if you deploy a new version of a file, you can tell the CDN to purge the old version from cache.

Caching strategies:

TTL (time-to-live): Each cached asset has a TTL (either set by origin via headers or default at CDN). For example, an image might have TTL of 1 day, meaning edge will serve it for 1 day before fetching a fresh copy. Tuning TTL is a strategy: long TTL for content that rarely changes (to get more cache hits), short TTL or none for frequently changing or dynamic content.
Cache Invalidation: When you need to update content immediately, you can invalidate it (explicitly clear it from caches) so that next request goes to origin. However, invalidations can be expensive, so an alternative is versioning the assets (put a version or hash in the filename/path so that new content uses a new URL and thus doesn’t conflict with cached old content).
Cache hierarchy / regions: Some CDNs have layers (edge POPs and regional caches) to improve cache hit ratio globally. As a user of CDN, you mostly don’t see this, but it’s good to know that CDNs are optimized to increase the chance a cache has the content either at that node or an upper-tier node.
Dynamic content and CDNs: Traditionally CDNs were for static files, but now they can also accelerate dynamic content by establishing optimized connections between edge and origin (keep-alive, reuse TLS handshakes, etc.). For instance, CloudFront can help even if the content isn’t cacheable by reducing TCP handshake overhead and using the AWS backbone for part of the journey. Also, CDNs can compress data and do things like TCP optimizations (multiplexing, etc.) to speed up even dynamic data.

CDN in system design interviews: If users are global, you should almost always mention a CDN for static content. E.g., “we will serve images and videos via a CDN to reduce latency for users and offload our servers.” If designing something like Netflix or YouTube, CDNs are absolutely critical (they heavily use CDNs to stream video bits from locations close to users). Even for API responses, a CDN can cache certain GET requests if they are public and heavy (e.g., a public leaderboard data that’s same for everyone). Additionally, mention using browser caching (setting headers so that the user’s browser caches files) – CDNs and browser caches together form a caching strategy.

Overall, CDNs and caching are about moving data closer to users and reducing redundant work. They are a form of scaling out read throughput – many more people can be served by caches than by the origin at once. The trade-off is cache consistency (you have to manage updates carefully). But for primarily static or globally replicated content, CDNs massively improve performance and are almost default in modern architectures.

Cloud Networking Fundamentals (AWS)

Amazon Web Services provides virtual networking capabilities that mirror many of the concepts from traditional networking, but in a software-defined, easy-to-manage way. Key topics include VPCs (Virtual Private Clouds), subnets and routing (including Internet Gateways and NAT), Security Groups vs NACLs for filtering, Route 53 for DNS, and Elastic Load Balancing options (ALB, NLB, CLB). Mastering these concepts is essential for designing secure and scalable AWS architectures.

VPC (Virtual Private Cloud)

An Amazon VPC is essentially a private network within AWS for your resources. When you create a VPC, you are carving out an isolated IP network (with a range you choose, e.g., 10.0.0.0/16) in which you can launch AWS resources (EC2 instances, RDS databases, Lambda functions (if in a VPC), etc.). By default, no one else can see or access your VPC – it’s like having your own logically separated section of the AWS cloud. As AWS says, “Amazon VPC is a logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network that you define.”. You have full control over that network’s configuration: IP address ranges, subnets, route tables, gateways, and security settings.

Key aspects of VPCs:

CIDR Range: You choose an IP address range in CIDR notation (IPv4, and optionally IPv6). For example, 10.0.0.0/16 gives you addresses 10.0.0.0–10.0.255.255 for your resources. This range should be chosen to avoid conflict with your other networks if you plan to connect them (common practice is to use private IP ranges that are not overlapping with on-prem networks if hybrid).
Subnets: Within a VPC, you create subnets (each subnet resides in a single AWS Availability Zone). Subnets further segment the IP range (for instance, a 10.0.0.0/16 VPC might be split into a 10.0.0.0/24 subnet in us-east-1a, 10.0.1.0/24 in us-east-1b, etc.). Subnets are typically designated as public or private – more on that in the next subsection.
Route Tables: VPC has routing logic – each subnet is associated with a route table that dictates how traffic destined for different IP ranges is forwarded (within VPC, to Internet Gateway, to peering connection, etc.). By configuring routes, you can connect subnets to the internet or to other networks.
Internet Gateway (IGW): If you want your VPC to access the internet, you attach an Internet Gateway to the VPC. This IGW allows outbound internet access and can allow inbound if routes and security permit. One IGW per VPC (and VPC can have at most one IGW attached).
DNS: VPCs come with an internal DNS resolution capability (plus you can use Route 53 Private DNS for custom domains). Hosts can resolve internal DNS names of EC2 instances (AWS provides a DNS server in VPC, usually at base of the IP range + 2).
Isolation: By default, a VPC’s networks are isolated from other VPCs and from on-prem networks. You can peer VPCs (connect two VPCs so they can talk over AWS backbone), or use AWS Transit Gateway to connect multiple VPCs and on-prem together. But absent those, a VPC is a closed world.
Default VPC: AWS accounts come with a default VPC in each region, where each AZ has a default subnet and instances launched without specifying a subnet go there. Default VPCs are convenient for quick starts (they have an IGW and auto-assign public IPs), but for production, you often create custom VPCs for better control (and perhaps to use only private subnets).

Design-wise, a VPC is analogous to having a virtual data center. You’d typically plan subnets for different tiers (public-facing vs internal). For example, a common pattern: a Public subnet for load balancers or bastion hosts (with internet access), and a Private subnet for application servers and databases (no direct internet ingress).

It’s worth noting that within a VPC, AWS provides an implicit router that handles routing between subnets (you don’t see it, but if subnets have routes to each other, traffic flows). You can imagine this as AWS’s internal network fabric connecting the subnets and implementing the route tables.

Subnets, Internet Gateways, and NAT (Public vs Private Subnets)

Subnets divide a VPC’s IP space and each lies in a single Availability Zone. They are the container where you actually place resources (when you launch an EC2 instance, you pick a subnet for its NIC). There are two main flavors:

Public Subnet: A subnet that has a route to an Internet Gateway (IGW). Instances in a public subnet can have public IPs (either auto-assigned by AWS or manually attached Elastic IP) and thus can send/receive traffic to the internet via the IGW. The IGW essentially performs NAT for instances’ public IPs – allowing them to be reached from the internet. Public subnets are used for resources that need to be directly accessible from the internet, like web servers, or for resources that need to initiate outbound internet connections without going through another proxy.
Private Subnet: A subnet that does not have a direct route to an IGW. Typically, private subnets are where you put backend servers, databases, etc., that you don’t want exposed to the internet. Instances in private subnets usually only have private IPs. But they often still need to reach the internet (for updates, external API calls, etc.). How can they if the subnet has no IGW route? The answer is via a NAT Gateway (or historically NAT Instance). A NAT device is placed in a public subnet and the private subnet’s route table is set to send “0.0.0.0/0” (internet-bound traffic) to the NAT, which in turn forwards it out to the IGW using its own public IP. This way, instances in private subnets can initiate outbound connections (like downloading patches or calling an AWS service API) but still no one from the internet can directly initiate a connection to them, because they have no public IP and no IGW route.

Internet Gateway (IGW): As mentioned, an IGW is attached to a VPC to enable internet access. It is horizontally scaled and highly available by AWS (no need to manage it). An IGW serves two purposes: it allows outbound traffic to the internet and inbound traffic from the internet for public IPs. For IPv4, the IGW also performs a one-to-one NAT: your instance’s public IPv4 is actually mapped to the instance’s private IP on the way in and out. For IPv6, NAT is not needed (since IPv6 addresses are globally unique), so the IGW is more of a router. In a route table, you make a route for 0.0.0.0/0 (and ::/0 for v6) pointing to the IGW to signify internet-bound traffic goes there. Only subnets with such a route are “public.” Instances in public subnets must have public IPs to be reachable from internet (or else they go out to net with no return path). Also, security groups/NACLs must allow the traffic. When designing, you may mention an IGW as essentially providing the VPC internet connectivity – without it, the VPC is isolated (which can be desired for strict private networks).

NAT Gateway: AWS’s managed NAT service. You place a NAT Gateway in a public subnet, give it an Elastic IP (public IPv4). Then private subnets route their internet traffic to this NAT. The NAT gateway then allows instances in private subnets to connect out to the internet, but prevents the internet from initiating connections with those instances. This quote encapsulates NAT’s essence: one-way access. The NAT device maps internal addresses to its own address for outgoing requests (port mapping). For IPv6, AWS offers an Egress-Only Internet Gateway which is similar concept (allows v6 out, not in). NAT Gateways are highly available within an AZ (and you typically deploy one per AZ for resilience). They’re fully managed (scale automatically but do incur hourly and data processing costs). Historically, one could also use a NAT Instance (an EC2 running Linux NAT software) which is cheaper but not HA by default and more maintenance. Nowadays NAT Gateway is recommended for ease.

Putting it together: A common AWS network setup:

VPC 10.0.0.0/16.
Public subnet 10.0.0.0/24 in AZ_a. Attached route table has route to IGW for 0.0.0.0/0. NAT Gateway deployed here (with IP, say 203.X.Y.Z).
Private subnet 10.0.1.0/24 in AZ_a. Attached route table has 0.0.0.0/0 route to the NAT Gateway (targeted by its ENI). No direct IGW route.
Similarly subnets in other AZs.
So, a web server in the public subnet can serve internet users (it has a public IP via IGW). An application server in the private subnet cannot be reached from internet, but it can reach out via NAT to, say, call an external API or reach AWS Systems Manager, etc.
The NAT gateway itself lives in public subnet and uses IGW to talk out. But external hosts cannot initiate to the private instance because they don't know any public IP for it (and if they did, the connection wouldn't be routed in due to NAT not allowing unsolicited inbound).

From a design perspective, public subnets are for your “edge” resources (like ALBs, bastion hosts for SSH, etc.), private subnets for internal servers and databases. This minimizes exposure – e.g., your database only accessible from app servers – and is a common interview point for securing cloud architectures.

One nuance: if an instance in a public subnet has no public IP, it’s effectively isolated (like a private instance) despite subnet being public, because IGW won’t have anything to do (it doesn’t NAT for an instance with only private IP). Conversely, if you somehow give an instance a public IP but put it in a subnet with no IGW route, that public IP is essentially useless (no route).

AWS Route Tables and summarizing IGW/NAT: Typically we have one main route table or custom tables. The internet route is only in the public ones. Also, AWS automatically adds routes for VPC’s own IP ranges so subnets talk to each other internally. You can also have routes to Virtual Private Gateways (for VPN to on-prem) or VPC Peering connections or Transit Gateway attachments, but that’s beyond scope here.

In summary, Internet Gateway = door to the internet (both directions) for the VPC, NAT Gateway = one-way door for private subnet instances to go out to internet (outbound only). Designing a secure app, you’d likely: “Place the web tier in public subnets (behind an ALB perhaps) and the business logic and database in private subnets. Use a NAT Gateway so that private instances can download updates. The public subnets have routes to IGW, private subnets have routes to NAT.” This ensures only the web tier is exposed and everything else isn’t directly reachable from outside.

Security Groups vs. NACLs (Network ACLs)

AWS provides two layers of network security within a VPC: Security Groups (SG) and Network Access Control Lists (NACLs). It’s important to understand the differences and how they complement each other.

Security Groups: As discussed earlier, they act as stateful firewalls at the instance level. Key properties:

Attached to network interfaces/instances. You can assign one or many SGs to an EC2 instance (or other resources like RDS, ENI).
Allow rules only: You specify what traffic is allowed (by protocol, port, source/destination). There is no “deny” rule; anything not explicitly allowed is denied by default.
Stateful: If an inbound rule allows traffic in, the response outbound traffic is automatically allowed out, and vice versa. You don’t need to write mirror rules for return traffic. This simplifies management and means SGs keep track of connections.
Operates at instance level: So if two instances in same subnet talk, their SGs still govern that communication. SGs don’t evaluate traffic that doesn’t involve their instances.
Order doesn’t matter: All rules are evaluated (they’re essentially an implicit allow list). There is no priority or numbering. If any rule matches the traffic, it’s allowed. If none do, it’s denied.
Can reference other SGs: e.g., allow inbound from instances in SG “AppServers”. This dynamic reference is super useful for microservice communication without fixed IPs.

Network ACLs: These act as a stateless firewall at the subnet level. They control traffic in and out of entire subnets. Key properties:

Attached to subnets. Each subnet must have a NACL (if you don’t create one, the subnet uses the VPC’s default NACL). One NACL can be associated with multiple subnets, but a subnet can only have one NACL at a time.
Allow and Deny rules: NACLs are basically access control lists with entries that either ALLOW or DENY traffic based on protocol, port, source/dest IP. You can explicitly deny traffic with NACLs (unlike SGs).
Stateless: This is crucial – NACLs do not keep track of connections. If you allow inbound port 443, you must also allow outbound ephemeral port range responses, otherwise the response will be blocked. Return traffic is not automatically allowed. This means one has to manage both directions. For example, typical NACL config might allow inbound 80/443 from anywhere, allow outbound 1024-65535 to anywhere (to cover return), etc.
Ordered rules: NACL rules are numbered 1–32766 and evaluated in order (lowest number first). The first rule that matches traffic wins (and it’s either allow or deny). There is an implicit deny if no rule matches. So the order and numbering of NACL rules matters (you often leave gaps in numbering to insert rules later, e.g., number by 10s).
Applies to all traffic crossing subnet boundary: If an instance in Subnet A talks to instance in Subnet B, the traffic leaving A hits A’s NACL outbound rules, and traffic entering B hits B’s NACL inbound rules. Also, traffic to/from internet: public subnet’s NACL inbound rules apply to incoming from IGW, outbound rules for leaving to IGW, etc.
Default NACL: By default (unless changed) is fully open (allow all in/out). Many people leave NACLs as default open and rely on SGs, because managing two layers can be redundant. Others lock them down as an additional safeguard.

Differences & Best Practices:

Granularity: SGs are per instance, NACLs per subnet. If you want a blanket rule for a whole subnet (e.g., deny all traffic from a malicious IP range across entire subnet), NACL is useful. If you want to isolate instance roles, SGs are better.
State: SGs being stateful means easier management of return traffic; NACLs stateless mean more work to configure properly (and potential to accidentally block responses).
Rules: SGs cannot explicitly deny; if you need to explicitly block something (e.g., block this specific IP from reaching your subnet) you use a NACL rule to deny it (SG could only not allow it, but SGs are tricky if other rules allow broad range).
Performance: Both are efficiently handled by AWS. NACLs might be at the edge of subnets, SGs at instance, but as a user you rarely worry about performance limits, except SGs and NACLs have quotas on number of rules (which are high by default).
Use cases: Security Groups are the primary defense in AWS setups (think of them as host firewall). NACLs can be used as an extra layer (like a network firewall at subnet border). Some orgs use NACLs to put an IP blacklist in place or to harden the environment in case someone misconfigures SGs. However, because SGs are so flexible and stateful, many solutions rely mainly on SGs.

To illustrate, let’s say we have a VPC with a public and private subnet:

We set the public subnet’s NACL to allow inbound 80,443,1024-65535 (ephemeral) from anywhere, and outbound 1024-65535 and 80,443 to anywhere (so basically allow web in, allow responses out). Private subnet’s NACL maybe allows inbound from public subnet IP range only, etc. But one could also leave NACL open.
Our web server SG allows inbound 80/443 from anywhere, and maybe SSH from a specific IP, and outbound all (SG outbound default is allow all). DB server SG allows inbound 3306 from the web server SG only, and outbound all. This way even if NACL is open, SGs enforce that DB only hears from app servers.

Stateful vs Stateless example: Imagine an HTTP request from a user to a web server: - NACL Inbound on public subnet: must allow TCP 80 from user’s IP. - SG on web server: must allow TCP 80 from user’s IP (or 0.0.0.0/0 if open to all). - Response flows out: - SG will allow it because SG is stateful (inbound was allowed, so outbound reply is auto-allowed). - NACL outbound on public subnet must explicitly allow ephemeral port traffic to the user's IP. If NACL outbound had a blanket allow all, fine. If not, you need a rule (e.g., allow 1024-65535). - If any of those checks fails, traffic is blocked.

Summary of differences:

“Security Groups work at the resource level and are stateful, while NACLs operate at the subnet level and are stateless.”.
Security groups are easier for most cases, NACLs for specific network-wide rules.
In an interview, if asked, good to mention: SGs are like instance firewalls, NACLs like subnet firewalls. AWS’s own docs highlight that difference in statefulness.

In practice, many designs simply use SGs to restrict traffic (which is often enough). But knowing NACLs exist and how to use them adds defense in depth. For example, to protect against an unlikely scenario of a compromised instance that modifies its SG (if compromised credentials), a NACL could still block certain traffic. Or to absolutely prevent any internet access on a subnet level, you could use NACL to deny 0.0.0.0/0 outbound. Also in some cases like restricting certain CIDR ranges environment-wide, a NACL is easier than updating many SGs.

Route 53 (DNS Management)

Amazon Route 53 is AWS’s cloud DNS service. It can manage both public DNS names for your domains and private DNS within your VPCs. The name Route 53 comes from TCP/UDP port 53, which is used for DNS.

Key features:

Domain registration: You can register domain names via Route 53 (AWS becomes your registrar). Or you can use Route 53 with domains from other registrars by updating name servers.
DNS zones: You create “hosted zones” for your domain, and within them, you create DNS records (A, AAAA, CNAME, MX, TXT, etc.). Route 53 then answers queries for your domain with those records. It’s an authoritative DNS service.
Highly available: It’s globally distributed – Route 53 uses a network of DNS servers across many locations. When someone queries your domain, typically they hit a Route 53 server close to them (via Anycast). This makes resolution fast and resilient.
Route 53 in VPC (Private Hosted Zones): You can have private DNS zones tied to your VPCs, so that, for example, internal.company.local resolves to internal IP addresses but only for queries from within those VPCs. This is great for microservices or internal endpoints, giving them friendly names.
Health checks and DNS-based routing: Route 53 can perform health checks (HTTP, TCP, etc.) on endpoints. It can then use that info in routing decisions. For example, you can set up DNS failover: if your primary site is down (health check fails), Route 53 will direct traffic to a backup site by changing which record is returned (say from primary IP to DR IP). This is one way to achieve cross-region failover at the DNS level.
Routing policies: Besides simple single-record responses, Route 53 supports:
- Weighted routing: You can assign weights to records. For instance, split traffic 70/30 between two IPs (maybe for A/B testing or a migration).
- Latency-based routing: If you have servers in multiple regions, Route 53 can detect where the user is coming from (based on DNS query source, which correlates with user location somewhat) and return the IP of the region with lowest latency to them. For example, users in Europe get the IP of your EU servers, users in US get US servers.
- Geo DNS: You can route based on geographic location of query (e.g., all users in EU get directed to an EU site; similar to latency but more policy-driven).
- Multi-answer (round robin): By default, multiple A records act as simple round-robin. Route 53 of course supports that if you just list multiple records.
- Multivalue answer routing: Route 53 can return multiple IPs (like round robin) and optionally health-check each, so it only returns healthy ones.
Integration with other AWS services: e.g., you can map a Route 53 record to an ELB, CloudFront distribution, or S3 static website easily (Alias records in Route 53). These alias records are nice because they map to AWS resource DNS names and Route 53 auto updates if underlying IPs change, etc., and they are free (no query charge) and can root-map a zone apex to an ELB (something you normally can’t do with CNAME due to DNS rules). For example, you can set myapp.com as an alias to an ALB. Route 53 will resolve it to the ALB’s IPs and keep them updated.

Route 53 and system design: If your service has a custom domain, you’ll use Route 53 (or another DNS) to host that domain’s records pointing to your load balancer or cloudfront, etc. With Route 53, you can also do clever things like blue-green deploy using weighted routing (send small % to new environment). Or region load balancing by latency. Or failover: e.g., if US-east-1 goes down, fail DNS over to US-west-2 where a backup setup is running. The propagation of DNS changes isn’t instantaneous for users (cache TTLs matter), but Route 53 allows TTL as low as 0 for alias records (meaning on next query it will fetch fresh, useful for health-check based failover to be quick).

One should mention that Route 53 is highly reliable – it’s designed with SLA of 100%. DNS in general is crucial infrastructure. Also mention the concept of resolver (clients, like stub resolvers, will cache results per TTL – something to keep in mind if doing fast failovers).

In interviews, typical mentions:

Use Route 53 to manage your domain’s DNS, pointing to the load balancer. Possibly leverage weighted routing for canary releases.
Use health-check based failover to a secondary site or to an “under maintenance” static page if main site down (Route 53 can serve an alternate record if health checks fail).
For internal service discovery within VPCs, use Private Hosted Zones in Route 53 rather than hardcoding IPs.

Elastic Load Balancing (ALB, NLB, CLB)

AWS offers Elastic Load Balancing (ELB) as a service with different types of load balancers. The main ones are:

CLB (Classic Load Balancer): Original generation, now legacy. It could do either HTTP/HTTPS (layer 7 basic) or TCP (layer 4) on one device. It doesn’t support many advanced features. AWS recommends ALB or NLB now instead. CLB is mostly kept for compatibility. It’s configured per instance instance or per AZ and doesn’t support things like HTTP/2, WebSockets nicely. Pricing is also not as cost-effective.
ALB (Application Load Balancer): Modern Layer 7 load balancer for HTTP/HTTPS and WebSocket. It’s what you use for web applications. It supports content-based routing (based on URL, host, headers), supports HTTP/2 and gRPC, WebSockets, integrates with AWS WAF, and can also do authentication via Cognito/OAuth, etc. ALB can route to multiple target groups (different microservices behind one ALB). It also can load balance to IP addresses and Lambda functions, not just instance targets. ALB provides detailed metrics per target group, including request counts, latency, etc. It also has fixed hostname per LB and no fixed IP (though you can combine with AWS Global Accelerator if you need static IP).
NLB (Network Load Balancer): Modern Layer 4 load balancer for TCP, TLS, UDP. It’s designed for extreme performance and can handle millions of requests per second with very low latency (~ microseconds of overhead). It’s also special in that it can preserve the source IP to the backend (ALB by default does not – it puts the client IP in X-Forwarded-For header instead). NLB is good for non-HTTP protocols or when you need static IPs per AZ (NLB provides an static IP in each AZ or you can assign your own Elastic IPs). It doesn’t have HTTP awareness – it’s for things like custom TCP protocols, or even balancing to an ALB (some use NLB to expose static IP that then forwards to ALB). It also has one useful feature: TLS pass-through or TLS termination at scale for non-HTTP (ALB only does HTTP/HTTPS). For example, you could use NLB to distribute TLS-encrypted traffic to IoT devices connecting on a custom protocol. NLB health checks are basic (like TCP connect or optional HTTP if you choose).

When to use which:

ALB – use for web applications and APIs (Layer 7). If you need to inspect HTTP headers, do routing based on path or host, or use AWS WAF, or offload HTTPS (TLS) and do redirects (http->https), etc. Example: serving a website or REST API with multiple microservices. ALB can route /api/users to one target group and /api/orders to another, for instance. It’s multi-tenant in that way.
NLB – use for very high throughput or low latency requirements, or non-HTTP protocols. For instance, a multiplayer game server using UDP, or needing to forward TCP to a database proxy. Also, if you require clients to see the original client IP as source (for IP-based allow lists at the server without using proxy headers), NLB preserves it at transport level. NLB also is good if you need a stable IP to give out (like DNS A record pointing to it) – ALB only gives a DNS name. NLB can be made internet-facing or internal. An example: you deploy an email server (SMTP uses TCP 25) – you could put an NLB to distribute SMTP traffic to multiple backends.
CLB – nowadays, rarely used unless you have old CloudFormation templates or need a quick TCP+HTTP in one LB but ALB+NLB combination can do better. CLB doesn’t support many target types (only instance, not IP, not Lambda). It also doesn’t support HTTP/2 or WebSockets well, and can’t do advanced routing. For new designs, you’d not choose CLB unless something prevents using ALB/NLB.

Scalability and elasticity: All these load balancers are managed by AWS – they automatically scale their capacity (adding more compute at the back end invisibly) as traffic grows. That’s why they’re called “Elastic”. You don’t manually provision the capacity (though with NLB you have less visibility; ALB you get some metrics). They are designed to be highly available across AZs if you subnet them appropriately.

Integration with ECS/EKS: ALB can directly integrate with container orchestrators (like as an ingress for Kubernetes or with ECS as a target group that automatically registers tasks). NLB too with certain capabilities.

Costs: ALB is priced per hour + LCU (capacity unit based on new connections, active connections, bandwidth, etc.). NLB is per hour + data. NLB often cheaper for pure data-heavy scenarios (because ALB’s LCU might charge more for lots of requests). CLB is per hour + data. A note: if you needed both layer 4 and 7 features, you might chain an NLB in front of ALB – but that’s advanced scenario.

In system design:

Use an ALB for serving HTTP(s) traffic to a fleet of servers. Good to mention that allows adding/removing servers seamlessly, health checks, and better utilization.
If expecting very spiky or extremely high load (like millions of IoT messages), mention NLB. Or if protocol isn’t HTTP (e.g., a game server using UDP).
If it’s internal microservice-to-microservice within a VPC, you can also use ALB (internal ALB that’s not internet-facing, which many do for central routing). Or sometimes just use service discovery without LB (depending).
Usually each public-facing service (set of endpoints) gets one ALB. Although ALB can host multiple apps via host-based routing, you might still use separate ALBs for isolation or if they are very high traffic individually.

Comparison quick recap: ALB operates at Layer 7 (HTTP) with smart routing and is ideal for web applications, NLB at Layer 4 for high performance and non-HTTP, and CLB is legacy providing basic load balancing functionality. In an AWS architecture, load balancers are the entry point for traffic (sitting in a public subnet if internet-facing), and they direct traffic to target instances in usually private subnets. They decouple clients from servers (clients hit LB DNS which stays same, even if servers change). Also, ALB can offload a lot of HTTP concerns (it can do HTTPS termination, including SNI for multiple certs, it can handle HTTP to HTTPS redirection, and you can attach WAF to it for filtering malicious requests). These features can simplify application code.

By understanding these networking principles, protocols, patterns, and AWS networking specifics, a senior engineer can design systems that are robust, scalable, and secure. In a system design interview, you would draw on these concepts to justify choices (like using TCP vs UDP, adding a CDN, segmenting networks with subnets and security groups, etc.). In practice, knowing how to configure and optimize these (like properly setting up VPC with least privilege, or using ALB vs NLB appropriately, or tuning DNS TTLs) is vital for running reliable services in the cloud.

Network Troubleshooting & Performance

Even with a solid design, networks can encounter issues. Troubleshooting those and optimizing performance are key skills. Common issues include high latency, packet loss, or DNS resolution failures. We also rely on tools like ping, traceroute, and DNS lookup utilities to diagnose problems. Finally, performance can often be improved by techniques like caching, using CDNs, reducing round trips, and tuning configurations.

Common Network Issues: Latency, Packet Loss, DNS Problems

Latency: This is the delay between sending a packet and getting a response (round-trip time). High latency (sometimes called high ping) can make applications feel slow. It can be caused by long geographic distances (speed of light delays), routing inefficiencies, or congested/overloaded links. For example, a user in Asia accessing a server in the US will have noticeable latency (maybe 200+ ms). Even within a data center, latency matters for high-frequency trading, etc. Network latency adds up with each hop a packet takes. Saturated networks (congestion) increase latency too (queues in routers). Latency mainly affects applications like voice/video calls (causing lag) and any interactive service (webpages taking longer to load pieces). Reducing latency often involves moving servers closer to users (CDNs, edge computing), optimizing routing (peering with networks), or using protocols that mitigate effects (TCP tuning, etc.). In cloud, ensuring your components are in the same region/AZ avoids unnecessary latency.
Packet loss: This means some packets are dropped and never reach the destination. Causes include network congestion (overflowing buffers, so routers drop packets), faulty hardware (corruption), wireless interference, etc.. Even a small packet loss percentage (e.g., 1%) can drastically reduce TCP throughput because TCP will retransmit and also slow down (thinking it’s congestion). Real-time apps like video streams see loss as glitches (since they often don’t retransmit). Loss in a web app results in timeouts or long waits as TCP recovers. Common causes in cloud might be over-subscribed links or unstable connections (like Wi-Fi). Solutions: identify and fix congestion (increase bandwidth or distribute load), error correction codes for media, or use protocols tolerant to loss. Monitoring tools measure packet loss over time; ideally it should be 0 in a healthy wired network. High latency and packet loss often come together under congestion – as a link gets too full, queues cause latency then overflow causing loss.
DNS issues: If DNS fails, users cannot reach the service by name (even if the service is up). DNS issues include misconfiguration (wrong records), propagation delays (updates not reached all resolvers), or an outage at the DNS provider. Also, if DNS is slow (a user’s ISP DNS is slow to respond), it adds to initial connection time. Another issue: DNS cache poison or stale records causing misrouting. In troubleshooting, one might find that ping domain.com fails but ping [IP] works – indicating a DNS resolution problem. Solutions: ensure your DNS records are correct and TTLs appropriate. Use multiple DNS servers for redundancy (Route 53 is highly available by itself). On user side, maybe use a faster public DNS. Within a system, sometimes DNS issues appear when services try to call others by name and can’t resolve due to VPC DNS settings or missing hostnames.

Other issues:

Throughput issues: maybe not enough bandwidth (hitting network limits, e.g., instance network cap in AWS, or NIC at 100%).
Jitter: variation in latency, problematic for steady streams (e.g., VoIP).
Connection exhaustion: running out of available TCP ports or NAT table entries, etc., which can manifest as inability to establish new connections.
Firewall misconfigurations: blocking traffic unintentionally.

When users report “the network is slow,” usually it is one of latency or loss or DNS (or the server is slow, but network gets blamed). So one systematically checks each.

Diagnostic Tools: ping, traceroute, and DNS lookup

ping: The most basic network test tool. It sends ICMP Echo Request packets to a target and measures if/when Echo Reply comes. Ping tells you if the target is reachable and how long the round-trip took. It also reports loss (if some pings don’t come back) and jitter (variation in times). For example, ping google.com might show time=20ms. If you get Request timed out, the host might be down or ICMP blocked (note: some servers/firewalls disable ping). Ping is great for a quick connectivity check. In system design context, you might not mention it explicitly, but as an SRE, you’d ping an instance to see if it’s alive on network. Ping uses ICMP which is a lightweight protocol at network layer. High ping times or lost pings indicate network issues like latency or packet loss.
traceroute (tracert on Windows): This tool maps the route that packets take to reach a destination and measures the latency at each hop. It works by sending probe packets with increasing TTL values. The first packet has TTL=1, so it expires at the first router which returns a TTL Exceeded message (identifying that router). Next TTL=2, the first router passes it, second router returns, etc. This way you discover each hop’s IP and round-trip time. Traceroute is extremely useful to pinpoint where latency is introduced or where a path is broken. For instance, if traceroute shows a huge jump in latency between hop 5 and 6, that segment might be the problem. Or if it stops responding at a certain hop, that might be where packets are not getting through (could be a firewall or a down router). In cloud networks, internal traceroutes can show if traffic is going through an unexpected route (like through a NAT instance or peering). It’s also useful to see if your traffic is going out to internet when it should stay internal, etc. Example: traceroute example.com might show the path through your ISP, then transatlantic link, etc. If troubleshooting a hybrid connection, traceroute can show if you’re going over VPN or over public internet. This tool operates at a lower level than application, so even if a web server is not responding, a traceroute might show that network is fine up to the server’s IP (meaning the issue is server app, not network).
nslookup / dig: These are DNS lookup tools. nslookup (available on both Windows and *nix) and dig (Unix) allow you to query DNS records. For example, nslookup www.example.com will ask the configured DNS server for that name and return the IP. You can also specify different DNS servers (nslookup www.example.com 8.8.8.8) to see if perhaps one resolver has a different answer (propagation issue). With dig, you can specify record types (MX, TXT, etc.) easily and get more detail. These tools help verify DNS is returning the expected results and how fast. For private DNS, running it on an instance inside a VPC can confirm internal names resolve. They can also show the authoritative answers vs cached via flags. As the Namecheap doc notes: “The nslookup command is a DNS lookup utility. You can use it to look up information for a selected hostname.”. In an AWS context, you might use dig txt id.server @resolver to find which Route 53 resolver you’re hitting (cool trick), or use these to debug Route 53 health check DNS failovers.
Other tools:
- telnet/nc (netcat): Often used to test connectivity to a specific port. E.g., telnet 10.0.2.5 3306 to see if you can reach a database port (if it hangs or fails, port might be blocked or server down).
- mtr (my traceroute): combines ping and traceroute continuously to show changing stats over time per hop.
- Network monitoring tools/services: like CloudWatch for AWS or custom logging of latency.
- Traceroute-like in AWS (Reachability Analyzer): not exactly a user tool, but AWS has Reachability Analyzer to virtually check network paths given your SGs/NACLs routes.
- ipconfig/ifconfig: to check local network config, sometimes a “network issue” is simply misconfigured IP or DNS on the host.

In an interview, if asked how to debug “service unreachable”, you might say: ping the instance (to see if it’s network or host), traceroute to see where it stops, check security group/NACL, check instance OS firewall, etc. If “site is slow”, you might use ping to see latency, or do a dig to ensure DNS isn’t slow, etc.

Performance Tuning Tips: Reducing Latency and Improving Throughput

To improve network performance (speed and reliability), consider:

Use of CDNs and Edge Caching: As discussed, serving content from a CDN dramatically reduces latency to users globally and offloads your servers. It’s often the number one improvement for geographically dispersed traffic. For example, after moving static assets to CloudFront, users in far regions see much faster load times because of nearer edge servers.
Implement Caching (Server and Client): Caching avoids repetitive data transfer. On the client side, setting proper cache headers so browsers don’t refetch the same resources every time (e.g., images, CSS with far-future expiry and cache-busting version in filename). On the server side, caching results of expensive operations in memory or using a service like ElastiCache (Redis/Memcached) reduces database calls, indirectly reducing network calls. Also, within microservices, if Service A calls Service B frequently for the same data, A can cache that data and reduce network chatter. Less network usage = better latency and throughput for other needs.
Minimize Payload Size: Smaller data transfers are faster. Use compression for text-based content (HTTP gzip or Brotli for JSON, HTML, CSS, etc.) – modern servers and ALBs can compress on the fly and browsers decompress. This can cut bandwidth usage significantly (JSON compresses well). For images, serve appropriately sized images (don’t send a 4K image to a mobile device if not needed) and use modern formats (WebP, etc.). For streaming video, use codecs that provide good quality in smaller bitrate. Also remove unnecessary metadata or whitespace (minify CSS/JS). All these reduce the bytes that travel over the network, thus reducing transfer time and possibly congestion.
Reduce Round Trips: Each request-response has overhead (latency). Therefore, reducing the number of separate requests improves overall performance. Strategies:
- Keep-Alive connections: Using persistent TCP connections (which HTTP/1.1 does by default) so that multiple HTTP requests can reuse the same connection, avoiding the handshake for each object.
- HTTP/2: Multiplexes many requests over one connection, so you can request 100 images in parallel without 100 separate TCP handshakes. Also compresses headers, etc.
- Batching requests: If you have many small requests, see if they can be combined. For example, rather than 10 API calls for different data on page load, have one API call that fetches all needed data in one go (reducing 10x network overhead to 1x).
- Caching DNS lookups: The first DNS lookup adds latency; ensure your clients cache DNS (they do by default respecting TTL). Also, using Keep-Alive and HTTP/2 means you reuse the TCP connection, saving the initial connect + TLS handshake time on subsequent requests. For APIs, consider using HTTP/2 or protocols like gRPC (which uses HTTP/2 under hood) to reduce overhead per call.
Use Appropriate Protocols: For example, if building a real-time chat, using WebSockets (which keep a single TCP connection open for bidirectional comms) can be more efficient than long-polling via many HTTP requests. If high throughput file transfer is needed and latency is high, consider parallel TCP streams or UDP-based protocols to better utilize bandwidth. Sometimes tweaking TCP settings (window size, etc.) on high-latency high-bandwidth links (like cross-ocean transfers) can help – but cloud load balancers often optimize that already.
Load Balancing and Horizontal Scale: Ensure that no single server is overloaded while others idle. A load balancer (ALB/NLB) can help distribute traffic evenly, preventing one node from saturating its NIC or CPU while others are free, thereby improving overall throughput and reducing response times. Also scaling out (more instances) can reduce response time if the bottleneck was server-side processing causing queueing of requests.
Network optimization and QoS: In some environments, you might prioritize certain traffic (Quality of Service) to ensure latency-sensitive packets get through first. AWS offers some features like prioritizing traffic in NICs for certain instance types (enhanced networking). These are advanced but can matter in specialized cases.
Monitoring and adapting: Use CloudWatch or other metrics to watch network throughput, latency, error rates. If you see high latency spikes, maybe auto-scale more instances or bring content closer. If you see high error rates/timeouts, perhaps increase timeouts or investigate if it’s network or application. Tools like AWS X-Ray can help pinpoint if slowness is in network calls between services.
Simulate conditions: Use latency injection or tools to test the app under high latency or packet loss to see how it behaves, then optimize accordingly (maybe use a retry with backoff on UDP, or tune TCP keepalive).
Take advantage of AWS global infrastructure: For example, AWS Global Accelerator provides a static IP and routes your traffic through the AWS global network to the closest endpoint, often improving latency vs public internet routing. Or deploying instances in multiple AWS regions and using latency-based Route 53 routing to serve users from nearest region (a step beyond just CDN, for dynamic content too).
Tune application patterns: e.g., if a client needs to fetch 1000 small items# Computer Networking Overview

Fundamental Networking Models

OSI Model (7 Layers)

Figure: The 7 Layers of the OSI Model, from Physical (Layer 1) to Application (Layer 7). Each layer has specific responsibilities and interfaces with the layers above and below.

TCP/IP Model

The TCP/IP model is the pragmatic model on which the modern Internet is built. It condenses the OSI layers into four or five layers: typically Link (Network Interface), Internet, Transport, and Application (some versions separate Link into Physical and Data Link, totaling five layers). While the OSI model is a theoretical reference, the TCP/IP model maps more directly to real protocols in use. For example, in the TCP/IP model, the Internet layer corresponds to IP (IPv4/IPv6) for routing, the Transport layer includes TCP and UDP, and the Application layer encompasses everything from HTTP to DNS. The TCP/IP model’s simplicity reflects the design of the Internet, where protocols are defined in these four layers and interoperability is key. In practice, when designing systems we often refer to TCP/IP layers; e.g., designing a solution “at the transport layer” likely implies working with TCP/UDP rather than inventing a new layer. The OSI model remains useful for conceptual understanding, but the TCP/IP model is now more commonly used in practice today, especially when discussing real-world networking (for instance, engineers often speak of “layer 4 vs layer 7 load balancing” in terms of TCP/IP and OSI equivalently). A senior engineer should understand both models: OSI for its vocabulary and thoroughness, and TCP/IP for its direct mapping to actual protocols and the Internet’s architecture.

Essential Networking Protocols

HTTP and HTTPS (Web Protocols)

TCP vs UDP (Transport Layer Protocols)

TCP is a connection-oriented, reliable protocol. It establishes a connection via a three-way handshake (SYN, SYN-ACK, ACK) before data transfer, and ensures all data arrives intact and in order. It achieves reliability with sequence numbers and acknowledgments, retransmitting lost packets and controlling flow. This means that if you send data via TCP, you either get it at the other end or get an error – TCP will automatically handle packet loss, retransmissions, and reorder out-of-sequence packets. It also provides congestion control to avoid overwhelming networks. The cost of these features is overhead (extra packets like ACKs) and latency (waiting for ACKs, etc.). Use cases for TCP include applications where accuracy is critical – e.g. web traffic (HTTP), file transfers, database connections – basically most application protocols on the Internet use TCP by default for its reliability.
UDP is a connectionless, “best-effort” protocol. It sends packets (datagrams) without establishing a prior connection and without acknowledgments. UDP does not guarantee delivery, ordering, or duplicate protection – packets may arrive out of order, or not at all, and UDP itself won’t inform the sender. Because of this simplicity, UDP has much lower overhead and latency. Use cases for UDP involve scenarios where speed is prioritized over reliability, or the application layer handles its own error correction. Examples: real-time media streaming (video calls, online gaming) often use UDP because a dropped packet is better than delaying the stream (and protocols like RTP add their own minor recovery or just live with occasional loss). Another use is DNS queries – DNS is typically over UDP for quick request/response (with the application retrying if needed). Also, IoT devices or any custom protocol that doesn’t need TCP’s guarantees may use UDP. In interviews, a classic question is when you’d use UDP over TCP – for real-time systems (where old data is irrelevant by the time it’d be retransmitted) or high-throughput systems on reliable networks.

IP Protocol (IPv4, IPv6 and Addressing/Subnetting)

IPv4: Uses 32-bit addresses (e.g., 192.168.10.150) which allow about 4.3 billion unique addresses. IPv4 addresses are typically written in dot-decimal notation (four octets). Due to historical allocation and inefficiencies, IPv4 addresses effectively ran out, leading to private addressing (RFC1918 private networks like 10.0.0.0/8) and widespread use of NAT (Network Address Translation) to allow multiple devices to share one public IP. IPv4 has been the workhorse of the Internet since 1983.
IPv6: Uses 128-bit addresses (written in hexadecimal colon-separated form, e.g. 2001:0db8:85a3::8a2e:0370:7334). This provides an astronomically large address space (approximately 3.4×10^38 addresses), essentially solving the address exhaustion problem by offering 1,028 times more addresses than IPv4. Besides address size, IPv6 incorporates improvements such as simplified header format, built-in IPsec security, and no need for NAT because there are enough addresses for end-to-end addressing. IPv6 addresses are longer (eight groups of four hex digits). For example, an IPv6 address might look like 2406:da1c::1. Transition to IPv6 has been gradual – many systems and cloud providers (like AWS) support dual-stack (both v4 and v6). For a system design, being aware of IPv6 is important (AWS allows creating IPv6-enabled VPCs, for instance), especially for future-proofing. One key interview point is that IPv6 does not require NAT due to abundant addresses; instead, routing and firewall rules control traffic. Another is compatibility: IPv4 and IPv6 are not directly interoperable, so many systems run both.

Addressing and Subnetting: An IP address has two parts: network prefix and host identifier. Subnetting is the practice of dividing a network into smaller networks (subnets) by extending the network prefix. Each subnet is identified by a network address and a subnet mask (or prefix length, e.g., /24). For example, 192.168.1.0/24 represents a subnet where the first 24 bits are network (192.168.1.0) and the remaining 8 bits are for hosts (256 addresses, of which a few are reserved). Why subnet? It improves routing efficiency, security, and management. A subnet (subnetwork) is a smaller network inside a larger network – it allows localizing traffic so it doesn’t have to traverse the entire network. By subnetting, network traffic can travel shorter distances without passing through unnecessary routers. For instance, in a data center, you might subnet by rack or by department so that most traffic stays local. From a design perspective, subnetting helps segment networks (e.g., separating a database subnet from a web subnet for security). In cloud environments like AWS, you must subnet your VPC IP range into subnets (often at least one public and one private subnet per availability zone). Each subnet has a CIDR block (range of IPs) and belongs to one availability zone.

In practice, when designing an AWS VPC, you might choose an IPv4 CIDR like 10.0.0.0/16 (gives 65k addresses) and then subnet it into /24s for different zones or services. IPv6 can also be enabled (AWS gives a /56 or /64 block) to allow IPv6 addressing. Many system design considerations (like load balancer addressing, NAT not needed for IPv6, etc.) tie back to IP addresses and subnets.

DNS (Domain Name System)

How DNS resolution works: DNS is a distributed, hierarchical system. There are different types of DNS servers that work together in a lookup:

Recursive resolver: This is the DNS client’s agent (often provided by your ISP or a public DNS like Google’s 8.8.8.8 or Cloudflare’s 1.1.1.1). When your computer needs to resolve a name, it asks a recursive resolver. The resolver will do the legwork of querying other DNS servers if the answer isn’t cached.
Root servers: There are 13 root server addresses (server clusters) worldwide. They know where to direct queries for top-level domains (TLDs). If a resolver doesn’t know the answer, it asks a root server which returns a referral to the appropriate TLD name server (e.g., for .com domains).
TLD name servers: These handle top-level domains like .com, .org, .net, country TLDs, etc. A .com TLD server will direct the query to the authoritative name server for the specific domain.
Authoritative name servers: These are the servers that actually host the DNS records for a domain (often managed by your DNS provider or registrar). For example, if you are resolving api.example.com, after the resolver queries the root and the .com TLD, it will reach the authoritative server for example.com, which will provide the IP address (A/AAAA record) for api.example.com.

DNS records: DNS records map names to data. Common record types include A (IPv4 address for a host), AAAA (IPv6 address), CNAME (alias one name to another), MX (mail exchange servers for email), TXT (arbitrary text, often for email security like SPF/DKIM), NS (delegates to a name server), and SOA (start of authority, domain metadata). For instance, an A record for www.example.com might point to 93.184.216.34. A CNAME for photos.example.com might alias to images.example.com (so that the latter’s A record is used). In system design, you use DNS to provide friendly endpoints for your services and to do things like load distribution (by returning multiple A records or using weighted DNS). DNS caching can sometimes cause stale data issues (e.g., if you change an IP but the old one is cached), which is why TTLs should be chosen carefully – short TTLs for highly dynamic services, longer for stable mappings.

Server roles: A single machine can be configured as different DNS roles:

Recursive resolver (often at ISP or local network – e.g., your Wi-Fi router often does simple DNS recursion for you).
Authoritative server for a domain (managed by the domain owner or DNS host). In AWS, Route 53 is an authoritative DNS service (more in AWS section). Also, operating systems have a stub resolver (the part that talks to the recursive resolver, usually via /etc/resolv.conf config or OS settings pointing to a DNS server).

Common Networking Components

Routers, Switches, and Gateways

A network switch operates mainly at Layer 2 (Data Link). It has multiple ports and forwards Ethernet frames between devices based on MAC addresses. Switches effectively create a network where all connected devices can communicate directly. They learn MAC addresses by examining incoming frames and only send frames out the port destined for the target MAC, which is more efficient than an old-fashioned hub (which would broadcast to all ports). Switches are fundamental for building local networks (e.g., all devices in an office floor might plug into switches). They isolate collision domains, meaning each link is its own segment, allowing concurrent communications. Some advanced switches (layer-3 switches) can perform routing functions too, but generally, a switch is about connecting devices on the same network. For example, in a data center, you might have a top-of-rack switch connecting all servers in that rack.
A router operates at Layer 3 (Network). Routers connect different IP networks and make forwarding decisions based on IP addresses. In simple terms, switches create networks, routers connect networks. Your home Wi-Fi router, for instance, routes between your home network (e.g., 192.168.0.0/24) and the internet (via your ISP). Routers use routing tables to decide which interface to send an outgoing packet to, based on the destination IP and network prefixes. They enable inter-network communication – without routers, networks would be isolated islands. On the Internet, routers owned by ISPs exchange routes using protocols like BGP. In system design, routers might not be directly discussed unless designing network architecture, but one should know that in AWS, for example, a VPC has an implicit router and you manage routes via route tables. If you have multiple subnets, the VPC router handles traffic between them (if allowed by NACLs etc.). A router can also do NAT (network address translation) as often done in home routers or cloud NAT gateways. A key point is routers separate broadcast domains – a broadcast in one subnet won’t go through a router to another. Gateway is a term often used interchangeably with router in the context of “default gateway,” which is simply the router that connects a local network to other networks (often the internet).
A gateway in networking is a broader term. It generally means a device that acts as an entry/exit point between two networks. Often a router serving as the “edge” of a network (connecting to an external network) is called a gateway. More specifically, a gateway joins two dissimilar networks, potentially translating between different protocols. For example, early networks had email gateways between different email systems, or a gateway could connect a TCP/IP network with an older protocol network. In IP terms, your default gateway is the IP address of the router your host sends traffic to when the destination is outside your local subnet. In AWS, an Internet Gateway (IGW) is what allows VPC traffic to reach the internet – effectively acting as the gateway between your private cloud network and the public internet. Another example: a API gateway (in application terms) acts as an entry point between external clients and internal services (though that’s higher-level than network layer). But at the network level, think of gateway as either a synonym for a router at the edge or a device that bridges different network systems. In summary, routers vs gateways: “While a router is used to join two similar networks, a gateway is used to join two dissimilar networks.” – in practice, many devices (including home routers) perform both roles, so the terms can blur. In exam or interview contexts, it’s good to mention a gateway might perform protocol conversion if needed and is the “last stop” before traffic leaves a network or the first point of entry.

Load Balancers & Reverse Proxies

A load balancer (LB) is designed to distribute incoming load across multiple servers (targets) to improve scalability and reliability. The load balancer presents a single address (IP or DNS name) to clients; behind that, it maintains a pool of servers. When requests come in, the LB chooses a server (based on a balancing algorithm: round-robin, least connections, etc.) and forwards the request. The client is unaware of which server handled it. Load balancers can operate at Layer 4 or Layer 7. A Layer 4 load balancer (e.g., AWS’s Network Load Balancer, or a TCP LB in a hardware device) looks at networking info (IP and port) and forwards packets without understanding the application protocol. It’s very fast and efficient, suitable for raw TCP/UDP traffic. A Layer 7 load balancer (e.g., AWS Application Load Balancer or an Nginx/HAProxy configured for HTTP) actually looks at the application protocol (HTTP, for instance) and can make smarter decisions – like routing based on URL path or HTTP headers, terminating HTTPS, or applying policies. Layer 7 LBs are essentially also reverse proxies, because they fully parse client requests and then issue their own requests to the servers. Load balancers often have health checks: they periodically ping the back-end servers (e.g., via HTTP) and if a server is down, they stop sending traffic to it. This provides high availability. In an interview, if you discuss scaling a web service, you’d likely introduce a load balancer so you can add more servers transparently. Also mention session stickiness if needed (some LBs can ensure the same client goes to the same server for session affinity).
A reverse proxy is a server that sits in front of one or more web servers, intercepting requests. It typically operates at Layer 7. Clients send requests to the proxy, which then forwards them to the appropriate server (possibly modifying them in the process) and then returns the server’s response to the client. The term “reverse” proxy distinguishes it from a “forward” proxy (which proxies outbound traffic for clients, like a corporate web proxy). A reverse proxy can perform load balancing, but it can also do caching, request/response rewriting, authentication, etc. For example, Nginx and HAProxy can act as reverse proxies. If you have an application running on multiple servers, you might put an Nginx reverse proxy in front to distribute requests (i.e., doing the job of a load balancer) and to serve static files or cache common responses. Reverse proxies are often used to terminate SSL/TLS – the proxy decrypts HTTPS and passes plain HTTP to internal servers – this offloads the CPU work of encryption from each server (this is common with Nginx or Apache as a TLS terminator). They can also add headers (like X-Forwarded-For to pass client IP to the server) or enforce policies (deny certain requests, etc.).

Firewalls, VPNs, and Security Groups

These components are all about security and controlled access in networking:

A firewall is a network security device (or software) that monitors and filters incoming and outgoing network traffic based on a set of rules. It’s like a gatekeeper that decides which traffic is allowed through and which is blocked, typically based on criteria like IP addresses, port numbers, and protocols. Traditional firewalls operate at network and transport layers (filtering IP packets), though more advanced ones (Next-Gen Firewalls) can inspect application-layer data. At its core, a firewall sits at the boundary of a network (or host) and blocks or permits traffic according to rules. For example, a firewall can allow web traffic (port 80/443) but block telnet (port 23) from outside. Firewalls can be hardware appliances or built into OS (like iptables in Linux or Windows Defender Firewall). In an enterprise, you might have a firewall at the network perimeter to block unauthorized access from the internet, and internal firewalls between sensitive network segments. In cloud (AWS), the concept of a firewall is implemented by Security Groups and Network ACLs at the VPC level (more on those soon). The key point: firewalls are usually stateful (they keep track of connections; if traffic is allowed one way, the response is auto-allowed) and default to deny everything not explicitly allowed – providing a secure default stance. A senior engineer should recall that firewalls protect by IP/port filtering and are crucial for defense-in-depth (e.g., even if an attacker gets a foothold in one server, a firewall can limit what else that server can talk to).
A VPN (Virtual Private Network) creates a secure, encrypted tunnel over a public network (like the internet) to connect remote computers or networks as if they were directly connected to a private network. It’s often used by remote employees to access corporate internal resources securely, or to link two offices over the internet. Essentially, a VPN encapsulates private network traffic inside encrypted packets so that it can traverse untrusted networks without exposing data. A common scenario: you have an AWS VPC and you set up a Site-to-Site VPN to your on-premise network – this encrypts traffic between your data center and AWS, and the two networks can communicate securely over the internet. Another: using a VPN client on a laptop to connect to the company network (through a VPN gateway appliance). VPNs are described as creating a “secure tunnel” between endpoints. For example, when connected, your computer might get an IP address from the company’s network and all your traffic to company IPs goes through the encrypted tunnel to the VPN server in the company network. VPN protocols include IPsec (at layer 3), OpenVPN or WireGuard (layer 4 over UDP), etc. In cloud services, AWS offers a managed VPN endpoint and also Client VPN service for end-user VPN access. For system design, if you have a scenario where services in different networks need to talk securely, you might mention using a VPN vs. exposing them to the internet. Also, VPNs add overhead (encryption) and latency, but significantly improve security by preventing eavesdropping. A simple way to think: “A VPN is a secure 'tunnel' between two or more devices used to protect private web traffic from snooping, interference, and censorship.”. In an interview context, you might consider VPNs when talking about hybrid cloud or accessing private services.
Security Groups (SGs) are an AWS cloud concept acting as virtual firewalls for your instances (at the instance or ENI level). If you run EC2 instances, each instance is associated with one or more security groups. A security group contains rules that allow inbound traffic (by default, everything not allowed is denied) and similarly outbound rules. Notably, security groups in AWS are stateful – if you allow inbound on port 443 from IP X, the response traffic to IP X is automatically allowed out, even if no outbound rule explicitly allows it. Security groups are attached to the network interface; thus, they filter traffic to/from that specific instance (regardless of which subnet it’s in). You might define a security group for web servers that allows inbound TCP 80/443 from anywhere (so the public can reach your HTTP/HTTPS service) and perhaps allows inbound TCP 22 from only your IP (for SSH). Another security group for a database might allow inbound TCP 3306 only from the web servers’ security group (using SG reference, a powerful AWS feature) and no public access. By default, if not configured, a security group denies all inbound traffic and allows all outbound. Think of a security group as a firewall at the instance level: you decide what traffic can talk to that instance. In system design with AWS, you’d mention security groups when discussing locking down access (e.g., “the database is in a private subnet and its security group only allows the app servers to connect to it”). Security group rules can reference other groups (e.g., allow inbound from SG “web-servers”), which is great for dynamic cloud environments (doesn’t require specifying IPs that might change). They work within the VPC, so if something outside AWS needs access, you’d use the public IP and the SG would see that IP. One common pitfall is forgetting that security groups default deny inbound – if your service is not reachable, check the SG rules. Another is not understanding that SGs are stateful (so you don’t need to add matching outbound rules for replies).

Networking Architectures & Patterns

Client-Server Architecture

Characteristics:

The server is typically always-on, listening on a well-known address/port for requests. Clients connect to the server’s address.
Servers can handle multiple clients concurrently (using multi-threading, async I/O, etc.), and clients can be served either one-to-one or one-to-many (one server serving many clients).
Clients usually do not share resources with each other directly; the coordination happens via the server.

Pros:

Centralized control: The server can manage data (e.g., a single source of truth in a database) and enforce security and access control. It’s easier to maintain and update one central service than many peers.
Easier to secure: You can put your server behind a firewall, add authentication at one point, etc., rather than trust arbitrary nodes.
Simpler clients: Clients can be lightweight (e.g., a thin client that just presents data), while heavy processing or state is managed by the server (e.g., web browser vs. web server + database doing the heavy lifting).
Well-suited for most web services: Browsers (client) always talk to web servers (server). In mobile apps, the app (client) calls a REST API (server). In a corporate network, user PCs (clients) query an Active Directory server (server). The model is conceptually straightforward.

Cons:

Scalability bottleneck: The server can become a single bottleneck. If you have 1 million clients and one server (or one cluster), that server must scale up (vertically or horizontally). This is where load balancing and clustering come in. But inherently, client-server concentrates load on the server side.
Single point of failure: If the server (or all servers in cluster) goes down, clients cannot function (no service). High-availability techniques (multiple servers, failover, etc.) are needed to mitigate this.
Cost of maintenance: The server infrastructure and upkeep is on the service provider side. In peer-to-peer, by contrast, each peer contributes resources.
Latency for distant clients: If the server is located in one region, clients far away might have higher latency. This can be mitigated by CDNs or having multiple server replicas globally (which then introduces complexity of syncing data).

Example patterns:

Two-tier: a client directly communicates with a server (e.g., a thick client app talking to a database server – though direct DB access by clients is rare in modern setups).
Three-tier (or n-tier): a common extension where you have a client, a middle-tier (application server), and a back-end (database). The client communicates with the app server, which in turn communicates with the database. This is still logically client-server between each layer (client → app server is one client-server relationship; app server → database is another).
Microservices: each microservice can be thought of as a server providing some API, and other services or clients call it. That’s effectively client-server on a smaller scale among services.

Peer-to-Peer Networks

Characteristics:

Decentralization: There’s often no single point of control. Resources (files, processing power) are distributed among peers.
Scalability: As more peers join, they also contribute resources, so capacity can scale with demand in an ideal scenario.
Peers find each other either through a discovery mechanism or overlay network. For example, BitTorrent uses tracker servers or DHT (distributed hash table) to let peers find others who have the desired file chunks – after that, peers download from each other rather than a central server.
Resilience: If some peers leave or fail, the network can still function (as long as there are enough other peers with the data). There’s no single server whose failure brings down service (though some P2P networks have semi-central components for coordination that could be points of failure).
Each peer often has equal privilege/responsibility, but in practice capabilities differ (some peers act as super-nodes, etc., to improve efficiency).

Pros:

Robustness: No central server to attack or fail – network can be highly fault-tolerant if data is well replicated on peers.
Resource utilization: It leverages computing resources of all participants. For example, in torrenting, each downloader also uploads to others, so the load of distributing a file is shared (unlike client-server where the server bears all load).
Scalability: In theory, P2P can scale better for certain applications (every new client also a server). E.g., distributing a large software update via P2P can be faster and cheaper because peers share the upload burden.
Reduced cost for the originator: The service provider doesn’t need to maintain huge server infrastructure; the users collectively provide it.

Cons:

Management and Security: Without central control, ensuring data integrity and security can be challenging. Peers might be untrusted. How do you ensure a peer isn’t distributing corrupted data? Many P2P systems use reputation or verification (e.g., BitTorrent pieces have hashes to verify integrity).
Complexity: Writing P2P systems is more complex (need to handle peers joining/leaving, discovery, incentives for sharing, etc.). Also handling things like firewalls and NAT (peers behind NAT routers might not accept incoming connections easily) adds complexity.
Inconsistent performance: Since peers are volunteers, some may have slow networks or go offline, causing variability. Also, without central servers, reaching data might take longer if the peers holding it are on slow links or far away.
Use cases limited: P2P is great for sharing static data (like files, or blockchain ledger where data replicates to everyone). For something like a search engine or real-time stock trading system, P2P is not a typical approach – those rely on central authority or database. So P2P is not a solution to all problems; it shines in specific scenarios (decentralized file storage, content distribution, certain collaborative networks).

Use cases and examples:

File sharing: BitTorrent protocol – when you download a Linux ISO via torrent, you get pieces from dozens of peers. There’s no single official server sending the file (though initially someone seeds it).
Blockchain networks (cryptocurrencies): Bitcoin or Ethereum network is P2P – each node communicates transactions and blocks to others. There’s no central bitcoin server – consensus is achieved by all peers verifying blocks.
VoIP/Video chat (partially P2P): Skype (in its original design) used a hybrid P2P where calls were direct P2P, but it also had some super-nodes for directory. Modern WebRTC allows browsers to establish P2P connections for video streaming (often still needs a signaling server to help them find each other, but actual media can flow peer-to-peer to reduce latency and server load).
Distributed computing: projects like BOINC (SETI@home) use P2P-ish models to harness computing power of many peers for a task.

CDNs (Content Delivery Networks) and Caching Strategies

Benefits:

Reduced latency: Because data travels a shorter distance from edge server to user, response is faster. The globally distributed nature of a CDN means the distance between users and the content is reduced, yielding faster load times.
Offloading origin traffic: Many user requests are served by edge cache, so the origin sees a fraction of the traffic. This helps scale – your origin needs capacity mainly for cache misses and dynamic content.
Bandwidth savings and cost: CDNs often have cheaper bandwidth at scale than your servers, and by caching, they reduce total data your servers send. They also can reduce data transferred by compressing files and optimizing content (minification, etc.).
Reliability: Good CDNs have many edge nodes; if one goes down or is overloaded, requests can be directed to another. Also, if your origin goes down temporarily, a CDN can often serve stale cached content (if configured to do so) to mask brief outages.
Security and DDoS protection: CDNs can act as a shield for your origin. They often include DDoS mitigation (absorbing flood traffic at the edge) and web application firewall features at the edge. CloudFront, for example, integrates with AWS Shield and WAF to protect against attacks.

Caching strategies:

TTL (time-to-live): Each cached asset has a TTL (either set by origin via headers or default at CDN). For example, an image might have TTL of 1 day, meaning edge will serve it for 1 day before fetching a fresh copy. Tuning TTL is a strategy: long TTL for content that rarely changes (to get more cache hits), short TTL or none for frequently changing or dynamic content.
Cache Invalidation: When you need to update content immediately, you can invalidate it (explicitly clear it from caches) so that next request goes to origin. However, invalidations can be expensive, so an alternative is versioning the assets (put a version or hash in the filename/path so that new content uses a new URL and thus doesn’t conflict with cached old content).
Cache hierarchy / regions: Some CDNs have layers (edge POPs and regional caches) to improve cache hit ratio globally. As a user of CDN, you mostly don’t see this, but it’s good to know that CDNs are optimized to increase the chance a cache has the content either at that node or an upper-tier node.
Dynamic content and CDNs: Traditionally CDNs were for static files, but now they can also accelerate dynamic content by establishing optimized connections between edge and origin (keep-alive, reuse TLS handshakes, etc.). For instance, CloudFront can help even if the content isn’t cacheable by reducing TCP handshake overhead and using the AWS backbone for part of the journey. Also, CDNs can reduce the amount of data transferred by reducing file sizes using minification and compression. Smaller file sizes mean quicker load times for users.

AWS CloudFront example: “Amazon CloudFront is a CDN service built for high performance, security, and developer convenience”. In practice, you’d set up CloudFront distributions for your static assets (perhaps using an S3 bucket as origin) and maybe one for your API (with shorter TTLs or no caching for dynamic API calls, but still using edge locations for TLS termination). CloudFront can also speed up sites which use TLS by optimizing connection reuse and using features like TLS False Start for handshakes. Overall, CDNs and caching are about moving data closer to users and reducing redundant work. They are a form of scaling out read throughput – many more people can be served by caches than by the origin at once. The trade-off is cache consistency (you have to manage updates carefully), but for primarily static or globally replicated content, CDNs massively improve performance and are almost default in modern architectures.

Cloud Networking Fundamentals (AWS)

VPC (Virtual Private Cloud)

An Amazon VPC is essentially a private network within AWS for your resources. When you create a VPC, you are carving out an isolated IP network (with a range you choose, e.g., 10.0.0.0/16) in which you can launch AWS resources (EC2 instances, RDS databases, Lambda functions (if in a VPC), etc.). By default, no one else can see or access your VPC – it’s like having your own logically separated section of the AWS cloud. As AWS says, “Amazon VPC is a logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network that you define.”. You have complete control to customize and design this software-defined network based on your requirements.

Key aspects of VPCs:

CIDR Range: You choose an IP address range in CIDR notation (IPv4, and optionally IPv6). For example, 10.0.0.0/16 gives you addresses 10.0.0.0–10.0.255.255 for your resources. This range should be chosen to avoid conflict with your other networks if you plan to connect them (common practice is to use private IP ranges that are not overlapping with on-prem networks in case of a VPN or Direct Connect link).
Subnets: Within a VPC, you create subnets (each subnet resides entirely within one AWS Availability Zone). Subnets further segment the IP range (for instance, a 10.0.0.0/16 VPC might be split into a 10.0.0.0/24 subnet in us-east-1a, 10.0.1.0/24 in us-east-1b, etc.). Subnets are typically designated as public or private – more on that below. Resources (EC2 instances, etc.) are launched into subnets.
Route Tables: A VPC has an implicit router, and you manage routing via route tables. Each subnet is associated with a route table that dictates how traffic destined for various IP ranges is forwarded (e.g., to the Internet Gateway, to a NAT gateway, or to a peering connection). By editing route tables, you connect subnets to the internet or to each other or to on-prem networks.
Internet Gateway (IGW): If you want your VPC to have internet access, you attach an Internet Gateway. This is a horizontally-scaled, redundant component that allows communication between your VPC and the internet. It doesn’t impose bandwidth constraints and is highly available. A VPC can have at most one IGW attached. Without an IGW, the VPC is isolated from the public internet (which can be desirable for strictly internal apps).
NAT Gateway: A managed service (living in your VPC) that enables instances in a private subnet to initiate outbound internet connections while preventing inbound connections directly to them. We cover this under subnets.
DNS: VPCs provide an internal DNS resolution service. Each instance can resolve public hostnames to public IPs and internal hostnames to private IPs. You can enable a DNS hostname option so that EC2 instances get a DNS name (useful for internal communication). Route 53 can also provide Private DNS zones tied to VPCs.
Isolation & Security: By default, a new VPC’s subnets can talk to each other (if routes exist) but not to other VPCs or on-prem networks unless you configure it (via VPC Peering, Transit Gateway, VPN/Direct Connect). Security Groups (and NACLs) provide traffic filtering inside the VPC. You could have multiple VPCs for isolation (e.g., prod vs dev), since by default they cannot reach each other’s resources unless explicitly connected.

Think of a VPC as your virtual data center. You have to plan IP ranges, subnets, and how to connect out or in. AWS provides defaults (e.g., default VPC) to get started, but custom VPCs are recommended for sophisticated setups. With a VPC, you have flexibility: you can peer with another VPC (even in different AWS accounts or regions, with some limitations), connect to on-prem networks via VPN or Direct Connect, attach endpoints for AWS services (so that calls to, say, S3 or DynamoDB don’t leave the AWS backbone), etc.

Design-wise, you might have one VPC per environment (production, staging, dev) or per application grouping. Within a VPC, using subnets to separate public-facing and internal resources is a common pattern described next.

Subnets, Internet Gateways, and NAT (Public vs Private Subnets)

Figure: Public vs. Private subnets in a VPC. In this AWS diagram, the subnet in AZ A is a public subnet because its route table directs internet-bound traffic (0.0.0.0/0) to an Internet Gateway, whereas the subnet in AZ B is a private subnet with no such route. An instance in the public subnet (AZ A) that has a public IP can communicate with the internet via the IGW, but an instance in the private subnet (AZ B) cannot reach the internet directly. Even if a private-subnet instance had a public IP, it wouldn’t be reachable, because the subnet’s route table lacks an IGW route. Typically, private subnets instead use a NAT Gateway in a public subnet for outbound internet access.

In AWS, subnets divide a VPC’s IP space and each lies in a single Availability Zone. There are two main flavors of subnets in a VPC:

Public Subnet: The subnet’s route table has a route to an Internet Gateway (IGW) attached to the VPC. This means instances in that subnet can reach the public internet and can be reached from the internet if they have a public IP address. Typically, when you launch an instance in a public subnet, you either assign it an Elastic IP or enable auto-assign public IPv4, so it gets a public IP. That public IP is mapped (1:1 NAT) to the instance’s private IP via the IGW. Public subnets are used for resources that need to be externally accessible: e.g., web servers, bastion hosts (jump boxes for SSH), or load balancers. Remember that just being in a public subnet doesn’t bypass security – Security Groups still control inbound access. Public subnet just makes it possible to communicate with the internet. Also note, an instance in a public subnet without a public IP remains effectively isolated (no one can reach it from internet, and it cannot initiate out except to other VPC resources).
Private Subnet: The subnet’s route table has no direct route to the IGW. Instances in a private subnet only have private IPs. They cannot directly access the internet, and the internet cannot reach them. This is ideal for backend systems like databases, internal application servers, caches, etc., that should not be exposed. However, often these instances do need outbound internet connectivity (for updates, downloading packages, calling third-party APIs, or accessing public AWS services that don’t have VPC Endpoints). The solution is to use a NAT Gateway (or historically a NAT instance). A NAT Gateway is placed in a public subnet, with a public IP. Private subnet route tables are configured to send all 0.0.0.0/0 traffic to the NAT Gateway. Thus, when a private instance initiates a connection to the internet, the traffic goes to the NAT, which then masquerades it as its own public IP and sends it out via the IGW. The reply comes to the NAT and then back to the instance. This allows outbound access but prevents inbound initiation to those private instances (the NAT will not forward unsolicited inbound traffic). In short, private instances can reach out to the internet (through NAT), but nothing on the internet can directly reach a private instance (since there’s no route or public IP pointing to it). Private subnets are a cornerstone of a secure AWS network design.

To use a NAT Gateway, you create it in a public subnet (one in each AZ for HA typically) and update private subnet route tables: 0.0.0.0/0 -> nat-gateway-id. NAT Gateways are managed by AWS (highly available in the AZ and scale automatically). They perform the IP/port translation. One must remember to also configure the Security Group of instances if they need to allow certain outbound flows (by default SG allows all outbound). Also, NAT Gateway itself uses an Elastic IP – ensure your IGW and route to IGW for the public subnet is in place.

Internet Gateway (IGW) is the VPC’s doorway to the internet. It’s attached at the VPC level (not per subnet). For a subnet to be public, it needs a route to the IGW. The IGW handles traffic to/from public IPs of instances: it translates between the instance’s private IP and its public IP (for IPv4). The IGW is horizontally scaled by AWS, and you don’t need to worry about capacity. It simply needs to be attached and routes set. Each VPC comes with a default route table; by default AWS puts the 0.0.0.0/0 -> IGW route in the main route table for default VPCs, but in a custom VPC you add it yourself in the public subnet’s route table. An IGW also supports IPv6 (where no NAT is needed – IPv6 addresses are public by nature, so the IGW is more of a router for v6).

Summary: In an AWS environment you typically design with at least one public subnet (with IGW route) for outward-facing components, and private subnets (no IGW route) for internal components. Private subnets use NAT gateways for outbound connectivity. Public subnets have internet connectivity potential, which you secure with Security Groups. This setup (often called a “2-tier VPC architecture”: public web tier, private app tier) is a common interview talking point for deploying a web service securely. For completeness: if something truly needs to be completely isolated (no internet even outbound), you would put it in a private subnet and either not use a NAT or even use NACLs to restrict all traffic except VPC internal.

Security Groups vs. NACLs (Network ACLs)

Security Groups are stateful, instance-level firewalls. They allow traffic based on rules you define and implicitly deny everything else. Key characteristics:
- Attached to instances (ENIs): SGs filter traffic to/from a specific resource. For example, an EC2 instance might have an SG allowing inbound TCP 22 from your IP and TCP 443 from anywhere.
- Allow rules only: You cannot explicitly deny traffic with SGs; you only specify what is permitted (any traffic not matching an allow rule is dropped).
- Stateful: If an SG allows inbound traffic on a port, the outbound response is automatically allowed, and vice versa. You don’t need to write separate rules for reply traffic. This makes configuration simpler – you just think “who can talk to this instance on which ports” and not worry about the response flow.
- No rule ordering: All rules are evaluated collectively (no priority). You can have multiple SGs on an instance; effectively the rules aggregate (with an OR logic for allows).
- References: SG rules can reference other SGs (allow traffic from instances in SG X). This is extremely useful: e.g., a database SG can allow MySQL access from the SG that the application servers belong to, rather than IP ranges which may change. This helps as your app tier scales out/in – any instance with that app SG will automatically fit the DB SG rule.
Network ACLs are stateless, subnet-level firewalls. They act as an optional additional filter on traffic entering or leaving a subnet. Key characteristics:
- Attached to subnets: All instances in that subnet are subject to the NACL rules (unless overridden by another NACL if you re-associate). By default, subnets use the VPC’s default NACL.
- Allow and deny rules: NACLs can explicitly permit or block traffic. For example, you could have a deny rule for a specific IP block (something SGs can’t do except by not allowing it).
- Stateless: If you open port 443 inbound on a NACL, you must also open ephemeral ports outbound for the responses. Return traffic isn’t automatically allowed. The same goes vice versa. So usually you create mirror rules: e.g., allow inbound 80,443; allow outbound 80,443 (for egress from server, not really needed normally), and crucially allow outbound 1024-65535 to cover response ports. This statelessness means NACLs need careful rule management but also makes them simpler in concept (each packet is checked against rules in the direction indicated).
- Rule order matters: NACLs have numbered rules (1–32766) and they are evaluated in order. The first matching rule decides to allow or deny. There is an implicit deny at the end if nothing matches. Typically you leave gaps (like use rule number 100, 110, 120 etc.) so you can insert later.
- Use case: NACLs are often left as default (which is allow all) and security is primarily enforced with SGs. But NACLs can be used for an extra layer. For instance, to block a known malicious IP range at the subnet level, or to ensure no one in private subnet can ever talk out to internet except via NAT (could deny all other egress).

Differences and best practice:

Security Groups are stateful and easier to manage for each service, making them the go-to for most traffic control in AWS. NACLs, being stateless and subnet-scoped, are more of a coarse filter or safety net. A common strategy:

Use Security Groups to allow the necessary ports between the specific components (this is your primary security mechanism).
Use NACLs perhaps to lock down entire subnet traffic patterns (for example, you might use NACLs to prevent any inbound from the internet to the private subnets as an extra guard, even though they have no IGW route, or to restrict outbound in public subnets to only certain ports).
Also, if you need to explicitly block something (SG can’t “deny”), a NACL deny rule can do that.

An example difference: Suppose an instance in a private subnet was compromised and tried to initiate traffic on an unusual port to exfiltrate data. If you had an outbound deny in the NACL for that port or foreign IP range, it would be blocked at subnet level regardless of SG. Conversely, SGs by default allow all outbound, so the traffic would go out unless you had locked down outbound SG rules (which many people don’t, they keep the default allow all outbound on SGs since they rely on inbound rules mostly and statefulness). So NACL can provide that additional egress control if needed.

Another example: You want to make absolutely sure no one can RDP into any server except through a specific bastion. You might set the NACL for private subnets to deny inbound TCP 3389 from all sources except the bastion’s subnet. Even if an SG was misconfigured, the NACL would block it. It’s belt-and-suspenders.

Security Group vs NACL summary: “Security Groups work at the resource level and are stateful, while NACLs operate at the subnet level and are stateless.”. You can say SGs are like host-based firewalls, and NACLs are like network edge ACLs on a router. Most AWS architectures lean on SGs for their flexibility and ease, using NACLs sparingly. But in an interview, it’s good to mention both and perhaps note that Security Groups are usually sufficient for most needs. If pressed: Security Groups for instance-level whitelisting, NACLs for subnet-level blacklisting (if needed).

Route 53 (DNS Management)

Amazon Route 53 is AWS’s managed DNS service. It provides domain registration, DNS record hosting, and health-checking/routing features. It is highly available and scalable. The name “53” comes from UDP/TCP port 53, the DNS port.

Key capabilities:

Public DNS hosting: You can create a “hosted zone” for your domain (e.g., example.com) and manage all DNS records (A, AAAA, CNAME, MX, TXT, etc.) for that domain in Route 53. Route 53 will be the authoritative nameservers for your domain (you get server names like ns-123.awsdns-45.com, etc., which you set at your registrar).
Domain registration: You can purchase domain names through Route 53, which integrates directly to create a hosted zone.
Private DNS: Route 53 can also create private hosted zones associated with one or more VPCs. These zones’ records are only answerable from within those VPCs (or via Resolver endpoints). This is great for internal service discovery. For example, you could have a private zone “internal.mycompany.com” and have records like db.internal.mycompany.com -> 10.0.5.25 (a private IP). Only your VPC’s instances will resolve that.
Routing policies: Route 53 can do more than standard DNS round-robin:
- Simple: standard DNS resolution (either single record or multiple and it’ll rotate them).
- Weighted: multiple records each with a weight; Route 53 will randomly select according to weights. You could route 10% of traffic to a new server and 90% to old, for instance (common in canary deployments).
- Latency-based: Route 53 has latency measurements to AWS regions. If you have servers in multiple regions, you can have Route 53 respond with the IP of the region that has the lowest latency to the user’s DNS resolver location. This is often used to direct users to the nearest region (similar to geolocation but using latency data).
- Failover: You can designate a primary and secondary record. Route 53 will periodically health-check the primary target; if it becomes unhealthy, Route 53 will start returning the secondary. For example, primary could point to an ALB in us-east-1, secondary to one in us-west-2. If entire us-east-1 app fails, Route 53 will failover to us-west-2 (after the health check fails and TTL expiry).
- Geolocation: You can route based on user’s geographic location (by continent, country, or even state for US) rather than strictly latency. Useful if you want to send all EU users to EU site not because of latency but due to data regulations, etc.
- Multi-value answer: It’s like simple routing but Route 53 can return multiple IPs (akin to round robin) and optionally health-check each and only return healthy ones. This is somewhat like client-side load balancing via DNS.
Health checks: Route 53 can monitor endpoints (HTTP, TCP, etc.). Health checks can be tied to record sets (for failover or multi-value). They can also be used just for monitoring purposes (CloudWatch alarms on them).
Integration with other AWS services: You can create alias records which map to AWS resources (ELB, CloudFront, S3 static website, etc.) without needing an IP. The alias is resolved by Route 53 internally to the right IPs and updated as they change. Aliases are also free of DNS query charges and can point a zone apex (root domain) to, say, an ELB (which CNAME normally can’t do at root).

Use in architecture:

Use Route 53 to manage the DNS for your application’s domain. e.g., myapp.com -> ALB DNS name (via an Alias record). That way, if the ALB’s IPs change, Route 53 seamlessly serves the new ones.
If doing multi-region active-active, use latency-based or geo DNS to send users to the nearest region.
If doing active-passive DR, use failover routing with health checks.
For microservices internally, perhaps use a Private Hosted Zone like service.local with records for each service pointing to the service’s load balancer or IP.
Route 53 is also often used to provide a level of global resilience: e.g., if an entire region fails, health checks fail and Route 53 can direct traffic to backup endpoints (either in another region or a static maintenance page on S3/CloudFront, etc.). The TTL on such records should be kept low (maybe 60 seconds) to allow quick switchover.

Route 53 operates on a global scale (you don’t “pick a region” for Route 53 – it’s AWS global). It’s highly reliable – it uses a global anycast network of DNS servers.

For interviews: mention Route 53 as the solution for scaling DNS and doing clever routing strategies. For instance, for a global service, “we’ll use Route 53 latency-based routing to direct users to the closest deployment region, improving performance.” Or “we use Route 53 health checks and failover routing to increase reliability by automatically shifting traffic if the primary cluster goes down.” It’s also one of those things that differentiates a design beyond just “use DNS” because of these advanced features. And of course, it integrates with AWS names (alias to an S3 or CloudFront can simplify serving a static site at root domain, etc.).

Elastic Load Balancing (ALB, NLB, CLB)

AWS offers multiple types of Elastic Load Balancers as managed services. The main ones are:

Application Load Balancer (ALB): A Layer 7 load balancer for HTTP and HTTPS (and WebSockets, gRPC). ALB is designed for web applications (hence the name). It can inspect HTTP headers and paths, and route to different target groups based on rules (host-based or path-based routing, and even HTTP header or method-based rules). It terminates TLS (HTTPS) and can optionally authenticate users via Cognito or OIDC, and it integrates with AWS WAF for security. ALBs support features like HTTP/2 and gRPC natively, and can also handle WebSocket upgrade connections. Use case: Any web/API service. For example, one ALB could route /api/* to microservice targets and /app/* to a different set of servers (content-based routing). ALB provides detailed metrics (e.g., request count, latency, error count). It does not give you static IPs; it has a DNS name that resolves to potentially changing IPs (usually one per AZ). But you typically use Route 53 alias to it or CNAME from your DNS. It’s highly scalable (AWS adds more infrastructure behind the scenes as needed). It can load-balance to EC2 instances, ECS containers (registering by instance or by IP), Lambda functions, IP addresses, and even support ALB-to-ALB routing (though not common).
Network Load Balancer (NLB): A Layer 4 load balancer for TCP, UDP, and TLS (at transport layer). NLB is designed for extreme performance and static IPs. It can handle millions of requests per second while maintaining ultra-low latencies (it’s basically pass-through at connection level). NLB preserves the client’s source IP to the backend (important for some applications needing the real client IP without using X-Forwarded-For headers as ALB does). NLB supports TLS termination now (so it can offload TLS, but it’s still not application-aware beyond that). It also supports some routing based on ports (e.g., you could have one NLB with listeners on different ports forwarding to different target groups, akin to ALB with paths). A big feature: NLB can be given static IP addresses (one per subnet/AZ it spans) or you can attach Elastic IPs. This is crucial for use cases where clients need an IP (for allow-lists or legacy systems). It also is the only choice for protocols other than HTTP/HTTPS when you need load balancing (e.g., load balancing SMTP, or MQTT, or any custom TCP/UDP protocol). Use case: Very high throughput or non-HTTP traffic. E.g., a multiplayer game server fleet could be behind an NLB on UDP ports, or a fleet of email gateways behind an NLB on TCP 25. Also, if you want to expose a service in a VPC to on-prem via static IP and direct connection, NLB could front it.
Classic Load Balancer (CLB): This was the original ELB (now deprecated in favor of ALB/NLB). It could do either HTTP(S) or plain TCP in one device. It’s effectively layer 4 (for TCP) or basic layer 7 (for HTTP – but ALB’s features are much richer). AWS still supports CLB for legacy reasons, but new designs seldom use it. It doesn’t support HTTP/2, has less flexibility (no host/path routing rules; you’d need multiple CLBs for multiple domains or use hostname in app), and its performance scaling is less advanced compared to ALB/NLB. It also does not support WebSockets well (only through TCP mode). Typically, ALB replaces CLB for HTTP use cases, and NLB replaces CLB for pure TCP use cases. Only scenario one might still use CLB: if you need a single load balancer that handles both HTTP and non-HTTP on the same instance (not common), or have not migrated an older CloudFormation that created a CLB. Pricing for CLB is different (per hour and per GB) and sometimes slightly cheaper if you have very low traffic, but generally ALB is more cost-effective now for web.

Comparison and design:

If designing a modern web service on AWS, you’d use an ALB fronting your EC2 instances or ECS containers. You’d point your DNS (Route 53) to the ALB (Alias record). That ALB would terminate SSL, distribute traffic, perhaps do path-based routing if you have microservices behind separate target groups, and maybe integrate with WAF.
If designing a service that requires TCP or UDP balancing (say a chat server using a custom TCP protocol, or need fixed IPs), use an NLB. You might also mention NLB if expecting extremely high request rates or needing source IP (though ALB via header covers IP need for HTTP).
If asked about preserving client IP at the application, note that ALB will put client IP in X-Forwarded-For header (common practice for HTTP), whereas NLB preserves it at network layer (useful for non-HTTP).
Both ALB and NLB support cross-zone load balancing (distributing traffic across AZs regardless of client’s entry point) optionally, and are integrated with AWS autoscaling (targets register/deregister).
Gateway Load Balancer (GLB): There’s a newer type for appliance load balancing (layer 3 for things like firewall virtual appliances). Unless specifically needed (like deploying a fleet of IDS/IPS appliances), you likely won’t mention it.

High availability: ELBs themselves are multi-AZ by default (you specify subnets in at least two AZs). AWS will create load balancer nodes in each AZ. If one AZ or LB node fails, others continue. So you get HA at infrastructure level.

Scalability: It’s elastic – ALB/NLB automatically scale capacity. You don’t see or manage that. However, note that NLB has a pretty high baseline capacity so can handle sudden bursts well, whereas ALB scales but very sudden massive bursts could see a bit of latency while scaling (AWS mitigates by providing “warm-up” or you can contact them if you expect a huge jump).

Integration: For example, AWS ECS (containers) can register tasks with ALB target groups (with dynamic port mapping) making service discovery easy. AWS EKS (Kubernetes) typically uses an ALB ingress controller to route traffic to services.

In an interview, when talking about AWS architecture:

Mention using an ALB for distributing incoming HTTP(s) requests to a cluster of application instances. This improves scalability (add instances behind ALB) and reliability (if one instance fails, ALB stops sending to it – and health checks catch that).
For the database tier, you might mention the AWS managed RDS endpoint or if custom DB on EC2, maybe an NLB could be used for active-passive failover IP (but typically we use RDS or other mechanisms).
If needing to handle millions of long-lived connections (like IoT MQTT), mention NLB because ALB has connection limits per node and adds more latency per hop.

Cost considerations: ALB charges per hour and per LCU (Load Balancer Capacity Unit – based on new connections, active connections, bandwidth). NLB charges per hour and per GB. Generally, for small traffic, ALB can be slightly more expensive; for extremely high throughput, NLB might be more cost-effective if LCU costs for ALB pile up. But cost rarely is a huge factor in choosing ALB vs NLB – functionality is.

To summarize:

Use ALB for Layer 7 (HTTP/HTTPS) with advanced routing, ideal for web applications and microservices (supports path/host-based routing, HTTP2, WAF, etc.).
Use NLB for Layer 4 (TCP/UDP) where performance and static IP or non-HTTP are needed (or extremely high throughput with simple balancing). It’s basically a “fast router” in the cloud.
CLB is legacy – in designs, prefer ALB or NLB as appropriate.
All are elastic and managed, so they simplify scaling and failover significantly (compared to rolling your own HAProxy cluster, for example).

By understanding these networking principles, protocols, patterns, and AWS networking specifics, a senior engineer can design systems that are robust, scalable, and secure. In a system design interview, you would draw on these concepts to justify choices (like using TCP vs UDP, adding a CDN, segmenting networks with subnets and security groups, etc.). In practice, knowing how to configure and optimize these (like properly setting up a VPC with least privilege, or using ALB vs NLB appropriately, or tuning DNS TTLs) is vital for running reliable services in the cloud.

Network Troubleshooting & Performance

Common Network Issues: Latency, Packet Loss, DNS Problems

Latency: This is the delay between sending a packet and getting a response (round-trip time). High latency (sometimes called high ping) can make applications feel slow. It can be caused by long geographic distances (speed of light delays), routing inefficiencies, or congested/overloaded links. For example, a user in Asia accessing a server in the US will have noticeable latency (maybe 200+ ms). Even within a data center, latency matters for high-speed trading or real-time systems. Network congestion can also increase latency as packets queue in buffers. Symptoms of latency issues include slow page loads despite low server load, or voice/video call lag. Reducing latency often involves moving servers closer to users (CDNs, edge computing), optimizing network routes (using better peering or a faster backbone), or using protocols that mitigate latency (e.g., UDP-based protocols with less handshake overhead for certain tasks).
Packet loss: This means some packets never reach their destination. Common causes are congestion (routers/switches dropping packets when buffers are full), faulty hardware or poor signal (especially in wireless links), or software bugs. Effects: TCP will notice loss and retransmit, but this slows things down dramatically (and triggers congestion control to reduce speed). Even a few percent packet loss can reduce throughput substantially and cause stutters in streaming. Real-time UDP streams (VoIP, video) will experience glitches (e.g., momentary freezes, garbled audio) when packets drop because there’s no time to recover them. Diagnosing loss involves tools like ping (see if some pings don’t come back) or more advanced monitoring. Solutions: if due to congestion, increase bandwidth or distribute load; if due to bad link (wifi issues), improve signal or redundancy. In cloud networks, packet loss is usually low, but can occur under heavy load or misconfigured network. We design systems (especially distributed ones) to handle occasional packet loss (e.g., by retrying idempotent requests, or using application-level acks).
DNS issues: DNS problems can manifest as an inability to reach services by name. For example, if your DNS records are misconfigured or a DNS server is down, clients may not resolve the IP to connect to. High DNS lookup times can also add to user-perceived latency. A common issue is forgetting to update DNS after moving a service, leading to some users still hitting the old IP (if TTL was long or if a DNS server’s records lagged). Also, some corporate networks might block external DNS and require using their DNS – if an app doesn’t handle that, it might fail. Diagnosing DNS issues involves using tools like nslookup or dig to see what IP a name resolves to, or if it fails to resolve. Caching can hide issues (e.g., your local machine cached an IP but others can’t resolve due to an outage). Solutions: ensure redundancy in DNS (Route 53 is itself redundant globally). Use health checks with DNS failover if appropriate. Also, setting conservative TTLs for critical records can allow quicker changes if needed (though don’t set too low unnecessarily, as it increases lookup traffic). On the client side, if DNS is slow, sometimes switching to a faster resolver (like a public DNS service) can help, though in a system design you assume users have functioning DNS.

Other issues not explicitly in the heading:

Jitter: variation in latency, relevant in VoIP/gaming (even if average latency is fine, high jitter can cause choppy quality).
Throughput bottlenecks: maybe hitting a bandwidth cap (some instance types have an EPC bandwidth limit) or small TCP window sizes limiting throughput on high-latency links (for large file transfer, tuning window or using parallel streams might be needed).
Network collisions or loops: not typically an issue in modern switched networks, but misconfigurations can cause broadcast storms.
Port exhaustion: if a NAT device or an instance runs out of ephemeral ports for outbound connections, new connections fail. Could happen under heavy load with many outbound calls (though NAT Gateway in AWS is quite high capacity).
Misconfigured firewalls/security groups: blocking traffic that should be allowed, causing what appears as connectivity issues.

A systematic approach: If a service is unreachable, check DNS (resolves correctly?), then basic connectivity (ping/traceroute to see if it’s network path or maybe SG blocking ICMP), then check the service port (telnet/nc to port to see if connection is accepted or refused or timed out – timed out often means firewall drop, refused means service up but not accepting, etc.), and so on.

Diagnostic Tools: ping, traceroute, and DNS lookup

Ping: A basic tool to test reachability and measure round-trip time. Ping uses ICMP Echo Request and Echo Reply messages. If you ping server.com, it will send packets and report if replies come and how long they took. For example, ping 8.8.8.8 might show time=20ms, meaning 20 ms round-trip. Ping also tells you if packets were lost (e.g., “4 packets transmitted, 3 received, 25% packet loss”). Ping is great for checking if a host is up and network is basically working. Many servers disable responding to ping for security, but within a VPC, EC2 instances do respond to ping by default (unless a security group/NACL blocks ICMP). In troubleshooting, if ping fails (“request timed out”), it indicates either the target is down, unreachable (route issue), or blocking ICMP. If ping works and latency is low, network connectivity at IP level is likely fine. Ping’s limitations: it doesn’t test a specific port or application, just IP reachability. But it’s a good first step.
Traceroute (tracert on Windows): This tool maps the path packets take to a destination and measures the delay at each hop. It works by sending packets with gradually increasing TTL (time-to-live). The first packet has TTL=1, so the first router it hits returns a TTL exceeded error (revealing that router’s IP). Next with TTL=2 reveals the second router, and so on. Traceroute output looks like a list of hops with their IPs or DNS names and the latency to each (usually it sends 3 probes per hop and shows 3 timings). For example:
```
1  192.168.0.1  1.23 ms  1.10 ms  1.05 ms   (your router)
2  10.0.0.1     5.45 ms  5.50 ms  5.60 ms   (ISP local)
3  ... 
8  72.14.236.44 30.0 ms 30.5 ms 31.0 ms   (some Google server)
9  *  *  *  (hop 9 not responding)
10  8.8.8.8    32.0 ms 31.8 ms 32.5 ms   (destination)
```
Asterisks mean no response (could be a firewall or an unreachable hop). Traceroute helps locate where a problem lies: if it stops early, maybe a routing issue at that point. If one hop is very high latency compared to previous, that segment might be a bottleneck. In AWS, traceroute within a region often shows a few private IP hops (between subnets or AZs) and helps confirm traffic flow. If you suspect BGP or internet issues to a certain region, traceroute from different locations can reveal if traffic is going an odd route. Note that some firewalls/routers de-prioritize ICMP or UDP used by traceroute, so high latency at a single hop but normal after might just mean that router doesn’t prioritize TTL exceeded responses to traceroute itself but forwards actual traffic fine. Nonetheless, traceroute is indispensable for network debugging beyond your own network.
nslookup / dig: These are tools to query DNS records. With nslookup, you can specify a domain and optionally a DNS server to use. For example, nslookup example.com 8.8.8.8 asks Google DNS for that name. It will return the IP(s). If it fails, you know there’s a DNS resolution issue. You can also query specific record types (nslookup -type=MX example.com to get mail server records). dig (mostly on Linux/Mac, but available on Windows via BIND tools) provides more detailed output (including the DNS query time, which server answered, etc.). For example, dig www.example.com +trace will actually do a recursive resolution from root downwards, showing each step – useful to see if delegation is correct. Use in troubleshooting: If a web service is down, check if the domain still resolves to the correct IP and hasn’t expired or been hijacked. If a new deployment isn’t being hit, maybe the DNS still points to old load balancer – dig/nslookup will show that. Within AWS, if using Route 53, you might use nslookup internal.serviceName domain from an EC2 to see if the Route 53 private zone is working (it should resolve to a VPC IP). Or use dig @resolver for a specific DNS. According to Namecheap, “The nslookup command is a DNS lookup utility. You can use it to look up information for a selected hostname.” – which is exactly its use: verify what an authoritative answer is, or debug DNS propagation by trying different servers. In summary, nslookup/dig help confirm that DNS is returning expected results and how fast.
Telnet / Netcat (nc): Not mentioned in prompt but worth noting: You can test a TCP port by telnet host port or nc host port. If it connects (blank screen in telnet or success message in nc), the TCP connection succeeded, meaning the route is open and server is listening. If it hangs then “could not connect”, likely blocked or not listening. This is useful to differentiate between network vs service issues (e.g., ping might succeed but telnet to port 80 fails – means host up but service/port down or blocked).
Packet capture (tcpdump, Wireshark): In deep debugging, capturing packets on the host or using VPC Traffic Mirroring can help see what traffic is arriving or not. But in interview, you’d stick to higher-level tools.
Cloud provider tools: AWS has VPC Flow Logs (to see accept/deny of traffic and metadata on flows), Reachability Analyzer (to check if SG/NACL/routes allow a path between two resources). But these are advanced AWS-specific ones.

In practice, a combination is used. For instance, if a user can’t reach a service, you might:

Ping the server (to see basic connectivity).
Traceroute to see where it stops (maybe your traffic isn’t even reaching AWS, indicating a network issue in between).
If ping works, try telnet server 443 to see if the port is open. If not, security group or server issue.
Use nslookup to ensure you’re hitting correct IP.
Check instance’s SG/NACL.
Use curl on the instance itself to test local service, etc.

These tools are fundamental for diagnosing the earlier issues of latency (ping and traceroute to locate latency), packet loss (ping over time or mtr to see where loss occurs), and DNS (nslookup/dig). For optimization, there are also tools like iperf (to measure throughput between two points) or specialised services (AWS CloudWatch for monitoring network metrics, etc.).

Performance Tuning Tips: Reducing Latency and Improving Throughput

Designing a network and system for performance involves multiple techniques:

Employ caching at multiple levels: Caching is one of the most powerful ways to improve performance. On the client side, leverage browser caching by setting proper HTTP headers (so static files aren’t fetched repeatedly) – the first load might be slow but subsequent loads are instant from cache. On the server side or mid-tier, use caches for expensive computations or database queries (e.g., use Redis/Memcached to store results, reducing network calls to the DB). By returning cached results (from memory or a closer location) rather than recomputing or refetching across the network, you cut down latency and network usage dramatically. Also consider CDN caching as discussed (edge caching) – static assets served from a CDN are effectively cached geographically near users, reducing latency. Even dynamic content can sometimes be cached for short periods if it’s tolerable (e.g., cache a popular API response for 5 seconds).
Use a CDN or edge servers: As covered, a CDN like CloudFront will reduce latency by serving content from the nearest location and reduce load on origin. For global services, this is essential. CloudFront can also compress data and reuse connections to your origin (persistent connections), which cuts down latency for dynamic content too. If not a CDN, at least have servers in multiple regions to serve regional traffic (though that adds complexity with data sync). CDNs effectively give you a large-scale distributed presence without managing it yourself.
Reduce data transferred: Through compression and data format optimizations. Enable GZIP or Brotli compression for textual responses (HTML, CSS, JSON, etc.) – this can shrink payloads by 70% or more in many cases, directly reducing time to transmit. Use binary protocols (like Protobuf) instead of verbose text (XML/JSON) for internal service calls if appropriate. For images, use efficient formats (WebP/AVIF vs JPEG, or appropriate quality level) and only as large as needed (thumbnail vs full size). For videos, use streaming protocols and codecs that adjust bitrates. Less data means faster transfer, especially important on slower networks (mobile, etc.) and reduces packet loss impact by finishing sooner.
Minimize the number of round trips: Each request/response (especially if it includes a new TCP handshake or TLS handshake) adds latency. Techniques:
- Connection reuse: HTTP keep-alive ensures multiple requests use one TCP connection, avoiding repeated handshakes. Modern HTTP/2 goes further by multiplexing requests concurrently on one connection.
- Pipelining/Multiplexing: HTTP/2 multiplexing allows sending many requests in parallel without waiting, unlike HTTP/1.1 where browsers used to limit to 6 connections per domain.
- Batching: Combine multiple small requests or operations into one. For example, instead of 10 separate REST calls to get different pieces of data, design an API that returns all needed data in one call (or use GraphQL which can aggregate queries). Fewer calls = fewer context switches and protocol overheads.
- Prefetching: In some cases, anticipate what data will be needed next and fetch it before it’s requested (if bandwidth allows), so it’s ready when needed.
- Caching (again): caches also reduce round trips, effectively serving repeated needs locally.
Protocol choices and tuning: Use HTTP/2 or HTTP/3 for web where possible – they significantly improve performance over HTTP/1.1 by multiplexing and reducing head-of-line blocking (HTTP/3 even uses UDP + QUIC to avoid some TCP limitations). For internal service calls, consider gRPC (which uses HTTP/2) for efficiency. If dealing with high-latency links (like cross-continent), increasing TCP window size or using parallel streams can help saturate the link. There’s also the option of UDP-based protocols for specialized needs (like live video often uses RTP/UDP to avoid the overhead of TCP in lossy conditions). TLS session reuse: If using HTTPS, ensure clients support session resumption or TLS 1.3 (which has fewer round trips in handshake) to reduce handshake overhead.
Load balancing and scaling: Ensure you have sufficient back-end capacity so that the network isn’t waiting on overloaded servers. A load balancer can help distribute load evenly; autoscaling can add servers to handle more throughput. A system that’s overloaded might respond slowly or drop packets. For example, if one server gets too many requests and starts timing out, from the user perspective it’s like a network slowdown. So, proper scaling keeps performance optimal.
Use asynchronous and parallel operations: For example, on the client side (browser), load resources in parallel as much as possible. On the server side, handle I/O asynchronously to better utilize throughput (e.g., Node.js or Golang’s async handling can manage more concurrent connections efficiently, or in Java use NIO libraries).
Network infrastructure choices: In AWS, placing resources in the same availability zone can reduce latency (one AZ to another is low ms, but within AZ can be sub-ms). However, you trade HA (so often you span AZs; still, keep latency-sensitive components maybe pinned close by or use placement groups if needed for HPC). AWS also has enhanced networking (ENA) which provides higher PPS and throughput for certain instance types. If you have very chatty services, deploying them in the same VPC (or even the same instance as microservices or via local IPC) could avoid network hops entirely.
Performance monitoring and tuning: Continuously monitor with tools (CloudWatch, APM tools) to find bottlenecks. For example, if 95th percentile latency is high, see if it correlates with GC pauses on an app server or with network retransmits (CloudWatch has metrics for network throughput and errors on instances). Maybe enable TCP BBR congestion control on Linux for better throughput on long fat networks. Or adjust thread pools to fully utilize network IO.

In summary, to improve network performance:

Decrease distance/data travel time: CDNs, edge caching, regional deployment.
Decrease data size: compression, efficient encoding.
Decrease requests: batching, caching, persistent connections.
Parallelize where possible: concurrent fetches, pipelining.
Optimize software/hardware: use faster protocols (HTTP/2, UDP if needed), ensure no artificial waits, and scale out to avoid resource saturation.

A practical example: say we have a web page loading in 3 seconds. After analysis, we find 100ms of DNS, 500ms TLS handshake, then it loads 50 assets sequentially. To tune:

Use keep-alive or HTTP/2 so those 50 assets load in parallel over fewer connections.
Combine some files (or use HTTP/2 push or simply let HTTP/2 multiplex).
Serve assets via CDN near user to cut 500ms to maybe 50ms.
Enable TLS false start and session resumption to cut handshake time.
Compress images/CSS/JS to reduce download time. Result: page might load in under 1 second.

For back-end throughput scenario: transferring a 1GB file from region A to region B had 100ms latency and took 2 hours. To optimize:

Enable parallel TCP streams or increase TCP window to better fill the pipe (if bandwidth is available).
Possibly use S3 transfer acceleration or AWS Global Accelerator (which uses AWS’s backbone which might have lower latency/higher throughput).
With tuning, maybe get it down to minutes.

All these show that network performance is multi-faceted – from architecture decisions (where to host, how to distribute) to configuration (protocols, caching, load balancing) to code (making fewer calls, etc.). A senior engineer should consider these in design: e.g., mention use of CDNs, note you will compress data, design APIs to minimize chattiness, and ensure to profile/monitor to continually improve.

system-design networking

SerialReads

Computer Networking Overview

Computer Networking Overview

Fundamental Networking Models

OSI Model (7 Layers)

TCP/IP Model

Essential Networking Protocols

HTTP and HTTPS (Web Protocols)

TCP vs UDP (Transport Layer Protocols)

IP Protocol (IPv4, IPv6 and Addressing/Subnetting)

DNS (Domain Name System)

Common Networking Components

Routers, Switches, and Gateways

Load Balancers & Reverse Proxies

Firewalls, VPNs, and Security Groups

Networking Architectures & Patterns

Client-Server Architecture

Peer-to-Peer Networks

CDNs (Content Delivery Networks) and Caching Strategies

Cloud Networking Fundamentals (AWS)

VPC (Virtual Private Cloud)

Subnets, Internet Gateways, and NAT (Public vs Private Subnets)

Security Groups vs. NACLs (Network ACLs)

Route 53 (DNS Management)

Elastic Load Balancing (ALB, NLB, CLB)

Network Troubleshooting & Performance

Common Network Issues: Latency, Packet Loss, DNS Problems

Diagnostic Tools: ping, traceroute, and DNS lookup

Performance Tuning Tips: Reducing Latency and Improving Throughput

Fundamental Networking Models

OSI Model (7 Layers)

TCP/IP Model

Essential Networking Protocols

HTTP and HTTPS (Web Protocols)

TCP vs UDP (Transport Layer Protocols)

IP Protocol (IPv4, IPv6 and Addressing/Subnetting)

DNS (Domain Name System)

Common Networking Components

Routers, Switches, and Gateways

Load Balancers & Reverse Proxies

Firewalls, VPNs, and Security Groups

Networking Architectures & Patterns

Client-Server Architecture

Peer-to-Peer Networks

CDNs (Content Delivery Networks) and Caching Strategies

Cloud Networking Fundamentals (AWS)

VPC (Virtual Private Cloud)

Subnets, Internet Gateways, and NAT (Public vs Private Subnets)

Security Groups vs. NACLs (Network ACLs)

Route 53 (DNS Management)

Elastic Load Balancing (ALB, NLB, CLB)

Network Troubleshooting & Performance

Common Network Issues: Latency, Packet Loss, DNS Problems

Diagnostic Tools: ping, traceroute, and DNS lookup

Performance Tuning Tips: Reducing Latency and Improving Throughput