SerialReads

CDN Modern Extensions & Advanced Use-Cases (System Design Deep Dive)

Jun 08, 2025

Introduction: Modern Content Delivery Networks (CDNs) have evolved far beyond basic file caching. For engineers preparing for system design interviews, it’s important to understand advanced CDN extensions and use-cases. This deep dive covers ten key areas – from edge computing and stateful storage to media streaming, optimization, multi-CDN strategies, private CDNs, low-level performance tweaks, new HTTP features, and cloud provider touches. The focus is conceptual and vendor-agnostic (with AWS as an illustrative example), aiming to equip you with insights on how CDNs handle emerging demands and what trade-offs are involved.

1. Edge Compute and Serverless Functions at the Edge

Running code on CDN edges: Modern CDNs let developers deploy serverless functions that run in edge PoPs (points of presence) worldwide, minimizing latency by moving compute close to users. Major examples include Cloudflare Workers, AWS Lambda@Edge, and Fastly Compute@Edge. The key idea is to customize or generate content at the edge (for personalization, authentication, routing, etc.) without always hitting a distant origin server. This yields performance gains since requests from (say) Chicago can be handled in Chicago, rather than traveling to a central region. It also offloads origin load and enables novel architectures (like fully serverless applications using the CDN as the execution layer).

Cold-start mitigation: A challenge with serverless is cold starts (startup delay for a fresh function instance). Edge platforms use different runtimes to address this. For example, Cloudflare Workers uses V8 isolates (lightweight sandboxed scripts), which start in a few milliseconds, effectively eliminating cold start latency. In contrast, AWS’s Lambda@Edge (based on standard AWS Lambda in regional edge locations) historically had heavier Node.js VM startup, sometimes 1000ms+ delays on first invoke. AWS has introduced features like Provisioned Concurrency and SnapStart (snapshot/restore for Java) to reduce cold starts, but these add complexity. The trend is toward isolation technologies and runtime optimizations that make edge functions respond almost instantly worldwide.

Data locality and scale: Running code in many PoPs introduces data locality considerations. Code can execute globally, but if it depends on data (user profiles, DB writes), you must decide where that data lives (see Section 2 on state). Some providers have dozens to hundreds of PoPs (e.g. Cloudflare ~200+ cities), yielding very fine-grained locality; others run code in more limited regional locations. Design-wise, you might choose to accept a trade-off: e.g. doing reads/writes in one region (simpler consistency) versus replicating data to every edge (lower latency but harder consistency).

Billing granularity: Edge function pricing often differs from cloud functions. For instance, AWS Lambda charges by memory * time (GB-ms) including any idle wait, with a minimum of 50ms per invoke, plus request counts. In contrast, Cloudflare Workers moved to a model charging per CPU time used and not charging for idle wait or long duration. This finer-grained billing (measuring actual CPU milliseconds) can be more predictable and cost-effective for I/O-heavy tasks that spend time waiting on network/disk. System designers should be aware of these models: an edge platform that charges only for compute work (and in sub-millisecond increments) encourages different usage patterns than one with coarser billing intervals. Overall, edge compute enables highly dynamic CDN behavior, but choosing the right platform involves balancing cold-start performance, programming model (languages supported, sandbox constraints), and cost structure for the workload.

2. Edge Storage: Key-Value Stores and Durable Objects

Stateless vs stateful edges: Traditional CDNs are stateless – edge servers don’t permanently hold application state beyond cached copies. Modern use-cases demand some state at the edge (for personalization, counters, sessions, etc.), leading to edge key-value stores and “durable objects.” A key-value (KV) store at the edge (e.g. Cloudflare Workers KV, Fastly Edge Dictionary) lets you keep frequently-read data (feature flags, lookup tables) in many PoPs for fast reads. These systems favor eventual consistency: writes propagate to other PoPs after a short interval rather than instantly. For example, Cloudflare’s edge KV is eventually consistent, with updates typically visible in other regions within ~60 seconds. This is a deliberate trade-off – by tolerating a consistency window (stale reads), the system achieves high throughput and low read latency worldwide. If a value is updated in New York, a user in London might still get the old value for up to a minute.

Durable objects and strong consistency: In cases where you cannot accept stale data (e.g. a per-user counter or real-time game state), some CDNs offer durable objects or similar primitives. A durable object is essentially a single consistent endpoint (or mini-database) accessible at the edge. Cloudflare Durable Objects, for example, provide global uniqueness and serialization for a given key or entity – ensuring all interactions go through one location/instance to maintain consistency. This yields strong consistency but at the cost of higher latency for some users (if your object “lives” in one region) and throughput limits (all updates funnel through one instance). The fundamental physics at play: “It is impossible to simultaneously have strong consistency and worldwide low-latency access to a single piece of data.”. You must pick a point on that spectrum. Durable Objects pick consistency (every request sees latest state), whereas a multi-region KV cache picks latency (reads are local but might be outdated). Some architectures even combine both: e.g. writes routed through a durable object which then updates an eventually-consistent cache.

Write fan-out and replication: If you attempted to make a truly strongly-consistent global datastore by updating all edge locations on each write, the write fan-out would be enormous – hundreds of remote updates for each user action – and coordination would introduce unacceptable latency. That’s why systems avoid immediate global fan-out. Instead, they either centralize writes (durable object approach) or propagate lazily (KV caches expiring after TTL or on next read). When designing system interview answers, mention these constraints. The stateful vs stateless decision often appears in scenarios like “design a global counter” or “maintain user sessions at the edge.” A good answer will note that truly consistent edge state requires pinning user/session to a location or doing cross-region replication with an eventual consistency window. In summary, edge storage is evolving: simple global caches for mostly-read data, and new primitives for stateful logic when needed – each with trade-offs in consistency, complexity, and performance.

3. Dynamic Personalization and A/B Testing at the Edge

Edge logic for personalization: CDNs now enable dynamic content adaptations per user – without always hitting origin servers. This is crucial for use-cases like A/B testing, feature flagging, geo/personalized content, and cookie-based experiments. The idea is to make the CDN decision logic aware of user context (headers, cookies, device) and choose which content variant or response to serve. For example, an edge worker can read an “experiment” cookie and route the request to either the A or B version of a page. If no cookie is present (first visit), the edge code can randomly assign the user to a variant, set a cookie, and then consistently send that user to the same variant thereafter. This avoids flapping experiences and ensures each user sees a stable experience during the test.

Techniques: Common techniques include using HTTP headers and cookies as control inputs. A CDN can be configured to treat the presence of certain cookies or header values as part of the cache key, so different variants get cached separately. Alternatively, an edge function can act as a mini routing layer: inspect Cookie or Authorization headers and rewrite the request to fetch a specific backend or object (e.g. /featureX/enabled/… vs /featureX/disabled/…). Feature flags can also be delivered to the edge via a key-value store: for instance, an edge function checks a flag in memory to decide if a feature is on for this region or user group. Many feature flag services now offer edge SDKs or integrations to evaluate flags in CDN workers for minimal latency. Systems like Akamai EdgeWorkers with EdgeKV or Cloudflare Workers KV can store flag configurations at the edge and evaluate in microseconds, tailoring the response content.

Privacy implications: Personalization and A/B testing at the edge raise privacy considerations. Since decisions are made closer to the user, often using user identifiers or cookies, engineers must ensure compliance with data protection laws. For example, in the EU, setting cookies for A/B tests or personalization typically requires user consent (under ePrivacy laws), unless the test is strictly necessary for service. Additionally, if the CDN edge is operated by a third party, any user data (like IP, cookie IDs, or segmentation info) processed there might be considered data sharing with a processor – requiring contractual and legal safeguards (e.g. GDPR’s requirements, as seen in cases where using certain CDN services was ruled to violate GDPR due to transferring IP addresses without consent). In practice, CDNs provide options for privacy: you can implement per-region gating (to keep EU user data in EU edges only), or use techniques like field-level encryption (discussed later) to ensure sensitive data is encrypted even at the edge. For interview purposes, mentioning privacy shows awareness: e.g., “We’d use edge logic for fast personalization but ensure no PII is exposed at the edge nodes, perhaps by using anonymized tokens or obtaining necessary user consent for tracking cookies.”

Traffic splitting math: An often-overlooked detail is how to split traffic evenly (say 50/50) in a distributed system. At small scale, purely random assignment might by chance skew results, and at huge scale it averages out – but a robust approach is to use a deterministic hashing of a user ID or cookie. For instance, hash the user’s ID and use modulo 100 to get a percentage bucket – if <50 then variant B else variant A. This ensures the percentage is consistent and doesn’t fluctuate each request, and it doesn’t rely on a central coordinator. If no user ID, the edge can generate a random number once and set it in a cookie (which is effectively what the example above does). For very high traffic (millions of users), statistical evenness is easier to achieve; for lower traffic, careful monitoring or adjusting the split might be needed to hit exact ratios. Another factor is sticky routing: ensuring once a user is assigned A or B, they stay on it. Edge solutions achieve this via cookies (the edge sets a cookie “variant=A” on first response). From then on, every edge node that this user hits will see that cookie and route them consistently. All these techniques allow A/B tests and personalized experiences to be served quickly from edge cache while still dividing users and gathering experiment data accurately.

4. Media Delivery Patterns: VOD vs Live Streaming

Not all content is simple files – video and real-time media are huge in modern system design. CDNs have special strategies for video on demand (VOD) versus live streaming. The primary difference is that VOD content is static (pre-recorded files) which can be fully cached, whereas live video is generated in real-time (like a sports event) and must be delivered with minimal delay to possibly millions of viewers concurrently.

HLS/DASH and packaging: Over the past decade, HTTP-based adaptive streaming has become the norm for both VOD and live. Two dominant formats are HLS (HTTP Live Streaming) and MPEG-DASH. Both involve breaking video into small segments (few seconds each) and providing a manifest file that lists segment URLs for different quality levels. CDNs excel at these because segments are just HTTP resources, nicely cacheable at edges. Packaging is the process of converting a single video feed into those segmented, multi-bitrate formats. In VOD, this is done offline (you might store an MP4 and your packager generates all the HLS/DASH files which are then cached). In live, packaging is continuous: as the live encoder produces video, a packager (like AWS Elemental MediaPackage or others) creates HLS/DASH segments on the fly and updates manifests. CDNs then rapidly distribute those to viewers. CMAF (Common Media Application Format) has emerged to unify this – it uses a standardized container (fragmented MP4) so that the same segmented files can be used for both HLS and DASH with different manifest wrappers. CMAF also introduced low-latency modes which both protocols can leverage.

Low-Latency HLS (LL-HLS) and Low-Latency DASH: Traditional HLS/DASH have SegmentDuration * 3 seconds of latency (e.g. 6-second segments lead to 20+ seconds behind live). Low-latency extensions cut this to ~2-5 seconds by using techniques like HTTP chunked transfer, partial segments, and quicker playlist updates. LL-HLS, for example, breaks each 6-second segment into smaller chunks that can be delivered as they’re encoded, and uses HTTP/2 push or preload hints for faster client fetching. In practice, LL-HLS can reach ~2 seconds glass-to-glass latency (comparable to or better than broadcast cable). These modes require CDN support: edges must handle rapidly updating manifests, allow many small chunk requests, and maybe even use HTTP/2 server push (though push usage has waned – see section 9). Many CDNs have optimized for LL-HLS and low-latency CMAF by enabling fine-grained caching and even specialized packet handling to ensure the first chunk of a segment can be streamed to the client immediately upon encode.

WebRTC and real-time: When interactivity is needed (latency < 1s), protocols like WebRTC come into play. WebRTC isn’t a file-based streaming; it’s a peer-to-peer media protocol (using UDP, RTP) originally for video conferencing. It achieves sub-second latency, but it’s not inherently scalable to millions of users by simple caching, because it’s a bidirectional session-based protocol. CDNs have adapted by acting as WebRTC relays or SFUs (Selective Forwarding Units): essentially, an edge server can receive a WebRTC stream from a broadcaster and replicate it out to many subscribers. This is more akin to a media server mesh than a cache. WebRTC at scale often needs a distributed network of media servers – which can be co-located with CDN PoPs for reach. However, even with infrastructure, WebRTC struggles to reach huge audiences due to overhead per stream and lack of caching; it’s typically used for interactive video (e.g. video chats, webinars with small audiences, or real-time cloud gaming). A common design is to convert WebRTC to HLS/DASH when you want to go to very large audiences: e.g. a live event might use WebRTC for sub-second latency to a few hundred key participants, but use LL-HLS for thousands of general viewers (with ~2-3s latency). WebRTC’s strength is speed: it can achieve ~500ms latency and full bidirectionality, but it does not scale easily – beyond ~50 concurrent viewers, a pure P2P approach fails and you need servers. Even with servers, reaching thousands is complex. By contrast, HTTP-based streams can reach millions via caching, at the cost of a bit more delay. As one source succinctly puts it: WebRTC is the fastest for real-time, but not for scale, whereas LL-HLS/CMAF can serve thousands with only ~3s delay.

Edge transcoding: Another advanced concept is performing transcoding at the edge. Transcoding means converting a video stream into different codecs or bitrates. Usually this is done at centralized encoders (like MediaLive in AWS or in a broadcaster’s data center) due to its high CPU cost. However, there are scenarios where doing it at the edge could help – for example, if an origin is pushing a single high-quality stream, edge servers in various regions could downscale it to lower bitrates locally, reducing the need to send multiple versions across the world. Some CDNs or research projects explore using powerful edge servers or GPU-accelerated edge instances for on-the-fly transcoding or transmuxing (changing format) close to viewers. This could reduce backhaul bandwidth and adapt to regional network conditions in real time. It’s not yet common in mainstream CDN offerings due to complexity and cost, but conceptually it might appear in an interview as “how would you reduce live video latency or save bandwidth?” – one answer could be “transcode at edge PoPs to avoid sending all renditions over the backbone.” Also, with WebAssembly and high-performance computing at edge, one could imagine running video optimizers or scene-specific encoders at edge in the future.

In summary, for VOD use static files and maximize caching (CDN can cache entire video files or segments indefinitely). For live, design around segmenting and small chunks, use CDN for multi-second latency at scale, or specialized media servers for ultra-low latency interactive streams. Know your protocols: HLS/DASH (with CMAF) for general streaming, LL-HLS/LL-DASH for low-latency streaming, and WebRTC for real-time interactive needs – and that CDNs support these in different ways.

5. Adaptive Content Optimization (Compression, Images, Video)

CDNs increasingly act as optimization layers, not just dumb pipes. They can automatically compress and transform content to reduce payload size and improve performance for end-users.

Image format conversion (WebP/AVIF): Images are often the bulk of webpage bytes. Modern image codecs like WebP and AVIF offer significantly better compression than legacy JPEG/PNG. Many CDNs now dynamically serve images in these formats based on the client’s capabilities. For instance, if a browser advertises support for WebP in the Accept header, the CDN can convert a JPEG to WebP on the fly and cache that variant. AVIF (based on the AV1 video codec) can achieve even smaller sizes at equal quality, though it’s slower to encode. The benefit is big: JavaScript bundles compressed with Brotli are ~14% smaller than with Gzip; likewise, images in WebP/AVIF can be 25-50% smaller than JPEG for similar quality. (For example, one might mention that AVIF often achieves ~30% reduction over an equivalent quality JPEG – a huge win at scale.) The CDN is a great place to do this because it already has the image bytes and computing resources at the edge. Services like Cloudflare Polish, Akamai Image Manager, Fastly IO, etc., all do this content-aware optimization. In an interview, if asked how to reduce bandwidth or speed up load times globally, a valid point is “use the CDN to convert images to next-gen formats and even resize or compress them based on device.” Many CDNs let you request an image with query params or an API to resize, compress, and format-shift on demand – these are cached so the expensive operation is done once per image per variant.

Perceptual quality and metrics: Simply compressing more can save bytes but might degrade user-visible quality. Advanced systems use perceptual quality metrics (like SSIM, PSNR or Netflix’s VMAF for video) to guide optimization. For images, techniques like “perceptual compression” attempt to remove data not noticeable to the human eye. Some CDNs allow setting a target quality score – e.g. “adjust JPEG quality so that SSIM > 0.99” – ensuring minimal visual difference. While we might not dive deep in an interview, mentioning “perceptual image optimization” shows you know it’s not just about cranking quality to low, but finding an optimal point where file size is minimized with acceptable quality loss.

Text compression (Brotli, etc.): On the text side (HTML, CSS, JS, JSON), Brotli (br content-encoding) is a newer compression algorithm outperforming gzip. All modern browsers support it, and CDNs widely implement on-the-fly Brotli compression for clients that accept it. Brotli at its best settings can produce files significantly smaller than gzip – e.g. HTML 20%+ smaller and CSS 17% smaller than gzip equivalents. This translates to faster loads. Notably, Brotli shines for static assets because it can compress more densely (it has 11 levels, with level 11 being very CPU-intensive but yielding smallest size). A CDN can pre-compress your assets at level 11 and serve those, saving bandwidth on every request. An example stat: HTML files compressed by Brotli are ~21% smaller than gzip. That’s why you’ll often see Content-Encoding: br from CDN responses. For dynamic responses, some CDNs use a slightly lower Brotli level to balance CPU and latency. In system design, enabling Brotli is a quick win for performance.

Adaptive bitrate and video optimization: Similar principles apply to video. Beyond just packaging and codecs, CDNs can adapt or hint at different video streams based on network conditions. One approach is server hints / client hints – where the client or server communicates capabilities (e.g. device pixel ratio, save-data preference, current bandwidth). For example, if a user is on a low bandwidth, the CDN might choose to serve a lower resolution or more compressed video chunk. This happens in adaptive streaming players automatically (the client requests different quality chunks), but future protocols may allow the server to get more involved.

HTTP/2 103 Early Hints and resource hints: Another optimization tool is using HTTP protocol features to speed up asset loading (tied to Section 9). The CDN can send an Early Hints response (status 103) to tell the browser about critical sub-resources (CSS, JS) before the final page response is ready. This is like a “head start” – it’s not compression, but it reduces idle time in the browser. For instance, as soon as an HTML request hits the CDN and the CDN might still be fetching from origin, it can immediately send a 103 with Link: </style.css>; rel=preload so the browser starts fetching style.css while the HTML is on its way. This can improve page load by ~30% in some cases. CDNs often implement this either by analyzing common patterns or via directives from origin. Similarly, older HTTP/2 server push attempted to push resources outright, but that had adoption issues (discussed later). So now “server hints” in the form of Early Hints (103) and <link rel=preload> are the preferred way to have the server/CDN assist the client in fetching resources sooner.

In summary, a modern CDN can shrink payloads (images, video, text) and guide the client to be more efficient. These optimizations yield faster, lighter experiences – crucial talking points for web performance. Always consider whether you can “flip a switch” in the CDN to get these benefits (often you can, with little effort, compared to rewriting application code).

6. Multi-CDN Orchestration

No single CDN is 100% best in performance or reliability everywhere, all the time. Large-scale services (Netflix, Facebook, news sites during big events, etc.) increasingly use multiple CDN providers in tandem. Multi-CDN strategy is about orchestrating traffic across several CDNs to improve resiliency and sometimes performance/cost.

RUM-based steering: One advanced method uses Real User Monitoring (RUM) data to decide which CDN is performing best for users in each region/ISP. For example, a multi-CDN system might embed a small script or image in web pages that triggers real users to download tiny files from different CDNs. This measures latency/bandwidth in real conditions. These performance beacons are collected and aggregated per geography and network (e.g. “in Verizon AS701, CDN-A latency 50ms vs CDN-B 80ms”). Based on that, the system updates routing decisions. Companies like Cedexis (now part of Citrix) pioneered this approach. As one media tech manager describes, they built a homegrown solution similar to Cedexis using a RUM client on all their web properties that makes the browser download a 1.5MB test object from each CDN, then “we aggregate performance and stack-rank CDN performance at the state/country and ASN level”, feeding those results into a decision engine. This can then automatically steer new user requests to the fastest CDN for that network. RUM steering gives a dynamic, data-driven way to route traffic.

DNS vs client-side switching: The actual switching can happen via DNS or in client code. DNS-based multi-CDN is common: you use a smart DNS service as your site’s authoritative DNS. When a request for www.example.com comes in, the DNS service (with knowledge of client’s approximate region or IP) responds with a CNAME or IP that corresponds to one of the CDNs based on the current steering logic. For example, users in Europe might get eu.cdnA.example.com while US users get us.cdnB.example.com if CDN-B is better in US at the moment. Solutions like NS1, Dyn, or Route53 Latency records can implement this. Cedexis Openmix specifically allowed writing custom logic to choose a CDN per request using RUM data. Client-side approaches involve the page (or app) itself making the choice. For instance, a user could be given multiple CDN hostnames in a JavaScript config, and the JS pings each to see which is fastest, then directs subsequent resource loads to that one. This can be more granular (per user decision) but has more moving parts and slight delay on first load. Some peer-to-peer or hybrid solutions (like Streamroot mentioned in a discussion) even use client-side logic to orchestrate CDN and P2P delivery together. In practice, DNS-based is simpler and sufficient in most cases.

SLA arbitration and failover: Multi-CDN also provides redundancy. If one CDN has an outage or performance issue, traffic can be quickly cut over to another. The orchestration layer often includes health checks – e.g., continuous probes or error-rate monitoring. If CDN-A starts failing requests in a region, the system detects that and routes new requests to CDN-B. “Fail-fast” policies mean you don’t wait for a complete outage; even 5% failure or a significant latency spike can trigger shifting traffic. Many multi-CDN setups have an agreed priority order or weights, but will override those if a provider is in trouble. Some CDN providers have outage notification APIs or you can integrate with systems like PagerDuty. The DNS TTLs are kept short (e.g. 30 seconds) so changes propagate quickly. Alternatively, some do this at the application level: the app knows multiple base URLs for content and can retry on a second CDN if the first fails.

Challenges: One challenge in multi-CDN is maintaining consistent behavior and features. Each CDN has its own capabilities (custom headers, caching rules, SSL configurations). A lowest common denominator approach is often needed, as noted by engineers: “it can be challenging because you’re forced to limit your configuration to the lowest common denominator of features, and testing consistency across all vendors is hard.”. For example, if one CDN doesn’t support HTTP/2 Server Push or a certain cache invalidation API, you may choose not to rely on that feature at all in your architecture. Another challenge is that multi-CDN is often achieved via a separate layer (DNS or a load balancer), which adds complexity and potentially cost.

Cost and business logic: Sometimes multi-CDN is used to optimize costs – leveraging cheaper traffic rates in certain regions from one provider, or using one CDN as primary and only spilling over to another when traffic spikes beyond contract limits (to avoid overage fees). This requires careful configuration and sometimes real-time traffic management (dynamically shifting percentages).

In summary, multi-CDN orchestration gives you redundancy (if one CDN fails, your site stays up) and potentially performance wins (choosing the best path for each user). In an interview, if asked about ensuring high availability or handling global traffic, multi-CDN is a valid strategy: mention using DNS-based load balancing with RUM feedback, health monitoring, and the trade-off of increased complexity. Real-world large platforms do exactly this to meet their 99.99% uptime and performance targets.

7. Private and Federated CDNs (Enterprise and Hybrid Models)

Not every CDN is a public, standalone service – there are scenarios for private CDNs, enterprise CDNs, and federated CDN networks. Understanding these helps in designs where a company might want more control over content delivery or where multiple network operators collaborate.

Private CDNs / eCDN: A private CDN refers to a CDN infrastructure owned and used by a single organization for its content (as opposed to a CDN provider serving many customers). For example, a large enterprise might deploy caching servers in their branch offices or data centers to serve internal content (training videos, software updates) efficiently – this is often called an enterprise CDN (eCDN) when deployed within a corporate network. Private CDNs can be as small or large as needed – “as simple as two caching servers or large enough to serve petabytes”. The distinguishing factor is these caches only serve that organization’s users or content, not the general public. Netflix Open Connect is a great real-world example: Netflix built its own CDN, placing servers inside ISP networks loaded with Netflix content. Those are private Netflix-owned PoPs, serving only Netflix video to Netflix customers. In system design terms, a private CDN might come up when a company doesn’t want to rely on third-party CDNs for cost or privacy reasons, or for internal applications. If designing a solution for a global enterprise app, you might propose deploying regional edge servers (private CDN nodes) that host the app closer to employees, improving performance without using an external CDN (which could be a compliance issue if data is sensitive).

Federated CDNs: On the other end, federated CDN refers to multiple CDN or network operators interconnecting and pooling resources to act like a single larger CDN. This often involves telecom operators. Imagine many ISPs each have their own small CDN for their region; a content provider could either connect separately to all, or those CDNs could federate such that one can hand off content to another in regions they don’t cover. A historical example: in 2011 a group of telecoms formed the Operator Carrier Exchange (OCX) to peer their CDNs and better compete with giants like Akamai. The idea was that as a federated group they could offer content providers a one-stop reach to all their networks’ subscribers. Federations address the fact that telcos “own” the last mile to users and can cache deep in the network (great performance advantages), but each telco CDN alone only covers its subscribers. By teaming up, they increase the footprint available to content publishers. In interviews, you might mention federated CDNs in context of industry trends – e.g., “Telcos might form a federated CDN to leverage each other’s regional strengths, presenting a unified service to content providers.” The Streaming Video Alliance’s Open Caching is a modern initiative in this vein: it defines APIs for CDNs to interoperate so content can flow across different providers seamlessly. For example, if you’re using CDN-A but your user is in an ISP that has its own cache (CDN-B), open caching could let CDN-A hand off the content to be served from CDN-B’s cache, closer to the user – improving efficiency.

Hybrid PoPs and security zones: Some enterprises do a hybrid CDN approach: they use a public CDN for general content but augment it with private caches for specific needs. For instance, an e-commerce might use a public CDN globally but have a private cache in a certain country for compliance (ensuring citizen data/images never leave the country). These private nodes can be integrated with the CDN’s routing via custom DNS or CNAMES (so they act like just another PoP for that region). Another scenario is security zones – say you have content that is highly sensitive, you may not trust it to be on a multi-tenant CDN node. Certain CDNs offer “dedicated PoPs” or isolated caches for such content (at higher cost). Or an enterprise might run a mini-CDN in its DMZ and only after a user passes auth, then use the public CDN for less sensitive assets. Essentially, mixing and matching to meet security, performance, and cost requirements.

Use cases: Private CDNs (eCDNs) are often used for internal video streaming (like CEO broadcasts to employees) – vendors like Microsoft, Cisco, etc., have products for that. Federated CDNs are relevant for large-scale content distribution where no single network covers all audiences (common in regions where local telcos want to retain traffic).

In a design discussion, if asked how to serve content to, say, users in a country with data residency laws, you could respond: “Use a private CDN node in-country as part of our network, so user data doesn’t leave jurisdiction – essentially an enterprise CDN cache for that region.” Or if asked how to extend a CDN into environments with limited connectivity (like cruise ships or remote offices), an offline/private edge cache that syncs content during connectivity windows might be a solution (a form of private CDN).

To summarize, not all CDNs are one-size-fits-all public networks. Private CDNs give control to content owners (at cost of running infrastructure), and federated CDNs create broader networks by alliance. Both concepts highlight flexibility in content delivery networks’ evolution beyond the classic centralized model.

8. eBPF, XDP, and Kernel Bypass for Performance

CDNs at their core are about moving bytes efficiently. At the scale of millions of requests per second, even small kernel overheads add up. Hence, modern CDN servers employ advanced OS-level and kernel-bypass techniques to optimize the “fast path” of serving content.

eBPF and XDP: eBPF (extended Berkeley Packet Filter) is a technology in Linux that allows running sandboxed code in the kernel, often used for custom packet processing. XDP (Express Data Path) is an eBPF mode that operates at the earliest point a packet arrives (even before the kernel’s networking stack). CDNs use eBPF/XDP for ultra-fast packet filtering and routing. For example, Cloudflare has discussed using XDP to handle DDoS mitigation and load balancing right in the network driver. When a packet comes in, an XDP program can decide very quickly to drop it (if it’s malicious) or redirect it to a specific server thread, avoiding a lot of overhead. Cloudflare’s edge servers run an XDP-based layer 4 load balancer that spreads traffic across cores/servers with minimal fuss. The result is millions of packets per second handled on a single machine with low CPU. For system design, if latency or packet rate is a concern, one might mention “using eBPF/XDP to implement custom load balancing or filtering in-kernel for speed.”

Zero-copy sendfile/splice: Normally, when a server reads a file from disk and sends to network, data goes through multiple copies (disk to kernel buffer to user space to kernel network buffer to NIC). Zero-copy techniques aim to eliminate unnecessary copies, usually by letting the kernel directly send data from disk cache to the socket. The UNIX sendfile() syscall is one such mechanism – it allows sending a file’s contents over a socket without copying it into user-space. This is perfect for CDNs serving large files because it stays in kernel space, saving CPU and memory bandwidth. Another is splice(), which can move data between two sockets or a file and socket via a pipe buffer without copying to user space. Cloudflare engineers referred to these as “holy grail” for proxies – “sendfile and splice avoid copying data to userspace” thereby reducing context switches and CPU usage. Modern CDN servers (or proxy servers like Nginx) use sendfile heavily for static content. If designing a high-performance file server, definitely mention enabling sendfile or similar zero-copy, which allows 10Gbps+ throughput per server with low CPU.

TLS encryption offload: TLS (HTTPS) is CPU-intensive due to encryption/decryption. There are a few strategies: hardware offload (using SSL/TLS accelerator cards or the NIC), kernel offload (Linux has kTLS which can handle encryption in the kernel, compatible with sendfile so you can sendfile encrypted data out), or optimized user-space libraries. Some CDNs offload TLS to specialized threads or even separate machines (in early days) so that the caching layer isn’t bogged down. Today, CPUs are quite capable, so many just use highly optimized libraries (like BoringSSL, etc.) possibly with AES-NI instructions. But kernel TLS (kTLS) is interesting: it lets the kernel handle the encryption if you use sendfile, so you still don’t copy data to userland just to encrypt – the kernel can pull from disk and encrypt directly into NIC. Facebook/Meta contributed a lot to kTLS in Linux for this reason. For an interview, you could mention “use TLS session resumption, HTTP/2, and possibly kernel TLS to optimize HTTPS performance, and consider terminating TLS as early as possible at the edge to reduce handshake latencies.” Also, CDNs often use persistent connections and HTTP keep-alive to amortize TLS cost over many requests.

User-space TCP/IP (kernel bypass): In some cases, the ultimate performance path is to bypass the kernel networking stack entirely and handle packets in user space. Frameworks like DPDK (Data Plane Dev Kit) or using technologies like AF_XDP sockets allow an application to receive and send packets directly to the NIC with zero or one copy, bypassing Linux’s general stack. This can greatly increase packets per second (PPS) throughput by avoiding kernel overhead and context switches. Some high-end CDNs or network appliances use this for specialized tasks. However, writing a full TCP/IP stack in user-space is complex. There are projects (like mTCP, or Quic implementations in userland, etc.) and some CDNs might do this for UDP or QUIC traffic. But for TCP, Linux is pretty optimized, especially with eBPF tweaks (like TCP congestion control in BPF, etc.). A middle-ground is using things like SOCKMAP/sockhash in eBPF which can splice connections together in kernel (for proxying). Cloudflare wrote about using eBPF sockmap to do TCP proxy splicing – basically, once a connection to origin is set up, move the data directly kernel-to-kernel between client socket and origin socket, reducing copies and context switches in the proxy.

Real-world impact: These optimizations let a single machine handle tens of gigabits of traffic and huge concurrency. For example, anycast (used by many CDNs) means one data center might suddenly get a flood of traffic; kernel bypass and eBPF help survive that by ensuring minimal per-packet overhead. Cloudflare noted they use XDP to drop unwanted DDoS packets at 10 million pps rates before it hits the main application. In design terms, if you talk about scaling a CDN node, mentioning using sendfile and tuning the kernel (like high ulimit, use of epoll, minimizing context switches, maybe pinning interrupts to CPUs, etc.) shows you understand how to make a server handle massive I/O efficiently.

In summary, eBPF/XDP = custom high-speed packet handling (drop or redirect packets early); zero-copy and kernel TLS = send data with minimal CPU by skipping user-kernel copies and leveraging kernel/hardware for crypto; user-space networking = sometimes used for ultimate performance in niche cases, but complexity is high. These are the secret sauce of CDN performance – often invisible to end users but crucial for cost and speed at scale.

9. HTTP/2 Server Push vs HTTP/3 and Early Hints

Web protocol features have evolved to speed up page loads. Two related concepts are HTTP/2 server push and HTTP/103 Early Hints (plus the idea of using preload hints). Understanding their fate is useful in system design, as it teaches lessons about browser behavior and practical performance.

HTTP/2 Server Push: Introduced with HTTP/2, server push allowed the server (or CDN) to send resources to the browser proactively, without the browser explicitly requesting them. The classic example: as soon as the server sees an HTML request, it could push down the associated CSS and JS files, guessing the client will need them, thus saving the round-trip delay. It sounded great on paper, but in practice server push had problems. One major issue was caching and duplication: the server might push a resource the client already has cached (wasting bandwidth), or push too much, overwhelming the client. There was no simple feedback mechanism to know what a browser already has or needs. Tuning push became complicated (developers had to ensure not to push unneeded assets). As a result, adoption was very low – at peak only ~1% of websites used HTTP/2 push. Chrome engineers observed that it often didn’t help or could even hurt performance in real sites. Additionally, HTTP/3 (QUIC) initially included a similar push feature, but clients (browsers) and servers largely didn’t implement it. Google and others signaled an intent to remove push support. Indeed, by 2022, Chrome deprecated HTTP/2 push and disabled it by default in Chrome 106. That effectively means server push is dead on arrival – if the dominant browser won’t use it, content providers can’t rely on it. The takeaway: even though the spec offered server push, “it was not used much, with only ~1.25% of sites using it; analyses showed unclear or negative performance gain, and many HTTP/3 implementations dropped it, so it’s been retired.”.

HTTP/103 Early Hints: In place of push, a simpler and safer approach emerged – Early Hints (status code 103). Instead of actually sending the resource bytes, the server/CDN sends a hint telling the client “hey, you will likely need these resources, you should start fetching them yourself now.” This keeps control with the client: the browser will check its cache (if it has the resource, it won’t refetch) and decide whether to fetch, rather than being forced to accept a pushed resource. Early Hints are typically used to send <link rel="preload"> headers before the main response. For example, a CDN or origin, upon receiving a request, immediately streams a 103 with links to CSS/JS, then continues generating the 200 OK HTML. The browser, in parallel, starts loading those linked assets. This can save a few hundred milliseconds to a second on initial page load, improving performance significantly in some cases. Early Hints are much simpler: they piggyback on the well-understood preload mechanism and don’t create stateful push streams. They’re seeing adoption – Cloudflare enabled them by default for many customers (with reported success), and browsers like Chrome and Safari have implemented support. Essentially, Early Hints achieve the core goal of server push (mobilize the client to get resources earlier) but without the pitfalls.

Preload and resource hints: Even without 103, developers could embed <link rel="preload" href="/main.css"> in the HTML head to tell browsers to start fetching something early. The limitation was the browser only sees that after getting the HTML. With Early Hints, you’re moving that instruction up in time. Another related mechanism is Preconnect or DNS-prefetch hints, which a server can also send in 103 or in HTML headers (to establish connections early). These all work together to reduce round trips.

Lessons learned: For an interview, this topic is often about showing that you stay current with web optimization trends. A strong answer might say: “HTTP/2 server push turned out to be not very effective – it’s being removed from Chrome due to lack of clear benefit. Instead, an alternative called Early Hints (103) is used, which lets servers suggest resources for preload. This is less error-prone because the browser stays in control and won’t download things it doesn’t need. So in my design, I’d likely rely on preload hints and maybe configure the CDN to use Early Hints rather than trying to use HTTP/2 push.” This shows an understanding of how theory met reality.

Also, if discussing HTTP/3/QUIC – note that HTTP/3 doesn’t have push actively used, and that the performance gains of HTTP/3 come more from lower latency transport than application-layer tricks. Instead of server push, new ideas like client-driven content negotiation and better caching (or just building a critical CSS/JS inlined strategy) have become the norm.

In summary, HTTP/2 push is deprecated (very low usage and being removed), and 103 Early Hints + Preload is the modern way to accomplish that early loading behavior. CDNs can automate a lot of this (e.g., Cloudflare can analyze a site and push hints for the critical assets). So when designing a system or optimizing a web service, it’s wiser to lean on hints and good caching rather than server push. This evolution underscores that sometimes simpler, more observable mechanisms win over complex “magic” ones in web protocol design.

10. AWS CloudFront and Media Services (Illustrative Examples)

Finally, let’s touch on a few AWS offerings that exemplify the above concepts in a cloud vendor context. While staying vendor-neutral in design is good, mentioning these shows you know real-world services and how they map to our discussion:

These AWS features illustrate how big cloud providers integrate CDN capabilities with edge computing, media handling, and security. They serve as concrete examples if you want to compare or contrast approaches. For instance, you might compare CloudFront Functions to Cloudflare Workers (similar concept of lightweight edge scripts) or note that AWS’s media pipeline achieves low-latency live streaming by integrating with CloudFront (just as one could design a custom solution similarly).

Conclusion: CDN technology is a broad and evolving field. Modern system design questions may touch on any of the areas we’ve covered: deploying logic at the edge, managing data consistency across distributed caches, personalization strategies, delivering rich media efficiently, optimizing payloads, using multiple CDNs for reliability, extending CDNs into private domains, squeezing the most out of servers with kernel bypass, and leveraging new web protocols. A well-prepared engineer should be able to discuss these concepts, cite examples, and reason about trade-offs. Ultimately, a CDN is all about moving content faster and smarter, and the “advanced use-cases” we’ve explored are all various means to that end. By combining these techniques appropriately, one can design systems that are performant, resilient, and suitable for the next generation of scale.

system-design