Transition from Gen 12 to Gen 13 Servers
Cloudflare's shift to the Gen 13 server fleet marks a deliberate move to optimize compute performance at the edge. Unlike the cache-heavy design of Gen 12 servers powered by AMD EPYC GenoaX processors, the Gen 13 systems prioritize core scalability over per-core cache. This shift posed challenges, particularly for workloads that previously depended on large caches for efficient request handling. However, these challenges were addressed through a comprehensive redesign of Cloudflare's core software stack.
The introduction of the FL2 framework, a Rust-based rewrite of the core request handling layer, played a pivotal role in overcoming these limitations. By eliminating dependencies on large caches, FL2 enabled the new servers to fully exploit the increased core density without sacrificing latency. This architectural overhaul was instrumental in maintaining strict SLA compliance while achieving higher throughput.
Architectural Highlights of AMD EPYC Turin
The AMD EPYC 5th Generation Turin processors used in Gen 13 servers bring substantial core count improvements, scaling up to 192 cores with simultaneous multithreading (SMT) providing 384 threads. This is a significant leap compared to the 96-core limit of GenoaX processors. Such a high density of cores supports parallel processing at an unprecedented scale for Cloudflare's edge computing needs.
Turin also introduces improvements in instructions-per-cycle (IPC) through Zen 5 architecture, enhancing computational efficiency. Additionally, the processors are more power-efficient, consuming up to 32% less wattage per core compared to their predecessors. Coupled with DDR5-6400 memory support, this architecture ensures that the higher core count is adequately fed with data, reducing bottlenecks in memory bandwidth.
Cache Tradeoffs and Mitigation Strategies
A key design tradeoff of the Turin architecture is the reduction in per-core cache. The 192 cores share 384MB of L3 cache, equating to 2MB per core-just one-sixth of what GenoaX processors provided. This presents a challenge for cache-locality-dependent workloads, which historically benefited from the larger cache sizes in Gen 12 servers.
Cloudflare's solution was to transition to a software architecture that could better exploit core scalability. FL2 was developed with this specific goal, allowing for workload optimization by distributing tasks more efficiently across the increased number of cores. This minimizes the performance impact of reduced cache while maximizing throughput.
Power Efficiency and Scalability
The power efficiency of the Turin architecture is a critical factor in enabling Gen 13 servers to scale. By consuming less power per core despite the higher core density, these processors align with Cloudflare's operational goals of delivering high performance while managing energy consumption.
This efficiency also contributes to improved thermal management, reducing the need for aggressive cooling solutions. As a result, the new server fleet can accommodate higher compute workloads without incurring significant operational overhead, aligning with sustainability objectives.
Implications for Edge Computing
The Gen 13 deployment underscores a shift in Cloudflare's edge computing strategy. By prioritizing core scalability over cache size, the company has positioned itself to handle more concurrent operations and adapt to future demands. The FL2 framework is a cornerstone of this strategy, ensuring that software can keep pace with advancements in hardware.
These changes not only enhance performance but also future-proof the infrastructure against emerging workloads that demand high parallelism. The combination of architectural improvements and software optimization demonstrates a meticulous approach to sustaining performance gains at the edge.