AI Infrastructure

AI infrastructure is moving beyond the chip itself, and three new announcements show how the pressure is spreading across memory, storage, and metadata layers. Samsung, Kioxia, and Peak:AIO each point to the same challenge: keeping data moving fast enough for larger AI workloads and more demanding environments. Samsung has started sending samples of its 12-layer HBM4E to major global customers. That move follows the company’s earlier mass production of HBM4, placing the next generation into customer testing sooner than many expected. HBM4E pushes memory bandwidth higher High Bandwidth Memory stacks DRAM dies vertically and uses wide parallel connections to maximize data throughput. It is paired with GPUs and domain-specific processors in chiplet-based systems to feed training and inference workloads. Samsung says HBM4E reaches a stable pin speed of 14 gigabits per second, with performance that can scale to 16Gbps. The company also says bandwidth per stack can reach 3.6 terabytes per second, more than 20 percent above HBM4. The 12-layer version is said to deliver 48GB of capacity, more than 30 percent higher than the previous generation. Samsung is also preparing 32GB 8-layer and 64GB 16-layer variants to match different customer needs. Market conditions help explain the urgency. Samsung says demand for HBM remains strong while supply capacity is still not enough, keeping prices elevated and pushing large users such as hyperscale data centers to rely on long-term contracts. Kioxia extends enterprise storage from data centers to space At HPE Discover, Kioxia will highlight SSDs designed for both terrestrial computing and space-based workloads. The company will also showcase PCIe 5.0 Enterprise and Datacenter products, EDSFF, and 24G SAS, also known as SAS-4. Kioxia says the lineup offers higher performance, greater storage density, and better power efficiency than earlier generations. Its SSDs are already used across HPE platforms, including enterprise servers, digital storage, mobile computing, and modern data centers. One of the most notable examples is Kioxia’s role in HPE’s Spaceborne Computing efforts. That includes work on the International Space Station and planned lunar exploration missions. The emphasis shows how enterprise storage is now judged by more than speed and capacity alone. It must also hold up in operating conditions that are far more demanding than a standard data hall. Peak:AIO and Los Alamos bring metadata into the AI bottleneck debate In storage software, Peak:AIO and Los Alamos National Laboratory launched Lattice at the IEEE International Conference on Massive Storage Systems and Technology. The open-source pNFS metadata server is designed to address one of the most persistent bottlenecks in AI and high-performance computing infrastructure. Peak:AIO says AI workloads now require extremely fast, continuous, and reliable parallel access to large datasets. GPU computing has advanced quickly, but the storage layer feeding those GPUs, especially the metadata architecture that coordinates access, has not moved at the same pace. The company also cites Cast AI data showing average GPU utilization of 5 percent across 23,000 production clusters. In that view, the problem is not the GPU hardware itself, but the software behind it. Lattice is built as a Linux-based, user-space metadata server with a focus on scale, modularity, and distributed coordination. The project is open-source, supported by the community, and released under the Linux Foundation. Its control plane is divided into four layers: Protocol State Plane, Lattice Core, MD Catalog Authority, and Data Server Control Plane. That structure is intended to make metadata services elastic, able to run on commodity hardware, and scalable from a single server to more than 1,000 metadata servers. Taken together, the three announcements show that AI performance is no longer defined only by faster accelerators. Memory bandwidth, enterprise storage resilience, and metadata orchestration are all becoming decisive parts of the same competition.

Samsung HBM4E Moves First, While Storage and Metadata Race to Keep Up