CSM
ProShares Large Cap Core Plus
stock BATS ETF

> Introducing Cluster-Scale Memory (CSM) for low latency workloads.
>
> Today's AI chips using HBM can’t achieve SRAM-level decode speeds due to memory subsystem and interconnect bottlenecks. SRAM-only chips have lower FLOPs density and memory capacity, sacrificing throughput.
>
> You’re forced to make a tradeoff: serve at much slower speeds, or run at low batch sizes and suffer from higher costs.
>
> When running large MoE models, token routing across experts requires sending data through a deep memory hierarchy and a networking switch to reach a destination expert.
>
> Each memory layer inherently adds latency; thus, the best layer is no layer.
>
> We’ve designed a new architecture that creates a shared low-latency memory pool across the entire scale-up domain.
>
> We use a proprietary ultra-low-latency, high-bandwidth interconnect to enable dramatically faster memory access across chips.
>
> Our HBM/SRAM hybrid design solves both memory capacity and mem2mem latency, enabling high throughput and interactivity simultaneously.
>
> CSM improves latency and avoids today's cost, reliability, yield, thermal, and compute tradeoffs of SRAM-only chips, 3D DRAM chips, or optics.
Fascinating stuff, I argued many moons ago about the need for a new type of memory in between HBM and SRAM, well it looks like someone has done it. Too bad it's not Nvidia.
https://www.reddit.com/r/NVDA_Stock/comments/1l0c3qt/nvidia_needs_to_make_an_ultralow_latency/

> Introducing Cluster-Scale Memory (CSM) for low latency workloads.
>
> Today's AI chips using HBM can’t achieve SRAM-level decode speeds due to memory subsystem and interconnect bottlenecks. SRAM-only chips have lower FLOPs density and memory capacity, sacrificing throughput.
>
> You’re forced to make a tradeoff: serve at much slower speeds, or run at low batch sizes and suffer from higher costs.
>
> When running large MoE models, token routing across experts requires sending data through a deep memory hierarchy and a networking switch to reach a destination expert.
>
> Each memory layer inherently adds latency; thus, the best layer is no layer.
>
> We’ve designed a new architecture that creates a shared low-latency memory pool across the entire scale-up domain.
>
> We use a proprietary ultra-low-latency, high-bandwidth interconnect to enable dramatically faster memory access across chips.
>
> Our HBM/SRAM hybrid design solves both memory capacity and mem2mem latency, enabling high throughput and interactivity simultaneously.
>
> CSM improves latency and avoids today's cost, reliability, yield, thermal, and compute tradeoffs of SRAM-only chips, 3D DRAM chips, or optics.
Fascinating stuff, I argued many moons ago about the need for a new type of memory in between HBM and SRAM, well it looks like someone has done it. Too bad it's not Nvidia.
https://www.reddit.com/r/NVDA_Stock/comments/1l0c3qt/nvidia_needs_to_make_an_ultralow_latency/