When you think you had an handle on things, some new technology comes up and throws you a curve ball. Let’s look at some of the early claims of 3D XPoint (pronounced 3D Crosspoint).
- 8x to 10x greater density than DRAM
- Think about DIMM (?) modules with 320GB of storage
- Current technology is using 2 layers, more could be possible, further increasing this number
- 100x faster than NAND latency wise
- If I take a Samsung XS1715 NVMe drive, that would put it at 900ns read/250ns write
- 10x more performance than NAND over PCIe/NVMe
- With NVMe drives reaching about 3GB/s, that would mean 30GB/s per drive. Let that sink in for a moment.
- Xeon SkyLake CPUs are supposed to have up to 48 PCIe 3.0 firstname.lastname@example.orgGB/s, yielding 47.2GB/s… Hmm we might have a problem here!
- For a 2 sockets system, that would mean about 3 drives and you’re not getting that data out on the network at all as there are no PCIe lanes left
- For each drive, you would need 240Gbps of network bandwidth
- In practice you would have systems where 1 socket is dedicated to the 3D XPoint drive and another for the NIC (PCIe lane wise)
- Compared to SAS SSD (i.e. HGST 1600MM), those drives are about 30x faster
- Writes have almost no impact on durability
- This always have been a concerned for NAND users
- 3D XPoint is supposed to be affordable, what that means in practice remains to be seen
- It will be available in 2016, which is pretty darn soon
As some of you may know, I’m currently looking at 100Gb networking for our next generation Storage Spaces Direct network. Keeping with 3D XPoint in mind, HDR Infiniband is almost a minimum (200Gbps) but that only comes out in 2017 or so. Looking at the Ethernet roadmap we see 200Gb in 2019-2020 or 400Gb in 2017 (estimated). 800Gb is targeted for 2020. Looks like we’re going to live with a network bottleneck for quite sometime.
If you thought like me that NAND with NVMe really added a lot of pressure on the networking side of things, 3D XPoint brought that to a whole other level!
More info about the technology can be found here.