While I was digging through some older files (early 2013) on OneDrive, I found the following idea I had worked on which I thought was interesting, especially in regards to Nano Server, Storage Spaces Direct, NVMe, and containers. Other than the SoC that might be changed to something more x86 friendly, the rest might still be interesting. Here’s what the hardware diagram looked like:
A couple of interesting things:
– All “cluster” wide communications are happening over PCIe
– Uses RAM, flash and rotational media as tiers in a unified memory model
– Original design was using ARM SoC from NVidia to get more GPGPU horsepower
– The goal is to increase storage/compute density in a linear fashion
Here’s what the core software stack looked like:
– Virtualization is operating at the thread level
– All threads can be distributed across all physical nodes
– Possible to emulate VM behavior through device emulation
– The goal was to run application threads as close to the metal as possible while maintaining mobility and load distribution capabilities
– As mentioned above, thread can access memory tiers (RAM, flash, rotating media) transparently, simplifying the application development model
– As ARM SoC were used an X86 emulation layer was required. The goal was to leverage as much as possible the GPU through possibly OpenCL
Overall the goal/assumption was the more data you have, the more you need to scale your compute layer to process and derive value from your data. Azure Service Fabric could also bring us closer to this concept as the state can be distributed and synced across multiple cluster nodes, which is a way to have thread mobility across the hardware ultimately. Nano Server with Storage Spaces direct can give us part of the platform but what would remain to be done is the unified memory model with tiering across all memory technologies and standard stack to transparently use CPU and GPU compute resources.
I thought it was interesting to dust off that old idea!