Old Files – Distributed hardware and core software stacks

While I was digging through some older files (early 2013) on OneDrive, I found the following idea I had worked on which I thought was interesting, especially in regards to Nano Server, Storage Spaces Direct, NVMe, and containers. Other than the SoC that might be changed to something more x86 friendly, the rest might still be interesting. Here’s what the hardware diagram looked like:


A couple of interesting things:

– All “cluster” wide communications are happening over PCIe
– Uses RAM, flash and rotational media as tiers in a unified memory model
– Original design was using ARM SoC from NVidia to get more GPGPU horsepower
– The goal is to increase storage/compute density in a linear fashion

Here’s what the core software stack looked like:

Some of the highlights:

– Virtualization is operating at the thread level
– All threads can be distributed across all physical nodes
– Possible to emulate VM behavior through device emulation
– The goal was to run application threads as close to the metal as possible while maintaining mobility and load distribution capabilities
– As mentioned above, thread can access memory tiers (RAM, flash, rotating media) transparently, simplifying the application development model
– As ARM SoC were used an X86 emulation layer was required. The goal was to leverage as much as possible the GPU through possibly OpenCL

Overall the goal/assumption was the more data you have, the more you need to scale your compute layer to process and derive value from your data. Azure Service Fabric could also bring us closer to this concept as the state can be distributed and synced across multiple cluster nodes, which is a way to have thread mobility across the hardware ultimately. Nano Server with Storage Spaces direct can give us part of the platform but what would remain to be done is the unified memory model with tiering across all memory technologies and standard stack to transparently use CPU and GPU compute resources.

I thought it was interesting to dust off that old idea!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s