Have Skills, Will Travel Charles Armour

Technological Singularity – Today Only $500M!

This morning I read a great article from New Scientist on memristors and their biological equivalents, and boy did I get excited.  I've been following the convergence of biology and technology for quite some time as I'm interested in bringing about the singularity.  When the two fields converge enough, we'll have the ability to fully simulate the biological computers that we're so fond of (brains), and hence we'll have a proper playing ground for true artificial intelligence.

Current leading research in this matter tops out at around 10 thousand neurons and 30 million synapses, which is a far cry from the 100 billion neurons and 150 trillion synapses in a human brain.  That computer is a 22.8 TFLOP machine, while the fastest supercomputer in the world currently is around 1100 TFLOPs.  That still only provides enough computing horsepower to handle simulation of 500 thousand neurons and 150 million synapses.  We're a long, long way off.

Or are we?  Those systems are not designed from the ground up to simulate such things.  They use global memory and thread models, run in software rather than hardware, and are generally unsuited to the distributed computation that neuron simulation requires.  The physical capacity (scaling) isn't too terrible either, as long as you've got large eyes.  Here are a few assumptions (based on this page) for calculations (please correct me if I'm off base here!):

• A neuron can be represented by 3 floats - 1 excitation threshold, 1 excitation state, and 1 cooling factor
• A synapse can be represented by 2 addresses and 2 floats - 1 source, 1 destination, 1 memory (per memristors), and 1 efficiency factor
• The human brain contains some 100 billion (1.0E11) neurons.
• The human brain contains some 150 trillion (1.5E14) synapses.

A float is either 32b or 64b, and an address in this context would most certainly be 64b.

With these numbers we can ascertain the memory requirements of a brain simulation system, and those would be:

• 100 billion neurons at 12 bytes each is 1.2E12 bytes, or 1.2 terabytes.
• 150 trillion synapses at 24 bytes each is 3.6E15 bytes, or 3.6 petabytes.

Let's ignore the cost of processing and updating all this information for now, and see what it might cost to put together such a memory subsystem:

• Commodity pricing for DRAM is around $10/GB for the fast stuff (DDR3).
• You'd need around 4 million of these sticks (includes some wiggle room).  Can be had for about $40M.
• Commodity pricing for HDDs is around $0.10/GB for the big stuff, say 2TB drives.
• You'd need around 2000 of these HDDs (includes some wiggle room) for persistance.  Can be had for about $400K.

So we're at a little over $40M just in memory.  Obviously you'd get better rates at these volumes, though!  Let's double this price for backplanes and power supply, to a cool $80M in memory, power, cooling, and interconnects.  Remember that our interconnects are just passing synapse data over a 3D mesh like our brain, nothing toroidal or otherwise, and they benefit from extreme locality.

Now let's investigate processing at 100Hz (suggested by many studies as the maximum neural firing rate) with the use of FPGA chips (so they can still be reprogrammed, but run in hardware):

• Xilinx Virtex-6 FPGAs run at (up to) 1.6Ghz and are suitable for stream processing on up to 8 DRAM modules at once.
• A 512Mbit memory module/chip with a 64 bit (or larger) interface could read and write its entire contents about 100 times a second.
• I don't have pricing for the Virtex-6 chips, but let's assume that with their 75% off volume pricing they'd cost around $100 each.
• You'd need about 4 million Virtex-6 chips to process all 4 petabytes of state 100 times per second.  Can be had for $400M (or just build a fab).

So there you have it, it's possible to do brain simulation at the human brain scale for about $500 million dollars, $600 million if you decide you must have a synchronized clock system (which requires double the state memory for buffering.  If anyone has this kind of cash lying around, let me know and I'll make it happen!*

If not, don't worry!  I'm sure the government already has ten - these are a bargain compared to Stealth Bombers, and why build one when you can build ten at ten times the cost?

* I can't be held responsible if a $500M machine just sits there and blinks a lot, I'm sure there are many secrets to the brain we haven't discovered!