Architecture Num neurons Num synapses

Fly 100K = 10^5 10M = 10^7

AlexNet 650K = 10^6 60M = 10^8

Mouse 100M = 10^8100 B= 10^11

Human 100B = 10^111014 - 1015

If each synapse is 1 FLOP (i.e., can fire / not fire once per second),

Then human brain requires 10

15

flops = 1 petaflop.

10,000 current CPUs, costs $1000 / hron Amazon’s EC2

Or 10 latest GPUs (Nvidia V100), costs $30 / hron EC2 (2017) --$25 / hron GCP (2018)

Now let’s take a look at scale