htm and sdrs for newbies
TRANSCRIPT
NuPIC Bay Area Meetup
Mar, 25 2015
• HTM and SDRs for Newbies – Jeff Fohl ~20 mins• Open Source Community Updates – Matt Taylor ~10 mins• Nupic Demo ~30 mins
– Matt [ Application Demo ]– Subutai [ Research Demo ]– Chandan [ Walnut ]
Hierarchical Temporal Memory is the model that describes how information is stored and processed in the brain.
• An HTM is a hierarchy of nearly identical regions of neurons.
• Regions learn, infer, and recall time-based sequences.
• Regions are constantly predicting the next inputs.
• Regions detect anomalous input.
MemoryIn order to remember sequences, a region needs a way to store this information. How is this done?
How do we represent an SDR?It can be written as a vector with the following properties:• High dimensionality. It will usually contain at least 2,000
elements.• Binary. Each element in the vector is either “on” or “off”.• Sparse. Out of the total number of bits, only a small percentage
(usually 2%) will ever be active.• Distributed. The “on” bits will be spread across the vector.• Semantic. Each bit in the vector has meaning.
00010000000000100000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000100000000000000000010000000000000000000000000000000000000000000000000000000001000000000000100000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000001000000000000001000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000001000000000000000000000000000000000000000000000000000000001000000000100000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000001000000000000000000000000101000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000
This could be written out using 1s and 0s:
If you are using so many bits, why not use them all? By using only 2% at a time, aren’t you wasting information?
Q:
You would be, if you were processing the information in the way a traditional computer processes information. But, this is different.
A:
The number of patterns that can be represented by a 2,048 bit SDR with 40 active bits:
1084
(that’s way more than the number of atoms in the observable universe)
A 2,000 bit SDR, with 40 active bits, means we only need to store the indices of those 40 – not the entire 2,000 bits.
7,29,47,96,101,134,139,157,180,201,234,245,267,273,291,299,304,356,358,363,389,404,435,444,453,467,481,482,495,505,603,667,689,703,766,781,834,875,909,945