Hierarchical Temporal Memory
Hierarchical temporal memory (HTM) is a biologically constrained machine intelligence technology developed by Numenta. Initially described in the 2004 ebook On Intelligence by Jeff Hawkins with Sandra Blakeslee, HTM is primarily used today for anomaly detection in streaming data. The technology is based on neuroscience and the physiology and interaction of pyramidal neurons in the neocortex of the mammalian (in particular, human) brain. On the core of HTM are learning algorithms that may store, be taught, infer, and recall excessive-order sequences. Unlike most other machine studying methods, HTM continuously learns (in an unsupervised process) time-primarily based patterns in unlabeled data. HTM is sturdy to noise, and has high capacity (it will possibly be taught a number of patterns simultaneously). A typical HTM network is a tree-shaped hierarchy of ranges (not to be confused with the "layers" of the neocortex, as described beneath). These ranges are composed of smaller components called areas (or nodes). A single level within the hierarchy presumably comprises several areas. Higher hierarchy levels usually have fewer regions.
Increased hierarchy levels can reuse patterns discovered at the decrease ranges by combining them to memorize extra complicated patterns. Every HTM region has the identical basic perform. In studying and inference modes, sensory data (e.g. information from the eyes) comes into bottom-stage regions. In generation mode, the underside stage areas output the generated sample of a given category. When set in inference mode, a area (in each stage) interprets information coming up from its "child" regions as probabilities of the categories it has in memory. Each HTM region learns by figuring out and memorizing spatial patterns-combinations of enter bits that always happen at the identical time. It then identifies temporal sequences of spatial patterns which might be likely to happen one after another. HTM is the algorithmic element to Jeff Hawkins’ Thousand Brains Principle of Intelligence. So new findings on the neocortex are progressively integrated into the HTM model, which modifications over time in response. The new findings don't necessarily invalidate the earlier elements of the mannequin, so ideas from one era will not be necessarily excluded in its successive one.
During coaching, a node (or region) receives a temporal sequence of spatial patterns as its input. 1. The spatial pooling identifies (in the input) regularly observed patterns and memorise them as "coincidences". Patterns which are considerably comparable to one another are treated as the identical coincidence. A lot of potential input patterns are lowered to a manageable number of known coincidences. 2. The temporal pooling partitions coincidences that are likely to follow one another within the training sequence into temporal groups. Every group of patterns represents a "trigger" of the input sample (or "identify" in On Intelligence). The ideas of spatial pooling and temporal pooling are still fairly important in the present HTM algorithms. Temporal pooling just isn't but nicely understood, MemoryWave Guide and its which means has modified over time (because the HTM algorithms advanced). During inference, the node calculates the set of probabilities that a pattern belongs to every known coincidence. Then it calculates the probabilities that the input represents every temporal group.
The set of probabilities assigned to the groups is called a node's "perception" about the input pattern. This perception is the results of the inference that is passed to one or more "mother or father" nodes in the subsequent greater degree of the hierarchy. If sequences of patterns are much like the coaching sequences, then the assigned probabilities to the groups is not going to change as often as patterns are obtained. In a more normal scheme, the node's perception may be sent to the enter of any node(s) at any degree(s), however the connections between the nodes are still mounted. The upper-stage node combines this output with the output from other baby nodes thus forming its own enter sample. Since resolution in area and time is lost in each node as described above, beliefs formed by larger-degree nodes signify an even bigger range of space and time. This is meant to replicate the organisation of the bodily world as it is perceived by the human brain.