Minhuan Li
(李黾奂)

Be Vulnerable

© 2024. Min-Huan Li.

Neuroidal Model

[  BiologicalComplexity  ] 10 Apr 2021

This is a summary about paper C. H. Papadimitriou, S. S. Vempala, Unsupervised Learning through Prediction in a Model of Cortex. This paper is a follow-up work of Prof. Valiant’s neuroidal model. Generally, people want to understand why human brains can do some amazing jobs like memorization, association. Traditional computer algorithms work in fashions that seem completely incompatible with the way brain actually work. The neuroidal model is one kind of minimalist quantitative neural model.

In the origin Valiant model, the cortex is modelled as a random directed graph of neuroids (abstract neuron-like elements) connected via directed edges called synapses. The algorithms running on this platform are vicinal by which it is meant that they are local in a very strict sense: the only communication from other neuroids that a neuroid is allowed to receive is the sum of the action potentials of all currently firing neuroids with synapses to it. This is the only outside information a neuroid can use in deciding what to do next. Valiant goes on to posit that real-world objects or concepts can be represented by sets of neuroids. Such sets are called items, and they constitute the basic element of Valiant’s theory: all \(r\) neuroids of an item firing (or perhaps some overwhelming majority thereof) is coterminal with the corresponding concept being “thought about.” Valiant shows that such items can be combined through the basic operations of JOIN and LINK, creating compound new items from, respectively simply connecting, existing items. For example, if A and B are items (established sets of about \(r\) neuroids) then Valiant shows how to create, via a vicinal algorithm, a new item JOIN(A, B) which stands for the combination, or conjunction, of the two constituent items.

Based on Valiant’s work, authors of this paper contribute formulation and implementation of a new operation on items that we call predictive join or PJOIN. This operation PJOIN extends JOIN in that, if only one of the constituent elements of PJOIN(A, B) fires, say A fires, then the structure will predict B, and initiate a sequence of downstream events whose purpose is to check this prediction.

This is how PJOIN is implemented to create PJOIN(A,B):

  1. Create C=JOIN(A, B), by executing the vicinal algorithm of the previous section.
  2. Then create a chosen half of the neuroids of item C called “predictive” C: \(C_{P}\). The neuroids of C that are not in \(C_{P}\) now enter a total state called OPERATIONAL, ready to receive firings from A and/or B as any neuroid of JOIN(A, B) would.
  3. Suppose now that A and B are predictive joins as well, and therefore they have parts \(A_{P}\) and \(B_{P}\). Then perform \(\operatorname{LINK}\left(C_{P}, A_{P}\right)\) and \(\operatorname{LINK}\left(C_{P}, B_{P}\right)\), linking the predictive part of C to the predictive parts of its constituent items. After these LINK creations, the working synapses from the relay neuroids to the neuroids of \(A_{P}\) and \(B_{P}\) will be in a memory state PARENT identifying them as coming “from above.” Such synapses will have a strength larger than the minimum required one (say, double), to help the neurons identify firings from a parent.
  4. After this is done, \(C_{P}\) enters the total state P-OPERATIONAL, where all strengths of synapses from A and B to \(C_{P}\) (but not to the rest of C ) are doubled. The effect is that the neuroids of \(C_{P}\) will fire if either A fires, or B fires, or both. This concludes the creation of PJOIN(A, B) Notice that it takes four steps (we discuss in the last section one possible way to simplify it to three steps). After the creation of the predictive join is complete, indeed, if only one of A or B fires, then \(C_{P}\) will fire, initiating a breadth-first search for the missing item.

And this is how the PJOIN(A,B) operates:

  1. If both inputs A and B fire simultaneously, then PJOIN(A, B) will operate as an ordinary JOIN.
  2. As soon as one of A and B fires - suppose that A fires - then \(C_{P}\) fires (because of the doubled synaptic strengths), an event that will cause \(B_{P}\) to fire downwards in the next step. Notice that \(A_{P}\) will not fire as a result of \(C_{P}\) firing, because it is in a refractory state. Also, after the firing of A, the neuroids in C double the strength of the synapses coming from B (that is, from neuroids that do not come from a parent and are refractory), and all of C enters a total state called PREDICTING B. Thus, if henceforth the predicted item B fires, all of C will fire.
  3. \(C_{P}\)’s firing will cause \(B_{P}\) to fire downwards in the next step; \(A_{P}\) does not fire as a result of \(C_{P}\) firing, because it is in a refractory state. After this, \(C_{P}\) enters a PASSIVE total state, in which it will ignore firings of the \(E_{P}\) part of any item E=PJOIN(C, D) further above. Again, this is accomplished by setting the strength of these synapses (which are the ones in the PARENT memory state) to zero.
  4. If henceforth the predicted item B fires, then C fires and all of A, B, C go back to the OPERATIONAL (P-OPERATIONAL for \(\left.C_{P}\right)\) total state.
  5. Finally, if one of C ‘s parents (by which we mean any item of the form PJOIN(C, D) fires “downwards,” then the neurons in \(C_{P}\) will fire (we think of them as firing “downwards”), thus propagating the search for the predicted items.

Comments