# A Brain that works on Images – Where the Images are Neurotransmitter Maps

Professor Douglas Greer has some original ideas on how the cortex works, and he has made a working program based on them, the source-code of which you can get for free. The most revolutionary idea is that instead of looking at the firing of a group of neurons as a representation of an object, we should look at the distribution of neurotransmitter at synapses as the representation.

First we should define some basic terms here, so that we can understand his theory.

1. Manifolds:

A manifold is like a landscape with some measure on it. For example, if you took a map of Manhattan, and listed the heights of all buildings at all locations, you would have a 2-dimensional manifold. At certain points the heights would be zero (such as Central Park) at other points they would be large (like at the location of the Empire State Building).

A contour map showing the heights of mountains and valleys would also be a manifold. Manifolds can be 1 dimensional, 2 dimensional, 3 dimensional on up. A contour map of the United States is a continuous manifold – it isn’t divided up into discrete points, it’s just a continuous shape with slopes at any point varying from positive to zero to negative (with an occasional abrupt discontinuity like a cliff). Manifolds can be discrete as well, and give that computers are digital, even continuous manifolds must often be approximated by discrete points. If you are wading into a cold swimming pool on a hot day, your skin can be thought of as a manifold with a measure of temperature, where your legs are colder than your upper body.

1. S.R. flipflop

One interesting aspect of Greer’s ideas is that he takes human inventions, and finds counterparts to them in the brain. The S.R. flipflop is one example, wavelets are another. Let’s look at the S.R. flipflop:

A logic gate is a device that takes in 2 inputs. The output is a logical operation on those inputs. For instance, an OR gate outputs a ‘1’ (for TRUE) if either or both of its inputs are ‘1’. Intuitively, this is like saying “if I take a nap on the beach, or I go to a tanning salon, I will get a tan.”. If either or both premises are true, then the conclusion will be true. Using an AND gate is like inferring: “If there is a sale on the 3 remaining iPads at ‘Best Buy’, and I am first in line when it opens, I can purchase all three”. In other words, both premises have to be true, for the conclusion to be true.

A NOT gate has only one input. If you feed it TRUE, the output is FALSE, and vice versa. (Or if you feed it 0, the output is 1, and vice versa).

A NOR gate inverts the conclusion or an OR gate, it negates it. It is like an OR gate in series with a NOT gate.

You can create truth tables for these gates. For an OR, the truth table has 3 columns, the first two being the inputs, and third being the output. It would look like this:

0,0,0
0,1,1
1,0,1
1,1,1

The first row shows that if neither premise is true, then the conclusion is not true either.

The basic building block that makes computer memories possible, and is also used in many sequential logic circuits is the flip-flop or bi-stable circuit. Just two inter-connected logic gates make up the basic form of this circuit whose output has two stable output states. When the circuit is triggered into either one of these states by a suitable input pulse, it will ‘remember’ that state until it is changed by a further input pulse, or until power is removed.

The SR flip-flop can be considered as a 1-bit memory, since it stores the input pulse even after it has passed. Flip-flops (or bi-stables) of different types can be made from logic gates and, as with other combinations of logic gates, the NAND and NOR gates are the most versatile, the NAND being most widely used. This is because, as well as being universal, i.e. it can be made to mimic any of the other standard logic functions, it is also cheaper to construct.

Here again is the diagram of a S/R (set / reset) flipflop.

The output of each gate is connected to one of the inputs of the other gate.

The circuit has two active low inputs marked S and R, ‘NOT’ being indicated by the bar above the letter, as well as two outputs, Q and Q. Table 5.2.1 shows what happens to the Q and Q outputs when various inputs are applied to either the S or R inputs.

Each gate here is a NAND gate, that means it has the truth table of an AND, with a negation afterwards.

Below I explain what happens when you apply an input – this is important to understand because Greer thinks an analogous mechanism works in the brain:

Let suppose that the initial S and R inputs are 0, which means their negations are 1 (Their negations are fed into the flipflop, as you can see in the diagram)

Suppose you apply a one to the S input. S-bar (negation) becomes 0. This will make the Q output become logic 1. (Q is considered the value of the flipflop, so at the moment the value is 1. There is also an output coming out from the lower gate, which is supposed to be the negation of whatever Q happens to be at the moment.

Why did the zero at S-bar lead to a 1 at Q? because an AND gate outputs a zero if any of its inputs are 0. This is a NAND gate, so the output 0 is inverted to become a 1. The 1 comes out of the upper gate at Q, and also gets fed into the lower gate. Remember we said that R is 0, so R-bar is 1, so we now have two ones feeding in as inputs into the lower gate. An AND of 2 ones is a one, but this is a NAND gate, so the one gets inverted to be a zero, and comes out as Q-bar. That’s good, because we want Q-bar to be the negation of Q. The zero also gets fed into the upper gate as an input, so along with S-bar, that means that the inputs are a 0 and a 1 and we know that NAND’s output a one in this case.

The interesting part now is that returning the S input to logic 1 has no effect. The 0 pulse has been ‘remembered’ by the Q. S-bar may now be zero, (the other input is also zero), so the NAND output remains as 1.

Q is reset to 0 by logic 0 applied to the R input (R means reset).

Even when R returns to logic 1 the 0 on Q is ‘remembered’ by Q.

There are problems with the SR Flip-flop. For instance, there is a state where it keeps flipping output values. This happens if the inputs change from 0,0 to 1,1 together.

Basically the SR Flip-flop is a simple 1-bit memory. If the S input is taken to logic 0 then back to logic 1, any further logic 0 pulses at S will have no effect on the output.

In Douglas Greer’s model of the cortex, instead of bits being fed into this setup, entire images are passed on the S and the R lines. If the images are ‘reciprocal’, (just like true and false are reciprocal), then an entire image is remembered on the Q output.

You might wonder how images could be reciprocal, but remember that any logical operation can be represented as a truth table, which means that are associating various combinations of values with another value. Neural nets are all about learning associations, so you can create a truth table to make Greer’s memories work. One advantage of these types of memories is that if the image is imperfect, or somewhat noisy, it will converge on the correct image.

1. Wavelets

Another human-invented technique that the brain may also use is ‘Wavelets’. If you look at an image, it may have repeating areas. For instance, an image of a checkerboard obviously does. There is a certain frequency that the black squares repeat at. It turns out that even less-obvious examples of images can be represented as a combination of frequencies. If you look at a sine wave:

You notice that it has both a frequency and an amplitude. An image can be converted, via a Fourier transform, to a set of coefficients of sine (and cosine) waves at different frequencies. You can plot the coefficients on the Y-axis, and the frequencies on the x-axis. This is “frequency space”. The transform can also be used to recreate the original image, and even clean it up a little. For instance, ‘noise’ is often at higher frequencies than the details of the image, so you could remove some of the high frequencies before recreating the image. Even if the higher frequencies are not due to noise, they might be due to small details that are hard to make out anyway, and you can compress an image by removing them from the Fourier transform before you recreate the image.

Fourier transforms have disadvantages. For instance, you are limiting yourself to using waves that go on forever in both directions, and those waves are poor at picking up some types of local features.

Wavelets were invented to make up for the deficiencies. A wavelet (there are many types) might look like:

This wavelet does not go on forever in both directions, in fact, it can be centered at a particular spot. So you can go through an image, with wavelets such as this, trying to fit them at regular intervals. Where they fit well, you will get a large coefficient, where they fit poorly, you will get a low coefficient.

You can then go through the image one more time, but with a dilated form of the wavelet (think of stretching it sideways). In the above example it would cover more space on the image, and the  peaks and valleys would be more spread out.

So with each pass across the image, you have another set of coefficients. For a particular spot on the picture where a wavelet was sampled, given a particular dilation, there will be just one coefficient.

Greer’s idea is that images in the brain can be represented by perhaps parallel layers, each layer corresponding to one dilation of the wavelet. He notes that in your retina, there are neurons that respond when a cell is not firing, but the cells in a circle around it are firing. This can be thought of as a wavelet. He also notes that some of these neurons have a wide receptive field, and others have a smaller one, so the dilation of the wavelet differs.

(It has been shown that different size fields project to different areas, so you have a fine-detail wavelet representation in one layer, and a broader view in another.)

So what is Greer’s theory?

First, on manifolds:

Physical quantities such as the intensity of light impinging on the surface of the retina, the air pressure on the eardrum over time, the position and temperature on the skin surface and the forces over entire muscle cross-sections, are all functions defined on manifolds. We refer to these functions as images or fields.

(Note that an image, in his terminology, is not just a picture taken through a camera, it can be any map of some measure).

Secondly on flip-flops:

He notes that a flip-flop…

can be viewed as a dynamical system with two fixed point attractors that correspond to “0” and “1”. The set of all values that converge toward a particular attractor is referred to as an attractor basin.

We construct a dynamical system analogous to an SR flipflop where the bits are replaced by images and the logic gates are replaced by feed-forward image association processors. The convergence of the system toward a reciprocal-image attractor is then demonstrated.

It is important that images be attractors–over time their components change to converge on one prototype image– for the same reason that computers deal with 1’s and 0’s and not fractional voltages.

One example of mapping between manifolds is from the image on the surface of the retina to a model of three-dimensional physical space. This transformation is the shape extraction problem of computer vision. It has been described as the inverse of computer graphics, which transforms three-dimensional spatial representations to a two-dimensional visual image.

Another example of a mapping would involve hearing.

Tonotopic maps, which map audio frequencies to locations in the brain, have been documented and studied for some time. Some of these resemble audio spectrograms which plot time in the horizontal direction and frequency in the vertical direction. In the computational manifold approach, a child learning new nouns is learning associations between visual images and audio images.

In motor control, a manifold is involved as well:

Fibers within the same muscle may pull the limb in different directions, but close-by fibers pull in a similar direction

Consequently, the two-dimensional image of the muscle cross-section smoothly maps to a manifold in a tangent bundle that describes the forces exerted on the limb. Images on the surface of the cross-section can be used to describe the efferent nerve signals that control the contraction of the muscle fibers as well as the afferent signals that convey the fiber length and tension.

In other words, one manifold coming into the brain might be the signals (arranged as coming from a surface) indicating the tension of your various muscle fibers. Another manifold, this one coming from your brain to your body, might be the signals directed at controlling your muscle fibers. In his article, he says that the ability to carry out motions (such as grasping an object) given your physical state can be looked at completely as mapping between ‘images’.

Smell also involves a manifold.

The olfactory bulb responds to odors with a spatial map generated by a distributed assembly of specific and general molecular receptors. As a result, different mixtures of odor molecules produce unique odor maps. Moreover, there are general similarities between the retina and olfactory bulb in the cellular circuits producing lateral inhibition and receptive fields, which indicate conserved mechanisms of neural processing in vision and olfaction. This is consistent with an image association model of sensory information processing in a frequency space such as one generated by the continuous wavelet transform.

Though computers use Boolean logic (true/false logic), and operations such as addition and multiplication on binary digits, Greer points out that binary digits are abstract mathematical concepts.

If someone were to take apart a typical desktop computer searching for ones and zeros, they might be disappointed to find only analog resistors and transistors. The digital circuit specifications themselves refer only to acceptable ranges of voltages on the inputs and outputs. These specifications are, in effect, a contract with the digital circuit designer that defines a guaranteed behavior. This concept of equating symbols with behavior can be extended to computational manifold automata.

In the SR flip flop, the circuit acts as a dynamical system with two attractors corresponding to “0” and “1”. The two binary digits are, in effect, voltage ranges where the circuit conceptually falls into one of two “energy wells”. These fixed-point attractors create the stability required for an actual physical realization to operate in the presence of the inevitable noise and transient errors in the inputs.

Greer defines a Λ-map (Λ from the Greek word Logikos) as a feed-forward process that accepts one or more images as inputs and produces a single associated output image.

We can create a circuit analogous to an SR flip-flop by replacing the bits with images and replacing the NAND gates with Λ-maps. The recurrent connections are shown below where the exterior and interior Λ-maps are labeled ΛE and ΛI. We refer to the two outputs as reciprocal images and use the term psymap to refer to the structure containing two recursively connected Λ-maps.

In other words, the Logikos (Λ) map maps one or more images to an output image, and the psymap combines 2 Logikos map building blocks in an analogy to the S-R flipflop.

Then Greer says:

Rather than having only two attractors for “0” and “1”, as is the case for an SR flip-flop, a psymap can have any number of attractors. Each reciprocal-image attractor can be constructed from two arbitrarily chosen reciprocal images. Let q = ΛE(p, s) and p = ΛI(q, r) denote the exterior and interior Λ-Maps shown in Fig 5(b) and let Null denote a predefined “blank” image. For an arbitrary collection of image pairs (ai, bi) we can create a new attractor by adding associations to both Λ-Maps so that ΛE(bi, Null) = ai and ΛI(ai, Null) = bi. In addition, we can define associations for the S and R inputs – for example ΛE(X, si) = ai where X is any image – that allow us to force the psymap to the (ai, bi) state.

I think he means that something like the diagram below becomes possible – since NULL and b(i) gives a(i), and NULL and a(i) gives b(i).   Here images a(i) and b(i) act like reciprocals.

Professor Greer sees the various psymaps as connected via a type of bus (see diagram) Each line in the diagram is conveying an entire image.  Psymaps can obtain images from the bus, or can output images to the bus for other psymaps to pick up.

Neurons connect to other neurons via synapses. If at a particular instant, you look at the neurotransmitter concentration in all these synapses, you have a measure on a map, or a manifold. Each synapse has its own measure on the map of neurotransmitter concentrations. Greer sees each manifold as a state that combined with inputs can make a transition to another state.  In everyday life, a state-transition diagram (for an ATM card) might look like this:

Greer’s analogy:

As an informal metaphor, we can imagine a room with a collection of flat-screen displays which are labeled 1 through N. Each display screen is a flat surface, i.e. a rectangular two-dimensional manifold, which we label Mi. The visual image currently being displayed on a screen can be represented by a function fi(x,y) where x and y parameterize the horizontal and vertical dimensions of the screen.

So each TV screen has a 2D image, which varies over x and y

The entire set of images currently being displayed on all screens is the system state.

At the core of an automata is a state transition function which determines the next state based on the current state and any inputs (the inputs would be images as well). The current state is the collection of images currently being displayed on the flat-panel screens. The next state is also a set of images which are uniquely defined by the current-state images and the input images. Since the inputs, outputs and current state are functions, we refer to the mapping between the current state and the next state as the next state transformation Z. Another transformation, T, generates the functions defined on the output manifolds based on the current state and the inputs.

In the diagram below, each PE is a neuron making a bridge between two manifolds. The PE’s shown are part of Logikos maps Λi and Λe.  Each PE maps chemical concentration at its dendrites to chemical concentrations at its axons. The vertical rectangles are manifolds.  In this particular diagram, the output is fed back to the input, and the output image is meant to be the same as the input image.   There  is a time delay in making the  trip through the mapping, so if we call the image by the letter ‘Q’, then the output is Q(t+1). The output is fed back into the input with hope that it  will converge to the prototype symbols Q.

Even though the dendritic and axonal trees of each PE cover only local limited areas, the recurrence allows their effect to spread over the entire image. This works because each PE creates a neurotransmitter map that affects several other PE’s (neurons) in the opposite Λ-map. These in turn form many local loops by feeding back into the original PE.   (note that in the illustration the arrows are labeled with Λi and Λe – each neuron is part of a Logikos map.)
A Λ-map can accept multiple input images by aligning them topologically as shown in the next image. One or more of the input images can serve as a control mechanism by regulating how the other images are processed. Image masks that overlay the multiple input images can be used to control the association formation process and to focus attention on specific regions

So how would Greer’s theory apply to thinking?

For one thing, Greer says that we can view the recognition of unique individuals in the external world as the process of the psymap dynamical system moving toward a unique attractor.

On the topic of human language he says this.

Words spoken by many different people with a variety of vocal characteristics have the same spelling and meaning. These similar but distinct sounds are all mapped to the same symbol.

So when you hear the word “tomato” in a Scottish accent versus  a southern accent, the word converges in your mind to a prototype that you are familiar with.

There has long been a debate on how to define the border between concepts.  He says that the symbols can be defined as either sets or attractors.  (For instance, for the symbol “chair”, you might list everything you can think of that  fits the definition, and that  would  be a set).

These definitions are related, but while using sets may be easier initially, it is more difficult to maintain in practice…

As the dimension of the state space is increased, chaotic attractors are more likely to be created [5]. The boundaries between these attractors will be fractal, so while the sets corresponding to the attractor basins can be defined in theory, in practice they are impossible to precisely locate.

Imagine a high-resolution photograph of a room containing assorted items. If several people were asked to describe the room, their narratives would have significant differences in the words used and the grammatical constructions.

Consequently, it is difficult to define the photograph in terms of sets. However, if we consider words to be CMA attractors in a dynamical system, then we can describe the process as each person having a “visual mask” that changes in shape and moves around the photograph of the room, bringing various items into “focus”. The resulting image portion may come close to a reciprocal-image attractor that is associated with a distinct word. The images that form the system state then move toward that attractor, eventually causing that word to be spoken. In this approach, there is no need to define the actual sets. The symbols are defined by the system behavior….

Letters are visual glyphs that are associated with sounds. When we read, we not only understand the meaning of the words, but we instinctively know when the words rhyme. The words may evoke mental images that are then transformed back into other sounds.

The presence of reciprocal-image attractors in connected psymaps can generate a “recognition cascade”. In one psymap, a small portion of an image moving near an attractor can generate a sequence of images moving toward that attractor. Its outputs may then cause the state of other psymaps to move toward related attractors. In this way, the slightest “hint” may evoke an intricate and detailed recollection.

The above is a summary of just one of Greer’s articles.  If you look at his patent on his website (gmanif.com), you will see more of his theory.   There are interesting implications of looking at the brain as a manipulator of maps.

Sources:

1. The Computational Manifold Approach to Consciousness and Symbolic Processing in the Cerebral Cortex – Douglas S. Greer
2. Patent: METHOD OF GENERATING AN ENCODED OUTPUT SIGNAL USING A MANIFOLD ASSOCIATION PROCESSOR HAVING A PLURALITY OF PAIRS OF PROCESSING ELEMENTS TRAINED TO STORE A PLURALITY OF RECIPROCAL SIGNAL PAIRS

(both sources can be found at Professor Greer’s company website: http://gmanif.com/.

1. You can obtain his free source-code at: http://gmanif.com/contact.html#Sapphire