Visual Processing Part 2

(Note this is part 2 of a 2 part series of posts on Visual Processing)

Figure. Rabbit-duck illusion from Wikipedia that can only be interpreted as one or other at any one time.[i]
Like many cortical structures, the visual cortices are bilateral and straddle both the right and left hemispheres of the brain. However the response of the visual processing system is much more lateralized than other sensory systems such as olfactory and auditory systems.[ii] This means that it is possible to target a more specific and precise area when using visual stimuli.

An illustration of various types of stimuli and their evoked responses by different visual cortices can be seen in a study on visual processing of shapes by Joakim Vinberg. Vinberg created stereoscopic and moving images of an object (a star), a hole in a surface, edges, surfaces and random noise. He then measured the response of different visual cortices, V1, V2, V3, V3a V4 and MT, to each of the stimuli.[iii]

The study revealed that the MT area was least responsive area tested for the stereo presentation of a hole, while the V1 area was most responsive. Similarly The V3 area responds more strongly to random moving stimuli than the V3a area.

Figure 4. The visual stimuli used by Joakim Vinberg to produced differing levels of response in specific visual cortices.

Some striking example of targeting specialized functional cortical regions is demonstrated by the work of Nancy Kanwisher.[iv] Using fMRI Kanwisher has identified cortical regions that consistently respond to specific categories of visually perceived objects. The regions identified respond twice as strong to their preferred stimuli as they do to any other type of object.

For example a small area of the bottom surface of the cerebral cortex, which is activated when a face is perceived.[v] This area responds to face-like arrangements of eyes, noses and mouths and also things like cartoon images of faces. Its perception is size, location and viewpoint of the face-like stimuli.

Another identified area adjacent to the collateral sulcus in parahippocampal cortex is activated when presented with any scene type of layout. The scene may be indoor or outdoor, but seems to require spatial information being presented.

An area adjacent to the visual motion area MT is activated by images of other people’s bodies or body parts. It even responds to images of the bodies of animals and stick figures.

And finally a very small region was shown to respond to stimuli that were written words and strings of consonant letters, but did not activate for numerical digits, line drawings or letters from unfamiliar alphabet systems.

Figure. Specialized visual perception areas identified using fMRI by Nancy Kanwisher.

This ability to create a more finely tuned response, along with wide bandwidth are great advantages to VBCI. Together they may be used to create a robust, fast and noninvasive means of communicating information from computers into the human mind.

[i] “Rabbit–duck illusion,” Wikipedia,, (April, 26, 2017).

[ii] M. S. Gazzaniga, R. B. Ivry, and G. R. Mangun, Cognitive Neuroscience: The Biology of the Mind, 3rd edition (W. W. Norton, 2015) 408.

[iii] Joakim Vinberg1 and Kalanit Grill-Spector, Representation of Shapes, Edges, and Surfaces Across Multiple Cues in the Human Visual Cortex (J Neurophysiol 99: January, 2008) 1380–1393.

[iv] Kanwisher, N. (2010). Functional specificity in the human brain: A window into the functional architecture of the mind. Proceedings of the National Academy of Sciences, 107(25), 11163-11170. doi:10.1073/pnas.1005062107

[v] Kanwisher , “Functional specificity,” 11164.

Visual Processing Part 1

(Note this is part 1 of a 2 part series of posts on Visual Processing)

To explore the possibility of a Visual BCI (VBCI) we need to discuss how the human visual processing system works. In early embryotic stages, what will become the retina is indistinguishable from other parts of the proto brain. During retrogenisis a protrusion develops out of the neural tissue that eventually separates and becomes the eye.[i] The rods, cones and ganglion cells of the retina are neurons.

As light stimulates rod and cone cells they send signals to their ganglion cells, which in turn transmit to the optic nerve and on to the main corpus of the brain. The optic nerve delivers the signals to the visual cortex, which is divided into many cortical areas each specialized in processing particular types of visual information.

It is important to note that the human brain dedicates a very large amount of real estate to visual processing. Nearly one third of the surface of the cortex is used exclusively for vision. About two thirds of the volume of the entire brain are in some way used by the visual processing system.[ii] Some neurons are multimodal and made use of by other functional specialized networks within the brain. Through these there is a pathway from visual input to areas of the brain that are not directly involved in vision.

Just like in the Dynamic Core and the other processes that contribute to it, the visual processing system has reentrant interconnections between individual visual cortices that feed forward into each other, slightly modifying the functionality of each other with every cycle. The state of your perception directly affects what you perceive. When one discerns an optical illusion that can be interpreted in two different ways, once one way is perceived it becomes more likely to continue to be perceived.

[i] Ikuo K. Suzuki1 and Pierre Vanderhaeghen, Is this a brain which I see before me? Modeling human neural development with pluripotent stem cells (The Company of Biologists Ltd, Development 142, doi:10.1242/dev.120568, 2015) 3145.

[ii] Valentin Dragoi, Visual Processing: Cortical Pathways (Neuroscience Online)

Brain Computer Interfaces Part 2

(Note this is part 2 of a 2 part series of posts on BCIs)

The alternative method is the non-invasive BCI.  Typically this is a tight fitting cap enhanced with a multitude of sensors. Donning a cap is preferable to surgery in terms of risks and upgradability, however it is also burdened with disadvantages. A non-invasive BCI can only detect neural activity near the surface of the cortex close to the skull. Signals in deeper areas are more difficult to distinguish. This is also a one-way communication. Signals come out of the brain to the computer but the computer is not able to send anything back.

The greater issue with both types of BCI is the narrow bandwidth they provide. Either type of BCI has a maximum bandwidth of less than 0.5 bits per second. Compared other forms of computer input this is relatively slow. For example using a mouse or joystick the rate is around 11 bit/s and auditorily received spoken English is approximately 38 bit/s. Reading is notable in that it has a relatively high maximum rate of around 42 bit/s.[i]

The fact that reading has such a relatively high bit rate should not be surprising considering the human visual processing system has a bandwidth astronomically larger than any of these channels. The retina can process more than 1010 bit/s.[ii] After the initial retinal processing the optic nerve is capable of carrying around 3 x 106 bit/s. Since the visual processing system is able to handle one million times more information than current BCI technology it seems an intriguing potential alternative path into the mind.

[i] Kacprzyk, Janusz, and Witold Pedrycz. Springer Handbook of Computational Intelligence (Dordrecht: Springer, 2015), 760.

[ii] Anderson, Charles H. Van Essen, David C. and Olshausen, Bruno A., “Directed visual attention and the dynamic control of information flow,” Neurobiology of Attention, eds. Laurent Itti, Geraint Rees & John K. Tsotsos (Burlington, MA: Elsevier, 2005) 12.

Brain Computer Interfaces Part 1

(Note this is part 1 of a 2 part series of posts on BCIs)

Our relationship with AI is shaped by the manner and limitation of how we interact with it. Currently this typically through a verbal interaction such as a user might have with Siri on their iPhone. But if we are to augment our consciousness with AI this mode is inadequate.

A BCI is a direct communication path between the brain and an electronic device. The promise of BCI technology is that will become faster, and more intuitive then keyboards or vocal interaction. This would allow AI to be a fluid extension of ourselves. BCI technology exits today but it does not yet have this capability. The two basic categories of BCI are invasive and non-invasive.

Implantation of an invasive BCI is surgical procedure where electrodes are placed through the skull into the brain. This type of procedure is sometimes performed as treatment for neurodegenerative diseases such as Parkinson’s or epilepsy. However the implanted electrode used for these surgeries is simpler in that it just stimulates the brain rather than facilitating the transfer of information.

Science fiction films like The Matrix depict invasive BCIs as an advanced method to connect to a computer, however there are unambiguous disadvantages to invasive BCIs. Complications can arise from brain surgery such as infection and hemorrhaging. Also typical computer hardware is updated every three to four years. Having frequent brain surgery to update your hardware is impractical and imprudent. Furthermore a 2016 Pew Research Center survey found that 69% of Americans are opposed to any type of “brain chip implant.”[i] Widespread use of invasive BCIs have many hurdles to overcome. They do have one great advantage. An invasive BCI has the is capable of two-way communication. Signals can be both delivered to deep inside the brain and sent from the brain to the outside world.

[i] Cary Funk, Brian Kennedy and Elizabeth Podrebarac Sciupac, U.S. Public Wary of Biomedical Technologies to ‘Enhance’ Human Abilities (July 26, 2016)

Human Consciousness, Part 3

(Note: This part 2 of a 3 part series of posts on Human Consciousness)

By continuously running multiple neural groups in parallel the brain is capable of processing thoughts and perceptions incredibly quickly. However not every reentrant process can or should continuously contribute to consciousness and the Dynamic Core. For example we need to notice when an object enters our visual field, but it would be a debilitating distraction if we needed to be consciously continuously observing every visual detail. Some parts of the brain are continuously monitoring these inputs from our senses but by necessity there is a process, which selectively limits them from our awareness. A synaptic threshold throttles the access of these additional reentrant processes to our conscious attention. If a process’s level of activation breaches this threshold it begins to contribute to consciousness.

Figure 2. Subconscious processes operating in parallel outside the Dynamic Core.


This is necessary because the there is limit to the resource of attention within the mind. In his Global Workspace Theory as Bernard Baars suggests, “Consciousness is closely associated with the ‘limited capacity’ aspects of the brain. Limited capacity mechanisms include immediate memory, and the selectivity of attention.”[i] Each reentrant structure is specialized to gain efficiency in a specific mode of neural processing. These circuits can be running in the background without contributing to consciousness. When one of these unconscious processes exceeds a synaptic threshold it forces some other neural circuit out of the Dynamic Core and out of our attention.

This model agrees with many of our familiar experiences. A common example is being aware that you know a fact but not being able to pull the information into your conscious mind. This “tip of the tongue” phenomenon is do to the subconscious process that has the information not being able to break the synaptic threshold to be able to contribute to the Dynamic Core of consciousness. Similarly we can change contributory neural networks by changing our physical activity, for example by taking a walk. This will alter the actively contributing core processes and can allow subconscious processes to come to the fore.

When we sleep many of the usual contributing core processes are inactivated. This allows our stream of consciousness to be dominated by cycles of thoughts normally buried deep within the subconscious. These buried processes might manifest as earworm song that continually plays in your mind, or an obsessive anxiety that is judged inconsequential in the light of day. These types of processes should not be receiving any attention at all. Normally they would not. They only can when the more important processes are shut down and not competing for the threshold of attention.

[i] Bernard J. Baars, In the Theater of Consciousness: Global Workspace Theory, A Rigorous Scientific Theory of Consciousness (Journal of Consciousness Studies 4, No. 4, 1997) 294.


Human Consciousness, Part 2

(Note: This part 2 of a 3 part series of posts on Human Consciousness)

For my purposes I will view as consciousness at its most fundamental as including self-awareness that is achieved by a continuous chain of thoughts, each based largely on its predecessor. This view is compatible with the natural theory of consciousness proposed by Gerald Edelman. At the heart of Edelman’s theory is the concept of reentrant processes where individual functional areas of the brain interact in a loop repeatedly signaling each other. But this differs from a simple feedback mechanism in that with every iteration of the cycle simultaneous reentrant loops of between other parts the brain have the opportunity to contribute to the stream of consciousness.

A reresentation of processess contributing to the Dynamic Core.

Edelman proposes that consciousness is a massively parallel process with reentrant operations occurring simultaneously. He has stated that the primary interaction is between the thalamic nuclei and the basal ganglia. It must be noted that more recent work has thrown doubt on the roles of the basal ganglia and the thalamus in the process of consciousness.[i] Injuries and degenerative diseases in these areas do not always correlate to a loss of consciousness.

There are many similarly functioning reentrant loops only one is the seat of consciousness. A particular combination of them manifests as a specific state of consciousness. This is what Edelman calls the Dynamic Core.[ii] Paired areas of the brain are both massively interconnected to each others neurons and also have a pathway which is capable of contributing to the dynamic core. The neural networks of each of these lesser reentrant loops are optimized to perform specific processing functions. Some of the functionally specialized neural areas that might contribute to the Dynamic Core include sensory processing, such as for visual or auditory input; memories; language processing; and value classification. These diverse components are all combined into a unified scene in the mind.

[i] Koch, “Neural correlates,” 310.

[ii] Edelman, Naturalizing Consciousness, 5522.

Human Consciousness, Part 1

(Note: This part 1 of a 3 part series of posts on Human Consciousness)

The enigma of consciousness has been pestering scientists, philosophers and theologians for millennia. In a recent review Christof Koch of the Allen Institute for Brain Science defined consciousness as “having an experience — the subjective, phenomenal ‘what it is like’ to see an image, hear a sound, think a thought or feel an emotion.”[i] This definition allows for a very broad inclusion many edge conditions of what might be considered semi-conscious states such as where there has been great damage to the brain or in dreaming sleep. Presumably Koch does not want to dismiss those afflicted with near vegetative states if there is some activity in the areas of the brain, which correlate with consciousness. However with this over broad basic premise he has lost the essential essence of what is meant by conscious.

René Descartes wrote, “Cogito ergo sum,” that is “I think, therefore I am.” Self-awareness is intrinsic to consciousness. One may be able to think without being self aware, in that thinking is neural activity that is evaluating your perceptions. But perception is not consciousness. Descartes’ “I am” is the paramount statement. Without a sense of self we are automatons that merely react to our environment.

Koch asserts that dreams are “conscious experiences.”[ii] Sleep serves many purposes including the consolidation of skills and knowledge learned in the waking state.[iii] That there is neural activity among the neurons and pathways used by conscious processes is of no surprise. At times we may even become conscious while sleeping, but that does not equate to conscious and dreaming being one in the same.

Self-awareness is a stream of thought. It requires a chain of thoughts with each being substantially based off its immediate predecessors. Perception and awareness of one’s environment contribute to consciousness but alone these are not sufficient. Most people can recall a dream where they were some one else, or had the point view of an omniscient observer of an unfolding story. While these episodes may make use of shards of the system of neurons used by consciousness, they cannot be called a conscious experience.

[i] Koch, C., Massimini, M., Boly, M., & Tononi, G. (2016). Neural correlates of consciousness: progress and problems. Nature Reviews Neuroscience, 17(6), 307-321. doi:10.1038/nrn.2016.61, 307

[ii] Koch, “Neural correlates,” 309.

[iii] Harrison, Y. (2012). The Functions of Sleep . In Harrison The Oxford Handbook of Sleep and Sleep Disorders. Oxford: Oxford Univ. Press.

Web Site Refresh

I have finally gotten around to refreshing this web site. I have wanted to put a blog on here for some time, but deciding exactly what the topic would be has been a bit of a pickle. I wanted to include some ideas I came up with for my capstone, but also be able to post on topics beyond that. For now I am just sub-titling it “Augmenting Human Consciousness with Artificial Intelligence.” If that becomes inappropriate at some point perhaps I’ll change it. But you have to start somewhere and just get to the creative act.