Next Article in Journal
Background Subtraction for Moving Object Detection in RGBD Data: A Survey
Previous Article in Journal
Applications of Laboratory-Based Phase-Contrast Imaging Using Speckle Tracking Technique towards High Energy X-Rays
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

The Potential of Cognitive Neuroimaging: A Way Forward to the Mind-Machine Interface

Lee Kong Chian School of Medicine, Experimental Medicine Building, Nanyang Technological University, 59 Nanyang Drive, Singapore 636921, Singapore
*
Authors to whom correspondence should be addressed.
J. Imaging 2018, 4(5), 70; https://doi.org/10.3390/jimaging4050070
Submission received: 23 March 2018 / Revised: 29 April 2018 / Accepted: 10 May 2018 / Published: 14 May 2018

Abstract

:
Bridging the human mind with an external system implicitly or explicitly has been the aspiration of researchers working in the field of cognitive neuroimaging. Identifying the potential of various imaging techniques in identifying and mapping different regions of the brain in relation to their functions is the key to eliminating the difficulties in developing a mind-machine interface (MMI). Communication technology has flourished to the extent that wireless MMI applications can be designed to virtually control machines like wheelchairs, artificial limbs, etc. A cornucopia of diversified works on cognitive imaging is required to move the preliminary MMI models forward, thus engendering a technologically advanced system which can be operated directly by the brain. This article provides an overview of various aspects of cognitive neuroimaging and its potential applications in the development of a mind-machine interface.

1. Introduction

Functional neuroimaging portrays the regions of the brain that are responsible for specific cognitive functions. The basic reason for this imaging approach is to understand the relation between various activities of the brain with respect to its cohorts. Neuroimaging has helped us comprehend the fundamental concept that intricate mental functions are best expressed as a blend of different rudimentary operations. This may not be restricted to a single region of the brain but can be a result of the combined work of various regions and neuronal networks. Matching these basic operations with the locations of the brain is a significant objective of advanced cognitive imaging research. The development of several cognitive imaging modalities such as functional magnetic resonance imaging (fMRI), electroencephalography (EEG), electrocorticography (ECoG), near-infrared spectroscopy (NIRS), magnetoencephalography (MEG), and positron emission tomography (PET), as well as hybrid modalities [1] (refer to Figure 1), has paved a way forward for the mind-machine interface (MMI).
The roots of cognitive neuroimaging lie within the work of Angelo Mosso (a 19th century Italian physiologist) and his experiment monitoring of the pulsations of the brain in human subjects while conducting a defined task, during which he noticed an increase in the circulation locally in the brain [2]. Later, in 1890, Charles Roy and Charles Sherrington confirmed the relationship between blood flow and brain function [3]. Further, in 1924 the first human electroencephalography (EEG) which recorded the electrical activity of brain, opened the window to harness brain signals as carriers of information for direct brain-computer communication [4,5]. Cognitive neuroimaging, together with various other inter-disciplinary approaches of mechatronics, communication, cybernetics, etc. helped in the development of a communication pathway between brain and an external device. Since the 1970s, research and development on MMI have influenced neuroprosthetics, cyberphysical systems, neurogaming, and several other cognitive disciplines.
The purpose of functional imaging is to map the cognitive functions with the spatial locations of the brain. This can be achieved by different strategies and models like double dissociation [6], which shows that different regions of brain correspond to different cognitive functions. There could be a common region of activation owing to activities which will give information regarding the network of neurons involved in a specific behavior. The cognitive neuroimaging technique not only requires imaging technology but also a presiding knowledge of various cognitive, behavioral, and emotional processes associated with the central neuronal network, and this knowledge gap is a major barrier in development of fully-fledged MMI applications. It is an emerging field of neuroscience which demands a deluge of research and analytical experiments to be performed to understand better the brain as a processing organ and its neuronal setup. Various invasive, partially invasive, and non-invasive techniques have been widely used to clearly define various cognitive concepts and their localization in the brain.
A prospective application of cognitive neuroimaging will be its use in designing a mind-machine interface [7]. This imaging technique can potentially identify various regions of cognition which can be used to control an external system. For example, Ludovico et al. used functional magnetic resonance imaging (fMRI) to control a robotic arm based on the imagination of subjects [8]. A junction between neuron and silicon in the leech Retizus cell for stimulation was developed by Fromherz et al. [9]. This cellular level interface entails the potential for making some well-advanced complex brain-machine devices. Cognitive functions on different loci of the human central processing system (the brain), can be used to control different tasks mentally with the help of machine/motor models. This will bring about a revolution in neuroscience. This review focusses on discussing the strategies and different techniques of cognitive neuroimaging, with its potential applications in the development of the mind-machine interface.

2. Functional Neuroimaging—The Concept

Functional neuroimaging relies on a blood oxygen level-dependent (BOLD) signal [10]. The principle is that the consumption of oxygen by neurons in the regions of brain involved in an activity is high when compared to other neurons not involved in the activity. This leads to a regional increase in blood flow in the oxygen-depleted region. The increase is not to equalize the oxygen concentration but to remove the metabolic waste and by products from the neurons [11]. Cognitive processes in the brain largely follow this BOLD signal phenomenon, and it can be used to image the brain with the regions involved in activation in contrast to the regions not involved. This does not mean that a region is responsible for a specific process only. Regions of the brain selective in responding to different visual stimulus corroborate the fact that specific areas of brain are responsible for specific cognitive functions [12,13]. PET and fMRI are the two main imaging modalities used in cognitive neuroimaging. The primary reason for the change in the blood flow around the area with neuronal activity is believed to be the production of neuronal spikes. However, it was proved by comparing the response using fMRI from a pair of groups that the local field potential is also involved in the production of blood oxygen level-dependent signals [14]. The cause for the change in blood flow in the brain region involved in cognition has been debated for a long time. A good understanding on the process behind the blood flow changes is required to comprehend the concept of cognitive imaging. The change in blood flow during an activity in the regions of brain is not only attributed to the oxygen or energy depletion in that region but to the various signaling processes of the brain. Also, the change in blood flow is an indication of activity within regions of the brain but not the change in blood flow in a specific location of the brain. This is because the increase in blood flow occurs over a larger area [15]. Imaging of cognitive functions in a region of interest is possible because of the difference in the magnetic property of hemoglobin in blood flowing in the vessels supplying the brain [16,17]. fMRI depends on the above-mentioned property of hemoglobin. PET requires a contrast agent to be administered to the subject for imaging the brain. The oxyhemoglobin of blood is non-paramagnetic, whereas the deoxyhemoglobin present in the blood is paramagnetic i.e., it produces a considerable output upon interaction with an external magnetic field [18]. This gives rise to a BOLD signal contrast where the magnetic property of blood entering and leaving a region of brain differs. This gives a natural contrast in the image of the fMRI imaging modality [19].
Various techniques have been employed in imaging the cognitive activities of the brain either individually or in association with one or more modalities. PET, fMRI, PET/CT, MRI etc. are used in direct imaging of the brain activity. Also, methods like MEG, EEG, ECoG and intra-cortical neuronal recording (ICNR) have been used in association with the imaging modalities to better understand the underlying process of cognition temporally in the brain [20].

3. Mind-Machine Interface—The Concept

Some cognitive functions of the brain, if they have a specific locality of control, can be used to build a mind-machine interface which can control functionalities just by generating thoughts. The Brain-Machine Interface (BMI) can be classified as invasive, partially invasive, and non-invasive [21]. Each method has its advantage like spatial resolution, temporal resolution, response time, ease of training etc. which can be utilized for MMI applications as shown in Figure 2. The capacity of cognitive imaging methods can be extended to be applied in designing a mind-machine interface with the help of electronic and mechanical systems. A motor movement based mind-machine interface can be used to assist physically disabled patients by translating the motor signals into commands to control an external system(refer to Figure 3) [22,23,24].

4. Cognitive Neuroimaging in Action

4.1. Functional MRI

Functional magnetic resonance imaging is the most widely used imaging modality for imaging the brain, especially the functional imaging of the brain. It mostly does not require a radio tracer to produce contrast; rather it depends on natural contrast from the BOLD signal. Many clinical trials and experiments have been done over the last three decades to understand the cognitive role of the brain using functional MRI. The distribution of flow of blood inside the brain is asymmetric as the grey matter receives higher amount of blood than the white matter. Though the flow to the grey matter is like that of the heart muscle, which is the most fueled organ, brain tissues do not contain any storage for oxygen. They only rely on the continuous cerebral blood flow, which is the reason for brain fatigue when the oxygen supply to brain is hindered [25]. fMRI works on the principle of nuclear magnetic resonance, which is the base for all MRI modalities [26]. The spatial resolution of the functional MRI is better than the temporal resolution, with a range of several millimeters in voxel size, which is advantageous in cognitive neuroimaging. However, the temporal resolution of functional MRI is poor as the blood circulation changes very rapidly i.e., in seconds [27]. An event-related functional MRI study on prefrontal cortex revealed that anatomically distinct sub-regions inside the cortex are involved in sequential processing of a particular task [28]. fMRI is extensively used in the study of memory, one of the important cognitive functions of brain. The involvement of state-related and item-related processes in the retrieval of memory is supported using a fMRI study where regions of each process have been identified [29]. The involvement of basal ganglia in the temporal processing of information is studied using fMRI study [30]. fMRI has been extensively used solely in cognitive imaging, but recent developments include clubbing of two or more modalities with the fMRI modality. EEG-fMRI is one such combination which gives an incisive information about various cognitive processes processing simultaneously in the brain. EEG cannot be fused with fMRI as it is prone to interference by magnetic signals of fMRI, but both the techniques can be performed at different times with the same experiment. EEG gives a temporal information of brain activity, which is poor in regular fMRI [31]. EEG-fMRI has been proven to be richer in information than a stand-alone fMRI [32]. Another modality that is like EEG is magnetoencephalography (MEG). fMRI and MEG together are used for improving temporal resolution of the image. An experiment combining fMRI and MEG showed the existence of sub-divisions for different functions on the orbitofrontal cortex [33]. A motion sensor for finger has been used to study the recovery of patients with stroke by associating the device with the MEG/fMRI modality. It was also used to find the regions of sensory motor function activation in the brain [34]. A combination of three modalities viz, fMRI, MEG, and EEG has also been used in an experiment study of memory processing where the fMRI and MEG revealed details of the localization of activation in the brain and the time course of activation, respectively. These two information were confirmed using EEG [35]. The potential of fMRI imaging can be extended to its use in creating an advanced non-invasive mind-machine interface. Through neuro-feedback, it is possible for humans to take a voluntary control over their brain activity, which is an advantage in developing a mind-machine interface. Seung-Schik Yoo and Ferenc A. Jolesz demonstrated in their fMRI study that subjects could increase their cortical activity specific to hand-motor movement with the help of a high-resolution fMRI image provided as visual feedback [14]. Later, Seung-Schik Yoo et al., with the help of the previous study, designed a mind-machine interface in which the movement of a cursor through a two-dimensional maze on computer screen was controlled by thoughts of subjects. Only four commands for four directions were assigned and each command was attributed to a spatially distinct BOLD signal activity in the brain [36]. A further advancement by Jong-Hwan Lee et al. is the design of a robotic arm-based MMI using an fMRI signal with two degrees of freedom of movement (left and right). The subjects could operate a robotic arm by performing an imagery task which activated the somato-motor region of the brain, as shown in Figure 4 [37] and the real-time fMRI signals of the activated region were converted into commands for the robotic arm movement.

4.2. PET

Positron emission tomography (PET) is yet another efficient imaging tool which utilizes a radio tracer that emits positron upon interaction with the electron. The commonly used radio tracer for PET is fluorodeoxyglucose (FDG) in which de-oxy glucose is linked to an F-18 radioisotope. The basic concept of PET imaging depends on the process of glucose uptake by cells as an energy source. The cell utilizes the glucose in FDG which sets free the radio tracers, resulting in the production of gamma rays due to the interaction of the free positron with the electrons [38]. PET has wide applications in neuroimaging studies. It has a similar function as in fMRI where the regional blood flow changes can be measured and imaged. A PET study in subjects performing phasic tasks ranging from low to highly demanding intellectual levels supported the fact that the efficiency of processing by the brain decreases as the difficulty of task increases. This is observed in the form of reduction of accuracy and time of response [39]. Importance of the kind of mental rotation involved in a certain motor tasks has been studied using PET [40]. The existence of a working memory mechanism as posited in a hypothesis has been supported experimentally using PET image analysis in normal human subjects. Activation of brain areas of vision and motion, which is the direct result of cerebral blood flow changes, is imaged using PET [41]. Statistical maps or statistical parametric maps (SPMs) are used in place of region-of-interest method for localization of an activated area in brain involved in cognition [42]. The experimental study on regional cerebral blood flow (CBF) by Bookheimer et al. using positron emission tomography showed that there are two different mechanisms involved in the cognitive process of silent reading and reading out loud. Also, it disproved the hypothesis of visual center for word processing [43]. The participation of the frontal cortex in auditory deviant event processing shown using a PET experiment also revealed that there is a link between the fronto-temporal region and the left half of the brain hemisphere [44]. Scarce research on PET-based mind-machine interfaces has been carried out. Experiment on rats conducted by Yunqi Zhu et al. using PET imaging-based MMI implies its usefulness in MMI applications [45]. The use of radio tracers stands as a barrier in the application of this imaging in MMI systems.

4.3. Micro-Electrode Array

An array of electrodes, in the range of micrometers in diameter, is used for directly recording either single-unit neuronal spikes or local field potential [46]. The micro-electrode array technique is invasive as the electrodes are embedded deep inside the brain tissue. This electro-physiological signal obtained using this method has good resolution and accuracy. However, the discomfort due to the insertion of electrodes creates a difficulty in applying this technique for MMI interfaces. Also, there is a chance of damaging the neurons during the procedure [47]. To overcome this, local field potentials are measured at a distance sufficiently far from the site of neuronal activation, where the cumulative signal from a group of neurons is recorded. A micro-electrode array containing 100 electrodes studied by Nordhausen et al. was efficient in recording the cortical spikes, with a signal to noise ratio of 5.5:1 [48]. However, the tissue around the region where the electrode was inserted was damaged due to the perfusion. MMIs based on signals recorded by invasive electrodes will have better signal strength and resolution [49] but are difficult to use due to the damaging effect of electrodes on tissues and incisions.

4.4. EEG

Electro-encephalography is a non-invasive/invasive modality used to monitor the brain’s electrical signals associated with various cognitive processes. It consists of electrodes placed on the scalp to record the minute-intensity electrical signals passing through the neuronal network of brain [24,50]. A motor imagery-based experiment on subjects to move a virtual bar on the computer screen by only thinking about limb movement and hand movement (right) was carried out by Guger et al. in 99 subjects, with and without feedback. EEG recordings were taken in real-time to record the corresponding signals of the task given to those subjects. The results showed tremendous ease of Brain-computer Interface (BCI) operation by the subjects in a time limit of less than 30 min [51]. EEG electrodes are generally bulky and contain too many leads. A novel comb-shaped EEG electrode design (wired and wireless) with good signal quality has been developed for an imagery-based BCI system similar to the previously mentioned experiment [52]. EEG-based MMIs can be used to control an external machine or motor, which can aid patients with motor disability in handling objects just by thinking, as shown in Figure 5 [53]. The brain cortical signal associated with the hand movement has been used to control an artificial hand, by recording the EEG signal associated with the brain region during that movement task. This EEG signal is processed and translated into commands for an external robotic hand movement, which aids the recovery of post-traumatic injury patients in their rehabilitation [54]. By far, EEG is the most suitable modality of application for MMI because of its temporal resolution and relatively better ease of use compared to other modalities.

4.5. MEG

Magnetoencephalography is another non-invasive type of modality that is used to monitor and record magnetic signals from the brain [55]. These signals are produced in the form of magnetic flux because of flow of the post-synaptic electric current in the neurons of the brain. It does not contain electrodes attached to subjects, which reduces discomfort during recording. MEG has a good spatio-temporal resolution when compared to EEG and the distortion of signals is very low in MEG as compared to EEG, as the magnetic flux can travel out of the skull without damping. MEG can distinguish sensory-motor rhythm of each finger movements with a better spatial resolution [56]. A MEG-MMI with a better spatial filtering algorithms has better results in terms of accuracy and ease of training than EEG-MMI [27]. Using MEG, the trajectories of movement in space can be predicted with an exactly similar pattern to that of the original trajectory drawn by the subjects, which is a promising sign for the scope of MEG in developing an advanced stand-alone MMI [57]. Movement of hands in post-stroke patients suffering motor disability using MEG mu-rhythm through an orthosis was successfully performed by subjects during an experiment [22]. Though MEG has greater potential, it cannot be used for MMI applications conveniently as it is magnetically susceptible and works inside a magnetically shielded room.

4.6. ECoG

Electrocorticography is a partially invasive technique of recording the brain’s electrical signal by placing a patch electrode on the surface of the brain as shown in Figure 6 [58]. It is partially invasive as it requires a part of the skull to be removed to directly place the electrode on the surface of the brain. ECoG-MMI is efficient and has better spatial resolution than EEG-based MMIs [59]. The first ECoG-MMI was used to virtually control a cursor movement on a display screen. The subjects were first asked to perform three tasks, followed by imagining the performance of those three tasks. A better result compared to EEG-MMI was obtained for ECoG and the time taken to familiarize with the MMI system was predominantly low [60]. ECoG recorded during individual finger movements has a potential in aiding the development of prosthesis [61]. A direct MMI can be developed with the help of single-channel electrocorticography coupled with a wavelet-packet algorithm which can distinctly detect different motor signals corresponding to each finger movement [62].

4.7. NIRS

Another budding technique in cognitive neuro-imaging which has a capability to contribute to design an efficient mind-machine interface is near-infrared spectroscopy also known as functional near-infrared spectroscopy (f-NIRS). NIRS is a non-invasive optical imaging technique that can be used to image the hemodynamic signals of the brain as in fMRI where BOLD signals are imaged i.e., it measures the hemoglobin concentration changes in the blood which arise due to regional cerebral blood flow (CBF) during brain activity (refer to Figure 7). A standalone NIRS could not be used to image the regions of brain activity as quantification of the hemoglobin changes is not possible. Several methods to quantify the changes have been developed [64]. NIRS is used to image the brain during locomotion and regions responsible for that has been identified [65]. The possibility of a NIRS-based MMI has been confirmed by Sitaram et al. in their experimental study. Five human subjects were asked to perform hand motor imagery tasks during which the hemodynamic changes in the regions specific to those tasks in the brain was recorded using multi-channel NIRS and pattern classification of the signals was obtained using support vector machine and hidden Markov model classifiers [66]. Coyle et al. proposed a novel NIRS-based MMI model based on their experimental study. A group of subjects were asked to perform simple hand motor related tasks both physically and imaginatively. A single-channel NIRS was used to record the signal change associated with the task based on which the size of virtual ball on the screen provided as a visual feedback changed. This stands as convincing evidence for the development of NIRS-based MMI [67]. Kazuki et al. proposed a NIRS-based BCI system which can control the movement of an external robotic arm. The movement of the arm corresponded to the NIRS signal in response to the motor signal recorded in real time [68].

4.8. Other Imaging Techniques

fMRI and EEG are the most used neuroimaging modalities for MMI. Several novel imaging tools and techniques have been developed and have a greater scope in the future, as much information on the actual mechanism of cognition in the brain is still not clear. One such technique is electrical impedance tomography (EIT) in which the change in impedance in the brain is imaged using EEG leads. The basic principle behind the technique is that blood in the blood vessels of brain has a different impedance when compared to the brain cortical areas. This contrast is used to image the brain [69]. Though novel, this technique cannot be used in localization of region of interest due to its poor signal to noise ratio (SNR), but it is efficient in imaging the impedance changes during various cognitive activities. This technique has a good temporal resolution as the EEG is involved and spatial resolution can be improved in the future by addition of several models or algorithms which will support MMI research [70]. Diffusion optical tomography (DOT) is used in place of NIRS as it overcomes the problem of quantifying changes in oxy and de-oxy hemoglobin molecules. Both of these optical imaging techniques are based on Beer-Lamberts law [71]. The functional connectivity of the brain at rest was supported by a DOT study based on the fMRI model, and was confirmed by comparison with a regular functional MRI of the same study [72]. Apart from normal optical imaging, photoacoustic functional imaging has been used in a rat brain in an in vivo imaging study. This technique depends on optical and acoustic tools [73]. The optical tool is laser, which produces a thermo-elastic expansion resulting in production of acoustic waves that are detected using an ultrasound probe. Information, including location of lesion in rat brain, is studied using photoacoustic tomography (PAT). Further, functional use of this PAT is also studied by producing stimuli to the whiskers of the rat. An image subtraction of normal PAT from PAT with stimuli yields the region of cerebral activity [74,75].

5. Significance of Cognitive Imaging in Neural Disease Diagnosis and Treatment

The applications of functional cognitive imaging in diagnosing and treating neuro-cognitive disorders are profound. In particular, it has a dominant contribution towards Alzheimer’s disease (AD). Both fMRI and PET are used in the diagnosis of various neural disorders. A study of amyloid PET tracer Pittsburgh compound B (PiB) in patients with mild cognitive impairment revealed that there is a greater retention of this tracer in those patients when compared with normal and Alzheimer’s disease human subjects [76]. A novel study using PET to categorize presence of AD has been done using 18[F]AV-1451, which is the tau protein-specific radiotracer. The study revealed that categorization of a patient as tau-positive or -negative can be performed by performing AD-prone region-specific imaging rather than overall imaging of brain. This quantification of AD using PET is first of its kind and it could be a potential clinical method in early detection of AD. Arterial spin labelling MRI (ASL-MRI) is used to determine changes in regional perfusion to brain in patients with AD and mild cognitive impairment (MCI), in comparison with human subjects with normal cognition. The same study was also done using PET and SPECT, and the results showed a regional decrease in perfusion to brain with AD patients. Also, ASL-MRI has been recently used to study the postdrome effects of migraine that have been explored less until now. The study showed about 1557 changes in regional CBF with a difference in processes involved for premonitory and postdrome phases [77]. The outcomes obtained with ASL-MRI were similar to those obtained from PET and single-photon emission CT (SPECT) studies of the same subjects with the same conditions [78]. fMRI study of BOLD signal images in a group of individuals in relation to working memory in Huntington’s disease (HD), serves as a better diagnostic method to find pre-manifest Huntington’s disease i.e. to diagnose the disease well before its significant manifestation—it can be used fifteen years before a regular clinical diagnosis can be made. The absence of activation in the striatal region of the brain serves as a significant factor for pre-manifest HD [79]. Apart from neural disorders, functional imaging is also used in brain injury studies. High-resolution diffusion tensor imaging (DTI) in patients with brain injury is the only modality which can image the injured brain in acute conditions. Other modalities, including DTI, were used to find the difference in the injured part after and before healing during one study [80]. Squarcina et al. used DTI to show the micro structure differences in the white matter of subjects with bipolar disorder (BD) and schizophrenia (SCZ) through diffusion index values. This is a potential area of study as there is a conception that both BD and SCZ exhibit similar pathological symptoms and epidemiology [81]. Another study using DTI revealed that immune T-cell infiltration can be measured in patients with brain metastases, which stands as a non-invasive approach to monitor immune therapy response for such patients [82]. Non-imaging techniques are also used in analyzing subjects with neural disorders. Cognitive impairment in patients with multiple sclerosis was studied using EEG, and the fact that the impairment was caused by the de-linking or disconnection between regions in the brain was confirmed by comparison with the MRI results in a similar study [83]. An MRI study for a group of autistic patients of different ages revealed functional and anatomical information—the growth of the brain in subjects at younger ages was greater, and it decreased as the age increased in the subjects [84].

6. Limitations in Functional Imaging Based MMI

Though studies and experiments on several functional imaging based mind-machine interfaces have been performed successfully, each has its own limitations [63]. For example, EEG has very poor spatial resolution and causes discomfort due to convoluted electrode wires [85]. Also, localization of the brain activity using MEG/EEG is difficult to determine [86]. EEG is affected by the presence of other signals such as the electrophysiological signal of heart from electrocardiogram (ECG), the musculo-electrical signals of eye movement from the electro-oculogram (EOG) and the interference from the power-line [87]. In addition to that, the use of electrodes on the skin causes discomfort and irritation. On the other hand, MEG has relatively better spatial resolution and has less signal disruption as it measures the flux which is not distorted by the skull. However, the MEG system is very bulky and directly applying it for the use of human-computer interface would be difficult. MEG requires a magnetically shielded room since it is affected by the noise, to the level that even the Earth’s magnetic field could dominate the brain’s magnetic signal [88]. However, the acquired signal patterns can be used later in operating an interface system during a task by pattern recognition. The main disadvantage of the fMRI is the cost of the fMRI setup for BCI and the huge size of the equipment. In addition, the temporal resolution of fMRI is low when compared to the EEG-based MMI [89,90]. Although MRI has been shown to be a potential system for BCI application due to its higher spatial resolution, it measures the indirect signal i.e., the BOLD signal, which can be unreliable for interfacing with systems. Also, it causes discomfort to the subjects due to the high magnetic field and the requirement of immobility for hours during acquisition. In the case of ECoG, the procedure is demanding as it involves invasive processes for placing the electrode which may only be applied to patients with traumatic brain injury [91]. NIRS, on the other hand, though similar to fMRI, has a poorer signal to noise ratio than fMRI, and positioning of probes influences the signal detection [92]. NIRS also has poor performance compared to the other modalities as it measures the brain signal indirectly using optical system.

7. Future Scope

MMIs in future will depend on cognitive imaging techniques, which are developing quickly. These imaging techniques can contribute solely or in association with two or more functionalities in flourishment of MMI-based applications as shown in Figure 8. MMIs will be used in various sectors such as defense research, space research, prosthetics, and rehabilitation [93], etc. Gerson et al. demonstrated the use of EEG in rapid target image recognition using a simple pattern classifier [94]. They used the classifier to prioritize two target images from a triage of 100 images by taking the unique EEG response elicited in the cortex when the subject looks at it. This is a good example of how EEG can be combined with systems to improve human’s quality of life and its possible application in defense patrols. Novel imaging techniques like photoacoustic imaging and calcium imaging are extensively being probed and in future they will provide much information about cognition that is currently unknown to scientific society [95,96]. Wireless MMI applications will have a wider demand as they can be used to control systems from anywhere just by mental manipulation [97]. A state-of-the-art auditory signal-based MMI has been developed by Hseih et al. for homecare use of visually challenged people [98]. They recorded the evoked responses exhibited by two different audio signals for each ear using EEG. The response can take the values of Right-Right (RR), Left-Left (LL), Left-Right (LR) or Right-Left (RL) as per the discretion of the subject in listening with the left or right ear in the first and second stage. These four different combinations are then used to send appropriate signals from the BCI system to the smart phone app through Bluetooth. Based on the configuration, each pair can be assigned to control a specific appliance like a television, computer, telephone etc. A solid example of translating the response to an audio input into an application for MMI shows the potential these technologies hold in changing the modern perspective of human assist systems. Non-invasive MMIs are expected to be utilized mainly for MMI applications as they reduce discomfort [99]. However, the strength of brain signals acquired for MMI studies using invasive functionalities like ECoG and multi-electrode arrays is appreciable when compared to non-invasive methodologies. For example, signals acquired from the micro-electrode array implanted in the motor cortex of a paralyzed patient were used to simulate a flight operation [100]. The velocity signal recorded from the electrodes to actuate motor movements of the patient’s prosthetic limb was translated to act as input flight control using neuro-processors. This stands as an accurate demonstration of mind-controlled mobility systems that can revolutionize the future.

8. Conclusions

The possibilities for current cognitive neuroimaging techniques are so vast that a stand-alone non-invasive mind-machine interface could be developed in a few years with the help of fast-growing research in the functional imaging and neuroprosthetics fields. Present motor signal-based robotic arm movement is a precursor to a fully developed MMI which, on behalf of the human, would be able to perform various functions without the muscular involvement of the humans. A better use of hybrid imaging modalities can enhance the performance of the MMI. Even hybrid MMIs can be built where two or more imaging modalities can be used to detect the brain signal with greater precision and to operate the MMI system with a faster response. The ability to control and work different functionalities only by thinking is the long-term goal of researchers, and the advancement in the functional imaging field is increasing at a rapid pace, such that soon activity- or function-specific signals of brain can be distinguished with 100% precision from other signals in the brain. This will be an important advance for successfully developing a multi-functional mind-machine interface.

Author Contributions

G.P. and S.M. contributed in designing the layout. G.P. and A.M.N. collected the literature and prepared the manuscript. G.P. contributed in preparing and incorporating the figures. G.P. and S.M. contributed to editing and formatting the manuscript. S.M., P.P. and B.G. contributed to reviewing, revising and guiding in preparation of manuscript title, layout, and content. All authors read and approved the final manuscript.

Acknowledgments

B.G. and P.P. acknowledges Lee Kong Chian School of Medicine, Nanyang Technological University MOE Start-Up Grant.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Padmanabhan, P.; Nedumaran, A.M.; Mishra, S.; Pandarinathan, G.; Archunan, G.; Gulyas, B. The Advents of Hybrid Imaging Modalities: A New Era in Neuroimaging Applications. Adv. Biosyst. 2017. [Google Scholar] [CrossRef]
  2. Sandrone, S.; Bacigaluppi, M.; Galloni, M.R.; Cappa, S.F.; Moro, A.; Catani, M.; Filippi, M.; Monti, M.M.; Perani, D.; Martino, G. Weighing brain activity with the balance: Angelo Mosso’s original manuscripts come to light. Brain 2014, 137, 621–633. [Google Scholar] [CrossRef]
  3. Friedland, R.P.; Iadecola, C. Roy and Sherrington (1890): A Centennial Re-examination of “On the Regulation of the Blood-Supply of the Brain”. Neurology 1991, 41, 10–14. [Google Scholar] [CrossRef]
  4. Haas, L.F. Hans Berger (1873–1941), Richard Caton (1842–1926), and electroencephalography. J. Neurol. Neurosurg. Psychiatry 2003, 74, 9. [Google Scholar] [CrossRef]
  5. Vidal, J.J. Toward Direct Brain-Computer Communication. Annu. Rev. Biophys. Bioeng. 1973, 2, 157–180. [Google Scholar] [CrossRef]
  6. MacDonald, A.W.; Cohen, J.D.; Stenger, V.A.; Carter, C.S. Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science 2000, 288, 1835–1838. [Google Scholar]
  7. Min, B.K.; Marzelli, M.J.; Yoo, S.S. Neuroimaging-based approaches in the brain-computer interface. Trends Biotechnol. 2010, 28, 552–560. [Google Scholar] [CrossRef]
  8. Minati, L.; Nigri, A.; Rosazza, C.; Bruzzone, M.G. Thoughts turned into high-level commands: Proof-of-concept study of a vision-guided robot arm driven by functional MRI (fMRI) signals. Med. Eng. Phys. 2012, 34, 650–658. [Google Scholar] [CrossRef]
  9. Schatzthauer, R.; Fromherz, P. Neuron-silicon junction with voltage-gated ionic currents. Eur. J. Neurosci. 1998, 10, 1956–1962. [Google Scholar] [CrossRef]
  10. Blackstone, J.J.; Long, J.C. Shifting the debate on geoengineering. Science 2010, 329, 1466–1468. [Google Scholar]
  11. Gazzaniga, M.S.; Ivry, R.B.; Mangun, G.R. Cognitive Neuroscience: The Biology of the Mind. Q. Rev. Biol. 2009, 84, 196–197. [Google Scholar]
  12. Downing, P.E.; Chan, A.W.; Peelen, M.V.; Dodds, C.M. Domain Specificity in Visual Cortex. Cereb. Cortex 2006, 1453–1461. [Google Scholar] [CrossRef]
  13. Aguirre, G.K.; Singh, R.; D’Esposito, M. Stimulus inversion and the responses of face and object-sensitive cortical areas. Neuroreport 1999, 10, 189–194. [Google Scholar] [CrossRef]
  14. Mukamel, R. Coupling Between Neuronal Firing, Field Potentials, and fMRI in Human Auditory Cortex. Science 2005, 309, 951–954. [Google Scholar] [CrossRef]
  15. Attwell, D.; Iadecola, C. The neural basis of functional brain imaging signals. Trends Neurosci. 2002, 25, 621–625. [Google Scholar] [CrossRef]
  16. Pauling, L.; Coryell, C.D. The Magnetic Properties and Structure of Hemoglobin, Oxyhemoglobin and Carbonmonoxyhemoglobin. Proc. Natl. Acad. Sci. USA 1936, 22, 210–216. [Google Scholar] [CrossRef]
  17. Pauling, L. Magnetic properties and structure of oxyhemoglobin. Proc. Natl. Acad. Sci. USA 1977, 74, 2612–2613. [Google Scholar]
  18. Ogawa, S.; Lee, T. Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc. Natl. Acad. Sci. USA 1990, 87, 9868–9872. [Google Scholar] [CrossRef]
  19. Raichle, M.E.; Mintun, M.A. Brain Work and Brain Imaging. Annu. Rev. Neurosci. 2006, 29, 449–476. [Google Scholar] [CrossRef]
  20. Nicolas-Alonso, L.F.; Gomez-Gil, J. Brain computer interfaces, a review. Sensors 2012, 12, 1211–1279. [Google Scholar] [CrossRef]
  21. Lebedev, M.A.; Nicolelis, M.A.L. Brain–machine interfaces: Past, present and future. Trends Neurosci. 2006, 29, 536–546. [Google Scholar] [CrossRef]
  22. Buch, E.; Weber, C.; Cohen, L.G.; Braun, C.; Dimyan, M.A.; Ard, T.; Mellinger, J.; Caria, A.; Soekadar, S.; Fourkas, A.; et al. Think to Move: A Neuromagnetic Brain-Computer Interface (BCI) System for Chronic Stroke. Stroke 2008, 39, 910–917. [Google Scholar] [CrossRef]
  23. Sarasola-Sanz, A.; López-Larraz, E.; Irastorza-Landa, N.; Klein, J.; Valencia, D.; Belloso, A.; Morin, F.O.; Spüler, M.; Birbaumer, N.; Ramos-Murguialday, A. An EEG-Based Brain-Machine Interface to Control a 7-Degrees of Freedom Exoskeleton for Stroke Rehabilitation. In Converging Clinical and Engineering Research on Neurorehabilitation II, Proceedings of the 3rd International Conference on NeuroRehabilitation (ICNR2016), Segovia, Spain, 18–21 October 2016; Ibáñez, J., González-Vargas, J., Azorín, J.M., Akay, M., Pons, J.L., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 1127–1131. ISBN 978-3-319-46669-9. [Google Scholar]
  24. Ramos-Murguialday, A.; Schürholz, M.; Caggiano, V.; Wildgruber, M.; Caria, A.; Hammer, E.M.; Halder, S.; Birbaumer, N. Proprioceptive Feedback and Brain Computer Interface (BCI) Based Neuroprostheses. PLoS ONE 2012, 7, e47048. [Google Scholar] [CrossRef] [Green Version]
  25. Buxton, R.B. Introduction to Functional Magnetic Resonance Imaging: Principles and Techniques. Energy 2002, 24, 523. [Google Scholar] [CrossRef]
  26. Leksell, L.; Leksell, D.; Schwebel, J. Stereotaxis and nuclear magnetic resonance. J. Neurol. Neurosurg. Psychiatry 1985, 48, 14–18. [Google Scholar]
  27. Fomby, P.; Cherlin, A.J. Family Instability and Child Well-Being. NIH Public Access. 2011, 72, 181–204. [Google Scholar]
  28. Wagner, A.D.; Maril, A.; Bjork, R.A.; Schacter, D.L. Prefrontal contributions to executive control: FMRI evidence for functional distinctions within lateral Prefrontal cortex. Neuroimage 2001, 14, 1337–1347. [Google Scholar] [CrossRef]
  29. Donaldson, D.I.; Petersen, S.E.; Ollinger, J.M.; Buckner, R.L. Dissociating state and item components of recognition memory using fMRI. Neuroimage 2001, 13, 129–142. [Google Scholar] [CrossRef]
  30. Nenadic, I.; Gaser, C.; Volz, H.-P.; Rammsayer, T.; Häger, F.; Sauer, H. Processing of temporal information and the basal ganglia: New evidence from fMRI. Exp. Brain Res. 2003, 148, 238–246. [Google Scholar] [CrossRef]
  31. Ahmad, R.F.; Malik, A.S. Simultaneous EEG-fMRI Data Acquisition during Cognitive Task. In Proceedings of the 2014 5th International Conference on Intelligent and Advanced Systems (ICIAS), Kuala Lumpur, Malaysia, 3–5 June 2014. [Google Scholar]
  32. Ahmad, R.F.; Malik, A.S.; Member, S.; Kamel, N.; Member, S.; Reza, F. Object Categories Specific Brain Activity Classification with Simultaneous EEG-fMRI. In Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 25–29 August 2015; pp. 1825–1828. [Google Scholar]
  33. Northoff, G.; Richter, A.; Gessner, M.; Schlagenhauf, F.; Fell, J.; Baumgart, F.; Kaulisch, T.; Kötter, R.; Stephan, K.E.; Leschinger, A.; et al. Functional dissociation between medial and lateral prefrontal cortical spatiotemporal activation in negative and positive emotions: A combined fMRI/MEG study. Cereb. Cortex 2000, 10, 93–107. [Google Scholar] [CrossRef]
  34. Menon, C. The Use of an Meg/fMri-compatible Finger Motion sensor in Detecting Different Finger actions. Front. Bioeng. Biotechnol. 2016, 3, 1–9. [Google Scholar] [CrossRef]
  35. McDonald, C.R.; Thesen, T.; Carlson, C.; Blumberg, M.; Girard, H.M.; Trongnetrpunya, A.; Sherfey, J.S.; Devinsky, O.; Kuzniecky, R.; Dolye, W.K.; et al. Multimodal imaging of repetition priming: Using fMRI, MEG, and intracranial EEG to reveal spatiotemporal profiles of word processing. Neuroimage 2010, 53, 707–717. [Google Scholar] [CrossRef]
  36. Lee, J.-H.; Ryu, J.; Jolesz, F.A.; Cho, Z.-H.; Yoo, S.-S. Brain–machine interface via real-time fMRI: Preliminary study on thought-controlled robotic arm. Neurosci. Lett. 2009, 450, 1–6. [Google Scholar] [CrossRef]
  37. Marchesotti, S.; Martuzzi, R.; Schurger, A.; Blefari, M.L.; del Millán, J.R.; Bleuler, H.; Blanke, O. Cortical and subcortical mechanisms of brain-machine interfaces. Hum. Brain Mapp. 2017, 38, 2971–2989. [Google Scholar] [CrossRef]
  38. Tomography, P.E. Positron Emission Tomography; Springer: Berlin, Germany, 2002; Volume 27, ISBN 1852337982. [Google Scholar]
  39. Jonides, J.; Schumacher, E.H.; Smith, E.E.; Lauber, E.J.; Awh, E.; Minoshima, S.; Koeppe, R.A. Verbal Working Memory Load Affects Regional Brain Activation as Measured by PET. J. Cognit. Neurosci. 1969, 462–475. [Google Scholar]
  40. Kosslyn, S.M.; Digirolamo, G.J.; Thompson, W.L. Mental rotation of objects versus hands: Neural mechanisms revealed by positron emission tomography. Psychophysiology 1998, 35, 151–161. [Google Scholar]
  41. Ross, M.; Errandonea, D.; Boehler, R. Melting of transition metals at high pressure and the influence of liquid frustration: The early metals Ta and Mo. Phys. Rev. B 2007, 76, 184118. [Google Scholar] [CrossRef]
  42. Friston, K.J.; Frith, C.D.; Liddle, P.F.; Frackowiak, R.S.J. Comparing functional (PET) images: The assessment of significant change. J. Cereb. Blood Flow Metab. 1991, 11, 690–699. [Google Scholar] [CrossRef]
  43. Blaxton, T.A.; Zeffiro, T.A.; Gabrieli, J.D.; Bookheimer, S.Y.; Carrillo, M.C.; Theodore, W.H.; Disterhoft, J.F. Functional mapping of human learning: A positron emission tomography activation study of eyeblink conditioning. J. Neurosci. 1996, 16, 4032–4040. [Google Scholar]
  44. Müller, B.W.; Jüptner, M.; Jentzen, W.; Müller, S.P. Cortical Activation to Auditory Mismatch Elicited by Frequency Deviant and Complex Novel Sounds: A PET Study. Neuroimage 2002, 17, 231–239. [Google Scholar] [CrossRef]
  45. Zhu, Y.; Xu, K.; Xu, C.; Zhang, J.; Ji, J.; Zheng, X.; Zhang, H.; Tian, M. PET Mapping for Brain-Computer-Interface-Based Stimulation in a Rat Model with Intracranial Electrode Implantation in the Ventro-posterior Medial Thalamus. J. Nucl. Med. 2016, 57, 1141–1145. [Google Scholar] [CrossRef]
  46. Moran, D. Evolution of brain–computer interface: Action potentials, local field potentials and electrocorticograms. Curr. Opin. Neurobiol. 2010, 20, 741–745. [Google Scholar] [CrossRef]
  47. Polikov, V.S.; Tresco, P.A.; Reichert, W.M. Response of brain tissue to chronically implanted neural electrodes. J. Neurosci. Methods 2005, 148, 1–18. [Google Scholar]
  48. Nordhausen, C.T.; Maynard, E.M.; Normann, R.A. Single unit recording capabilities of a 100 microelectrode array. Brain Res. 1996, 726, 129–140. [Google Scholar]
  49. Maynard, E.M.; Nordhausen, C.T.; Normann, R.A. The Utah intracortical Electrode Array: A recording structure for potential brain-computer interfaces. Electroencephalogr. Clin. Neurophysiol. 1997, 102, 228–239. [Google Scholar]
  50. Michel, C.M.; Murray, M.M.; Lantz, G.; Gonzalez, S.; Spinelli, L.; Grave de Peralta, R. EEG source imaging. Clin. Neurophysiol. 2004, 115, 2195–2222. [Google Scholar] [CrossRef]
  51. Guger, C.; Edlinger, G.; Harkam, W.; Niedermayer, I.; Pfurtscheller, G. How many people are able to operate an EEG-based brain-computer interface (BCI)? IEEE Trans. Neural Syst. Rehabil. Eng. 2003, 11, 145–147. [Google Scholar] [CrossRef]
  52. Lin, B.-S.; Pan, J.-S.; Chu, T.-Y.; Lin, B.-S. Development of a Wearable Motor-Imagery-Based Brain–Computer Interface. J. Med. Syst. 2016, 40, 71. [Google Scholar] [CrossRef]
  53. Wolpaw, J.R.; McFarland, D.J. Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl. Acad. Sci. USA 2004, 101, 17849–17854. [Google Scholar] [CrossRef]
  54. Fok, S.; Schwartz, R.; Wronkiewicz, M.; Holmes, C.; Zhang, J.; Somers, T.; Bundy, D.; Leuthardt, E. An EEG-based brain computer interface for rehabilitation and restoration of hand control following stroke using ipsilateral cortical physiology. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2011, 6277–6280. [Google Scholar] [CrossRef]
  55. Cohen, D. Magnetoencephalography: Detection of the Brain’s Electrical Activity with a Superconducting Magnetometer. Science 1972, 175, 664–666. [Google Scholar] [CrossRef]
  56. Braun, C.; Schweizer, R.; Elbert, T.; Birbaumer, N.; Taub, E. Differential activation in somatosensory cortex for different discrimination tasks. J. Neurosci. 2000, 20, 446–450. [Google Scholar] [CrossRef]
  57. Georgopoulos, A.P.; Langheim, F.J.P.; Leuthold, A.C.; Merkle, A.N. Magnetoencephalographic signals predict movement trajectory in space. Exp. Brain Res. 2005, 167, 132–135. [Google Scholar] [CrossRef]
  58. Schalk, G.; Leuthardt, E.C. Brain-Computer Interfaces Using Electrocorticographic Signals. IEEE Rev. Biomed. Eng. 2011, 4, 140–154. [Google Scholar] [CrossRef]
  59. Freeman, W.J.; Holmes, M.D.; Burke, B.C.; Vanhatalo, S. Spatial spectra of scalp EEG and EMG from awake humans. Clin. Neurophysiol. 2003, 114, 1053–1068. [Google Scholar] [CrossRef]
  60. Leuthardt, E.C.; Schalk, G.; Wolpaw, J.R.; Ojemann, J.G.; Moran, D.W. A brain-computer interface using electrocorticographic signals in humans. J. Neural Eng. 2004, 1, 63–71. [Google Scholar] [CrossRef]
  61. Shenoy, P.; Miller, K.J.; Ojemann, J.G.; Rao, R.P.N. Finger Movement Classification for an Electrocorticographic BCI. In Proceedings of the 2007 3rd International IEEE/EMBS Conference on Neural Engineering, Kohala Coast, HI, USA, 2–5 May 2007; IEEE: Piscataway, NJ, USA, 2007; pp. 192–195. [Google Scholar]
  62. Graimann, B.; Huggins, J.E.; Levine, S.P.; Pfurtscheller, G. Toward a direct brain interface based on human subdural recordings and wavelet-packet analysis. IEEE Trans. Biomed. Eng. 2004, 51, 954–962. [Google Scholar] [CrossRef]
  63. Abdulkader, S.N.; Atia, A.; Mostafa, M.-S.M. Brain computer interfacing: Applications and challenges. Egypt Inform. J. 2015, 16, 213–230. [Google Scholar] [CrossRef]
  64. Hoshi, Y. Functional near-infrared optical imaging: Utility and limitations in human brain mapping. Psychophysiology 2003, 40, 511–520. [Google Scholar]
  65. Miyai, I.; Tanabe, H.C.; Sase, I.; Eda, H.; Oda, I.; Konishi, I.; Tsunazawa, Y.; Suzuki, T.; Yanagida, T.; Kubota, K. Cortical mapping of gait in humans: A near-infrared spectroscopic topography study. Neuroimage 2001, 14, 1186–1192. [Google Scholar] [CrossRef]
  66. Sitaram, R.; Zhang, H.; Guan, C.; Thulasidas, M.; Hoshi, Y.; Ishikawa, A.; Shimizu, K.; Birbaumer, N. Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. Neuroimage 2007, 34, 1416–1427. [Google Scholar] [CrossRef]
  67. Coyle, S.; Ward, T.; Markham, C.; McDarby, G. On the suitability of near-infrared (NIR) systems for next-generation brain-computer interfaces. Physiol. Meas. 2004, 25, 815–822. [Google Scholar] [CrossRef]
  68. Yanagisawa, K.; Asaka, K.; Sawai, H.; Tsunashima, H.; Nagaoka, T.; Tsujii, T.; Sakatani, K. Brain-computer interface using near-infrared spectroscopy for rehabilitation. In Proceedings of the 2010 International Conference on Control Automation and Systems (ICCAS), Gyeonggi-do, Korea, 27–30 October 2010; pp. 339–354. [Google Scholar]
  69. Cheney, M.; Isaacson, D.; Newell, J.C. Electrical Impedance Tomography. SIAM Rev. 1999, 41, 85–101. [Google Scholar] [CrossRef]
  70. Tidswell, T.; Gibson, A.; Bayford, R.H.; Holder, D.S. Three-dimensional electrical impedance tomography of human brain activity. Neuroimage 2001, 13, 283–294. [Google Scholar] [CrossRef]
  71. Boas, D.A.; Gaudette, T.; Strangman, G.; Cheng, X.; Marota, J.J.; Mandeville, J.B. The accuracy of near infrared spectroscopy and imaging during focal changes in cerebral hemodynamics. Neuroimage 2001, 13, 76–90. [Google Scholar] [CrossRef]
  72. White, B.R.; Snyder, A.Z.; Cohen, A.L.; Petersen, S.E.; Raichle, M.E.; Schlaggar, B.L.; Culver, J.P. Resting-state functional connectivity in the human brain revealed with diffuse optical tomography. Neuroimage 2009, 47, 148–156. [Google Scholar] [CrossRef]
  73. Wang, L.V.; Hu, S. Photoacoustic Tomography: In Vivo Imaging from Organelles to Organs. Science 2012, 335, 1458–1462. [Google Scholar] [CrossRef]
  74. Wang, X.; Pang, Y.; Ku, G.; Xie, X.; Stoica, G.; Wang, L.V. Noninvasive laser-induced photoacoustic tomography for structural and functional in vivo imaging of the brain. Nat. Biotechnol. 2003, 21, 803–806. [Google Scholar]
  75. Yang, S.; Xing, D.; Zhou, Q.; Xiang, L.; Lao, Y. Functional imaging of cerebrovascular activities in small animals using high-resolution photoacoustic tomography. Med. Phys. 2007, 34, 3294–3301. [Google Scholar] [CrossRef]
  76. Forsberg, A.; Engler, H.; Almkvist, O.; Blomquist, G.; Wall, A.; Ringheim, A.; Bengt, L. PET imaging of amyloid deposition in patients with mild cognitive impairment. Neurobiol. Aging 2008, 29, 1456–1465. [Google Scholar] [CrossRef]
  77. Bose, P.; Karsan, N.; Zelaya, F.; Goadsby, P. 1557 Alterations in cerebral blood flow during the postdrome phase of a migraine attack captured with arterial spin labelled (asl) mri. J. Neurol. Neurosurg. Psychiatry 2017, 88. [Google Scholar] [CrossRef]
  78. Johnson, N.A.; Jahng, G.; Weiner, M.W.; Miller, B.L.; Chui, H.C.; Jagust, W.J.; Gorno-tempini, M.L.; Schuff, N. Radiology Pattern of Cerebral Hypoperfusion in Alzheimer Disease and Mild Cognitive Impairment Measured with Arterial Spin-labeling MR Imaging: Initial Experience. Radiology 2005, 234, 851–859. [Google Scholar]
  79. Georgiou-karistianis, N.; Stout, J.C.; Domı, J.F.; Carron, S.P.; Ando, A.; Churchyard, A.; Chua, P.; Dymowski, A.R.; Poudel, G.; Egan, G.F. Functional Magnetic Resonance Imaging of Working Memory in Huntington’s Disease: Cross-Sectional Data From the IMAGE-HD Study. Hum. Brain Mapp. 2014, 1864, 1847–1864. [Google Scholar] [CrossRef]
  80. Narayana, P.A.; Yu, X.; Hasan, K.M.; Wilde, E.A.; Levin, H.S.; Hunter, J.V.; Miller, E.R.; Kumar, V.; Patel, S.; Robertson, C.S.; et al. Multi-modal MRI of mild traumatic brain injury. NeuroImage Clin. 2015, 7, 87–97. [Google Scholar] [CrossRef]
  81. Squarcina, L.; Bellani, M.; Rossetti, M.G.; Perlini, C.; Delvecchio, G.; Dusi, N.; Barillari, M.; Ruggeri, M.; Altamura, C.A.; Bertoldo, A.; et al. Similar white matter changes in schizophrenia and bipolar disorder: A tract-based spatial statistics study. PLoS ONE 2017, 12, e0178089. [Google Scholar]
  82. Zakaria, R.; Platt-Higgins, A.; Rathi, N.; Radon, M.; Das, S.; Das, K.; Bhojak, M.; Brodbelt, A.; Chavredakis, E.; Jenkinson, M.D.; et al. T-Cell Densities in Brain Metastases Are Associated with Patient Survival Times and Diffusion Tensor MRI Changes. Cancer Res. 2018, 78, 610–616. [Google Scholar] [CrossRef]
  83. Van Schependom, J.; Mieke, D.; Cleynhens, K.; Marie, B.D.; De Keyser, J.; Nagels, G. Detection of cognitive impairment in MS based on an EEG P300 paradigm. In Proceedings of the 2013 International Workshop on Pattern Recognition in Neuroimaging (PRNI), Philadelphia, PA, USA, 22–24 June 2013; pp. 3–7. [Google Scholar] [CrossRef]
  84. Courchesne, E.; Karns, C.; Davis, H.; Ziccardi, R.; Carper, R.; Tigue, Z.; Chisum, H.J.; Moses, P.; Pierce, K.; Lord, C.; et al. Unusual brain growth patterns in early life in patients with autistic disorder: An MRI study. Neurology 2001, 76, 2111. [Google Scholar] [CrossRef]
  85. Weiskopf, N.; Sitaram, R.; Josephs, O.; Veit, R.; Scharnowski, F.; Goebel, R.; Birbaumer, N.; Deichmann, R.; Mathiak, K. Real-time functional magnetic resonance imaging: Methods and applications. Magn. Reson. Imaging 2007, 25, 989–1003. [Google Scholar] [CrossRef]
  86. Liu, A.K.; Belliveau, J.W.; Dale, A.M.; Raichle, M.E. Spatiotemporal imaging of human brain activity using functional MRI constrained magnetoencephalography data: Monte Carlo simulations. Proc. Natl. Acad. Sci. USA 1998, 95, 8945–8950. [Google Scholar] [CrossRef]
  87. Rajya Lakshmi, M.; Prasad, T.V.; Chandra, P.V. Survey on EEG Signal Processing Methods. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 2014, 4, 84–91. [Google Scholar]
  88. Taulu, S.; Hari, R. Removal of magnetoencephalographic artifacts with temporal signal-space separation: Demonstration with single-trial auditory-evoked responses. Hum. Brain Mapp. 2009, 30, 1524–1534. [Google Scholar] [CrossRef]
  89. Sitaram, R.; Caria, A.; Veit, R.; Gaber, T.; Rota, G.; Kuebler, A.; Birbaumer, N. FMRI brain-computer interface: A tool for neuroscientific research and treatment. Comput. Intell. Neurosci. 2007, 2007, 25487. [Google Scholar] [CrossRef]
  90. Kim, S.; Richter, W.; Ugurbil, K. Limitations of Temporal Resolution in Functional MRI. Magn. Reson. Med. 1997, 34, 631–636. [Google Scholar]
  91. Wilson, J.A.; Felton, E.A.; Garell, P.C.; Schalk, G.; Williams, J.C. ECoG factors underlying multimodal control of a brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 2006, 14, 246–250. [Google Scholar] [CrossRef]
  92. Cui, X.; Bray, S.; Bryant, D.M.; Glover, G.H.; Reiss, A.L. A quantitative comparison of NIRS and fMRI across multiple cognitive tasks. Neuroimage 2011, 54, 2808–2821. [Google Scholar] [CrossRef]
  93. Daly, J.J.; Wolpaw, J.R. Brain–computer interfaces in neurological rehabilitation. Lancet Neurol. 2008, 7, 1032–1043. [Google Scholar] [CrossRef]
  94. Gerson, A.D.; Parra, L.C.; Sajda, P. Cortically Coupled Computer Vision for Rapid Image Search. IEEE Trans. Neural Syst. Rehabil. Eng. 2006, 14, 174–179. [Google Scholar] [CrossRef]
  95. Smetters, D.; Majewska, A.; Yuste, R. Detecting action potentials in neuronal populations with calcium imaging. Methods 1999, 18, 215–221. [Google Scholar] [CrossRef]
  96. Hu, S. Listening to the Brain with Photoacoustics. IEEE J. Sel. Top. Quantum Electron. 2016, 22, 117–126. [Google Scholar] [CrossRef]
  97. Lin, C.-T.; Ko, L.-W.; Chang, C.-J.; Wang, Y.-T.; Chung, C.-H.; Yang, F.-S.; Duann, J.-R.; Jung, T.-P.; Chiou, J.-C. Wearable and Wireless Brain-Computer Interface and Its Applications. In Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience. FAC 2009; Springer: Berlin, Germany, 2009; pp. 741–748. [Google Scholar]
  98. Hsieh, K.L.; Sun, K.T.; Yeh, J.K.; Pan, Y.U. Home care by auditory Brain Computer Interface for the blind with severe physical disabilities. In Proceedings of the 2017 International Conference on Applied System Innovation (ICASI), Sapporo, Japan, 13–17 May 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 527–530. [Google Scholar]
  99. Cincotti, F.; Mattia, D.; Aloise, F.; Bufalari, S.; Schalk, G.; Oriolo, G.; Cherubini, A.; Marciani, M.G.; Babiloni, F. Non-invasive brain–computer interface system: Towards its application as assistive technology. Brain Res. Bull. 2008, 75, 796–803. [Google Scholar] [CrossRef]
  100. Kryger, M.; Wester, B.; Pohlmeyer, E.A.; Rich, M.; John, B.; Beaty, J.; McLoughlin, M.; Boninger, M.; Tyler-Kabara, E.C. Flight simulation using a Brain-Computer Interface: A pilot, pilot study. Exp. Neurol. 2017, 287, 473–478. [Google Scholar] [CrossRef]
Figure 1. Different imaging modalities contributing towards the development of the mind-machine interface in a clockwise direction from the top right corner: near-infrared spectroscopy (NIRS), functional magnetic resonance imaging (fMRI), positron emission tomography (PET), magnetoencephalography (MEG), electroencephalography (EEG), and electroencephalography (ECoG).
Figure 1. Different imaging modalities contributing towards the development of the mind-machine interface in a clockwise direction from the top right corner: near-infrared spectroscopy (NIRS), functional magnetic resonance imaging (fMRI), positron emission tomography (PET), magnetoencephalography (MEG), electroencephalography (EEG), and electroencephalography (ECoG).
Jimaging 04 00070 g001
Figure 2. Comparison of different imaging modalities. ICNR: intra-cortical neuronal recording.
Figure 2. Comparison of different imaging modalities. ICNR: intra-cortical neuronal recording.
Jimaging 04 00070 g002
Figure 3. Block diagram depicting the basic concept of utilizing the brain signal to control a particular device connected to a part of the body which is damaged or partially functional. The signal from the brain is converted into meaningful commands with the help of a system that moves the artificial robotic arm with reference to the type of brain signal.
Figure 3. Block diagram depicting the basic concept of utilizing the brain signal to control a particular device connected to a part of the body which is damaged or partially functional. The signal from the brain is converted into meaningful commands with the help of a system that moves the artificial robotic arm with reference to the type of brain signal.
Jimaging 04 00070 g003
Figure 4. Brain activity associated to the sense of agency for BMI control. Group results showing regions with stronger activations (p < 0.05 False discovery rate (FDR), masked with any effect contrast at p < 0.001) when subjects reported feeling in control as opposed to not feeling in control. The activity over the basal ganglia region was associated with positive blood oxygen level-dependent (BOLD) response, unlike that over the Anterior cingulate cortex (ACC) and the left superior frontal gyrus, which showed a negative response to the stimulation. Adapted with permission from [37].
Figure 4. Brain activity associated to the sense of agency for BMI control. Group results showing regions with stronger activations (p < 0.05 False discovery rate (FDR), masked with any effect contrast at p < 0.001) when subjects reported feeling in control as opposed to not feeling in control. The activity over the basal ganglia region was associated with positive blood oxygen level-dependent (BOLD) response, unlike that over the Anterior cingulate cortex (ACC) and the left superior frontal gyrus, which showed a negative response to the stimulation. Adapted with permission from [37].
Jimaging 04 00070 g004
Figure 5. Experimental design of the EEG-MMI. (A) Timing of an experimental trial. Each trial starts with a baseline of 3 s followed by an auditory instruction period. Then, 2 s after the instruction a “Start” cue is presented and 5 s later there is an “End” cue. (B) BCI participant wearing the 128 EEG channel cap seated with the hand attached to the orthosis showing the components used during all tasks. (C) A close look at the orthosis with the fingers attached. (D) Schematic of the 128 channels. The 61 channels used during the experiments are shaded in grey. Adapted with permission from [24]. MMI: mind-machine interface.
Figure 5. Experimental design of the EEG-MMI. (A) Timing of an experimental trial. Each trial starts with a baseline of 3 s followed by an auditory instruction period. Then, 2 s after the instruction a “Start” cue is presented and 5 s later there is an “End” cue. (B) BCI participant wearing the 128 EEG channel cap seated with the hand attached to the orthosis showing the components used during all tasks. (C) A close look at the orthosis with the fingers attached. (D) Schematic of the 128 channels. The 61 channels used during the experiments are shaded in grey. Adapted with permission from [24]. MMI: mind-machine interface.
Jimaging 04 00070 g005
Figure 6. ECoG invasive patch electrode placement. Adapted with permission from [63].
Figure 6. ECoG invasive patch electrode placement. Adapted with permission from [63].
Jimaging 04 00070 g006
Figure 7. The NIRS signal is processed to move the cursor point (red dot) on the monitor through the maze using the visual feedback that modulates the thoughts of the subject with respect to the position of the cursor.
Figure 7. The NIRS signal is processed to move the cursor point (red dot) on the monitor through the maze using the visual feedback that modulates the thoughts of the subject with respect to the position of the cursor.
Jimaging 04 00070 g007
Figure 8. Prospective applications of a mind-machine interface.
Figure 8. Prospective applications of a mind-machine interface.
Jimaging 04 00070 g008

Share and Cite

MDPI and ACS Style

Pandarinathan, G.; Mishra, S.; Nedumaran, A.M.; Padmanabhan, P.; Gulyás, B. The Potential of Cognitive Neuroimaging: A Way Forward to the Mind-Machine Interface. J. Imaging 2018, 4, 70. https://doi.org/10.3390/jimaging4050070

AMA Style

Pandarinathan G, Mishra S, Nedumaran AM, Padmanabhan P, Gulyás B. The Potential of Cognitive Neuroimaging: A Way Forward to the Mind-Machine Interface. Journal of Imaging. 2018; 4(5):70. https://doi.org/10.3390/jimaging4050070

Chicago/Turabian Style

Pandarinathan, Ganesh, Sachin Mishra, Anu Maashaa Nedumaran, Parasuraman Padmanabhan, and Balázs Gulyás. 2018. "The Potential of Cognitive Neuroimaging: A Way Forward to the Mind-Machine Interface" Journal of Imaging 4, no. 5: 70. https://doi.org/10.3390/jimaging4050070

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop