Using multiple behavioral and neuroimaging approaches, our laboratory seeks to explore human brain network organization within a translational framework. In doing so, we believe that a systems approach is crucial in filling the gaps in our understanding of the human brain with all its complexity arising from the need for multiple levels of analysis – from neurons to cognition and from cognition to behavior.
An overarching theme that is guiding our work is to arrive at a set of universal principles governing all levels of analysis – the neural, cognitive, and behavioral expressions of the human mind. So, we pay close attention to the structure-function relationship of brain regions to elucidate the link between the brain and the mind. In this quest, we believe that a combined effort from multiple facets of brain research is necessary. On top of the need for inquiries both at theoretical and experimental levels, various approaches to analyzing research data is becoming increasingly more important. Two main research approaches employed at our lab are computational and systems models. Due to its reliance on analysis by synthesis, computational models have the potential to provide a unified perspective on the underlying biology of a wide range of behaviors including those associated with neuropsychiatric disorders. We also consider neural network topology as an important brain model bearing on the systems approach. This approach is particularly effective in uncovering how neural circuitries spanning multiple neuronal populations collectively work together and give rise to complex, higher-level functions of cognition. These approaches have allowed us to latch on a multi-scale and multi-level analysis of brain activity.
One important focus in our work is a push towards deciphering individual brains. Traditionally, human neuroimaging has had to combine measurements from multiple brains and generate average brain data. This group-level analysis approach is problematic because various lines of research has increasingly shown over the recent years that there is meaningful variation across different brains that are collapsed by this type of technique. Acknowledging that understanding inter-individual variability inherent in brain network organization is paramount in bringing basic neuroscience research into the translational domains of precision medicine and other real-world settings directed at improving health and performance of individuals. We believe that this emphasis on the individual brain will help us more effectively interface basic neuroscience and real-world intervention settings such as clinical practice and education.
The most recent perspective in the brain sciences considers the brain as “brain connectomes”, which is a view regarding the brain as a complex network of neuronal populations linked by local and inter-regional connections. A systems view, accordingly, have often been employed, with a number of neuroimaging studies having investigated anatomical and functional connectivity of the brain. In addition to the traditional structural MRI, diffusion tensor imaging (DTI), a MRI-based neuroimaging technique used in estimating axonal fiber distributions of the brain, is often utilized to probe anatomical connectivity of the brain. fMRI is used for functional and effective connectivity.
To explore functional brain connectivity, which can be defined as temporal associations between brain regions, we employ the EEG/MEG and/or the popular functional MRI (fMRI). Not only the task-fMRI, but also the resting-state fMRI has been instrumental in the investigation of functional brain networks, such as the default mode network (DMN), and their implications for emotion, cognition, and behavior in health and disease are only beginning to open doors for exciting new research questions. EEG/MEG is particularly adequate for temporal synchronization with the functional imaging data, and combination of fMRI or EEG/MEG with TMS is often used for inferring causal networks.
Effective connectivity is defined as the influence one node (neuronal population) exerts over another node, within the particular network model of “causal dynamics”. Inferring effective connectivity involves estimating model parameters that best explain the observed BOLD or EEG/MEG signals. Effective connectivity is becoming increasingly more important in the analysis of neuroimaging data since the network model of causal dynamics is capable of denoting the terms of causal coupling. We use dynamic causal modeling to estimate effective connectivity.
This connectionist view of the brain culminates in the global brain network approach based on the graph theory that regards the human brain as a small-world of networks, which is both locally well-segregated and globally well-integrated. We aim to understand these brain networks through the use and development of various neuroimaging tools mentioned above. The importance of multimodal imaging comes from the understanding that the combination of multiple types of brain data would complement each other. To illustrate, spatiotemporal information of neural data can better be understood when fMRI, which has a high spatial resolution but relatively low temporal resolution, is combined with synchronous EEG data, which offers high temporal resolution but low spatial resolution. Or, molecular imaging modality such as PET can yield information about receptor-ligand interactions in the brain and thus can be an exceptional tool in understanding the neurobiological basis of behavior when combined with other functional modalities or anatomical data. Cautionary view, however, is needed with regards to the challenges arising with the multimodal approach. With the increase of the number of modalities, data complexity also increases, and each modality, as it becomes highly sophisticated, also presents with obstacles that complicate the translation from basic neuroscience to applied neuroscience, such as, issues regarding specificity of functional localization of brain regions, validity of a neurobiological model, and deployability of neural data. These issues should be addressed through rigorous assessment and optimization of the value of acquired neural data.
Brain networks under perturbed conditions, such as task-positive states, post-operative lesions, and sleep, reveal important information about neural mechanisms underlying health and disease that could be used to find novel biomarkers and treatment modalities for neuropsychiatric disorders such as depression, schizophrenia, cerebral palsy, and autism. Moreover, these systems and computational methodologies outlined above are paving the way for new frontiers in neural social sciences.
The configuration of the human brain system at rest, which is in a transitory phase among multistable states, remains unknown. To investigate the dynamic systems properties of the human brain at rest, we constructed an energy landscape for the state dynamics of the subcortical brain network, a critical center that modulates whole brain states, using resting state fMRI. We evaluated alterations in energy landscapes following perturbation in network parameters, which revealed characteristics of the state dynamics in the subcortical brain system, such as maximal number of attractors, unequal temporal occupations, and readiness for reconfiguration of the system. Perturbation in the network parameters, even those as small as the ones in individual nodes or edges, caused a significant shift in the energy landscape of brain systems. The effect of the perturbation on the energy landscape depended on the network properties of the perturbed nodes and edges, with greater effects on hub nodes and hubs-connecting edges in the subcortical brain system. Two simultaneously perturbed nodes produced perturbation effects showing low sensitivity in the interhemispheric homologous nodes and strong dependency on the more primary node among the two. This study demonstrated that energy landscape analysis could be an important tool to investigate alterations in brain networks that may underlie certain brain diseases, or diverse brain functions that may emerge due to the reconfiguration of the default brain network at rest.
Context-sensitive and activity-dependent fluctuations in connectivity underlie functional integration in the brain and have been studied widely in terms of synaptic plasticity, learning and condition-specific (e.g., attentional) modulations of synaptic efficacy. This dynamic aspect of brain connectivity has recently attracted a lot of attention in the resting state fMRI community. To explain dynamic functional connectivity in terms of directed effective connectivity among brain regions, we introduce a novel method to identify dynamic effective connectivity using spectral dynamic causal modelling (spDCM). We used parametric empirical Bayes (PEB) to model fluctuations in directed coupling over consecutive windows of resting state fMRI time series. Hierarchical PEB can model random effects on connectivity parameters at the second (between-window) level given connectivity estimates from the first (within-window) level. In this work, we used a discrete cosine transform basis set or eigenvariates (i.e., expression of principal components) to model fluctuations in effective connectivity over windows. We evaluated the ensuing dynamic effective connectivity in terms of the consistency of baseline connectivity within default mode network (DMN), using the resting state fMRI from Human Connectome Project (HCP). To model group-level baseline and dynamic effective connectivity for DMN, we extended the PEB approach by conducting a multilevel PEB analysis of between-session and between-subject group effects. Model comparison clearly spoke to dynamic fluctuations in effective connectivity – and the dynamic functional connectivity these changes explain. Furthermore, baseline effective connectivity was consistent across independent sessions – and notably more consistent than estimates based upon conventional models. This work illustrates the advantage of hierarchical modelling with spDCM, in characterizing the dynamics of effective connectivity.
The brain decoding using fMRI infers mental states associated with BOLD signal changes. This association is a converse operation to the traditional inference, i.e., depiction of brain activation elicited by stimuli. Although the fMRI-based brain decoding approach has recently received growing attention, the brain decoding itself is not a new concept. Numerous studies were performed to read one’s mind noninvasively by measuring brain activities, using mostly electroencephalogram (EEG).
This research area, which is often called brain computer interface (BCI), or brain machine interface (BMI), is successfully performed in various applications using EEG. However, fMRI is rarely recommended for conventional BCI applications since it is very expensive and lacks portability. Nonetheless, fMRI has a high spatial resolution covering the whole brain, not simply the surface. Therefore, fMRI has advantages that cannot be replaced by any other brain decoding techniques. Certainly, fMRI also has been used to detect human intentions for BCI. The goal of using fMRI for BCI is to present the usefulness of fMRI images as an index reflecting the various brain states as well as understanding how the neural system behaves or spatially represents information.
By decoding the brain using fMRI, we can expand our knowledge about the mechanism of encoding one’s mind. The advancement of brain decoding using fMRI can be attributed to the increased brain research. Again, brain decoding can improve knowledge of the brain itself. We expect a great impact of brain decoding on science and the public as shown in the evaluation of patients with the vegetative state. However, we have many challenges to overcome, including the gaps between the mind’s intention and brain neural activity, and between the neural activity and the measured BOLD signals.
In combination with brain decoding, fMRI processing in real time is a promising tool for assessing dynamic changes in brain states. A system that provides schemes to detect activated brain regions in real time is called real-time fMRI (rt-fMRI). Since rt-fMRI allows data analysis simultaneously with image acquisition, we have more freedom in expanding the brain decoding applications, which were not possible previously. The application area of rt-fMRI includes BCI, brain state monitoring, and neurofeedback. When compared to EEG-based BCI methods, rt-fMRI can allow prediction of brain status with superior reliability and a higher degree of freedom due to its high spatial resolution covering the whole brain. We are working on developing rt-fMRI system with individualized brain decoding schemes.
- Developing brain-based artificial intelligence algorithms
- Developing brain signatures using multivariate pattern recognition techniques
- Predicting functional networks of individual brains through hyper-alignment approach of deep learning
[Conference paper info]
Si-Baek Seong and Hae-Jeong Park, Brain Korea 21 PLUS Project for Medical Science, Yonsei University College of Medicine, Seoul, Republic of Korea
The conventional CNN , widely used for two-dimensional images, however, is not directly applicable to non-regular geometric surface, such as cortical brain. We propose Geometric CNN (gCNN) that deals with data representation over a spherical surface and renders pattern recognition in a multi-shell mesh structure. The classification accuracy for sex was significantly higher than that of SVM and image based CNN. It’s only use MRI thickness data to classify gender but this method can expand to classify disease from other MRI or fMRI data.
[Conference paper info]
Si-Baek Seong, Hae-Jeong Park
Department o Nuclear Medicine, Yonsei University College of Medicine
BK21 PLUS Project for Medical Science, Yonsei University College of Medicine, Seoul, Republic of Korea
Among many machine learning approaches, one of the most popular deep learning methods is the Convolutional Neural Network (CNN) (Krizhevsky et al. 2012, LeCun et al. 1998), which is a type of space variant artificial neural network. The conventional CNN is mainly optimized for two-dimensional images. However, in the field of neuroimaging research, brain cortex has been represented in terms of mesh-structured cortical sheet, topologically corresponding to a sphere. Therefore, a direct application of CNN to the mesh-based cortical representation is not a trivial problem. In previous study (Seong et al. 2017), we proposed a Geometric CNN (gCNN) method that deals with data representation over a geometric surface structure and renders pattern recognition in a multi-layer mesh structure. We evaluated the performance of the current method in the clinical application, in particular, diagnosis of Alzheimer’s disease using cortical thickness maps of Alzheimer’s Disease Neuroimaging Initiative (ADNI) database (http://adni.loni.usc.edu)
[Conference paper info]
Changwon Jang, Hae-Yoon Choi, Yoon Kyoung Choi, Hae-Jeong Park
BK21 PLUS Project for Medical Science, Yonsei University College of Medicine, Seoul, Korea
Institute of Human Complexity and Systems Science, System Science Center for Brain and Cognition, Yonsei University, Seoul, Korea
Dept. of Nuclear Medicine, Dept. of Radiology and Dept. of Psychiatry, Severance Hospital, Yonsei University College of Medicine, Seoul, Korea
Dept. of Cognitive Science, Yonsei University, Seoul, Korea
Can you predict individual functional brain networks related with task without actually obtaining task imaging data? Even if the same task is performed by multiple subjects, the pattern of individual brain networks involved in task is unique. To better understand these individual differences originating from intrinsic network architecture, we implemented a transformative function converting brain data between different persons through deep learning. The brain networks of a subject could be predicted from data from another subject through converting data into the different subject space. We called this method hyper-alignment and implemented it through deep learning and evaluated its performance.
In recent decades, a neuroscientific approach to topics traditionally thought to belong to humanities and social sciences has been rapidly gaining scientific support. With the advancement of neuroimaging technology, we now have tools to peak into the human brains while they perform a wide range of cognitive tasks and are engaged in social interactions.
As a multidisciplinary team, we are interested in answering questions that plagued the minds of scholars in the humanities and social sciences for centuries. How does sense of self emerge? What are the cognitive mechanisms underlying various decision-making processes? How do people form models of others, and how likely are those models to change in the face of contradictory information? Why are some people more autonomous, intrinsically motivated, or resilient than others? Would the brains of people with high social intelligence look different? Are there fundamental psychological differences between men and women? How do aesthetic experiences arise? How are concepts such as empathy and theory of mind constructed in our brains? How does lying look different from truth-telling in the brain? What kind of social factors help people produce their best work? Does cultural conventions and values look different in our brains?
In this line of our research, we particularly focus on understanding the ways in which brains make sense of social information and communicate with other brains and agents. Drawing upon cognitive neuroscience techniques such as fMRI and computational methods as well as behavioral measures and eye-tracking, we attempt to decipher neural mechanisms underlying psychological constructs relevant to various social contexts, such as interpersonal communication (theory of mind), social norm formation, cooperation, and human-robot interaction. We believe that by applying knowledge gained from these studies, we can devise more effective intervention measures within various clinical, educational, and corporate contexts – from treating neurodevelopmental disorders to conflict resolution and design of human assistance machine intelligences.
Some of the past and ongoing research include…
- Student performance measured with neuroscientific data in the context of two different types of social environments – autonomy-supportive vs. controlling
- Neural correlates of judgment on two different types of self-presentation – likability vs. competence
- Neural correlates of higher vs. lower communicative competence
- Neural correlates of interrogation anxiety under two different contexts – lying vs. truth-telling (true positive vs. false positive)
- Neural correlates of reward perception within differential social contexts – cooperative vs. competitive
- Cultural differences between social norms regarding politeness represented in brain activity
- Neural basis of cognitive inference involved in everyday conversations
- Neural basis of phonological processing
- Neural mechanism involved in embodied cognition
- Digital phenotyping – classifying individuals into neurobehavioral subtypes based on their phenotypes captured on virtual reality (VR) platforms
- Estimating neurocognitive and affective states of brains engaged in human-robot interaction
- Neural correlates of perception in aesthetic experiences
In answering these questions, technologies appropriate for social neuroscience such as real-time fMRI or hyperscanning have been utilized. Such brain science research in the humanities and social sciences overcomes the potential limitations of traditional psychological research relying on self-reports and the resulting subjectivity issue.
Many of our research projects in this line of work have allowed us to collaborate with various departments at Yonsei University and other institutions. We maintain close relationships with the Department of Journalism and the Department of Education. We have also collaborated with the Graduate School of Film and Digital Media at Yonsei University.