Deep CNN model for Hybridization of Human Cognitive Features and Neurobiology for Sentiment Analysis Based on Video Contents

Project Details


Over the past decades there is a remarkable growth in the field of video generation, analysis and visualization technologies. Due to this, extensive use of multimedia content in our daily life is noticed, and the unannotated or unstructured multimedia content has always been a key issue for research. With the advancement of technologies, today a lot of progress has been done in the field of computer vision for the development of reliable systems towards automatic multimedia processing. As multimedia content is mostly targeted to human being, researchers are nevertheless working for incorporation of subjective and cognitive aspect in existing systems to increase their usefulness. Thus, an automatic multimedia content analysis or tagging based on cognitive features remains an open problem. Over the past few years researches have implemented the concept of human visual models to design (or refine) the video processing systems. Literature reveals that some methods that were proposed for human visual modelling rely on the manual analysis of the video. Manual analysis is found challenging due to two key reasons: (a) It is slow attention intensive process where human operator is not able to cope up with massively generated multimedia contents; and (b) time consuming since video analysis involves user’s perception, thinking and action steps in sequence. These challenges can be overcome by creating a hybrid deep convolutional neural network model for process human cognitive features along with neurobiology. In this study, learning human cognition will be performed by discovering cognitive features using physiological signals (EEG signals). With the advancement of neuro and vision technology a deep understanding about processing of visual information by brain can be developed which helps in identifying the portion of brain is used in the process of recognition, attention navigation or decision making etc. For example, Pre-Frontal Cortex (PFC) of brain region plays major role in taking decision as well as to process visual attention i.e. to direct attention to subjectively most interesting and important stimuli. Hippocampus and Amygdala portion of brain handles formation and processing of memory. Further, the field of cognitive psychology has provided the way to understand human behavior by analyzing associated mental processes. Various brain activities can be decoded with the help of neural signal processing and brain computer interfaces (BCI), whereas understanding of neural correlates with human’s cognition is always been a topic of interest in neuroscience. Through this project an attempt will be made to simulate the human’s brain behavior to provide cognitive abilities to machines (computers or robots).
AcronymDEEPH-Sentiment Analyzer
Effective start/end date4/06/203/06/21


  • Machine Learnig
  • Deep Learning
  • Brain Computing Interface
  • Emotion Detection
  • Human Behavior Analysis
  • EEG singals
  • Robotics