ADVANCe Augmented dialogue tool based on verbal and non-verbal behaviour computing
ADVANCe is an ongoing Innosuisse project that uses the recent advances in machine learning and deep learning in the domain of video processing and especially human behaviour detection to build a tool to assist recruiters during face to face or video (recruiting) interviews. The goal of the project is to assist the recruiters by giving them cues on verbal and non-verbal indices of the interviewee to enhance the dialogue interaction quality.
The project includes the creation of a dataset of videos of real life interview settings. Those videos are recorded using an intel 3D camera. The videos are annotated using a custom developed video annotation tool.
Using this dataset a SaaS tool is developed to assist recruiters either using a recorded video of the interview, or with a real time analysis of a video 3D video stream. This is achieved by using state of the art machine learning models to extract the verbal and non-verbal cues of the video and analyse them to establish a correlation between visual and audio cues.