Video analysis

This project focuses on the use of computer vision tools to help neuroprosthetically equipped upper limb amputees grasp.
Tracking objects to be grasped in egocentric video: a temporal incremental learning approach

Technological basis of a video stream
Displacement estimation
CNN and attentional models for video
Introduction to continuous learning

Course taught in English.

Using egocentric videos, i.e. from wearable cameras, we are investigating different solutions based on convolutional neural networks for the detection and tracking of objects of interest [1].
The video represents a scene recorded by the camera attached to glasses, so we have a continuity of the scene. So, instead of recognizing and locating the object in each frame of the video, we can track the object detected in a few first frames.
This tracking/tracing is seen as the continuous adaptation of the pre-trained model for object detection to new data/frames that arrive over time in the video. We propose to achieve this using an incremental learning approach that we have designed [2]. The project will involve:

appropriating the proposed implementation of the Move-to-Data method
combining this method with initial object detection
evaluating this method on a subset of data from the Grasping-in-the-Wild corpus and proposing avenues for improvement.

Skills acquired: students will acquire notions and practice the incremental/continuous learning approach.
Bibliographical references:
[1] Iván González-Diaz, Jenny Benois-Pineau, Jean-Philippe Domenger, Daniel Cattaert, Aymar de Rugy:
Perceptually-guided deep neural networks for ego-action prediction: Object grasping. Pattern Recognit. 88: 223-235 (2019)
[2] Miltiadis Poursanidis, Jenny Benois-Pineau, Akka Zemmari, Boris Mansencal, Aymar de Rugy:
Move-to-Data: A new Continual Learning approach with Deep CNNs, Application for image-class recognition. CoRR abs/2006.07152 (2020)