Scientific Image and Video Mining

This project finished in 2013, now continued by Video Understanding.

Project Summary

We propose to focus on fundamental computer science research in computer vision and machine learning, and its application to archaeology, cultural heritage preservation, and sociology. We validate our project by collaborations with researchers and practitioners in these fields.

 Goals of the project

We address the following problems:

Mining historical collections of photographs and paintings with applications to archaeology and cultural heritage preservation. This includes for example the quantitative analysis of environmental dammage on wall paintings or mosaics over time, and the cross-indexing of XIXth Century paintings of Pompeii with modern photographs.

Mining TV broadcasts with applications to sociology. This includes automating the analysis and annotation of human actions and interactions in video segments to assist –and provide data for– studies of consumer trends in commercials, political event coverage in newscasts, and class- and gender-related behavior patterns in situation comedies, for example.

For every one of the problems we have in mind, indexing, searching and analyzing photo and video collections is a key issue. Recent advances in image analysis, computer vision, and machine learning promise an opportunity to automate, partly or completely, these tasks (e.g., annotation of photos and videos), as well as to access information whose extraction from images is simply beyond human capabilities (e.g., indexing of very large image archives). To fulfil this promise, we propose to conduct fundamental research in object, scene, and activity modeling, learning, and recognition, and to validate it with the development of computerized image and video mining tools at the service of sciences and humanities.