What's going on here?

Welcome to Machine Vision Search, a Media Ecology Project initiative

The Media Ecology Project

The Media Ecology Project (MEP) is a digital resource at Dartmouth that enables researchers across disciplines to 1) digitally access historical moving image collections; 2) build a dynamic context of research that enhances search and discovery within library and archive collections--a key value add; 3) develop new research uses over time by a wide range of users, and 4) contribute back to the library and archival community through the fluid contribution of metadata and other knowledge. We are working to realize a virtuous cycle by which new scholarship helps to ensure the preservation of historical media as public memory.

it work?

Machine Vision Learning:
The Technology

MVL is a prototype search system for identifying objects and actions in archival video. The software leverages image recognition algorithms to enable content-based search and metadata generation in video collections. The system uses google images generated from a given search word to train a neural network. Once trained, the network can find other examples of the concepts that were tagged in larger film libraries.

The Visual Learning Group researches methods to learn models of the real-world from visual data, such as images, video, and motion sequences. Our most recent work leverages the framework of deep learning to address challenging problems at the boundary between computer vision and machine learning. Projects include image categorization, action recognition, depth estimation from single photo, as well as 3D reconstruction of human movement from monocular video.

Human Generated

annotations from people like you

Machine Generated

annotations using image data to identify video tags

image results from a given search word are generated by GOOGLE IMAGES

Those image train a NEURAL NETWORK to identify actions and objects

Those actions and objects can then be identified in ARCHIVAL FOOTAGE

HUMAN ANNOTATIONS are added to machine generated data to increase archive information

is it

Libraries and Archives

The Machine Vision Search system has the potential for adaptation by educational institutions, libraries or anyone interested in sustained scholarship and preservation of film archives. The project will be an important tool used by scholars who need to find source texts to study. This new search strategy could change how video content is discovered online.

Machine Vision Search made possible by

Back to Top