News digest: Multi-document Multi-media Summarization
This deliverable contains three demonstrators:
-
Browsing demonstrator
The Browsing demonstrator provides a browsable hierarchical representation
of the news data corpus. We use a set of nine BBC broadcasts recorded
during the elections in Zimbabwe 2008. We provide two alternative ways for
exploring the data. One starts with a textual screen while the other one
displays images alone. With a tag cloud of named entities at the highest
level, the text centric approach provides easy access to the lower levels.
Each named entity is displayed with font-size and colour corresponding to
its importance within the news stories. Clicking on one entity, the next
level shows an overview of all stories that contain the corresponding entity.
The story overview shows the top words from the most likely textual topic
within the story, along with a salient keyframe, which has been detected
based on the textual summary and the cross-correlation between visual and
textual topics. Clicking on one of the stories, a complete view of all the
information associated with that story is shown. The news clip can be watched
and the full story as well as a textual summary is shown.
The other approach offers an image centric browsing experience. The
keyframes of each shot are arrranged in a regular rectangular grid based
on their visual similarity. This allows to easily grasp the visual content
of the dataset. As with the other approach we arrange the images in a three
level hierarchy which makes it easier to navigate to the desired part of the
dataset. Hovering with the mouse over an image shows the keywords as quick-tip,
while a click on the Play-Button displays again all the information associated
with that story, as mentioned above. Please note, due to copyright issues,
we cannot put the full news clips online. Thus we have replaced the videos
by the salient keyframes for this online demonstrator.
-
Linking and Summarizing demonstrator
The Linking and Summarizing demonstrator is at its very core a video player which simultaneously plays video with audio
while displaying the corresponding text transcript of the spoken dialogue. We use again the sample corpus of nine BBC news broadcasts
that are each roughly half an hour in length, covering the time period of the Zimbabwe 2008 elections.
Any broadcast can simply be played, but more sophisticated navigation options are possible.
For example, one can easily skip to any component new story within a broadcast.
Likewise, one can find similar news stories to the one playing, even if they are contained in a broadcast from a different day.
Finally, the video player also identifies key people within a news story, and highlights them when they are visible.
One can then find other occurrences of an individual in other news stories across different broadcasts.
Lastly, a textual summary of each story can be displayed. If wanted, the video can display the names assigned to the detected faces.
The work is based on our CBMI 2009 paper,
our Multimedia Tools and Applications paper (in press),
our CLIN 2009 paper
and SWSM 2009 paper.
-
Names and Faces demonstrator
Because the Linking and Summarizing demonstrator is not a Web based application, we built a separate demonstrator that shows the alignment
of names and faces in news pictures and their captions. (The web-based version of this demonstrator will be online soon.)
We use here the "Labeled faces in the Wild" dataset (Huang et al. 2007). The work is based on our IEEE Transactions on Multimedia paper (in press).
Contact: Bill Triggs (coordinator), Bill.Triggs@imag.fr,
phone +33 4 7651 4553
Laboratoire Jean Kuntzmann, 51 rue des Mathematiques,
38402 Saint Martin d'Heres, Grenoble, France
