Early Versus Late Fusion in Semantic Video Analysis C. G. M. Snoek, M. Worring, A. W. M. Smeulders In ACM International Conference on Multimedia 2005. [bibtex] [pdf] [url]

Semantic analysis of multimodal video aims to index segments of interest at a conceptual level. In reaching this goal, it requires an analysis of several information streams. At some point in the analysis these streams need to be fused. In this paper, we consider two classes of fusion schemes, namely early fusion and late fusion. The former fuses modalities in feature space, the latter fuses modalities in semantic space. We show by experiment on 184 hours of broadcast video data and for 20 semantic concepts, that late fusion tends to give slightly better performance for most concepts. However, for those concepts where early fusion performs better the difference is more significant.

@InProceedings{SnoekICM2005,
  author       = "Snoek, C. G. M. and Worring, M. and Smeulders, A. W. M.",
  title        = "Early Versus Late Fusion in Semantic Video Analysis",
  booktitle    = "ACM International Conference on Multimedia",
  pages        = "399–--402",
  year         = "2005",
  url          = "https://ivi.fnwi.uva.nl/isis/publications/2005/SnoekICM2005",
  pdf          = "https://ivi.fnwi.uva.nl/isis/publications/2005/SnoekICM2005/SnoekICM2005.pdf",
  has_image    = 1
}