Skip to main content

Research Repository

Advanced Search

Modeling spatial layout for scene image understanding via a novel multiscale sum-product network

Yuan, Zehuan; Wang, Hao; Wang, Limin; Lu, Tong; Palaiahnakote, Shivakumara; Lim Tan, Chew

Authors

Zehuan Yuan

Hao Wang

Limin Wang

Tong Lu

Chew Lim Tan



Contributors

Z. Yuan
Other

H. Wang
Other

L. Wang
Other

T. Lu
Other

C. Lim Tan
Other

Abstract

Semantic image segmentation is challenging due to the large intra-class variations and the complex spatial layouts inside natural scenes. This paper investigates this problem by designing a new deep architecture, called multiscale sum-product network (MSPN), which utilizes multiscale unary potentials as the inputs and models the spatial layouts of image content in a hierarchical manner. That is, the proposed MSPN models the joint distribution of multiscale unary potentials and object classes instead of single unary potentials in popular settings. Besides, MSPN characterizes scene spatial layouts in a fine-to-coarse manner to enforce the consistency in labeling. Multiscale unary potentials at different scales can thus help overcome semantic ambiguities caused by only evaluating single local regions, while long-range spatial correlations can further refine image labeling. In addition, higher orders are able to pose the constraints among labels. By this way, multi-scale unary potentials, long-range spatial correlations, higher-order priors are well modeled under the uniform framework in MSPN. We conduct experiments on two challenging benchmarks consisting of the MSRC-21 dataset and the SIFT FLOW dataset. The results demonstrate the superior performance of our method comparing with the previous graphical models for understanding scene images.

Citation

Yuan, Z., Wang, H., Wang, L., Lu, T., Palaiahnakote, S., & Lim Tan, C. (2016). Modeling spatial layout for scene image understanding via a novel multiscale sum-product network. Expert systems with applications, https://doi.org/10.1016/j.eswa.2016.07.015

Journal Article Type Article
Acceptance Date Jul 9, 2016
Publication Date 2016
Deposit Date Nov 15, 2024
Journal Expert Systems with Applications
Print ISSN 0957-4174
Electronic ISSN 1873-6793
Publisher Elsevier
Peer Reviewed Peer Reviewed
DOI https://doi.org/10.1016/j.eswa.2016.07.015