Automatically Generating Natural Language Descriptions of Images by a Deep Hierarchical Framework

IEEE Transactions on Cybernetics, Journal Year: 2021, Volume and Issue: 52(8), P. 7441 - 7452

Published: Jan. 5, 2021

Automatically generating an accurate and meaningful description of image is very challenging. However, the recent scheme caption by maximizing likelihood target sentences lacks capacity recognizing human-object interaction (HOI) semantic relationship between HOIs scenes, which are essential parts caption. This article proposes a novel two-phase framework to generate addressing above challenges: 1) hybrid deep learning 2) generation. In deep-learning phase, factored three-way machine was proposed learn relational features pairs hierarchically. this way, recognition problem transformed into latent structured labeling task. generation lexicalized probabilistic context-free tree growing innovatively integrated with generator transform descriptions task syntactic-tree process. Extensively comparing state-of-the-art captioning methods on benchmark datasets, we demonstrated that our outperformed existing in different ways, such as significantly improving performance HOI relationships scenes (RHIS) predictions, quality generated captions semantically structurally coherent manner.

Language: Английский

---

Generating image description by modeling spatial context of an image

2022 International Joint Conference on Neural Networks (IJCNN), Journal Year: 2015, Volume and Issue: unknown, P. 1 - 8

Published: July 1, 2015

Generating the descriptive sentences of a real image is challenging task in understanding. The difficulty mainly lies recognizing interaction activities between objects, and predicting relationship objects stuff/scene. In this paper, we propose framework for improving description generation by addressing above problems. Our includes two models: unified spatial context model an model. former, as centerpiece our framework, models 3D to learn human-object predict semantic these casts problems latent structured labeling problems, can be resolved mathematical optimization. Then based on relationship, generates through proposed lexicalized tree-based algorithm. Experiments joint dataset show that outperforms state-of-the-art methods co-occurrence analysis, recognition, generation.

Language: Английский

---

Vision-based multi-sensor fusion for robust unmanned aerial vehicles autonomous navigation

Published: Jan. 1, 2019

For Complete Set Of Sensors .

Language: Английский

---