Subject: Visual storytelling · Deep learning · Vision-to-language
Year: 2018
Type: Proceedings
Title: Stories for images-in-sequence by using visual and narrative components
Author: Smilevski, Marko
Author: Lalkovski, Ilija
Author: Madjarov, Gjorgji
Abstract: Recent research in AI is focusing towards generating narrative stories about visual scenes. It has the potential to achieve more human-like understanding than just basic description generation of imagesin-sequence. In this work, we propose a solution for generating stories for images-in-sequence that is based on the Sequence to Sequence model. As a novelty, our encoder model is composed of two separate encoders, one that models the behaviour of the image sequence and other that models the sentence-story generated for the previous image in the sequence of images. By using the image sequence encoder we capture the temporal dependencies between the image sequence and the sentence-story and by using the previous sentence-story encoder we achieve a better story flow. Our solution generates long human-like stories that not only describe the visual context of the image sequence but also contains narrative and evaluative language. The obtained results were confirmed by manual human evaluation.
Publisher: Springer, Cham
Relation: International Conference on Telecommunications
Identifier: oai:repository.ukim.mk:20.500.12188/23134
Identifier: http://hdl.handle.net/20.500.12188/23134
