|
|
|
|
|
|
|
|
|
|
|
|
|
|
| We propose replacing scene text in videos using deep style transfer and photometric transformations. Building on recent progress on still image text replacement, we present extensions that alter text while preserving the appearance and motion characteristics of the original video. Compared to the problem of still image text replacement, our method addresses additional challenges introduced by video, namely effects induced by changing lighting, motion blur, diverse variations in camera-object pose over time, and preservation of temporal consistency. We parse the problem into four steps. First, the text in all frames is normalized to a frontal pose using a spatio-temporal transformer network. Second, a reference frame is identified where the original text exhibits the most suitable appearance and geometry. Third, the text is replaced in the reference frame using a state-of-art still-image text replacement method. Finally, the new text is transferred from the reference to remaining frames using a novel pairwise image transformation network that captures lighting and blur effects in a temporally consistent manner. Results on synthetic and challenging real videos show realistic text transfer, competitive quantitative and qualitative performance, and superior inference speed relative to alternatives. We introduce new synthetic and real-world datasets with paired text objects. To the best of our knowledge this is the first attempt at deep video text replacement. |
 |
 |
Vijay Kumar B G, Jeyasri Subramanian, Varnith Chordia, Eugene Bart, Shaobo Fang, Kelly Guan, Raja Bala STRIVE: Scene Text Replacement In Videos In International Conference on Computer Vision (ICCV), 2021. (hosted on ArXiv) |
Acknowledgements |