Creating comics with AI

Comics challenge current creative AI technologies in several ways. Firstly, they combine sequential images with text in a unique and complex way. The creation process supposes a thorough understanding of script writing, lay-out, rhythm, image composition and dialogue writing, to name just a few of the involved steps. Secondly, many AI models are built for photorealistic purposes (both for image understanding as well as image generation), making it non-trivial how to translate or retrain them for line-art. Up until today we haven't seen any proper autonomously generated comics yet. The attempts done so far needed human assistance. However, recent AI progress made comic book generation within reach. Models like GPT-3 have produced incredible text, often indistinguishable from human-written text, and the mind blowingly artistic images in all kinds of different styles generated by the DALL-E 2 and Stable Diffusion text-to-image generators stun even the best artists. While fully automatic comic book generation is the long-term moonshot, I believe there are also plenty of opportunities for creating AI tools that can be used with comic artists in co-creative ways to empower their artistic process. The disseration will focus on the following tasks where AI could offer automated support to comic artists: • text-to-image composition: given a script, draw scenery, characters and text balloons in positions to inspire the artist.   • inking assistant: Turning pencil drawings into inked pages maintaining a specific inking style.   • color assistant: coloring black & white drawings with flat colors. Especially these repetitive and less creative tasks, if supported with AI, will speed up the creation process and make it more enjoyable for the artist. The comics industry that relies heavily on a lot of manual work could get a significant boost as a result.