Reflecting on best practices | Mandate Swiss Agency for Development and Cooperation SDC | Story SDC, Hynek Bures | Direction Hynek Bures | Graphics & animations Ehud Graf | Voice Fernando Tiberini | Music Léa Célestine Bernasconi | Text correction Sally Lowick | Languages English | 2023/4

2023 has been been the berth year of AI! Or so they say... So naturally, Ehud, graphic designer, and I wanted to explore those new possibilities: How to use this new tool for our creations? We got an opportunity to dive into the text-to-image generation while working on two animations for the SDC. When we started production, a full text-to-motion-picture workflow was not possible. Thus we focused on generating AI images which Ehud used for the animation. I really learnt a lot from Ehud, who spent hours perfecting the prompts. Let me describe our workflow here. Please keep in mind that these applications are evolving almost daily. What has been the reality for this production, might not apply in future:

• We used Midjourney (connected to the Discord application) for our image generation. The advantage with this application is that you can decide on the image aspect ratio; in our case 16:9.
• Our story plays in Latin America. Thus, we then decided on the image style. We researched popular artists from Latin America. Giving the AI a clear idea what you want is key for a consistent image style. The machine always strays away from your style. You always need to remind it what you want. Some of our prompts included up to 7 image links to keep the machine on track.
• The amount of images generated compared to what made it into the animation must be something like 20:1 ! Very often the machine generated the right image, but the faces were off, or just not the way we wanted them. You let the machine generate a few more variations till everything in the image is right.
• Sometimes Midjourney just didn't react to our prompts the way we wanted. Workarounds were necessary: We asked ChatGPT to rewrite our prompts and copy/pasted those into Discord, or...
• The background was not what we wanted. In some cases the solution was to open the image in Photoshop and use the background generation effect from Adobe.

Here an example of a prompt. Notice the many image links at the start of the prompt:

Text-to-image generators give us the opportunity to work with image styles which otherwise would be very expensive to generate. It lets us change an image in the video fairly quickly, without having to go back to the illustration board. But it is not like we saved a lot of time. Creating, adjusting prompts, and cleaning up the images continues be labor intensive.

We definitely look forward to the text-to-motion-picture applications that are rapidly developing. We are not afraid that they will take our job away, at least not that quickly. For now, all these applications still need the human factor to create a product that meets the client's needs.

We created 2 animations in the same style, each with its own focused story and message. Here a few beautiful screen shots from video 02:

Let me conclude with this wisdom: