After my initial weather-driven AI poetry experiments, I wanted to go to the next level to create more engaging videos. This time I used multiple AI tools to create the poetry, speech, music, and visuals. Every element, from words to images to sound, reflects the creative capabilities of AI in capturing the essence of weather and nature. The result is a ten-minute compilation showcasing poems generated by ChatGPT interpreting weather data with no text edits.
Visuals
To create the visuals, I created a JSON file containing the information saved from generating several weather poems. Then I wrote a Python script that looped through the file, asking ChatGPT to generate an image based on the poem text and its corresponding weather data.
I then took each image and the audio from each poem to create animations using Kaiber.ai.
Voices
To add some texture to the vocals, I modified my original weather poem process to exclude background music, leaving only video and speech. I also added a random choice between four voice models from ElevenLabs. I then used another Python script to add time-stretched tracks to the vocals, giving it a more otherworldly feel.
To be honest, my wife was not impressed with this decision: she had trouble making out the speech and recommended I at least create subtitles (which I did, but you may have to turn them on manually).
Music
All the background music was composed by AIVA.ai. I picked from one of a thousand tracks or sections of track I’ve generated over the course of a year, then put them together manually in Audacity.
Final Composition
Once I had my audio and visual elements, I put them together using the OpenShot video editor (in case you haven’t noticed, I like to use open source software whenever I can).
Conclusion
Overall, this was another really fun learning experience. I like what came out, flaws and all.
And there are definitely flaws.
The poems are interesting, but are clearly hallucinating at times. Granted, I gave ChatGPT permission to do so, so I shouldn’t be surprised.
I do wish I had more control over the animations, or better yet automate the process, but I don’t know how to do that. Yet. Same goes for the additional processing for the vocals and music.
More importantly I want my next project to have more of my poetic voice. This particular project has its merits and its own beauty – but it showcases more of my skills as a programmer than as a poet.
Actually…I’m going to take that back. I’m going to go out on a limb and posit that this is one of my voices, one that is an amalgamation of the programmer, the poet, the LLM, the external data, and the AI services.
I feel strongly that I must be in the loop when it comes to art that I claim is mine. Where I am in that loop, however, will depend on the art I am creating and why I’m creating it.
That won’t sit well with many creators or consumers of art.
To those folks I say this: So be it – I’ll see you in the marketplace of ideas.