What Happens When You AI Upscale a Video 51 Times?

Ever wondered what happens if you render out a 4K video in standard definition, then used AI upscaling to bring it back to 4K, only to re-render it out to do the entire process over and over again? What do 51 passes of AI upscaling and enhancement do to a piece of video footage?

I recorded a number of QHD video files, then edited the clips into a sequence of crossfading abstract colours and forms quickly moving in and out of focus. I am working under the assumption that AI models are trained on the kinds of images normal people make, and I wanted to give it something it wasn’t designed for, something it would struggle with.

I exported each stage at 768×432, which is close enough to standard def while maintaining the QHD aspect ratio. I used a high bitrate so compression artifacts wouldn’t steal the show, I wanted the focus to be on what the AI models were introducing, not what h.264 was taking away. I introduced some fake digital noise or grain with each step to further prevent compression jaggedness in areas of saturated colour. I also wanted the noise to introduce a bit of random into the image so texture wouldn’t be obliterated, since the AI upscaling includes noise reduction. I was originally intending on doing 50 AI passes, but when I was editing, I realized I was 15 seconds short of hitting the 1 hour mark, so I gave it one more pass to push beyond the 60 minute mark and make this a full length feature film!

This artwork is inspired by the piece “I Am Sitting In a Room” by Alvin Lucier. While Lucier wrote about the sound characteristics of the room taking over the sound and obliterating his original words, I always felt this overlooked the role of the technology itself in shaping the final sound. A microphone is going to emphasize, attenuate, and otherwise shape the sound it captures. The pre-amp will introduce some noise while filtering others. The analogue tape it is being recorded to will have it’s own quirks and limitations. A speaker playing back the recording will further shape the sound. Let’s take the room out of the equation and do the whole process digitally, where the real world can’t creep in, and it will only be the technology shaping the final results.

Source material is video footage of crumpled aluminum foil with three RGB LED portable lights shining on it.

Shot on a Nikon Z6 with the 105mm /2.8 MC

Video edited and rendered out in Vegas 19.

AI upscaling done in Topaz Video AI v6.2.2

AMD 780M doing the GPU AI work.

Published by

Kyle Clements

Kyle Clements is a Toronto-based artist and nerd. During his thesis at the Ontario College of Art and Design, Kyle began working on his Urban Landscapes series, a body of work that aims to capture the energy and excitement of life in the fast-paced urban environment. After graduating from OCAD in 2006, Kyle spent a year living in Asia to gather source material and experience in a different kind or urban environment. His work is vibrant and colourful. Whether painting the harsh Northern landscape, or capturing the overwhelming buzz of life in the city, his acrylic paintings hover between representation and abstraction.