{"id":1043,"date":"2025-05-20T12:51:45","date_gmt":"2025-05-20T16:51:45","guid":{"rendered":"http:\/\/kyleclements.com\/blog\/?p=1043"},"modified":"2025-05-20T12:51:46","modified_gmt":"2025-05-20T16:51:46","slug":"what-happens-when-you-ai-upscale-a-video-51-times","status":"publish","type":"post","link":"http:\/\/kyleclements.com\/blog\/2025\/what-happens-when-you-ai-upscale-a-video-51-times\/","title":{"rendered":"What Happens When You AI Upscale a Video 51 Times?"},"content":{"rendered":"\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"What Happens When You AI Upscale a Video 51X?\" width=\"660\" height=\"371\" src=\"https:\/\/www.youtube.com\/embed\/ioHdqqOLlM0?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<!--more-->\n\n\n\n<p>Ever wondered what happens if you render out a 4K video in standard definition, then used AI upscaling to bring it back to 4K, only to re-render it out to do the entire process over and over again? What do 51 passes of AI upscaling and enhancement do to a piece of video footage? <\/p>\n\n\n\n<p> I recorded a number of QHD video files, then edited the clips into a sequence of crossfading abstract colours and forms quickly moving in and out of focus. I am working under the assumption that AI models are trained on the kinds of images normal people make, and I wanted to give it something it wasn&#8217;t designed for, something it would struggle with. <\/p>\n\n\n\n<p>I exported each stage at 768&#215;432, which is close enough to standard def while maintaining the QHD aspect ratio. I used a high bitrate so compression artifacts wouldn&#8217;t steal the show, I wanted the focus to be on what the AI models were introducing, not what h.264 was taking away. I introduced some fake digital noise or grain with each step to further prevent compression jaggedness in areas of saturated colour. I also wanted the noise to introduce a bit of random into the image so texture wouldn&#8217;t be obliterated, since the AI upscaling includes noise reduction. I was originally intending on doing 50 AI passes, but when I was editing, I realized I was 15 seconds short of hitting the 1 hour mark, so I gave it one more pass to push beyond the 60 minute mark and make this a full length feature film! <\/p>\n\n\n\n<p>This artwork is inspired by the piece &#8220;I Am Sitting In a Room&#8221; by Alvin Lucier. While Lucier wrote about the sound characteristics of the room taking over the sound and obliterating his original words, I always felt this overlooked the role of the technology itself in shaping the final sound. A microphone is going to emphasize, attenuate, and otherwise shape the sound it captures. The pre-amp will introduce some noise while filtering others. The analogue tape it is being recorded to will have it&#8217;s own quirks and limitations. A speaker playing back the recording will further shape the sound. Let&#8217;s take the room out of the equation and do the whole process digitally, where the real world can&#8217;t creep in, and it will only be the technology shaping the final results. <\/p>\n\n\n\n<p>Source material is video footage of crumpled aluminum foil with three RGB LED portable lights shining on it. <\/p>\n\n\n\n<p>Shot on a Nikon Z6 with the 105mm \/2.8 MC <\/p>\n\n\n\n<p>Video edited and rendered out in Vegas 19. <\/p>\n\n\n\n<p>AI upscaling done in Topaz Video AI v6.2.2 <\/p>\n\n\n\n<p>AMD 780M doing the GPU AI work. <\/p>\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[21],"tags":[192,189,44,193,195,15,194,104],"class_list":["post-1043","post","type-post","status-publish","format-standard","hentry","category-kyleswork","tag-abstract","tag-ai","tag-art","tag-demo","tag-lucier","tag-photography","tag-technical","tag-video"],"_links":{"self":[{"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/posts\/1043","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/comments?post=1043"}],"version-history":[{"count":2,"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/posts\/1043\/revisions"}],"predecessor-version":[{"id":1045,"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/posts\/1043\/revisions\/1045"}],"wp:attachment":[{"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/media?parent=1043"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/categories?post=1043"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/kyleclements.com\/blog\/wp-json\/wp\/v2\/tags?post=1043"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}