Shot on Film or AI Slop?

Reviewing the AI Tools for Film Photography, Their Effectiveness, and Costs

11 min read by Dmitri.
Published on July 7, 2024. Updated on July 8, 2024.

Comments

Save

A film photo (Minolta TC-1, Kodak Ektar) labelled as “Made with AI” on Instagram. Shrimp design by brgfx/Freepik.

Film photography may seem contrary to all things digital, but that is not necessarily the case today. Our beloved analogue medium is in grand resurgence because of its strong presence online and you are reading this on a website.

AI is a hot and controversial topic. It’s talked about in virtually every niche, including ours. A few film photographers have already shared their opinions and feelings about the technology on PetaPixel, Fstoppers, Lomography, and 35mmc/+1.

Though I have feelings and opinions of my own, I’d like to instead focus this discussion on the practical applications of AI for film photography, its potential futures, and its costs. In this article: A practical use case: cleaning up dust and scratches. Can AI images replicate the film look? Copyright and intellectual property. AI’s energy and water consumption. AI slop. Support this blog & get premium features with GOLD memberships!

A practical use case: cleaning up dust and scratches.

Two years ago, before the AI hype took over the internet, I spoke to Daniela Ivanova, a researcher at the University of Glasgow and a film photographer. She was working on her Ph.D., which focused on using deep learning to remove dust and scratches from film.

Example restoration results. From (University of Glasgow) paper “Perceptual Loss based Approach for Analogue Film Restoratio” by Daniela Ivanova.

It was a promising project meant to automate image clean-up for black-and-white and slide films that can not be processed using Digital ICE.

But to work, it needed a large training set of images in pairs: with damage and without. Daniela knew that acquiring such a set would be a challenge, so she used specialized techniques, such as the U-Net architecture and generated damage to boost up the set numbers. I’ve sent over fifty pairs of my clean/dirty images, and she received a number of submissions from the community.

Alas, the research project did not yield a product. The GitHub repository appears abandoned.

Still, Daniela’s attempt at creating such a tool wasn’t the first, nor was it the last. Adobe had a dust and scratch clean-up feature as part of its Photoshop package for many years. I last tested it in 2021 while examining automation alternatives to doing the work by hand. It was terrible: instead of selectively removing film damage, the software would strip images of definition, rendering them useless.

A lot has changed since. Today, I downloaded a new AI package for Photoshop designed specifically for the job. Undoubtedly trained on more material than Daniela could ever have access to, it seemed promising.

But despite all the touted technological advancements, Adobe’s Photo Restoration failed to remove all the scratches. That would’ve been OK if it also didn’t remove detail from the image itself!

I tested it with a few photographs. Each time, the software relentlessly cut parts of the image out, such as tree branches, fence posts, shells from the beach, etc. In some cases, I saw a significant loss of sharpness across the entire image.

View this image in HD with GOLD

Adobe’s Photo Restoration Neural Filter removed parts of the image (right), highlighted in white: branches, parts of the fence, wires, and pavement edges. Note the large scratch in the middle of the image that was not fixed by the tool.

View this image in HD with GOLD

Adobe’s Photo Restoration Neural Filter removed parts of the image (on the left, I highlighted the bits that are missing from the output image on the right). Parts of the scratches remain. If you zoom in to the high-res version of this image, you’ll notice that the grain has been glossed over.

The app replaced physical damage with degradation across the entire image. In some cases, the results were halfway acceptable but still required about the same amount of work (or more) to clean up as with manual methods.

In other words, Adobe’s new AI tool for global dust and scratch removal is not working.

Can the tool be improved? I am not convinced that it can. Adobe has an enormous training corpus, and yet it can not distinguish fine elements from damage. Given that nearly all human-created media has already been used up for ML, this may be as good as it gets.

Neither AI models nor traditional algorithms can reliably identify film damage. But this doesn’t mean that they can’t be useful when directed by a human hand (like a brush). I’ve been using the Spot Healing tool for years, and the new AI-powered Remove tool looks promising. But neither can complete the task unsupervised. You still have to find dust and scratches and “paint over them” manually.

☝︎ Further reading: “How to Remove Dust and Scratches From Film.”

Can AI images replicate the film look?

The grain and colour profile of film has been a target of digital replication since the early days of Instagram. Its founder, Kevin Systrom, gained an audience with his in-app filters, made to turn low-res mobile snaps into photographs that looked like they were shot on his Holga.

Years passed since photographers switched to digital cameras, but the accurate, clinical look of modern sensors seems to have taken something away from the creative works. So we built tools to put that something, whether it’s texture or colour, back in. “The film look” for digital photos is a whole new industry, where a variety of apps are used to simulate specific emulsions or create generic film-like effects, such as “X-Pro.”

Companies that make this software claim that its output is indistinguishable from the real thing. Yet, even non-photographers seem to be able to tell the difference — whenever that matters.

Jason @grainydays reacts to his audience’s hot takes. This one caught my attention. youtube.com/watch?v=nE5iHJGUbqk

Because AI can generate computer code, some asked whether it can be used to create custom or better film simulation presets. Unfortunately for those looking to write code without actually writing the code, the quality of the generated output is quite low. Having used GitHub CoPilot (a ChatGPT plug for developers) for months professionally, I found myself spending as much time correcting errors as I saved with the tool.

Perhaps using AI to build film simulation tools isn’t the right approach.

+ “shot on 35mm film.”

I learned about #aicinema from an Ars Technica article where Benj Edwards interviewed Julie Wieland and shared her attractive images that looked like they were shot on film. Warm light, good stage design, some dust, some scratches, weird limbs, broken shadows, alien text, and a texture that resembles film grain.

Julie’s work looked interesting (though she did admit to manually adding edits in Photoshop after the images were generated), so I dug a little deeper to find tutorials on how to achieve this look.

“Shot on Kodak Portra film” prompt ending in Midjourney V6 by Tao Prompts.

This one by Tao Prompts focuses on key elements of lighting, stage design, location, and camera angles, with examples generated in Midjourney V6. Each example is captioned with the prompt that generated it, and many of those prompts end with “shot on 35mm film.”

Tao also used more specific prompt endings, including: “Arri Alexa film,” “Shot on Kodak Portra film,” “CineStill film,” “Shot on Agfa Vista film,” “Shot on Kodak Tri X film,” and “Shot on Kodak Portra 160 film.”

Curiously, using those endings did not make any difference; Tao admits it in his videos and yet uses them anyway. They all produced the same generic look with soft colours and identical textures.

In 10 years AI will be so good at creating photos and grain that you’ll have to show your negatives as proof you actually took the photo.

— A comment from the @grainydays video.

I doubt that machine learning models will ever accurately replicate specific film looks.

In the previous chapter, I described the difficulties of making limited but nuanced edits to images with AI and creating specialized datasets. Given that we already have algorithm-based digital presets that are better than language models and much easier to build, AI just doesn’t seem like the right tool for this.

Copyright and intellectual property.

Daniela had to go through the ethics process at her university and get explicit permission for the data I and others submitted to train her tool. I was happy to do my part so that the whole community could benefit from her open-source project.

However, many machine learning models were and still are trained on data from the web without consent. For example, Google downloads large parts of this website’s content, churns it through its various systems, and produces an output that summarizes my articles and sometimes yields a click from an interested visitor.

Many other tools also scrape this website and store its contents in their databases without my consent to train their language and image generation tools. But instead of yielding visitors, those tools provide summaries from this website and generate images from my and others’ works without links or attribution.

Of course, my consent isn’t required to read, download, or process my public archives. But if I were to change my mind, AI companies would not take a “no” for an answer. They do not respect industry-standard directives for content-scaping limits, and they are known to download hidden content — even if it’s behind a paywall.

No matter where you or I share our words, images, and film scans, they’re scraped, remixed, and sold as parts of paid subscriptions.

Google’s practices have been long accepted by the web as fair use. They showed small excerpts and paid back in traffic. While you or I may not get away with copying even the smallest bits of someone’s work for profit, Google gave back traffic, so it became accepted. But Google has been dialling that attribution back over the years by placing increasing numbers of ads at the top and now AI Overviews — a new feature that’s trained on the content but doesn’t provide a reason for the visitor to come to the website.

Adobe proclaimed its training sets to be ethical, but behind the veil, it used Adobe Stock images without the authors’ explicit consent or an option to opt-out. It also appears that the company tried to include the works of its paying customers in its training sets.

AI companies need our data to feed their models, so they take it. Sometimes, they get sued, and other times, they pay for that data (if it suits them). In some cases, they pay companies who do not create any content to hand over their users’ data without those users’ consent or an option to opt-out.

It’s still not certain whether those practices are legal.

However, it appears fairly certain that AI-generated works can not be granted copyright protections.

AI’s energy and water consumption.

Environmental impacts are notoriously difficult to assess, especially for opaque industries that use large networks of diverse data centers. However, it’s clear that training and running large AI models is extremely energy-intensive. Additionally, the computers that run those models are using a lot of water for cooling.

AI slop.

Finally, we can talk about Shrimp Jesus!

Despite the enormous amounts of data, power, and water consumed, AI models still have trouble distinguishing tree branches from scratches on film. They can produce likable images and text — but that also seems to fall apart when it comes to quality. However, they have no trouble powering bots to promote odd, generated images that spread like wildfire.

Screenshot source: twitter.com/TheHornetsFury/status/1767792068947329106

Perhaps the most popular example of such an AI slop is Shrimp Jesus. An AI-generated image of a religious figure morphed with a live shrimp body floating under the sea and a caption that reads “Made it with my own hands!” Below, there are thousands of comments, most of which spell a single word: “Amen.”

There are many versions of Shrimp Jesus, some way more disturbing than funny.

Unfortunately, Shrimp Jesus’ crustaceous charm fades quickly once you learn that it’s a symptom of forthcoming AI-powered misinformation & scams.

Of course, there’s no longer a need for a human to misinform us since AI can do the job automatically.

Earlier this year, Google debuted their AI Overviews tool with an ad featuring a film camera and its puzzled user. The photographer proceeds to find an answer using Google’s product only to receive bad advice. The machine told us to open the camera’s film door, which in real life would destroy the film.

Soon after, Google became famous for its suggestions to put glue on pizza and to eat a few small rocks a day.

As photographers, we know that there’s no “perfect” camera. There’s always a trade-off, even if it’s a Leica. The same goes for other tech, including AI.

It would’ve been really nice if AI could remove dust and scratches from my film with a single click. But it can’t, so I don’t want to use it. Unfortunately, tech companies are putting AI into everything.

I’d like to conclude this article acknowledging the genous of the engineers who built those imperfect tools.

This technology is difficult to comprehend on both scientific and practical levels (I tried). It extends far beyond the depths of Shrimp Jesus or generative AI as a whole. There are tons of life-saving applications, even some tools to fix the mess it created.

It’s not all bad. I just wish there was less of it.

☝︎ Further reading: “NFTs and Film Photography.”