Long Exposures - Creating Average Frames From Movies

I read a Guardian article about an artist who set up an analogue camera in front of their TV, set it to long exposure, and set a movie playing. The result was a rather wonderful collection of images.

You can see more of Jason Shulman's works

Is there a way to automate this process? Yes!

Here's my attempt at a "Long Exposure" of "Taxi Driver".

It's not the same as Shulmans's process, but I think it is rather charming. Here's a timelapse of how it was constructed.

I experimented with several different different methods (code at the bottom of the page) - here are some of my other experimental results.

Taxi Driver

The Muppet Movie

I also managed to create something similar to the originals.

That's a mid-point of the movie, heavily blended with previous frames.

Moana

Terence Eden is on Mastodon

@edent

Replying to @edentMoana is, unsurprisingly, very blue.
Still need a way to stop overly bright scenes washing out everything. Darken first, perhaps? pic.x.com/bamxfyc0ft

---

HOWTO

There's a couple of different ways to do this. These example work on Linux. Doing a digital average of all frames produces a different effect to Shulman's analogue process - but they are just as pleasing to my eye.

Naive and Slow

First, let's extract all the frames from a movie:

avconv -i movie.mkv -r 25 -ss 00:00:06 -t 01:23:45 %06d.png

• -r 25 this extracts at 25 frames per second (for PAL movies). Use avconv -i movie.mkv to see which framerate you should use.
• -ss 00:00:06 starts at 6 seconds in. This avoids the studio's logo.
• -t 01:23:45 captures 1 hour, 23 minutes, 45 seconds. Useful to avoid the credits at the end.
• %06d.png the extracted images will be numbered sequentially starting at 000000.png

To convert all those images to an average image, we use ImageMagick:

convert -limit memory 200MiB -limit map 400MiB *.png -average average.jpg

I've artificially limited the memory that the process can use. You can remove or adjust -limit memory 200MiB -limit map 400MiB depending on the speed of your system.

Fast and Pythony

We can use OpenCV to extract frames from the video. We can use PIL to average the difference between frames.

I tried several different options, but this produced the most pleasing results.

 Python 3from PIL import Image

from PIL import ImageChops

from PIL import ImageEnhance

import cv2



#   Video to read

vidcap = cv2.VideoCapture('test.mkv')



#   For saved images

filename = "test"



#   Which frame to start from, how many frames to go through

start_frame = 600

frames = 180000



#   Counters

count = 0

save_seq = 0

first = True



while True:

   #   Read a frame

   success,image = vidcap.read()

   if not success:

      break

   if count > start_frame+frames:

      break

   if count >= start_frame:

      if first:

         #   Extract the frame and convert to image

         average_image = cv2.cvtColor(image,cv2.COLOR_BGR2RGBA)

         average_image = Image.fromarray(average_image)

         old_image = average_image

         first = False

      if (count%100 == 0):

         #   Every 100 frames (4 seconds @ 25fps)



         #   Extract the frame and convert to image

         image = cv2.cvtColor(image,cv2.COLOR_BGR2RGBA)

         image = Image.fromarray(image)



         #   Calculate the difference between this frame and the last

         diff = ImageChops.difference(image, old_image)



         #   Store the image for use in the next itteration

         old_image = image



         #   Convert to greyscale and use that as the alpha channel

         gray_image = diff.convert('L')

         diff.putalpha(gray_image)



         #   Pick one!

         #average_image = Image.blend(average_image,image,0.1)

         #average_image = Image.alpha_composite(average_image,diff)

         average_image = ImageChops.lighter(average_image,diff)



         if (count%2500 == 0):

            #   Every 100 seconds (assuming 25fps)

            print("saving "+str(count))



            #   Darken the image slightly to prevent it getting washed out

            average_image = average_image.point(lambda p: p * 0.9)



            #   Show a preview of the image

            average_image.show()



            #   Save Image

            average_image.save(filename+str('{0:03d}'.format(save_seq))+".png")

            save_seq += 1



         if count == frames + start_frame:

            break

         count += 1



#   Save the very last generated image

average_image.save(filename+str(start_frame)+"-"+str(count)+".png")

On my cheap and crappy laptop, a 90 minute movie took around 15 minutes to render.

To create a timelapse of the images, with two frames per second.

avconv -r 2 -i example%03d.png example.mp4

To crossfade the images in the timelapse

ffmpeg -framerate 2 -i example%03d.png -vf "framerate=fps=30:interp_start=64:interp_end=192:scene=100" example.mp4

Got a better / faster / more beautiful way to do it? Let me know in the comments!