By 1851, Englishman Sir David Brewster and Frenchman Jules Duboscq had perfected stereophotography. It led to an explosion of creativity in 3D photography, with the London Stereoscopic and Photographic Company becoming one of the most successful photographic companies of the era.

There are thousands of stereoscopic images hidden away in museum archives. For example, here's one from Birmingham, UK:

You probably don't have a stereoscope attached to your computer, but the 3D depth effect can be simulated by animating the two images.

Fast forward to 2023 and the invention of Gaussian Splatting. Essentially, using computers to work out 3D information when given multiple photos of a scene. It is magic - but relies on lots of photographs of a scene. Then, in 2024, Splatt3r was released. Give it two photos from an uncalibrated source, and it will attempt to reconstruct depth information from it.

Putting the above photo into the demo software gives us this rather remarkable 3D model as rendered by SuperSplat.

I think that's pretty impressive! Especially considering the low quality and low resolution of the images. How accurate is it? The pub is "The Old Crown" in Digbeth and is viewable on Google Streetview.

It's hard to get a perfect measurement - but I think that's pretty close.

Interactive Examples

Here's the image above.

Here are the Shoshone Falls, Idaho - from a series of photos taken in 1874.

This is Li Hung Chang from a stereograph taken in 1900.

Of course, it doesn't always produce great results. This is Roger Fenton's 1860 stereograph of the British Museum's Egyptian Room (Statue of Discobolus). Even with a high resolution photograph, the effect is sub-par. The depth works (although is exaggerated) but all the foreground details have been lost.

Background

Regular readers will know that I played with something similar back in 2012 - using similar software to recreate 3D scenes from Doctor Who. I also released some code in 2018 to do the same in Python.

Both of those techniques worked on screenshots from modern 3D video. The images are crisp and clear - perfect for automatically making 3D models. But neither of those approaches worked well with old photographs. There was just too much noise for simple code to grab onto.

These modern Gaussian Splatting techniques are incredible. They seem to excel at detecting objects even in the most degraded images.

Next Steps

At the moment, it is a slightly manual effort to pre-process these images. They need to be cropped or stretched to squares, artefacts and blemishes need to be corrected, and some manual tweaking of the final model is inevitable.

But I'd love to see an automated process to allow the bulk transformations of these images into beautiful 3D models. There are over 62,000 stereographs in the US Library of Congress alone - and no doubt thousands more in archives around the world.

You can download the images and models from my CodeBerg.

OK, that's a lie. It's a pretty good tech demo of what one vision of the future could look like. But it is making a little bit of my brain itch.

What Government services⁰ could / should be run in the Metaverse?

Obviously, the answer is "none". Sure, you could create a virtual job centre, housing office, or DMV - but would you want to sit in a virtual waiting room for a couple of hours waiting for your name to be called? No. Even if the room was designed to look like a spaceship and the staffs' avatars were funky robots, it would be a pointless experience.

Or would it? One of the worst things about waiting on hold on the phone is the inability to see how fast a queue is moving. Would you like to see a visual representation of how long you had to wait? Would it calm and reassure you if your case-worker looked like a friendly muppet rather than a disembodied voice?

(There's a side issue of whether VR headsets are likely to be owned by people going to a job centre. At the moment, a Quest 2 is about the same sort of price as a mid-tier smartphone - so the equipment isn't out of reach for all but the richest. Many people at the job centre were previously in employment and would have had enough disposable income to buy one.)

The other things which tickles my fancy is the idea of using humans' natural spatial awareness and memory. One of the great things about browsing the Web in the Metaverse is the ability to physically place screens around you. No more alt-tabbing. Twitter is a small screen by your feet, the news is a big screen up and to the left, and so on. It quickly becomes really convenient to remember where tabs are in 3D space, rather than a one-dimensional browser tab bar.

If you're applying for a Government service, and need to provide evidence, would it be helpful for you to have all your files laid out in front of you?

Remember, the youth of today don't understand how file systems work. So having a large visualisation of their documents may be just what they need in order to present their evidence.

People who have played the amazing game "Papers Please" will recognise just how difficult it is to organise lots of documents on a small screen.

Is this sensible?

We shouldn't be recreating offline experiences - turning unlimited potential into a janky skeuomorph. Otherwise you end up with this abomination:

Homo Digitalis

@DigitalisHomo

This is how Walmart envisions Shopping in the #Metaverse.

Thoughts? 💭 pic.x.com/5l7khobse7

---

(OK, that's a 2017 tech demo rather than a real product)

Queuing up to speak to an advisor or handing over documentation are probably the stupidest examples I could come up with.

But there's something there.

It's very easy to go from filling in a paper form to filling in a web form.

And it's easy to go from an in-person chat to a typed-chat or video-chat.

So what are the governmenty things which can go from "X" to "X-but-in-the-Metaverse"?

A few ideas

Here are a few random ideas which might lend themselves to a VR experience. Remember - I am not a VR expert, nor do I have the ability to put any of these ideas into practice.

• Does an inspector need to visit an abattoir, farm, or factory? Would it be easier to ship a robot to the location and have it controlled by inspectors in a remote location?

• How do you test whether people know how to perform an emergency drill? Running an exercise in a Nuclear power station is expensive - so strap everyone in to a headset and have them play in a Digital Twin.

• UK driving tests already have a video-based hazard perception test. Could the entire driving test take place in VR?

• If you are offered council housing, would you like to be able to view the property in VR before deciding if you wanted to move?

• Would hustings and other public discussions benefit from being in the Metaverse? A remote video call is good - but being able to collaboratively look over a 3D model of proposed designs for a new shopping centre might be helpful.

• Should a jury in a trial use the Metaverse to visualise where a suspect was found? Or to see a visualisation of a reconstruction of a crime scene?

Some (most?) of these ideas are ridiculous. Some could be commonplace in the future. The Metaverse might explode like the Web or collapse like MySpace. But I think it's a good idea for any organisation to play with new technology and explore ideas. You never know what the future will bring.

Twitter can be amazing sometimes...

I was reading about "L'Inconnue de la Seine". The face of a young woman found drowned in the river Seine in the 1800s. Her death mask was enormously popular - with copies of it appearing all over the world. Her smile was compared to the Mona Lisa and her face decorated the popular salons of the day.

Her death mask was even the basis for the first "CPR Doll". Her death has helped save more lives than anyone can possibly count.

There are thousands of photos and drawings of the mask - but no 3D scans.

Terence Eden is on Mastodon

@edent

Is there a freely available 3D model of "L'Inconnue de la Seine"?

---

Twitter, being a wretched hive of scum and villainy machine for serendipity, soon found me someone who works somewhere that makes 3D scans of busts.

Daniel Pett

@DEJPett

Replying to @puntofisso@puntofisso @edent Not that I know of, but @nebulousflynn or @jnthnbck may have come across one on travels.

---

Jonathan Beck

@jnthnbck

Replying to @DEJPett@DEJPett @puntofisso @edent @nebulousflynn We worked on a series of death masks at @TeviotPlace, all downloadable here myminifactory.com/users/Anatomic…. I’ll look into the whereabouts of this one, has been on the to-do list for a while.

---

But they didn't have L'Inconnue in their collection.

Luckily, there was a museum nearby which had a copy of her face and, before the day was out, she'd been located!

Jonathan Beck

@jnthnbck

Replying to @StJohnsGate@StJohnsGate @edent @DEJPett @puntofisso @nebulousflynn @TeviotPlace Found her! pic.x.com/dhjcqp1hus

---

And promptly scanned!

Jonathan Beck

@jnthnbck

Replying to @edent@edent @StJohnsGate @DEJPett @puntofisso @nebulousflynn @TeviotPlace Welcome to the digital world, mademoiselle inconnue!

Got to stress test a new browser based photogrammetry pipeline we've been working on. Expect more user friendly experiences soon!😉 pic.x.com/pwur0ysvry

---

A few days later, and...

(Try dragging the object with your finger or mouse.)

L'Inconnue de la Seine in 3D

Back in 2013, I asked "Should Wikipedia Have 3D Models of Sculptures?" I speculated on how soon it would be before Pirate Museums of illicit and unauthorised scans appeared.

A modern mobile phone and a few minutes in a room with good lighting is all⁰ you need and - WHAM - you've got a 3D model.

Huge thanks to Jonathan Beck for the scanthe.world project.

This post will give you a few strategies for detecting 3D images using Python.

Firstly, some terminology.

3D videos are usually saved either as Side-By-Side images, or Over-Under images. Colloquially known as H-SBS and H-OU. Here's an example.

SBS

OU

Metadata?

My first thought was that 3D video may contain metadata which the TV picks up on. This is not the case. If I display a still image from a USB stick, my TV offers to convert it. Additionally, the TV sometimes makes mistakes! If it sees a scene which has substantially similar content on each side, it will offer to convert it.

Detecting the split

There's a very obvious horizontal or vertical line in the images. It's an optical illusion caused by graphical differences between the two sides. A computer can see this line using edge detection.

from PIL import Image
from PIL import ImageFilter
Image.open("Finding_Nemo_01.png").convert("L").filter(ImageFilter.FIND_EDGES).show()

Here are the edge detected versions.

You can see the line, right? It's incredibly unlikely that a random still from a movie would have a line in exactly that position - especially over several frames.

I think this is what sometimes causes my TV to make mistakes. When I play a split-screen game like Portal 2, the TV occasionally switched to 3D mode erroneously.

Detecting

But... how does a computer "see" that line? Especially when it isn't continuous?

We know where the line should be - either exactly halfway up or across - so we can look just at the specific pixels in those locations.

First, let's convert the image to monochrome. This means we just need to look at strong contenders for edges:

Converted with:

Image.open("Finding_Nemo_01.png").filter(ImageFilter.FIND_EDGES).convert("1").show()

We could count how many white pixels there are - but that's unreliable. Look at how noisy the image is.

Can we improve it? Sure! We need to turn of dithering when we convert it to mono.

Image.open("Finding_Nemo_01.png").filter(ImageFilter.FIND_EDGES).convert("1", dither=Image.NONE).show()

We can also shrink the image. This will bring some of the lines closer together, and make it slightly quicker to count the continuous white pixels.

filename = "samples/Finding_Nemo_01.png"
image         = Image.open(filename)
width, height = image.size

small         = image.resize((int(width/2), int(height/2)))
width, height = small.size

greyscale = small.convert("L")
edges     = greyscale.filter(ImageFilter.FIND_EDGES)
mono      = edges.convert("1", dither = Image.NONE)

mono.show()

Find the line

Converting a Pillow Image to an array is easy:

import numpy as np
pixels = np.asarray(mono)

But there's something annoying/confusing lurking there:

np.asarray(mono).shape
(1080, 1920)

Yup! It rotates the image around 90 degrees. No idea why! We'll need to convert it to check the vertical line. More on that later.

When converting to a black and white image (note - not greyscale) the arrays are filled with bools. That is True and False.

To grab the horizontal line, we look halfway through the array.

ou_data   = pixels[int(height/2)]

To grab the vertical line, we rotate the image, then look halfway through the array

pixels    = (np.asarray(mono.rotate(-90, expand=True)))
sbs_data  = pixels[int(width/2)]

There are now two ways to determine if a line is present.

Total number of pixels

This is the easy way. As you have seen from the images above, there are relatively few white pixels in any given column or row.

print("SBS total: " + str(np.sum(sbs_data)))
print("SBS mean: "  + str(np.mean(sbs_data)))

If more than, say, 25% of the pixels are lit up, we can assume that there is a straight line and this is a 3D image.

Continuous pixels

Some images are very noisy, so another strategy we can use is to count how many continuous white pixels there are along the centre horizontal line and along the centre vertical line.

I'm sure there's some fancy library for Run Length Encoding, but this simple loop is all we need.

def longest_line(boolean_array):
    counter = 0
    biggest = 0
    for i in boolean_array:
        if i == True:
            counter += 1
            if counter > biggest:
                biggest = counter
        if i == False:
            counter = 0

    return biggest

Results

Running on the SBS image

OU total:   15
OU mean:     2%
OU Length:   2
SBS total: 159
SBS mean:   29%
SBS Length: 59

Running on the OU image

OU total:  113
OU mean:    12%
OU Length:  77
SBS total:   9
SBS mean:    2%
SBS Length:  3

Certainty

Based on my unscientific sampling, if more than 10% of the sampled line is continuous, that's a good indication that there is a split, and this is a still from a 3D movie.

In a video we can sample every frame and if, for example, more than 5 seconds worth of frames have a line - assume that it is a 3D film.

Image Similarity

The above techniques don't work for every 3D image though.

You will have noticed that both halves of the image are substantially similar.

There are many complex ways to detect image similarity. The simplest is calculating the Mean Square Error

def mse(imageA, imageB):
    err = np.sum((imageA.astype("float") - imageB.astype("float")) ** 2)
    err /= float(imageA.shape[0] * imageA.shape[1])
    return err

We can split the image both horizontally and vertically, then see which pair has the smallest error.

from PIL import Image
import numpy as np

original = Image.open("Finding_Nemo_01.png").convert('RGB')

#   # Get the dimensions of the image
width, height = original.size

#   # Split into left and right halves.  The left eye sees the right image.
right  = original.crop( (0,       0, width/2, height))
left   = original.crop( (width/2, 0, width,   height))

#   # Over/Under. Split into top and bottom halves. The right eye sees the top image.
top    = original.crop( (0,        0, width, height/2))
bottom = original.crop( (0, height/2, width,   height))

#   # Calculate the similarity of the left/right & top/bottom.
left_right_similarity = mse(np.array(right), np.array(left))
top_bottom_similarity = mse(np.array(top),   np.array(bottom))

if (top_bottom_similarity < left_right_similarity):
    #   # This is an Over/Under image
    print("Over-Under image detected")
else:
    print("Side-By-Side image detected")

A more robust way is

pairs = zip(right.getdata(), left.getdata())
dif = sum(abs(c1-c2) for p1,p2 in pairs for c1,c2 in zip(p1,p2))
ncomponents = right.size[0] * left.size[1] * 3
print ("Difference (percentage):" + str((dif / 255.0 * 100) / ncomponents))

With both, the lower the difference, the greater the similarity.

Speed

On my crappy laptop, the basic line detection algorithm takes less than a third of a second - a quarter of second is spent reading the image from disk.

To split the image and compare it takes a similarly short amount of time. With sufficient hardware and code optimisation, 3D detection can be done in real time.

Conclusion

I don't know that's how my TV detects 3D, but I suspect it uses a similar process. A combination of edge detection and similarity comparison is a cheap and easy way to detect 3D features.

Putting it all together

from PIL import Image
from PIL import ImageFilter
import numpy as np

filename = "Finding_Nemo_01.png"

def longest_line(boolean_array):
    counter = 0
    biggest = 0
    for i in boolean_array:
        if i == True:
            counter += 1
            if counter > biggest:
                biggest = counter
        if i == False:
            counter = 0

    return biggest

def mse(imageA, imageB):
    err = np.sum((imageA.astype("float") - imageB.astype("float")) ** 2)
    err /= float(imageA.shape[0] * imageA.shape[1])
    return err

def difference(imageA, imageB):
    pairs = zip(imageA.getdata(), imageB.getdata())
    dif = sum(abs(c1-c2) for p1,p2 in pairs for c1,c2 in zip(p1,p2))
    ncomponents = imageA.size[0] * imageB.size[1] * 3
    return ((dif / 255.0 * 100) / ncomponents)

image         = Image.open(filename).convert('RGB')
width, height = image.size

small         = image.resize((int(width/2), int(height/2)))
width, height = small.size

greyscale = small.convert("L")
edges     = greyscale.filter(ImageFilter.FIND_EDGES)
mono      = edges.convert("1", dither = Image.NONE)

mono.show()

pixels    = np.asarray(mono)
ou_data   = pixels[int(height/2)]
ou_length = longest_line(ou_data)

print("OU total: " + str(np.sum(ou_data)))
print("OU mean: "  + str(np.mean(ou_data)))
print("OU Length: "+ str(ou_length))

pixels    = (np.asarray(mono.rotate(-90, expand=True)))
sbs_data  = pixels[int(width/2)]
sbs_length= longest_line(sbs_data)

print("SBS total: " + str(np.sum(sbs_data)))
print("SBS mean: "  + str(np.mean(sbs_data)))
print("SBS Length: "+ str(sbs_length))

width, height = image.size
#   # Over/Under. Split into top and bottom halves. The right eye sees the top image.
top    = image.crop( (0,        0, width, height/2))
bottom = image.crop( (0, height/2, width,   height))
#   # Calculate the difference of the top/bottom
top_bottom_mse = mse(np.array(top),   np.array(bottom))
print("Top/Bottom: " + str(top_bottom_mse))
print ("Difference (percentage):" + str(difference(top,bottom)))


#   # Split into left and right halves.  The left eye sees the right image.
right  = image.crop( (0,       0, width/2, height))
left   = image.crop( (width/2, 0, width,   height))

#   # Calculate the difference of the left/right.
left_right_mse = mse(np.array(right), np.array(left))
print("Left/Right: " + str(left_right_mse))
print ("Difference (percentage):" + str(difference(right,left)))

In this example, we'll be using Star Wars - The Last Jedi.

tl;dr? Here's the end result (this video is silent):

Grab the code on GitHub.

Let's go!

Take a screenshot of your favourite scene. Something with a clearly defined foreground and background. The brighter the image the better the results.

Split the image in two:

mogrify -crop 50%x100% +repage screenshot.png

As you can see, 3D movies compress the image horizontally. The separated screenshots will need to be restored to their full width.

mogrify -resize 200%x100% screenshot-*.*

The next step involves a little trial-and-error. Generating a depth map can be done in several ways and it takes time to find the right settings for a scene.

Here's a basic Python script which will quickly generate a depth map.

import numpy as np
import cv2
from matplotlib import pyplot as plt

leftImage  = cv2.imread('screenshot-0.png',0)
rightImage = cv2.imread('screenshot-1.png',0)

stereo = cv2.StereoBM_create(numDisparities=16, blockSize=15)
disparity = stereo.compute(leftImage,rightImage)
plt.imshow(disparity,'gray')
plt.show()

Depending on the settings used for numDisparities and blockSize, the depth map will look something like one of these images.

In these examples, lighter pixels represent points closer to the camera, and darker pixels represent points further away. Optionally, you can clean up the image in your favourite photo editor.

The next step is to create three-dimensional mesh based on that depth, and then paint one of the original colour images onto it.

I recommend the excellent pyntcloud library.

In order to speed things up, I resampled the images to be 192*80 - the larger the image, the slower this process will be.

import pandas as pd
import numpy as np
from pyntcloud import PyntCloud
from PIL import Image

Get the colour image. Convert the RGB values to a DataFrame:

colourImg    = Image.open("colour-small.png")
colourPixels = colourImg.convert("RGB")

Add the RGB values to the DataFrame with a little help from StackOverflow.

colourArray  = np.array(colourPixels.getdata()).reshape((colourImg.height, colourImg.width) + (3,))
indicesArray = np.moveaxis(np.indices((colourImg.height, colourImg.width)), 0, 2)
imageArray   = np.dstack((indicesArray, colourArray)).reshape((-1,5))
df = pd.DataFrame(imageArray, columns=["x", "y", "red","green","blue"])

Open the depth-map as a greyscale image. Convert it into an array of depths. Add it to the DataFrame

>depthImg = Image.open('depth-small.png').convert('L')
depthArray = np.array(depthImg.getdata())
df.insert(loc=2, column='z', value=depthArray)

Convert it to a Point Cloud and display it:

df[['x','y','z']] = df[['x','y','z']].astype(float)
df[['red','green','blue']] = df[['red','green','blue']].astype(np.uint)
cloud = PyntCloud(df)
cloud.plot()

Result

Here's the 192*80 image converted to 3D, and displayed in the browser:

Wow! Even on a scaled down image, it's quite impressive. The 3D-ness is highly exaggerated - the depth is between 0-255. You can either play around with image normalisation, or adjust the values of z by using:

df['z'] = df['z']*0.5

The code is relatively quick to run, this is the result on the full resolution image. The depth is less exaggerated here, although I've multiplied it by 5.

Creating Meshes

For quick viewing, PyntCloud has a built in plotter suitable for running in Jupyter. If you don't have that, or want something higher quality, viewing 3D meshes is best done in MeshLab.

PyntCloud can create meshes in the .ply format:

cloud.to_file("hand.ply", also_save=["mesh","points"],as_text=True)

Better Depth Maps

The key to getting this right is an accurate depth mapping. That's hard without knowing the separation of the cameras, or being able to meaningfully calibrate them.

For example, from this image:

We can calculate a basic depthmap:

Which gives this 3D mesh:

If you use a more complex algorithm to generate a more detailed map, you can get some quite extreme results.

Depthmap Code

I'm grateful to Timotheos Samartzidis for sharing his work.

import cv2
import numpy as np
from sklearn.preprocessing import normalize

filename = "screenshot"

img_left  = cv2.imread(filename+'-1.png')
img_right = cv2.imread(filename+'-0.png')

window_size = 15

left_matcher = cv2.StereoSGBM_create(
    minDisparity=0,
    numDisparities=16,
    blockSize=5,
    P1=8 * 3 * window_size ** 2,
    P2=32 * 3 * window_size ** 2,
    # disp12MaxDiff=1,
    # uniquenessRatio=15,
    # speckleWindowSize=0,
    # speckleRange=2,
    # preFilterCap=63,
    # mode=cv2.STEREO_SGBM_MODE_SGBM_3WAY
)

right_matcher = cv2.ximgproc.createRightMatcher(left_matcher)

wls_filter = cv2.ximgproc.createDisparityWLSFilter(matcher_left=left_matcher)
wls_filter.setLambda(80000)
wls_filter.setSigmaColor(1.2)

disparity_left  = left_matcher.compute(img_left, img_right)
disparity_right = right_matcher.compute(img_right, img_left)
disparity_left  = np.int16(disparity_left)
disparity_right = np.int16(disparity_right)
filteredImg     = wls_filter.filter(disparity_left, img_left, None, disparity_right)

depth_map = cv2.normalize(src=filteredImg, dst=filteredImg, beta=0, alpha=255, norm_type=cv2.NORM_MINMAX);
depth_map = np.uint8(depth_map)
depth_map = cv2.bitwise_not(depth_map) # Invert image. Optional depending on stereo pair
cv2.imwrite(filename+"-depth.png",depth_map)

As good as this code is, you may need to tune the parameters on your images to get something acceptable.

More models!

Here are a few of the interesting meshes I made from the movie. Some are more accurate than others.

Snoke's Head

Throne Room Battle

V-4X-D Ski Speeders

Kylo Ren's TIE Fighter Guns

Is TLJ really 3D?

Nope! The movie was filmed with regular cameras and converted in post-production.

As I've discussed before, The Force Awakens has some scenes which have some reasonable 3D, but it wasn't a great conversion. I think TLJ is done much better - but I wish the CGI was properly rendered in 3D.

Much of the 3D-ness is one or two foreground elements floating against a background. If you want real 3D models, you need something shot and edited for 3D - for example this Doctor Who 3D special:

Further Reading

Getting this working took me all around the interwibbles - here are a few resources that I used.

• Callibration of stereo cameras
• Epipolar Geometry and Depth Map from stereo images
• Disparity of stereo images with Python and OpenCV
• Optimizing point cloud production from stereo photos by tuning the block matcher
• Calculating a depth map from a stereo camera with OpenCV
• 3D Object Reconstruction using Point Pair Features
• Dr. Jürgen Sturm's "Generate Pointcloud"

Copyright

Star Wars: The Last Jedi is copyright Lucasfilm Ltd.
These 6 screenshots fall under the UK's limited exceptions to copyright.
Any code I have written is available under the BSD License and is available on GitHub.

It looks a little like a Microsoft Kinect - a single bar with two embedded cameras. The camera resolution is 720p on each lens.

It clips on easily to a phone via a padded screw clamp. You need to install a dedicate 3D camera app in order to make it work. In the box you get the camera, two USB-OTG cables, a tripod and clamp, and a carrying pouch.

Let's take it for a spin!

Sample shots

Video

As you can see, the 3D effect is prominent.

Video quality is 2560x720. Framerate is variable depending on how fast your phone is. My Nexus 6 managed just 16fps, my S6 got to the slightly better 25fps.

Because the cameras are independent, their exposure and white balance aren't linked. This also means that occasionally one camera may drop a frame or exhibit tearing - while the other side remains unaffected.

The cameras work best in bright light. Too much motion and the JPEG encoding can't keep up. You can stream raw YUYV video off the device - but you need to either lower the framerate or resolution if you want the USB bandwidth to be sufficient.

Audio is aac, 44100 Hz, mono, fltp, 67 kb/s You don't get stereo sound, which is a little bit of a disappointment. There are no microphones in the unit - audio comes from your phone's mic. I've removed the audio from the above videos

3D metadata isn't embedded - this is needed for sites like YouTube to recognise them as 3D videos.

TV Playback

I uploaded a sample video to YouTube - which has native 3D support. I was able to stream the video to my smart TV and watch it using 3D glasses.

Photo

The Svpro app doesn't let you take photos! A bizarre omission. I cheated and used the phone's screenshot tool to capture these. Click for full size.

App

Android doesn't natively support external cameras - so you need to download the Svpro app. It's basic - you can record video and play video. That's it!

No photo capabilities, no resolution or quality settings, no framerate control, no video editing or upload to YouTube.

Worse, the app in the Google Play Store doesn't support Android 7.0 - you need to download it directly from http://www.svpro.cc/app/.

I found that the app would not keep the screen on while I was recording. You may need to adjust your phone's display settings if you want to keep the screen active.

On the positive side, it doesn't ask for crazy permissions - although you will need to grant it USB access.

There are other apps on the play store which you can use, which show how good the Svpro app could be. For example, this is USB Dual Camera:

That's not a perfect app, but it gives you an idea of the potential.

The Svpro app also seems quite slow - which sometimes leads to disappointing video results.

Raspberry Pi Linux support

Although this camera isn't advertised as working on Linux, it is a USB camera - and it works brilliantly!

Running lsusb gives us:

Bus 001 Device 004: ID 1a40:0101 Terminus Technology Inc. 4-Port HUB
Bus 001 Device 005: ID 0ac8:9902 Z-Star Microelectronics Corp.
Bus 001 Device 006: ID 0ac8:9901 Z-Star Microelectronics Corp.

So this is a USB hub with two cameras attached to it! It shows up as /dev/video0 and /dev/video1 - and both cameras work perfectly!

It supports YUV 4:2:2 (YUYV) and MJPG for video capture and, pleasingly, each camera is 1280*720 native. No interpolation!

The Raspberry Pi website has a quick tutorial on capturing images.

You can capture the MJPG directly from the camera. This is the quickest and most efficient way to get 25fps video out of the device - although the filesize is rather large (5MB per second).

avconv -f video4linux2 -input_format mjpeg -i /dev/video0 -t 00:00:05 -c:v copy output.mkv

That will only capture from one of the twin cameras. How do we get both? Here's a lazy way to capture dual USB cameras simultaneously.

avconv -f video4linux2 -input_format mjpeg -i /dev/video0 -t 00:00:05 -c:v copy outputL.mkv & avconv -f video4linux2 -input_format mjpeg -i /dev/video1 -t 00:00:05 -c:v copy outputR.mkv

That worked on my Pi 2 Model B - I'll write a blog post soon explaining other ways to do this.

Technology

At the start of the decade, my mate Ket was building his own 3D cameras. Literally two cameras wired together and mounted next to each other.

It's incredible to think how quickly technology has progressed. For £65 you get a lightweight (50g) and tiny (92mm * 20mm * 20mm) camera pairing. The distance between the cameras is almost identical to the distance between my pupils. The view angle on each lens is 100°. Perfect for stereoscopic viewing.

But pushing all that video over a USB cable is tough. A more practical design may be to place the video processing within the unit - rather than relying on the USB bandwidth and phone speed being sufficient.

Downsides

There are a few little niggles with the Svpro.

• For phones, it is Android only. At the moment the USB OTG functionality only works with Android. So you iPhone users are out of luck.
• Dedicated Android app needed. An app like this needs to be polished, professional, and a joy to use. It isn't.
• 720p is fine for a cheap camera - and matches the resolution of most 3D displays - but a bigger resolution would be welcome.
• Framerate should be higher - although this is partially a limitation of the speed of the USB transfer and of your phone.
• No binaural sound. Audio is recorded from your phone's microphone, so there's no stereo effect.

Those are minor problems. The only other issue I found was that on some devices, it would capture BGR video - the red and blue channels swapped around. Which gave this weird effect: No idea why - I reported it to Svpro and they're looking into it.

To correct it using Linux requires ffmpeg's filter colorchannelmixer

ffmpeg -i in.mp4 -vf colorchannelmixer=rr=0:rb=1:br=1:bb=0 -acodec copy out.mp4

Recording from the Pi didn't exhibit these problems.

Buy it

If you want to get started with a 3D VR Camera - for playback on a Google Cardboard or other VR headset - this is a pretty good device. The app is basic, but the video quality is satisfactory if your phone is fast enough. The 3D effect is perfect. It's also a handy way to add stereo-vision to a Raspberry Pi robotics project.

• The 3D camera is available on Amazon UK for £65.
• You can also get it direct from the manufacturer for around $80.

Last year I managed to blag tickets to the premiere of The Force Awakens - in 2D. The next day, we saw it again in 3D. I've never been a huge fan of 3D films and, despite owning a 3D TV, think the format is a bit of a gimmick.

But the 3D Star Wars was something else! Most of the scenes really didn't benefit - but some of the special effects just popped out of the screen.

The first thing to note is that SWTFA was not filmed with 3D cameras. The film was converted to 3D in post-production.

A few years ago I documented a process to convert 3D movies into animated GIFs. I'm going to use this technique to show you just how fake some of the 3D is.

Take a look at the famous opening crawl. This has been created entirely digitally - you can see the star field moving in parallax to the text.

When viewed through 3D glasses, this gives the impression of the text floating in front of the stars.

SWTFA successfully mixes live action and computer-generated special effects. In this shot, the actors have been filmed on a regular 2D camera and the "blaster shot" has been composited on afterwards.

As with much of the conversion, the frame is split into three distinct depths.

1. Popping out of the frame is the laser bolt.
2. In the middle we have Poe and the Stormtroopers.
3. Everything else is background.

Take a look at the troopers in the background - notice how they don't move in relation to each other - or the buildings behind them. If this were a true 3D scene, we'd expect to see some relative movement, but there is none.

Another sign of 3D conversion is the way lights reflect off a surface. Take a look at any normal reflective surface close to you. Wink your eyes alternately. Notice how the reflections on the surface jump around quite dramatically? Now look back at the foreground Stormtroopers. The reflections on their armour stay completely still.

I had naively thought that all the computer-generated special effects would have been rendered in 3D - but I'm not convinced that's true. Here we see some TIE fighters pursuing the Millennium Falcon.

Here, the 3D effect is noticeably worse! There is a fighter at the front of the screen - literally everything else is in the background. The position of the Falcon relative to the crashed Star Destroyer is static. The debris on the left of the screen should moving relative to the background, but it isn't.

In fact, looking at the TIE in the foreground - it looks like a 2D image floating above the rest of the scene. There's no discernible depth to it.

You can see this in some of the other shots. Here's Rey exploring the abandoned Star Destroyer.

The front floats above the background. Despite this being - I assume - an entirely CGI shot, it is little better than a South-Park-style paper-cut animation. Even worse, take a look at the right hand side of the image. A large blank gap where the image has just been squashed.

This happens constantly throughout the film. Here's what looks like an scene with huge depth - at first glance:

Look at both the left and right side. They've been artificially shunted inwards, leaving huge gaps!

That isn't to say there are some great 3D moments in the film. The character of Maz Kanata is entirely computer generated - and she looks amazing!

But, for every shot like that, you get this hot mess.

The angle of the sparks seem to change radically from foreground to background. The Lightsabre seems almost curved. There's no depth to the main character. And what the hell is happening on the right hand side? It looks like the picture has been squashed.

Thoughts

At first, I was disappointed with this conversion. This wasn't the fully immersive 3D world I'd been promised!

But then I remembered how much fun it was to watch the movie. After 5 minutes of a normal 3D flick the depth becomes about as entertaining as technicolor; it fades away. Or, even worse, it becomes a distraction as your eyes keep trying to catch up with what the brain thinks it is seeing.

But SWTFA keeps popping out of the screen without being obnoxious. Most of the action sequences have an exaggerated foreground object standing out from a flattened background.

There are a few notable exceptions - like the BB-8 shot at the top of this page and my personal favourite:

The hologram blurs as it exits the front of the picture. Finn's head seems to move beautifully behind the rest of the technical read-outs. This is one of the few shots which looks like it was filmed in 3D and also had 3D CGI applied.

And, yet, for all that - I still love the film. It's a fun romp and the 3D, while not perfect, certainly helps some of the scenes.

Technical Notes

Here's how to create these animated gifs out of a 3D movie. These are mostly notes to myself!

ImageMagick can split an Over/Under movie into Left and Right images using:

mogrify -crop 100%x50% +repage 3D.png

For Side-by-Side you'll need to use:

mogrify -crop 50%x100% +repage 3D.png

This produces two files - 3D-0.png and 3D-1.png

You can, optionally, trim out the black borders with:

convert 3D-0.png -crop 1920x398+0+70 +repage 3D-0.png

(Substitute the correct resolution for your particular file.

Stretch the image back to full height using:

mogrify -resize 100%x200% 3D-0.png

You can make an animated GIF using:

convert +dither -delay 25x100 -loop 0 +repage 3D-0.png 3D-1.png 3D.gif

Or, you can make a movie using

avconv -r 1 -i 3D-%d.png 3D.ogv

This can be transcoded to webm and mp4 as needed

As part of my "Pirate Museum" I wanted to display 3D scans of statues using WebVR. The only problem is, these files are huge.

Take The Dancing Faun - at full resolution, that's around 230MB. Even on fast broadband that's a pain to download - and even on a fast computer it is slow to render. Once you add several models like that to a scene, everything grinds to a halt.

I wanted a simple way to turn a complex model into something simple - but still recognisable.

On the left you can see the face of the full sized model - on the right is it after being shrunk.

A surprising level of detail remains - and the filesize is a miniscule 1MB. Small enough for a floppy disk!

To do this, I used MeshLab - the Open Source 3D modelling software. Meshlab has a great GUI - but it can be slow to work with, especially if you're trying to change lots of models at once.

The first step is to create a filter script in the Meshlab GUI. This is what I settled on, a script to reduce any model to 32,000 faces.

<!DOCTYPE FilterScript>
<FilterScript>
 <filter name="Quadric Edge Collapse Decimation">
  <Param type="RichInt"   value="32000" name="TargetFaceNum"/>
  <Param type="RichFloat" value="0"     name="TargetPerc"/>
  <Param type="RichFloat" value="1"     name="QualityThr"/>
  <Param type="RichBool"  value="true"  name="PreserveBoundary"/>
  <Param type="RichFloat" value="1"     name="BoundaryWeight"/>
  <Param type="RichBool"  value="true"  name="PreserveNormal"/>
  <Param type="RichBool"  value="false" name="PreserveTopology"/>
  <Param type="RichBool"  value="true"  name="OptimalPlacement"/>
  <Param type="RichBool"  value="true"  name="PlanarQuadric"/>
  <Param type="RichBool"  value="false" name="QualityWeight"/>
  <Param type="RichBool"  value="true"  name="AutoClean"/>
  <Param type="RichBool"  value="false" name="Selected"/>
 </filter>
</FilterScript>

The files is then saved as 32k.mlx.

I followed Andrew Hazelden's blog post to run the script using Meshlab server.

meshlabserver -i original.obj -o small.obj -s 32k.mlx

After a few moments - depending on the speed of your kit and size of model - a reduced size version will be spat out.

It is important to note that the MeshLab work only changes the quality of the model - not its apparent size in 3D space.

Models can be permanently scaled using Obj-Magic. For example, to scale a model to be half its physical size, run: ./obj-magic -s 0.5 -o half.obj original.obj

Finally, we have the issue of rotating models in 3D space. Some models are oriented in surprising ways. This can be done with obj-magic if you already know how you want the model to be rotated. I found it easier to use Blender on some of the models to rotate them and re-centre them.

Blender is notoriously complicated - all you need to do is import the .obj. Select it in the right hand panel. Then select the Cube / Object menu. In there you'll find the controls to shrink and rotate the object.

Once done, select "Export" then "Wavefront (.obj)".

So, with that you can easily shrink 3D models to a more compact file size.

You can visit my Pirate Museum to experience the models.

A little backstory... The Europeana Carare Project has some gorgeous 3D scans of ancient monuments and heritage sites. The only problem is that the 3D files are trapped inside PDFs. Yup - did you know the PDF spec supported 3D models? Me neither.

The only way to view these is with the Official Adobe Viewer (install it on Linux following these instructions).

This is what it looks like:

Nifty, right?

So, how do we extract that 3D file?

1. Download and install the Pythong PDF Parser
2. Search for the 3D object:
   ./pdf-parser.py -t /3D input.pdf
3. You'll get a response like this:
   

   obj 48 0
    Type: /3D
    Referencing: 58 0 R, 59 0 R
    Contains stream
   
     <<
       /AN 58 0 R
       /Length 6066066
       /Subtype /PRC
       /Type /3D
       /VA 59 0 R
     >>
   

4. The Object ID (in this case 48) is what we need to extract.
5. ./pdf-parser.py -o 48 -d output.u3d input.pdf

You can download the U3D file.

Unfortunately, nothing opens it. I've tried Meshlab, Blender, FreeCAD, i3D Converter, online service Babel3D, and even Unity 3D on a Windows box.

Nothing! Not a single application would recognise it - let alone convert it.

There is a U3D library on GitHub - but I'm not skilled enough to integrate it into anything.

I've found an Android app which claims to convert U3D files - but it has no downloads, no reviews, and costs £6!

So, gentle reader, if you know of a way to extract and view 3D models from PDFs, please let me know!

I created that gif manually, but I wondered if it was possible to create such an animation in an automated fashion. Turns out, it's pretty easy! This relies on ImageMagick - which is a powerful image manipulation tool. All of these scripts work on Ubuntu - and they should work on any GNU/Linux disto as well as Windows and Mac.

Get Your Image

I just pressed "Print Screen" while watching a Half-Side-by-Side (HSBS) movie in full screen. You can use mplayer or VLC to capture the image if you wish.

For mplayer, you can run this simple command to grab a single frame at the 20 minute and 5 seconds mark

mplayer video.avi -frames 1 -vo jpeg -nosound -ss 20:05

You should end up with an image like this - which we'll call "2D.jpg".

Split the Image

We want the left and right image separated.

mogrify -crop 50%x100% 2D.jpg

This will result in two files: 2D-0.jpg and 2D-1.jpg.

Warning: this will destroy your original image. Make sure you have a backup.

Stretch

HSBS means that the aspect ratio of each image is squashed. We want to stretch each one back to full width.

mogrify -resize 200%x100% 2D-0.jpg
mogrify -resize 200%x100% 2D-1.jpg

The two JPGs files are replaced by images with the correct aspect ratio.

The split and stretch commands can be amalgamated into:

mogrify -resize 200%x100% -crop 50%x100% 2D.jpg

Animated It!

ImageMagick makes it really easy to create gifs - in this case, we want a 250ms delay between each frame.

convert -delay 25x100 -loop 0 2D-0.jpg 2D-1.jpg movie.gif

Or, if there are only the two JPGs in the directory:

convert -delay 25x100 -loop 0 *.jpg movie.gif

You can knock the file size down by about a quarter using dithering. Although the resulting image won't look so good.

convert +dither +map -delay 25x100 -loop 0 *.jpg movie.gif

Examples

A few interesting examples.

Here, again from Doctor Who, it's possible to see a marked difference between foreground and background.

In this example, I've take two sequential frames from The Great Gatsby, The shot is a slow zoom in. By combining the left and right of both frames, we get an interesting 3D effect.

Reconstructing Depth Information

How easy is it, I wonder, to turn a stereoscopic image into a 3D model. I used Stereoscan from AgiSoft (Linux, Mac, Windows) to turn this: Into:

Doctor Who Geek Side Note

Interestingly (well... to geeks like me) the shot of Capaldi's eyes is not in 3D. I can only assume they filmed his cameo separately. There are a few other shots which aren't in 3D - those of the old Doctors (including Eccleston) and the original title sequence.

I watched the episode live at home on my Panasonic TX-L37ET5B.

After the broadcast, I wanted to watch it again. That's where the problems started.

First, a quick lesson on how 3D TV broadcasts work. Generally speaking, most 3D TV is Half-Side-By-Side. That means the left and right image are horizontally compressed and combined into a single image.

The TV automatically detects that the image is split-screen and then turns the image into 3D. It displays the left image so it can only be seen by the left eye, and the right by the right. If we apply that to the above image, we get this 3D effect.

So far, so good! The BBC have the 3D version of Day of the Doctor on iPlayer - and my TV has an iPlayer app. The only problem is that the iPlayer app on the Panasonic doesn't trigger the TV's automatic 3D detection. Even when I force the TV to 3D mode, it persists in showing the original picture. I've no idea if this is a fault with Panasonic's coding, or with the BBC's.

Well, my phone can access the iPlayer - and it can output its screen over HDMI. How well would it work? The answer, not well!

Here's a screenshot from my Android phone from the same scene.

You'll notice two things with it. Firstly the image quality is dreadful. The image resolution is 832*468 and the bitrate appears to be very poor.

Secondly, there's a bloody great BBC logo in the top left! The BBC have known for ages that Digital Onscreen Graphics (DOGs) are annoying for their viewers - but still they persist!

The problem with this on 3D content becomes readily apparent. The DOG is stretched and appears to flicker in and out to the viewer.

In addition, all iPlayer audio content is only in stereo - not the lovely surround sound of the original broadcast.

So, I can have iPlayer through the TV which won't display in 3D, I can have it through my phone in low quality with a ridiculous DOG, or I can find a kindly soul on the Internet offering Doctor Who in 1080p 3D with surround sound via a dodgy download.

I bet they don't have these problems on Inspector Spacetime!

Looking at the abstract statues and carvings, I was struck not only by their beauty, but by how easy they would be to reproduce with a 3D printer. Ok - ok! I'm a little obsessed since building a RepRapPro for work - but hear me out.

Wikipedia has been collecting high-resolution scans of famous works of art - although not without controversy - and providing them for free to the public. For example, you can download a 90MB image of the Mona Lisa at 7,479px * 11,146px, detailed enough for just about any purpose.

The controversy with Wikipedia and the National Portrait Gallery was of volunteers downloading ultra-high resolution scans which had been created by the museum and were not originally intended to be set free.

Now we have a situation where many museums are actively laser scanning their artefacts - the Smithsonian have a great 3D collection. It's very easy to grab the Smithsonian's 3D scan of the Willendorf Venus; just check the source code of that page. Some time later, the 3D file can be converted to an STL and shared freely for anyone to view and download.

And, if galleries aren't scanning their own sculptures, what happens when Wikipedia volunteers start taking 3D photos of famous works of art? It wouldn't be the hardest thing in the world to sneak a Kinnect and laptop into a gallery and surreptitiously create a 3D mesh. Or, even, just take lots of photos at different angles and use software to determine the dimensions.

And that's exactly some enterprising chap has done. With nothing more than an iPhone and used the 123D Catch software, it takes less than five minutes to make a 3D model of Moore's "Old Flo".

That 3D file could easily be embedded into Wikipedia in much the same way that high-resolution photographs are today. This would allow visitors to experience an object in 3D in their browser. See, for example, how GitHub manages this.

With an ancient sculpture, I think that's acceptable. The 3D scan is a series of facts about an object. It's a slavish copy of a work of antiquity - no more deserving of copyright than a photocopy of a Shakespeare folio.

But what about Henry Moore? Most of his art was created in the second half of the 20th century, and he died in 1986. In most parts of the world, his copyright persists for at least another 20 years.

I'd love to see interactive 3D models of his work embedded in Wikipedia - but is it legal or moral for people to create such items?

I'm not a copyright expert. I feel uncomfortable that the 3D scan of "Old Flo" has been unilaterally released by the scanner as Creative Commons - Attribution - Share Alike. The scan is undoubtedly its own work - but just as taking a photo of a portrait doesn't necessarily create a new and copyrightable piece of work, I'm not sure that a straight scan of a sculpture somehow releases the object from the bonds of copyright.

Today, we are exactly where we were with MP3s in 1998. Converting art to .stl files is a niche sport for dedicated fans, it requires expensive hardware to reproduce the original art, and the quality isn't that great.

15 years later and our laws have barely caught up with the fact that people can reproduce movies and music in nano-seconds.

How will our copyright laws react when high quality models of sculptures are freely shared on underground art exchanges?

Museum Victoria - an Australian institution - has released a 3D scanned trilobite on Thingiverse.

I trimmed the model - to make the bed shallower and the sides neater - and hit print. I'm pretty impressed with the results:

My remixed model is also available on Thingiverse if you'd like to have a go at printing your very own fossil.

I'm impressed with the fidelity of the print. It came out much better than I expected. You can really see a lot of the detail. I think it could be an excellent way to introduce people to fossils, to really give them a chance to examin them and feel them - even if they can't get to a museum.

The GB3D Type Fossils Online project, funded by JISC, aims to develop a single database of the type specimens, held in British collections, of macrofossil species and subspecies found in the UK, including links to photographs (including 'anaglyph' stereo pairs) and a selection of 3D digital models.

What that means is that they have thousands of British fossils available as freely downloadable 3D models! Even better, all the models are Creative Commons (Attribution-NonCommercial-ShareAlike 3).

So, I decided to experiment with printing out a Fossil on the 3D Printer I built for my employer.

I chose a model more-or-less at random - an Ammonite from Dorset.

I used netFabb for Linux to slice the model along the Z-Axis. This ensured it had a smooth, flat foundation.

The resolution of this particular scan was quite low - and is taken from a plaster cast of the original small fossil - but even still, I'm impressed with how well the print turned out.

The printer got a little skew-whiff towards the end of the print, so I finished it early. Even so - it's easy to see how well some of the features have printed out.

Now I need to find some larger, and more interesting things to extrude!

The artist uses Lego as his medium, and the art he produces is breathtaking.

The mixture of 2D and 3D is incredible.

But there are aspects of the exhibition which left me with questions about the nature of art and creativeness. Take, for example, these statues (click to embiggen).

They are magnificent to behold, and they are stunning "in the flesh", yet - to me - they seem like slavish copies. Perhaps it's the jagged nature of Lego, or perhaps it's the familiarity of such famous artwork, or perhaps it's in the nature of Lego that anything created with it looks like it came from a kit - but they really feel inauthentic to me.

It feels like the artist has simply taken an existing 3D image and reproduced it bit-by-bit brick-by-brick. In effect, it's not creating a piece of art, it's building from a plan.

An excellent example of this is with the 2D art.

It really looks like a 2D image has been downloaded from Wikipedia, then run through a PhotoShop filter, then pieced together. It just doesn't feel like the craftsmanship of an artist. I took Rembrandt's self portrait of 1657, shrank it, brightened it, and made it full sized again. There's a very low level of difference between mine and the brick version.

What strikes me, is that there is some amazing art within the exhibition. So much original, conceptual, and beautiful art.

There are moments which brought forth incredible emotions within me.

And others which left me utterly cold at the seemingly automated nature of production.

Should art be difficult? There is a skill level required to produce, say, an oil painting - even if you are just copying the work of an old master. Perhaps Nathan Sawaya is copying existing art by eye - using his skill to convert paintings to bricks. I just can't escape the feeling that some - not all - of the pieces don't meet my rather woolly criteria for art.

The Art of the Brick is currently in Discovery Times Square and is well worth a look if you're in town.

That said, coins are very useful as calibrators for 3D printers. They come in various shapes, sizes, and thickness. More importantly, it's really easy to compare a 3D printed coin with one in your pocket. Or, even, just by looking at them.

I've spent the last few days creating a basic set of 3D coins - based on the UK's currency.

The Royal Mint maintains openly available specifications for all UK coins. Predictably, they don't have the coins in a readily available 3D format :-)

The faces and designs of the coins are also not available. Using a high resolution photo of the designs could be used to generate a height map suitable for printing - but I'll leave that for version 2!

One interesting thing to note, the 20p and 50p are not heptagons and pentagons as many people think. They are Reuleaux Polygons. The mathematics behind them is a little to taxing for a hungover Sunday morning, so I used Dan Newman's freely available OpenSCAD template from Thingiverse.

You can download the UK coins STL file from Thingiverse or fork the coins on GitHub. Hopefully, in the next week or so I'll print them out on the new 3D printer I've been building.

Amarillo Roll is a 3D game in the same style as Super Monkey Ball. You tip your phone to move the armadillo around the course. The graphics are stunning. Reminiscent of early Xbox games. The one thing the photos don't do is convey the speed of gameplay - it's fast!

The game is a tech demo from Qualcomm. You can download Armadillo Roll directly from the Marketplace to your phone.