Project 6: Auto-Stitching Photo Mosaics

Project 6 lays the basis for stitching photo mosaics.


For part 1, two major tasks had to be completed:
-Find the projective transform
-Warp the image

The first one requires singular value decomposition (SVD) to find the optimal transform between two sets of points. The first set of points corresponds to output locations of the image coordinates and the second set of points is the input coordinates.

The second part is a little trickier. Warping the image directly produces regions of unknown information. If you have a small image and you are warping to a big image, you have to sample each of the locations and apply them to the bigger image, but this leaves several pixels unfilled unless you map the smaller pixel to many larger pixels. This technique would look too discrete. The proper technique is to take the perspective warp and invert it. Then you take the output image and sample from the input image with the inverse warp. I add another trick and interpolate between pixels if I land inbetween 2 or 4 of them. I also resize the output image and map the entire warp inside the larger projected image (the output almost always gets larger).

The first results shown below represent image rectification where it appears as if the camera moved in the scene.

I first wanted to see if I could make the peleton (tour de france) look like they were coming at me instead of following each other in the peloton. No more drafting!

It was kind of effective. The depth of field on the shot wasn't good enough to see the guys in the back really well.

Next, I was at JPL and they wouldn't let me take a picture from the other side of the MSL. I can't quite move the camera that far, but I can make it look like I took the picture from the front (the back wall is now parallel to the image).

Now I want to mosaic images. First, let's see if I can mosaic a vertical and horizontal image.

The results turned out pretty good. A little blurry on the blending from imperfect point clickling, but not bad considering.

Next, I took some photos I had of Buffalo after a blizzard. It was cold that day...

These turned out pretty good because I used the building as the perspective, but all of the planes in front of the building moved and therefore blurred together (see the pole).

Finally, on to the good stuff, here you can see my disgusting room. My phone, even when the difference is just a few degrees and a few seconds, changed the way it took the image (in software). Surprisingly, it still looks really good expcept for the off color change.

Bells & Whistles!

First, off, let's see if we can add to existing mosaics:
Adding pieces to my room:

Adding images to the falls, the last one was more of the rock on the right, so it wasn't 'infinite planar,' which causes more of the distortion.

And even more room images (this last image didn't want to blend in real well):

And, I tried implementing the laplacian pyramid. As I added more levels, it just got worse. The power lines started disappearing, the edge of the image started to show up and the image started to white wash. First is 2 layers, 2nd is 3 layers, 3rd is 4 layers and the last one is 5 layers.

-Stephen Smith 12/6/2015