Panos

Taking Photos

Images were taken on an iPhone 15 Pro, iOS 17.6.1.
Photos were auto-adjusted by Apple's image algorithm; no special settings were used.
HEIC images were converted to JPG, and downscaled.
For example, many images had dimensions 5712 x 4284, and were reduced to 960 x 1280.
A majority of images were taken at aperature value 1.6638.

Here are some of the photos; these ones have manual corresopndances or points defined for part 4A.

Extracting Homographies from Points: Some Math

A homography is an image transformation given by a full matrix:
\( Hp = p' \rightarrow \begin{pmatrix} a & b & c \\ d & e & f \\ g & h & 1 \end{pmatrix} \begin{bmatrix} x \\ y \\ 1 \end{bmatrix} = \begin{bmatrix} wx' \\ wy' \\ w \end{bmatrix} \)

Note that the bottom right entry of \( H \) is a constant factor 1, and the resulting p' is not the normal affine coordinate. We can extract \( x', y' \) by dividing by the scaling factor \(w\).
With 8 degrees of freedom (8 variables), we need at least 8 equations to solve this system. Every point consists of two equations, so we need at least 4 points.
Because we typically select sample points or correspondances via eyeball or mouse clicking, there is a large amount of noise.
Just a few pixels off can wildly shift the homography's mapping.
Thus, we select more than 4 points to have an overdetermined system that is solvable by least squares.
Mathematically, we can rearrange the matrix to find our 8 variable system:

\( \begin{align} \begin{Bmatrix} ax + by + c &= wx' \\ dx + ey + f &= wy' \\ gx + hy + 1 &= w \end{Bmatrix} \\ \begin{Bmatrix} ax + by + c &= (gx + hy + 1) x' \\ dx + ey + f &= (gx + hy + 1) y' \\ \end{Bmatrix} \\ \begin{Bmatrix} ax + by + c &= gxx' + hyx' + x' \\ dx + ey + f &= gxy' + hyy' + y' \\ \end{Bmatrix} \\ \begin{Bmatrix} ax + by + c - gxx' - hyx' &= x' \\ dx + ey + f - gxy' - hyy' &= y' \\ \end{Bmatrix} \\ \begin{Bmatrix} ax + by + c + 0 + 0 + 0 - gxx' - hyx' &= x' \\ 0 + 0 + 0 + dx + ey + f - gxy' - hyy' &= y' \\ \end{Bmatrix} \\ \begin{Bmatrix} x & y & 1 & 0 & 0 & 0 & xx' & yx' \\ 0 & 0 & 0 & x & y & 1 & xy' & yy'\\ \end{Bmatrix} \begin{bmatrix} a \\ b \\ c \\ d \\ e \\ f \\ g \\ h \end{bmatrix} = \begin{bmatrix} wx' \\ wy' \\ w \end{bmatrix} \end{align} \)

Image Rectification

Here are some image rectifications!

Mosaic Stitching (Manual)

We can use image rectification to stitch together mosaics. Correspondances are used to define the projective transform that warps one image to the same plane as another image; the overlapping region can then be blended together.

Smoothing Images: Comments

You might've noticed in the last mosaic that there is a wedge shaped darkness in the middle of the sky.
The specific reason for that set of images is because the automatic contrast / brightness balancing done by the iPhone causes these two images to have significantly different brightness, despite being taken very close to each other.
For this project, several different blending methods were implemented; here is a small display.
Shown below are some Euclidean distance maps.

Part B: Automatic Mosaic Stitching

As described in class and specified by the paper 'Multi-Image Matching using Multi-Scale Oriented Patches'.
An automatic panoroma stitching function, that detects Harris corner features, uses Adaptive Non-Maximal Suppression algorithm to pare down corners and space them evenly across the image, and random sample consensus (RANSAC).
BELLS AND WHISTLES: We use multi-scale Harris corner features, by implementing a Gaussian pyramid.
As described in the paper, we subsample by a factor of 2 and use \(\sigma = 1.0\) for the Gaussian kernel.
Side note: Instead of a hard threshold, the implementation here opts for a pre-filtering mechanism before ANMS of "z-score pruning";
for most of the images, we use \(z=2\), and filter out pixels whose Harris values are less than \( \mu_h + z \sigma_h \).

Continued Results