Table of Contents:

Deadline

11:59PM, Wednesday, March 30, 2022

Introduction

The aim of this project is to implement an end-to-end pipeline for panorama stitching. We all use the panorama mode on our smart-phones–you’ll implement a pipeline which does the same basic thing!

This document just provides an overview of what you need to do. For a full breakdown of how each step in the pipeline works, see the course notes for this project.

System Overview

Here’s a system diagram, showing each step in your panorama-stitching pipeline:

A brief description of each step (you’ll implement the steps in bold):

  • Cylinder Projection: Project images onto a cylinder, to reduce distortion at the panorama’s edges. For this project, it’s optional.
  • Detect Corners: identify corner points in your images. (You can just use OpenCV’s cornerHarris)
  • ANMS: pick out the stronger corner points.
  • Feature Descriptors: create descriptors for the corner points, so they can be matched between images (in the next step). See the approach in the document provided above.
  • Feature Matching: Match feature descriptors from different images, to find possible point correspondences (helper code provided).
  • RANSAC and Homography Estimation: refine the feature point matches, and use the correspondences to estimate homographies between images. Homography helper code is also provided.
  • Image Warping (and Blending): Use the estimated homographies to warp the images onto one another, and apply blending to reduce the appearance of seams where they fit together. See description for implementing this.
    • For blending in this project, you can simply average the pixel values of overlapping images or take the maximum.

Point Distribution:

  • ANMS: 25pts
  • Feature Descriptors: 15pts
  • Feature Matching: 15pts
  • RANSAC and Homography Estimation: 20pts
  • Image Warping (and Blending): 25pts

Project Files and Starter Code

Please find the starter code at this link. This also includes three sets of “training images”. Twenty-four hours before the due date, we’ll distribute a “test set” of two more sets of images (look out for an announcement on Piazza).

When writing your program, you can assume that input images will always follow the filename convention “1.jpg”, “2.jpg”, etc.

Submission Guidelines

We will deduct points if your submission does not comply with the following guidelines.

File tree and naming

Your submission on Canvas must be a zip file, following the naming convention YourDirectoryID_hw2.zip. For example, xyz123_hw2.zip. The file must have the following directory structure:

  • YourDirectoryID_proj2.zip/
    • code/
      • panorama.ipynb
    • images/
      • input/
      • custom1,2/
      • (all of the other train and test images)
    • report.pdf

In the Jupyter notebook, your code should load the images in images/input/ and display a resulting panorama in the end.

Report

You will be graded primarily based on your report. We want you to demonstrate an understanding of the concepts involved in the project, and to show the output produced by your code.

Include visualizations of the output of each stage in your pipeline (as shown in the system diagram on page 2), and a description of what you did for each step. Assume that we’re familiar with the homework, so you don’t need to spend time repeating what’s already in the course notes. Instead, focus on any interesting problems you encountered and/or solutions you implemented.

Be sure to include the output panoramas for all five image sets (from the training and test sets). Because you have limited time in which to access the “test set” images, we won’t expect in-depth analysis of your results for them.

As usual, your report must be full English sentences, not commented code. There is a word limit of 750 words and no minimum length requirement.

Collaboration Policy

You are encouraged to discuss the ideas with your peers. However, the code should be your own, and should be the result of you exercising your own understanding of it. If you reference anyone else’s code in writing your project, you must properly cite it in your code (in comments) and your writeup. For the full honor code refer to the CMSC426 Spring 2021 website.

Acknowledgements

This fun homework was inspired by a similar project in UPenn’s CIS581 (Computer Vision & Computational Photography).