Computational Photography (CS 445) – Fall 2019

  

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: <i>The Key to the Fields</i> (Magritte 1936)

  Instructor:  Derek Hoiem (dhoiem)

 

  TAs: Wilfredo Torres Calderon (trrscld2)

           Yuan Shen (yshen47)

           Jae Yong Lee (lee896)  

 

  Office Hours:

            1 hour, held near Siebel 3407

Mondays 11:30AM (Derek)

Tuesdays  7PM (Jae)

Thursdays 5PM (Yuan)

Fridays 11AM (Wilfredo)

 

  Class Time and Place: Wed and Fri 2:00-3:30 at DCL 1320

 

  Syllabus

  Take-home questions

  Lecture Recordings

  Homework Submission (Compass)

  Piazza Discussion Board

 

 

  Textbook: Computer Vision: Algorithms and Applications by Rick Szeliski

                                                                                                           

  

   Projects

Project 1: Hybrid Images

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: leopard elephant

Results

Project 2: Image Quilting

    Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: C:\Users\Hoiem\Documents\Classes\ComputationalPhotography - Fall 2012\projects\quilting\face_toast_cropped.png

    Results

Project 3: Gradient Domain Fusion

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: cloning_teaser

Results

 

Project 4: Image-based Lighting

https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQJ6tcY1JugjgyMMpZMj9BgU6OaGyIv8jL4T8J-Qsgnn0UOrHT2tw

Results

 

Project 5: Video Stitching

Results

 

 

Final Project

Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: http://t2.gstatic.com/images?q=tbn:ANd9GcSEXEDpOJHSX2MOS0mvolJmNhoGKiQZ6t4ky5dyEio1mph1-L1x

Results

 

 

 

  

  Class Schedule   (subject to change)

Date

Topic

Link

Reading/Notes

Aug 28 (Wed)

Introduction

ppt ; pdf

 

Basics of Working with Images

 

 

Aug 30 (Fri)

Pixels and image filters

ppt ; pdf

S3.2 (linear filtering)

S3.3 (non-linear filtering)

Sept 4 (Wed)

Thinking in frequency

Derek away Sept 1-5. Lecture by Jae.

ppt ; pdf

’15 video

S3.4 (fourier transforms)

S2.3.3 (compression)

FFT Demo (original) ; Python demo by Jae

Sept 5 (Thurs)

Linear Algebra Tutorial at 5-6:50PM in DCL 1320 by Yuan. Use of Jupyter Notebook covered at start.

review ; demo

Another linear algebra review

 

Sept 6 (Fri)

Templates and image pyramids

ppt ; pdf

S3.5.2 (image pyramids)

S8.1.1 (pyramid alignment)

Other reading:

   Burt Adelson 1983 – The Laplacian Pyramid

Sept 11 (Wed)

Light and color

ppt ; pdf

S2.2 (light), S2.3.2 (color)

or Forsyth and Ponce Ch 3

Sept 13 (Fri)

Histograms and color adjustment

ppt ; pdf

S3.1 (histograms and color adjustment)

Color balance demo

Sept 17 (Tues)

Project 1 (Hybrid images) due

 

 

The Digital Canvas: Coloring, Blending, Cutting, Synthesizing, and Warping Images

 

 

Sept 18 (Wed)

Growing: Texture synthesis and hole filling

ppt ; pdf

Texture Synthesis – Efros Leung (1999)

Image Quilting – Efros Freeman (2001)

Image Analogies – Hertzmann et al. (2001)

Patch Match – Barnes et al. (2009)

Sept 20 (Fri)

CS 497 only: Team and Topic Proposal

 

 

Sept 20 (Fri)

Cutting: Intelligent Scissors and Graph Cuts

ppt ; pdf

Intelligent Scissors – Mortenson Barret (1995)

GrabCutRother et al. (2004)

Sept 25 (Wed)

Pasting: Compositing and blending

ppt ; pdf

Poisson Image Editing – Perez et al. (2003)

Burt and Adelson, A multiresolution spline with application to image mosaics, ACM ToG (1983) 

Sept 27 (Fri)

Image warping (translation, rotation, scale, etc.)

ppt ; pdf

S3.6 (warping)

Alignment demo

Oct 1 (Tues)

Project 2 (Image quilting) due

 

 

Oct 2 (Wed)

Image morphing

ppt ; pdf

 

Modeling the Physical World

 

 

Oct 4 (Fri)

The Pinhole Camera

ppt ; pdf

S2.1.5 (3D to 2D projection)

Oct 9 (Wed)

Single-view Metrology + Camera Model continued

ppt ; pdf

HoiemSavarese-SVG Chapter

Oct 11 (Fri)

Single-view 3D Reconstruction

ppt ; pdf

Tour into the picture (Horry et al. 1997)

Oct 15 (Tues)

Project 3 (Gradient domain fusion) due

 

 

Oct  16 (Wed)

Image-based Lighting: ray tracing, environment maps, light probes

ppt ; pdf

Reading (do read this):

Debevec, Image-based Lighting Tutorial (2005)

Oct  18 (Fri)

Image-based Lighting cont.: HDR light probes, relighting

ppt ; pdf

Optional Reading:

Debevec & Malik, “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997

Debevec, Rendering Synthetic Objects in Real Scenes, 1998

Oct 18 (Fri)

CS 497 only: Detailed Project Plan

 

 

Oct  23 (Wed)

The image as a virtual stage

ppt ; pdf

Rendering Synthetic Objects (Karsch et al. 2011)

Correspondence and Recognition

 

 

Oct  25  (Fri)

Matching and alignment with interest points

ppt ; pdf

Grauman/Leibe Draft Chapter on Local Features Optional: Lowe - SIFT paper

Oct 29 (Tues)

Project 4 (Image-based lighting) due

 

 

Oct 30  (Wed)

Automatic Photo Stitching and RANSAC

ppt ; pdf

Brown Lowe 2007 ; S9 (stitching);  slides

Nov 1 (Fri)

Object recognition, retrieval, and augmented reality

ppt ; pdf

Grauman/Leibe Draft Visual Vocabularies; S14.3

Nov 6 (Wed)

Opportunities of scale: texture synthesis, multi-view reconstruction, im2gps, tiny images, etc.

ppt ; pdf

Reading: Hays & Efros, Scene Completion Using Millions of Photographs

 

More Topics of Interest

 

 

Nov 8 (Fri)

Generating and detecting fakes  

ppt ; pdf

Pix2pix

Cycle GAN

Deep fakes in news

Nov 12 (Tues)

Project 5 (Video stitching) due

 

 

Nov 13 (Wed)

How the Kinect works

ppt ; pdf

Nov 15 (Fri)

Computational approaches to cameras

ppt ; pdf

Adelson & Bergen, The Plenoptic Function and the Elements of Early Vision (1991)

Ng et al., Light Field Photography with a Hand-held Plenoptic Camera (2005)

Levin et al., Image and Depth with a Coded Aperture (2007)

Nov 18 (Mon)

Final Project Proposal due

(for CS 497 students, this is a midpoint report)

 

 

Nov 20 (Wed)

Exam Review 

ppt ; pdf

Example questions

Nov 22 (Fri)

Exam, normal class time/place

 

 

Nov 27, 29

NO CLASS - Thanksgiving Break!

 

 

Dec 4 (Wed)

Understanding Faces

 

Dec  6 (Fri)

Video Magnification

 

 

Dec 11 (Wed)

Last day – cutting edge and wrap up

 

Dec 16 8-11am (Mon)

Final Poster Presentations (final exam period). Not yet finalized whether we will have posters or just submit web pages.

 





 


 

 

 

Credits and Course Notes

I am grateful for the many preceding efforts to design excellent courses and course notes for this topic.  I am particularly indebted to Alyosha Efros for borrowing much of his course and project design, to Rick Szeliski for making his book available, to James Hays for course suggestions and materials, and to Steve Seitz, Lana Lazebnik, Silvio Savarese, among others for their slides (as credited within).  Anyone who would like to use my slides is more than welcome; please do credit the original sources where appropriate.


Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: Description: GoStats.com