Syllabus

Lectures

Homework and Exams

Piazza

Compass

Lab 3

In lab 3, you will create a series of one-feature classifiers for images, using colorspace features, gradient features, and matched filtering.

• Code, test code, data, and solutions.
• Images of the solutions, created using make_cool_plots.py.
• Autograder submission site.

The ZIP Archive, and the recommended Code-Writing Process

In the ZIP archive, you will find the following directory hierarchy:

• submitted.py -- this is the ONLY file that you will submit, and the only file on which you will be graded. It defines a python class called Dataset that loads images from the data directory. The only images that are loaded are those from the classes you specify, which might be ['fire','water'], ['airplane','skyscraper'], or ['beetle','bicycle']. You'll need to write the following functions:
  1. set_ypbpr - convert the raw RGB color space into a YPbPr color space.
  2. set_rowconv - convolve each row with the differencer [1,0,-1], and with the averager [1,2,1].
  3. set_gradient - compute the horizontal and vertical gradient of each image using the Sobel mask.
  4. set_matchedfilters - design matched filters for each of the two classes. The matched filter is just the average of all images in that class, with the outermost two rows and outermost two columns removed, then flipped left-to-right and top-to-bottom.
  5. set_matches - convolve each image with each matched filter, keeping only the valid output samples.
  6. set_features - length-3 feature vector per image. Each features combines data over the whole image using the Frobenius norm (square root of the sum of the squares of all pixels):
     1. The color feature, for each image, is the norm of its Pb color plane, minus the norm of its Pr color plane.
     2. The gradient feature, for each image, is the norm of the horizontal (Gx) gradient of the luminance (Y) color plane, minus the norm of the vertical (Gy) gradient of the luminance (Y) color plane.
     3. The matched feature, for each image, is the norm of the matches of all color planes for class 1, minus the norm of the matches of all color planes for class 0.
  7. set_accuracyspectrum - The "accuracy spectrum" is a list of the possible accuracies you could get, for each feature, by thresholding the feature at any possible value. Specifically, for each of the three features (0<=f<=2), the classifier you want to test is ``choose class 1 if feature[f] >= threshold[f], otherwise choose class 0.'' It shouldn't take you long to realize that the only values of threshold[f] you need to test are the values of feature[f] for every image in the database --- i.e., just 12 different values.
  8. set_bestclassifier - Choose, for each feature, the best possible threshold, and the best possible polarity. A polarity=+1 classifier has the form listed under set_accuracyspectrum. A polarity=-1 classifier has the form "choose class 1 if feature[k] < threshold[k], otherwise choose class 0." The accuracy of the polarity=-1 classifier is just 1-accuracy(polarity=+1), with the same threshold.
  The results that you obtain, for the nperclass=6 cases, should match those given in solutions/*.npz.
• data/CLASS_NUM.jpg -- these are the image files; they have been downloaded using an on-line search engine. CLASS is one of {fire, water, airplane, skyscraper, beetle, bicycle}. NUM is one of [0001,0012].
• requirements.txt lists the python packages that are used by the autograder. You can install them by typing pip3 install -r requirements.txt.
• make_cool_plots.py --- generates plots that might be useful to debug your code. After you have written some functions in submitted.py, you can generate the corresponding plots, e.g., for the beetle vs. bicycle case with 12 images per class, type python3 make_cool_plots.py beetle bicycle 12. Outputs will be placed in the directory make_cool_plots_outputs; they should look like the images provided in the solution image set
• run_tests.py --- once your code is producing reasonable output plots using make_cool_plots.py, try running python3 run_tests.py. This will call the test routines in tests/test_sequence.py, using scoring utility functions in score.py, and compare the results to the hashed solution criteria in solutions/*.json.

How to Submit

When you're ready to submit, go to Gradescope.

• Submit only the one file, submitted.py. Any other files you submit will be ignored.
• You may submit as many times as you like, until the deadline. Only your last submission will count toward your course grade.