Lab 10: Computer Vision

NetID:
In this lab, we will have a look at a couple of simple examples of image processing for computer vision.
Run each line of code in each section and observe their effects on the image inputs.

Part 1

Some Basic Image Processing Operations

DominantColors

DominantColors returns dominant colors in
image
.
In[]:=
DominantColors

Separate an image into the regions of its dominant colors:
In[]:=
DominantColors
,Automatic,{"CoverageImage","Color"}

ColorQuantize

ColorQuantize reduces the number of distinct colors in the image:
In[]:=
ColorQuantize
,3
In[]:=
ColorQuantize
,5
Quantize using a specific list of colors:
In[]:=
ColorQuantize
,{Black,Purple,Green}

Binarize

Binarize creates a binary image: every pixel above a threshold value is 1 or white and every pixel below is 0 or white.
In[]:=
Binarize

Binarize is like ColorQuantizing to black and white:
In[]:=
ColorQuantize
,{Black,White}
Try this fun example:
In[]:=
BinarizeGradientFilter
,1,Method{"BlackFraction",.8}

EdgeDetect

Remember the vertical edge filter from class. Let's apply it to an image using image convolution:
In[]:=
ImageConvolve
,
-1
0
1
-2
0
2
-1
0
1

Or directly use the built-in function EdgeDetect. EdgeDetect includes some other processing as well as the basic edge detection filter. So it seems to do a better job of finding edges in
image
and returns the result as a binary image.
In[]:=
EdgeDetect

Try another example:
In[]:=
EdgeDetect


Problem 1

You can apply multiple image processing operations on an input:
In[]:=
EdgeDetectBinarize

You can do arithmetic operations with the image output, like addition or subtraction:
In[]:=
ColorQuantize
-EdgeDetect

Use an image of your own choice and apply any two of the four image processing operations you have learned above.
​

Part 2

What does the model see?

Step 1

Check what the image identification machine learning model says about this image:
In[]:=
ImageIdentify


Step 2

Now blur the image a bit
In[]:=
blurredImage=Blur

​
You can use blur as follows, with a number after the comma next to the image, which blurs the image over a pixel radius specified by the number.

Step 3

Now try to identify the blurred image

Problem 2

ImageIdentify seem to work correctly on the following:
Keep increasing the pixel radius for blurring till the point when ImageIdentify is no longer able to identify the image correctly. What is the value of the pixel radius when ImageIdentify fails? What does ImageIdentify identify the image as at this point?

Part 3

Create some pop art with image segmentation algorithms.

Step 1

Here is an image:

Step 2

The ClusteringComponents function finds clusters of pixels in an image. You can provide the number of clusters you want in the image. The code below will find only 3 clusters, which may not produce a very interesting result.

Problem 3

Use colorize to create a visually interesting output from an image of your choice. You will have to increase the number of clusters in step 2 to get more clusters that will be colored differently.

Part 4

Segment an image of a street scene

Neural networks can be used to segment the images scene by a self-driving car's camera on the road.
Download a pre-trained neural network model from Wolfram neural net repository:
Define a list of labels that can be applied to objects in the image:
You will need this evaluation function to process the output of the neural net:
Here is a test image:
Here are the different objects detected in the image:
Segment and colorize the image:

Problem 4

Try it on the following image ( https://en.wikipedia.org/wiki/Paris_Street;_Rainy_Day ):

Extra Credit

Warning: The video processing code may take a while to run.
Download an example video with the following piece of code:
Extract one frame from the video:
Find ImageBoundingBoxes for objects in the image:
Find ImageBoundingBoxes for objects in every frame of the video:

Problem 5

Download the following video: https://pixabay.com/videos/swans-ducks-water-white-bird-1287/
Import the video. From the menu use Insert -> File Path to find the path to your downloaded file:
Use the code provided earlier to find the bounding boxes for all the swans and ducks on the water:
Comment on how well the algorithm is able to follow the birds as they float on the water.
​

Submitting your work

1
.
Publish your notebook
1
.
1
.
From the cloud notebook, click on “Publish” at the top right corner.
1
.
2
.
From the desktop notebook, use the menu option File -> Publish to Cloud
2
.
Copy the published link
3
.
Add it to the top of the notebook, below your netID
4
.
Print to PDF
5
.
Upload to Gradescope
6
.
Just to be sure, maybe ping your TA Sattwik on Slack that you have submitted.

​