Lab 9: Sense, Compute, Communicate, Actuate
Lab 9: Sense, Compute, Communicate, Actuate
NetID:
Link to published notebook:
In this lab, we will have have a look at a couple of simple systems that uses the Sense-Compute-Communicate-Actuate loop.
Part 1: A Facial Recognition App
Part 1: A Facial Recognition App
You have been tasked with developing a facial recognition app that will verify if it is you or someone else. The app would be installable on any device and allow you to access only if it was satisfied that its you. For simplicity we will only have it play a message or a sound.
Sense
Sense
The first thing we need to do is to sense some input from the environment.
Problem 1
Problem 1
What kinds of inputs would the app need? Which sensors should it use?
Answer
Answer
Here is some code that will take a picture with your device camera:
In[]:=
pic=CurrentImage[]
Compute
Compute
Next, we need to perform some computation on the sensor data.
Problem 2
Problem 2
Based on your answer to the previous question, what do we need to compute? Think in terms of what part of the input you have obtained is really important for this app -- all of the data you have sensed (in entirety) or only part of it.
Answer
Answer
Here is some code that can identify a face in an image:
In[]:=
FindFaces[pic]
In[]:=
FindFaces[pic,"Image"]
Communicate
Communicate
The next step is communicate. We might need to communicate with external systems. Maybe it has to communicate with a pre-trained model.
Problem 3
Problem 3
In the context of the face-recognition app, what sort of information you might have collected beforehand? How would you have used that information collected earlier? Do you need to communicate with the system that has the previously-collected information stored? What do you need to send to the system from what you have sensed right now?
Answer
Answer
Some preparations needed beforehand (Work with a partner)
Some preparations needed beforehand (Work with a partner)
Since we do not already have a trained model, let’s build one.
You have to train the model with pictures of yourself and a few pictures of another person, so that it is able to recognize you (vs. "not you").
Later you can assume the trained model is available on your device.
You have to train the model with pictures of yourself and a few pictures of another person, so that it is able to recognize you (vs. "not you").
Later you can assume the trained model is available on your device.
User CurrentImage to take 3-4 pictures of yourself and 1 image of a classmate:
CurrentImage[]
Add the photos to make a comma-separated list as below:
photos={,,,,,}
Use the following code with FindFaces to isolate the face from the images:
FindFaces[#,"Image"]&/@photos
Here’s an example
Here’s an example
FaceRecognize can train a machine learning classifier on the images of people’s faces:
FaceRecognize,,
For example, here is a model trained on 3 images of Thor and 1 image of Loki:
In[]:=
frModel=FaceRecognize
"Thor",
"Thor",
"Thor",
"Loki"
Out[]=
ClassifierFunction
Let's test the model:
In[]:=
frModel
Out[]=
Thor
In[]:=
frModel
Out[]=
Loki
Test on an image of someone the model has not seen before:
In[]:=
frModel
Out[]=
Missing[Anomalous]
Now you try
Now you try
Don’t forget to test
Don’t forget to test
Test your model on your own image:
In[]:=
frModel[FindFaces[CurrentImage[],"Image"]]
Test your model on your classmate’s image:
frModel[FindFaces[CurrentImage[],"Image"]]
Test your model on someone's image that your model has not seen before:
frModel[FindFaces[CurrentImage[],"Image"]]
Actuate
Actuate
Finally, we need to actuate parts of our system to make it interact with the environment.
Problem 4
Problem 4
What are some things that can be actuated on a laptop or mobile device? What kind of signals do we need to send to these parts?
Answer
Answer
I created the function “recognize” that will “speak” only if it is given your name within quotes. For any other input, it will play a sound.
In[]:=
recognize["Thor"]:=Speak["Hello Thor, welcome back to Asgard?"]
In[]:=
recognize[anyoneElse_]:=EmitSound[Sound[{SoundNote["G"],SoundNote["C"]}]]
Feed the output of frModel into recognize and see what it does:
In[]:=
recognizefrModel
In[]:=
recognizefrModel
Problem 5
Problem 5
Think of some other things the app could do in response to the input.
Answer
Answer
Part 2: Your Own Design
Part 2: Your Own Design
Now, you will design your own device that performs all four steps in a loop: Sense, Compute, Communicate, and Actuate. Think of such a device, and how it would work.
Problem 6:
Problem 6:
What is this device? What would you name it? What does it do?
Answer:
Answer:
Sense
Sense
Problem 7:
Problem 7:
What kinds of inputs would your device need? Which sensors would it use?
Answer:
Answer:
Compute
Compute
Problem 8:
Problem 8:
What does your device need to compute? Try to be as precise as possible.
Answer:
Answer:
Communicate
Communicate
Problem 9:
Problem 9:
What does your device need to communicate with? What kind of information does it need to send? How does it send this information?
Answer:
Answer:
Actuate
Actuate
Problem 10:
Problem 10:
What parts of your device need to be actuated? What kind of signals do you need to send to these parts?
Answer:
Answer:
Submitting your work
Submitting your work
1
. Publish your notebook
1
.1
.From the cloud notebook, click on “Publish” at the top right corner.
1
.2
.From the desktop notebook, use the menu option File -> Publish to Cloud
2
.Copy the published link
3
.Add it to the top of the notebook, below your netID
4
.Print to PDF
5
.Upload to Gradescope
6
.Just to be sure, maybe ping your TA Sattwik on Slack that you have submitted.