You are using a browser not supported by the Wolfram Cloud
Supported browsers include recent versions of Chrome, Edge, Firefox and Safari.
I understand and wish to continue anyway »
WOLFRAM NOTEBOOK
Lab 8: Sense, Compute, Communicate, Actuate
Lab 8: Sense, Compute, Communicate, Actuate
NetID: <Please fill in>
Note: Please do not rush to evaluate all code by hitting shift+enter. If you do not understand the logical progression of how we are attempting to solve the task, chances are you will run into more frustrating errors.
Part 1: A Facial Recognition App
You have been tasked with developing a facial recognition app that will verify if it is you or someone else. The app would be installable on any device and allow you access only if it is satisfied that the user is indeed you.
Our suggestion is to use Wolfram Language to create a really simple prototype of the actual app. Also it would be helpful to breakdown the design according to the Sense-Compute-Communicate-Actuate loop you learned about in class.
Our suggestion is to use Wolfram Language to create a really simple prototype of the actual app. Also it would be helpful to breakdown the design according to the Sense-Compute-Communicate-Actuate loop you learned about in class.
Sense
Sense
The first thing we need to do is to sense some input from the environment.
Problem 1
Problem 1
1) What kind of input would the app need for facial recognition?
2) Which sensor, already available on your computer, can be used for this purpose?
2) Which sensor, already available on your computer, can be used for this purpose?
Answer
Answer
Hint
Hint
Compute
Compute
Next, we need to perform some computation on the sensed data.
Problem 2
Problem 2
Based on your answer to the previous question, what sort of computation do you need to perform on the input?
(Think in terms of the specific part of the input you have sensed, that would be useful for the app.
Do you need the whole image or a very specific part of the image?)
(Think in terms of the specific part of the input you have sensed, that would be useful for the app.
Do you need the whole image or a very specific part of the image?)
Answer
Answer
Hint
Hint
Communicate
Communicate
The next step is to “communicate”.
The app might need to communicate with external systems.
Or maybe it can be set up with a pre-trained model internally with which it can communicate and get more information about the sensed input.
The app might need to communicate with external systems.
Or maybe it can be set up with a pre-trained model internally with which it can communicate and get more information about the sensed input.
Problem 3
Problem 3
In the context of the face-recognition app, what sort of a pre-trained model would you like to use?
(Think in terms of machine learning - supervised or unsupervised? If supervised - regression or classification?)
What will be your training data? What, if needed, would be the labels to identify each example in the training data?
How would the app use this model when presented with a new example of sensed input?
(Think in terms of machine learning - supervised or unsupervised? If supervised - regression or classification?)
What will be your training data? What, if needed, would be the labels to identify each example in the training data?
How would the app use this model when presented with a new example of sensed input?
Answer
Answer
Training the face recognition model (Work with a partner)
Training the face recognition model (Work with a partner)
Since we do not already have a trained model, you can use built-in machine learning functions in Wolfram Language to build one.
For the training data for your model, you can use a few pictures of yourself and a few pictures of another person. Each picture should be labeled either “you” or “not you”, so that a classifier model can be trained to recognize you (vs. "not you").
Later such a model can be installed on your device, so that the app can check-in with it to decide whether a sensed image is your face or not.
For the training data for your model, you can use a few pictures of yourself and a few pictures of another person. Each picture should be labeled either “you” or “not you”, so that a classifier model can be trained to recognize you (vs. "not you").
Later such a model can be installed on your device, so that the app can check-in with it to decide whether a sensed image is your face or not.
User CurrentImage to take 4-5 pictures of yourself and 2-3 pictures of a classmate:
CurrentImage[]
Copy paste the photos into the code below to make a comma-separated list:
photos={,,,,,}
Use the following code with FindFaces to isolate just the face from the images:
FindFaces[#,"Image"]&/@photos
Here’s an example
Here’s an example
FaceRecognize can train a machine learning classifier model on the images of people’s faces:
FaceRecognize,,
For example, here is a model trained on 3 images of Thor and 1 image of Loki:
In[]:=
frModel=FaceRecognize
"Thor",
"Thor",
"Thor",
"Loki"
You can test the model:
In[]:=
frModel
In[]:=
frModel
You can also try the model on an image of a face the model has not seen before:
In[]:=
frModel
Now you try
Now you try
Copy the FaceRecognize code from above and edit it.
Instead of Thor and Loki, use images of your face and your classmate’s.
Instead of Thor and Loki, use images of your face and your classmate’s.
Don’t forget to test
Don’t forget to test
Test your model on a new example of your own image:
In[]:=
frModel[FindFaces[CurrentImage[],"Image"]]
Test your model on a new example of your classmate’s image:
frModel[FindFaces[CurrentImage[],"Image"]]
Test your model on someone's image that your model has not seen before:
frModel[FindFaces[CurrentImage[],"Image"]]
Actuate
Actuate
Finally, we need to design an actuation step in our prototype. This will show how the final app might interact with the environment based on the sensed data.
Example of Actuation: Play Audio
Example of Actuation: Play Audio
I created the function “recognize” that will “speak” only if it is given your name within quotes. For any other input, it will play a sound.
In[]:=
recognize["Thor"]:=Speak["Hello Thor, welcome back to Asgard?"]
In[]:=
recognize[anyoneElse_]:=EmitSound[Sound[{SoundNote["G"],SoundNote["C"]}]]
Feed the output of frModel into recognize and see what it does:
In[]:=
recognizefrModel
In[]:=
recognizefrModel
Problem 4
Problem 4
1) What are some things that can be actuated on a laptop or mobile device?
2) What kind of signals do we need to send to these parts?
3) Copy over code above and edit it so that it does “speaks” a welcome message when presented with the image of your face and emits an alert sound when shown the image of your classmate.
2) What kind of signals do we need to send to these parts?
3) Copy over code above and edit it so that it does “speaks” a welcome message when presented with the image of your face and emits an alert sound when shown the image of your classmate.
Answer
Answer
Problem 5
Problem 5
List some other things you might want the face-recognition app to do, in response to the input, when it is actually deployed on a device.
Answer
Answer
Part 2: Design Your Own Sense-Compute-Communicate-Actuate Device
In this part, you will design your own device that performs all four steps in a loop: Sense, Compute, Communicate, and Actuate. Think of such a device, and how it would work.
Problem 6:
Problem 6:
What is this device? What would you name it? What does it do?
Answer:
Answer:
Sense
Sense
Problem 7:
Problem 7:
What kinds of inputs would your device need? Which sensors would it use?
Answer:
Answer:
Compute
Compute
Problem 8:
Problem 8:
What does your device need to compute? Try to be as precise as possible.
Answer:
Answer:
Communicate
Communicate
Problem 9:
Problem 9:
What does your device need to communicate with?
What kind of information does it need to send?
How does it send this information?
What kind of information does it need to send?
How does it send this information?
Answer:
Answer:
Actuate
Actuate
Problem 10:
Problem 10:
What parts of your device need to be actuated?
What kind of signals do you need to send to these parts?
What kind of signals do you need to send to these parts?
Answer:
Answer:
Submitting your work
1
.Ensure you have filled in your NetID at the top of the notebook
2
.Save the notebook as a PDF file (Alternately, "Print to PDF" but please ensure the PDF looks ok and is not garbled)
3
.Upload to Gradescope
4
.Just to be sure that your submission was received, maybe email your TA (sattwik2@illinois.edu) that you have submitted.