WOLFRAM NOTEBOOK

Lab 9: Sense, Compute, Communicate, Actuate

NetID:
Link to published notebook:
In this lab, we will have have a look at a couple of simple systems that uses the Sense-Compute-Communicate-Actuate loop.

Part 1: A Facial Recognition App

You have been tasked with developing a facial recognition app that will verify if it is you or someone else. The app would be installable on any device and allow you to access only if it was satisfied that its you. For simplicity we will only have it play a message or a sound.

Sense

The first thing we need to do is to sense some input from the environment.

Problem 1

What kinds of inputs would the app need? Which sensors should it use?

Answer

Here is some code that will take a picture with your device camera:
In[]:=
pic=CurrentImage[]

Compute

Next, we need to perform some computation on the sensor data.

Problem 2

Based on your answer to the previous question, what do we need to compute? Think in terms of what part of the input you have obtained is really important for this app -- all of the data you have sensed (in entirety) or only part of it.

Answer

Here is some code that can identify a face in an image:
In[]:=
FindFaces[pic]
In[]:=
FindFaces[pic,"Image"]

Communicate

The next step is communicate. We might need to communicate with external systems. Maybe it has to communicate with a pre-trained model.

Problem 3

In the context of the face-recognition app, what sort of information you might have collected beforehand? How would you have used that information collected earlier? Do you need to communicate with the system that has the previously-collected information stored? What do you need to send to the system from what you have sensed right now?

Answer

Some preparations needed beforehand (Work with a partner)

Since we do not already have a trained model, let’s build one.
You have to train the model with pictures of yourself and a few pictures of another person, so that it is able to recognize you (vs. "not you").
Later you can assume the trained model is available on your device.
User CurrentImage to take 3-4 pictures of yourself and 1 image of a classmate:
CurrentImage[]
Add the photos to make a comma-separated list as below:
photos={,,,,,}
Use the following code with FindFaces to isolate the face from the images:
FindFaces[#,"Image"]&/@photos

Here’s an example

FaceRecognize can train a machine learning classifier on the images of people’s faces:
FaceRecognize
example
1
name
1
,
example
2
name
2
,
For example, here is a model trained on 3 images of Thor and 1 image of Loki:
In[]:=
frModel=FaceRecognize
"Thor",
"Thor",
"Thor",
"Loki"
Out[]=
ClassifierFunction
Input type: Image
Classes: Loki,Thor
Let's test the model:
In[]:=
frModel
Out[]=
Thor
In[]:=
frModel
Out[]=
Loki
Test on an image of someone the model has not seen before:
In[]:=
frModel
Out[]=
Missing[Anomalous]

Now you try

Don’t forget to test

Test your model on your own image:
In[]:=
frModel[FindFaces[CurrentImage[],"Image"]]
Test your model on your classmate’s image:
frModel[FindFaces[CurrentImage[],"Image"]]
Test your model on someone's image that your model has not seen before:
frModel[FindFaces[CurrentImage[],"Image"]]

Actuate

Finally, we need to actuate parts of our system to make it interact with the environment.

Problem 4

What are some things that can be actuated on a laptop or mobile device? What kind of signals do we need to send to these parts?

Answer

I created the function “recognize” that will “speak” only if it is given your name within quotes. For any other input, it will play a sound.
In[]:=
recognize["Thor"]:=Speak["Hello Thor, welcome back to Asgard?"]
In[]:=
recognize[anyoneElse_]:=EmitSound[Sound[{SoundNote["G"],SoundNote["C"]}]]
Feed the output of frModel into recognize and see what it does:
In[]:=
recognizefrModel
In[]:=
recognizefrModel

Problem 5

Think of some other things the app could do in response to the input.

Answer

Part 2: Your Own Design

Now, you will design your own device that performs all four steps in a loop: Sense, Compute, Communicate, and Actuate. Think of such a device, and how it would work.

Problem 6:

What is this device? What would you name it? What does it do?

Answer:

Sense

Problem 7:

What kinds of inputs would your device need? Which sensors would it use?

Answer:

Compute

Problem 8:

What does your device need to compute? Try to be as precise as possible.

Answer:

Communicate

Problem 9:

What does your device need to communicate with? What kind of information does it need to send? How does it send this information?

Answer:

Actuate

Problem 10:

What parts of your device need to be actuated? What kind of signals do you need to send to these parts?

Answer:

Submitting your work

1
.
Publish your notebook
1
.
1
.
From the cloud notebook, click on “Publish” at the top right corner.
1
.
2
.
From the desktop notebook, use the menu option File -> Publish to Cloud
2
.
Copy the published link
3
.
Add it to the top of the notebook, below your netID
4
.
Print to PDF
5
.
Upload to Gradescope
6
.
Just to be sure, maybe ping your TA Sattwik on Slack that you have submitted.
Wolfram Cloud

You are using a browser not supported by the Wolfram Cloud

Supported browsers include recent versions of Chrome, Edge, Firefox and Safari.


I understand and wish to continue anyway »

You are using a browser not supported by the Wolfram Cloud. Supported browsers include recent versions of Chrome, Edge, Firefox and Safari.