Part 3: Future

Sense-Compute-Communicate-Actuate

Part 1

Technologies of Today and Near Future

Many technologies share this theme of a Sense-Compute-Communicate-Actuate loop.

Sense

Use sensors to pick up signals - get information about the environment

Compute

Use computation using that information

Communicate

Send results of computation (or some of the sensed information) to other systems or to humans

Actuate

Take some action: physically move (with help of motors) or send a signal

Loop

Go through the four-step process again and again

Today’s topic: Sensing

The Loop in Humans

Sense

◼
  • see
    • ◼
  • hear
    • ◼
  • touch
    • ◼
  • smell
    • ◼
  • taste

    • Compute

    In the brain
    ◼
  • Visual cortex
    • ◼
  • Auditory cortex
    • ◼
  • Sensory cortex

    • Communicate

    ◼
  • Facial expression
    • ◼
  • Gestures
    • ◼
  • Voice

    • Actuate

    ◼
  • Muscular and skeletal movements

    • The Loop in a Machine: Drone Delivery Systems

    Sense

    ◼
  • GPS location

    • Compute

    ◼
  • Process sensed data
    • ◼
  • Decisions based on the results

    • Communicate

    ◼
  • with other drone systems
    • ◼
  • base camp
    • ◼
  • destination

    • Actuate

    ◼
  • Control rotors to land safely or hover
    • ◼
  • Deposit package

    • The Loop in a Machine: Autonomous Vehicles

    Sense

    ◼
  • Camera; LIDAR
    • ◼
  • GPS
    • ◼
  • others

    • Compute

    ◼
  • Processing video and images
    • ◼
  • Location information
    • ◼
  • Audio

    • Communicate

    ◼
  • Communicate with cloud
    • ◼
  • Communicate with human

    • Actuate

    ◼
  • Turn wheels
    • ◼
  • Speed up
    • ◼
  • Slow down
    • ◼
  • Basically drive the car

    • The Loop in a Machine: Digital Assistants

    E.g. Amazon Alexa

    Sense

    Wake up: “Alexa”, “Hey Siri”,”Hey Google”

    Compute

    Matching signals

    Communicate

    Communicate with the database, cloud, information on the web.

    Actuate

    Turn on screen, speaker, light

    ​

    The Loop in a Machine: Smart Treadmill

    Sense

    Motion sensor, microphone, medical sensors

    Compute

    To interpret voice commands, suggest exercise routine based on information collected, find music matching your speed and gait

    Communicate

    Record of previous workout, music database, medical data lookup

    Actuate

    Stop, slow down, incline, change speed

    The Loop in a Machine: Cleaning Robots

    Sense

    Sense obstacles, sense motion, camera

    Compute

    Distance to obstacle, direction of motion

    Communicate

    Among multiple cleaning robots

    Actuate

    Move, change direction, turn vacuum on or off
    SLAM: Simultaneous Localization and Mapping

    Sensors

    Camera: IR, thermal, LIDAR, RADAR

    Microphones: audible, ultrasound

    IMU (Inertial Measurement Unit): accelerometer, gyroscope, magnetometer

    Wireless: GPS, WiFi, Radio signals

    Assorted: Pressure, humidity, proximity, temperature, chemical traces

    The Humble Camera

    Source: Khan Academy course “Pixar in a Box”
    ◼
  • The Pinhole Camera: https://youtu.be/jhBC39xZVnw
    • ◼
  • Camera Lenses: https://youtu.be/Q5Jat06VM6o

    • The Smartphone Camera

    Source: Structure and Imaging Principles of a Smartphone Camera
    ◼
  • Article: https://www.utmel.com/blog/categories/optoelectronics/structure-and-imaging-principle-of-smartphone-camera
    • ◼
  • Video: https://youtu.be/fkG3D0WXRAw