mechatronics with machine learning (+ deep...

Mechatronics with Machine Learning

04/10/2017

Prof. Seungchul Lee

(+ Deep Learning)

Introduction

• Since 2013 July: UNIST

– iSystems Design Lab.

• 2010, Ph.D. from the University of Michigan, Ann Arbor

– S. M. Wu Manufacturing Research Center

– The Center of Intelligent Maintenance Systems (IMS)

• 2007, M.S. from the University of Michigan, Ann Arbor

• 2005, B.S. of Electrical Engineering from Seoul National University

• 2001, B.S. of Mechanical Engineering from Seoul National University

2

My Research Areas

• Mechatronics (mechanics + electronics)– Undergraduate can do this with fun

– Robotics

– Hardware + coding

– Circuit + digital logic + signal processing + control

• Machine Learning and Deep Learning– A level of graduate study

– New research field for mechanical engineering

– Will bring lots of potentials

– Math + coding

3

Today

• Physical Computing

• Kinematics in Robotics

• Parallel Manipulator (Delta Robots)

• Dynamics and Control

• Cyber-Physical Systems (CPS)

• Signal Processing

• Intelligence with Machine Learning

• Deep Learning

• Epilog

4

Arduino and Raspberry Pi

• Open-source platform used for building electronics projects

• Micro-controller/micro-computer

• Physical computing

• Easy to use even for non-professionals

• Maker Faire

5

Physical Computing: Sensors and Actuators

• Sensor: allow the microcontroller to receive information about the environment

• Actuator: devices that cause something to happen in the physical world

6

Flex Sensor and Servo Motor

7

Sensors + Actuators + Microcontroller

8

3D Printed Prototype

Today

• Physical Computing

• Kinematics in Robotics

• Parallel Manipulator (Delta Robots)

• Dynamics and Control

• Cyber-Physical Systems (CPS)

• Signal Processing

• Intelligence with Machine Learning

• Deep Learning

• Epilog

9

(Industrial) Robot Arms

10

Robot Arms

11

1 2 3

Kinematics

12

• “Introduction to Robotics” Course in Mechanical Engineering

Robot Playing Piano

• Inverse kinematics

13

Today

• Physical Computing

• Kinematics in Robotics

• Parallel Manipulator (Delta Robots)

• Dynamics and Control

• Cyber-Physical Systems (CPS)

• Signal Processing

• Intelligence with Machine Learning

• Deep Learning

• Epilog

14

Delta Robot

15

Inverse Kinematics of Parallel Manipulator

16

work space

Dancing Robot with Signal Processing

• Dancing with music (rule-based and -programmed)

• Signal processing (FFT)

17

Multi-Agent System

18

• Wireless network

• Internet of Things

• Control strategy

• Cooperation

• 국립부산과학관전시예정

Mobility: Mecanum Wheel

19

Mobile Delta Robot

20

Idol Robot Group (?)

21

• Korean idol girl groups

• Dance with changing formation

• Mobile: position and orientation control

• Localization + Robot Navigation (→ self driving car)

Robot and Light

22

23

Today

• Physical Computing

• Kinematics in Robotics

• Parallel Manipulator (Delta Robots)

• Dynamics and Control

• Cyber-Physical Systems (CPS)

• Signal Processing

• Intelligence with Machine Learning

• Deep Learning

• Epilog

24

Control: PID

• Require more advanced system design

• From open-loop to closed-loop systems

• 코딩으로어떻게구현하는지는대부분모른다

25

Inverted Pendulum

26

• A famous control problem

• Feedback control: – angle from encoder command to servo motor

Ball and Plate

27

• Position Feedback: vision in the loop– Location identification via Computer Vision

Drone and Dynamics

• Quadcopter

28

(Simpler) Unicopter

• Angle from encoder

• DC motor

Drone Control

30

• Drifting Issue: Hovering and Tracking via vision

• Aerial Surveillance & Monitoring, Observation with Drone

Reinforcement Learning

• Require more advanced control scheme– State space representation

– Optimized control scheme (LQR)

– Deep learning (reinforcement learning): totally different approach

31

Today

• Physical Computing

• Kinematics in Robotics

• Parallel Manipulator (Delta Robots)

• Dynamics and Control

• Cyber-Physical Systems (CPS)

• Signal Processing

• Intelligence with Machine Learning

• Deep Learning

• Epilog

32

New Concept: Cyber-Physical System (CPS)

• Physical and virtual components are deeply intertwined

33

Augmented Reality (AR) and Virtual Reality (VR)

34

35

(Big) Data InformationIntelligence

• Signal Processing (DSP) • Data Analytics

• Machine Learning and Deep Learning

Machines

Today

• Physical Computing

• Kinematics in Robotics

• Parallel Manipulator (Delta Robots)

• Dynamics and Control

• Cyber-Physical Systems (CPS)

• Signal Processing

• Intelligence with Machine Learning

• Deep Learning

• Epilog

36

Noise Reduction (Smoothing)

37

Detection of Abrupt Changes

38

Vibration Issue on SRT High Speed Train

39

Vibration Issue on SRT High Speed Train

40

Measured Data: SRT

41

Measured Data: KTX

42

Kalman Filter

43

Today

• Physical Computing

• Kinematics in Robotics

• Parallel Manipulator (Delta Robots)

• Dynamics and Control

• Cyber-Physical Systems (CPS)

• Signal Processing

• Intelligence with Machine Learning

• Deep Learning

• Epilog

44

Robot Playing Piano

45

Robot Playing Piano

46

How to Make Robots Intelligent

• It was programmed by users

• From music notes to autonomously playing– Artificial Intelligence from Computer Science

47

Perception with deep learning

Machine Learning for Machine Intelligence

48

Classification

Regression

Clustering Dim reduction

Deep Learning for Machine Intelligence

49

CNN

RNN

Web-based Monitoring Dashboard

50

Probability of Classification

normal

unbalance

misalignment

Motor in Robot

• Load (anomaly) generation– Anomaly is induced through manual press

51

Demo for Unsupervised Learning

52

Th

reshold

Anomaly Detection

53

normal abnormal

Anomaly Detection

54

normal abnormal

Data Science and Analytics in Engineering

• Industrial 4.0

55

Plant and facility Manufacturing

Today

• Physical Computing

• Kinematics in Robotics

• Parallel Manipulator (Delta Robots)

• Dynamics and Control

• Cyber-Physical Systems (CPS)

• Signal Processing

• Intelligence with Machine Learning

• Deep Learning

• Epilog

56

Convolutional Neural Networks (CNN)

• 이미지분류에서높은성능을보인 CNN 기법을진동신호기반결함진단에사용제안

57

Training Data

Feature Extraction Classification

DiagnosticsDeep Learning

CNN on STFT Image

• 기본 CNN구조를활용하기위하여신호를이미지화 (STFT spectrogram)

58

STFT image

signal

Dimension Reduction

• Principal Component Analysis (PCA)– Dim reduction without losing too much information

59

u1

u2

1

1

2

xu

x

Dimension Reduction

• PCA in time signals

60

61

Provide compressed representations

Deep Learning: Autoencoder

Tutorials on Deep Learning

62

Today

• Physical Computing

• Kinematics in Robotics

• Parallel Manipulator (Delta Robots)

• Dynamics and Control

• Cyber-Physical Systems (CPS)

• Signal Processing

• Intelligence with Machine Learning

• Deep Learning

• Epilog

63

Failure, Success and the drive to keep Creating

64

Failure, Success and the drive to keep Creating

65

Version 1 Version 2

http://isystems.unist.ac.kr/

All materials (codes + hardware design) are available

66