ee3682011spring_guestlecture_part2
Post on 04-Dec-2015
215 Views
Preview:
DESCRIPTION
TRANSCRIPT
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 1
Mobile image processing – Part 2
Mobile application development on AndroidProject 1: “Hello Image”, reading/writing/modifying imagesProject 2: “Viewfinder EE368”, processing viewfinder framesProject 3: “CVCamera”, utilizing OpenCV functionsMobile application examples
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 2
Recognizing video at a glance
(1) User snaps a photo of screen.
(2) Our system identifies video andframe within the video.
(3) User resumes video on the phone.
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 3
http://www.stanford.edu/~dmchen/mvs.html
Recognizing video at a glance
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 4
Mobile device versus personal computer
Feature Motorola DROID Typical PCScreen size 4.6 in × 2.4 in 25 in × 16 in
Processor speed 550 MHz 3.0 GHzRAM 256 MB 4 GB
Permanent storage 133 MB internal8 GB external flash 500 GB disk
Internet access Typically 3G orWiFi at hotspots WiFi or wired Ethernet
Telephony Core feature Supplementary feature
Camera 5 MP camera embedded External webcam
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 5
“Viewfinder EE368” project
Goals of this projectLearn how to access frames from the viewfinderLearn how to modify camera parametersLearn how to augment the viewfinder frames
Step-by-step instructions available in Tutorial #1:http://ee368.stanford.edu/Android
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 6
Android manifest file
…
…<application android:icon="@drawable/icon"
android:label="@string/app_name"><activity android:name=".ViewfinderEE368"
android:label="@string/app_name"android:screenOrientation="landscape“android:theme="@android:style/Theme.NoTitleBar">
<intent-filter><action android:name="android.intent.action.MAIN" /><category android:name="android.intent.category.LAUNCHER" />
</intent-filter></activity></application>
<uses-sdk android:minSdkVersion=“7" /><uses-permission android:name="android.permission.CAMERA"/>
Set the proper hardware permissions
Landscape mode without title bar
Main Activity
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 7
Class hierarchy for “Viewfinder EE368”
Viewfinder EE368
(Activity)
Preview(View)
Draw on Top
(View)
Manage two views and manage program open/close
Handle incoming viewfinder frames and adjust camera parameters
Augment viewfinder frames with a new layer of information on top
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 8
Viewfinder class: full screen mode
Viewfinder EE368
Icon and title bars visible Icon and title bars hidden
public void onCreate(Bundle savedInstanceState) {...getWindow().setFlags(
WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN
);requestWindowFeature(Window.FEATURE_NO_TITLE);...
}
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 9
Class hierarchy for “Viewfinder EE368”
Viewfinder EE368
(Activity)
Preview(View)
Draw on Top
(View)
Manage two views and supervise program open/close
Handle incoming viewfinder frames and adjust camera parameters
Augment viewfinder frames with a new layer of information on top
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 10
Timeline of events on the mobile device
Time
New frame New frame
Capture
Augment
Preview
DrawOnTop
Capture
Augment
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 11
Preview class: frames go down two paths
Preview(View)
Viewfinder frames
…
Display on phone screen
Forward to custom callback function
myCamera.setPreviewCallback(new PreviewCallback() {public void onPreviewFrame(byte[] data, Camera camera){ Pass data to DrawOnTop class }
});
Data in YCbCr 4:2:0 formatw
h
w/2 w/2
h/2
h/2
YCb Cr
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 12
Preview class: set camera parameters
Preview(View)
Camera.Parameters parameters = myCamera.getParameters();parameters.setPreviewSize(640, 480);parameters.setPreviewFrameRate(15); parameters.setSceneMode(Camera.Parameters.SCENE_MODE_NIGHT); parameters.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);myCamera.setParameters(parameters);
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 13
Class hierarchy for “Viewfinder EE368”
Viewfinder EE368
(Activity)
Preview(View)
Draw on Top
(View)
Manage two views and manage program open/close
Handle incoming viewfinder frames and adjust camera parameters
Augment viewfinder frames with a new layer of information on top
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 14
DrawOnTop class: YCbCr to RGB conversion
84.27625641.516
25608.298
58.13525612.208
25629.100
25608.298
921.22225658.408
25608.298
−⋅
+⋅
=
+⋅
−⋅
−⋅
=
−⋅
+⋅
=
CbYB
CrCbYG
CrYR
w
h
w/2 w/2
h/2
h/2
YCb Cr
w
h
RGB
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 15
DrawOnTop class: create some paint brushes
myPaintRed = new Paint();myPaintRed.setStyle(Paint.Style.FILL);myPaintRed.setColor(Color.RED);myPaintRed.setTextSize(25);
myPaintGreen = new Paint();myPaintGreen.setStyle(Paint.Style.FILL);myPaintGreen.setColor(Color.GREEN);myPaintGreen.setTextSize(25);
myPaintBlue = new Paint();myPaintBlue.setStyle(Paint.Style.FILL);myPaintBlue.setColor(Color.BLUE);myPaintBlue.setTextSize(25);
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 16
DrawOnTop class: draw histogram bars
Bin 0 Bin 1 Bin 2 Bin 3 Bin 4
// Draw red rectangleRect rect = new Rect(left x, top y,
right x, bottom y);myCanvas.drawRect(rect, myPaintRed);
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 17
Viewfinder EE368 demo
http://ee368.stanford.edu/Android
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 18
Real-time debugging using DDMS (1)
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 19
Real-time debugging using DDMS (2)
File system on device
Messages on the device
0123456789ABC
Device
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 20
Taking a screenshot of the phone (1)
0123456789ABC
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 21
Taking a screenshot of the phone (2)
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 22
Class Project: Mobile Image Completion
H. Wang and A. Lin, Spring 2010http://ee368.stanford.edu/Project_10
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 23
Class Project: Mobile HDR Imaging
J. Mathe and T. Wong, Spring 2010http://ee368.stanford.edu/Project_10
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 24
Mobile image processing – Part 2
Mobile application development on AndroidProject 1: “Hello Image”, reading/writing/modifying imagesProject 2: “Viewfinder EE368”, processing viewfinder framesProject 3: “CVCamera”, utilizing OpenCV functionsMobile application examples
top related