Automatic Camera Alignment • The current manual PTR Alignment of Halo camera

configurations take too long, complex, and prone to errors. The PTR assembly is the only mechanical component in Halo rooms that contains a source of variability- (backlash of gears when changing directions is the source of the variability). By creating an image processing API that can be utilized by existing client services, the manual PTR alignment process can be replaced with an automated sequence. The alignment process can be speedup, human errors can be eliminated, complexity reduced and a method for periodic alignment validation can be achieved. An automated process would achieve world-wide repeatability and consistency resulting in a continuous quality view for multipoint connections and thereby, reduce the need for a human resource to visit a room

• Project Objective Statement (POS):– By identifying the table arc with respect to an

established location in the view, iteratively move the Pan, Tilt, and Rotate motors in increments to achieve the continuous table line in the array of monitors. Use the automated inspection before alignment to determine if view is within specification and then once again after to verify that alignment meets specification. Perform an automated PTR alignment with the aid of table targets by initializing a PTR sequence that aligns center view then left and right views to establish a continuous table line in array of monitors. Calibration PTR values will be automatically stored after successful completion of the automated sequence and retrieved and applied at request for initialization if required.

• Technology Assessment:– Leveraged Technology:– Existing PTR macros. – Room services; Device Service, Device Manager, and

Database Service.– Jeff Thielman’s image capture hardware for MatLab

image capture and image processing and remote calibration.

– New Technology:– An image processing application and API that replaces

graphic overlays and human guidance to position Halo camera FOV for 2person-2point, 2person-multipoint, and 4person- multipoint

• Major Deliverables:• Image processing API for Device Manager Client application to accomplish (2) key operational

functions.– Operational Phase 1: Overnight verification: An ability to run an application from the calibration studio to

perform a semi automated alignment check to verify if views are within alignment specification.– Operational Phase 2: PTR operations check: An ability to run an application from the calibration studio to

perform semi automated selective moves of PTR stages in each direction to determine if PTR stages are functional and operational.

– Automated sequence: Based upon image processing API.• Metrics and Goals:• Report position in pixel coordinates of known reference feature to base PTR moves• Measurement of known feature to establish magnification/ zoom • Reference saved values for positioning and comparing against ‘golden standard’• Sequence of operation for 2P-2P, 2P-multipoint, and 4P-multipoint alignment that provides;

– Global repeatability of adjustments for installation and support teams.– Reduce time for alignment from 2 hrs to 10 min.– Reduce errors made by operators- – Generate new values to be saved for the calibrated view.– Eliminate additional instrument interfaces.– Reduce errors caused by mental gymnastics now being preformed.

Automatic camera alignment

2-person 2-point

Starting PointStarting Point Starting Point

2-person Multi -point

Pan to Center Pan to Center

Same

2-person Multi -point

Zoom OutZoom Out Zoom Out

Tilt

Pan

Rotate

Same ChairSame Chair

A AB B

A Height = A HeightB Height = B Height

??

??

2-person Multi -point

ZOOM

????

2-person Multi-point

Center View is duplicated for

multi-point

Same Chair

Same Chair

A B

A Height = A HeightB Height = B Height

? ?

4-person Multi -point

ZOOM

????

4-person Multi-point

?B A?

Wide Angle Zoom duplicated multiple times.

Step 1 Encoder Limits4 Person Multi-Point

Center Camera1. Full Zoom Out2. Pan to Left Limit3. Get PTRZ Motor Encoder Count (Left X Limit)4. Pan to Right Limit5. Get PTRZ Motor Encoder Count (Right X Limit)6. Tilt to Upper Limit (Top Y Limit)7. Get PTRZ Motor Encoder Count (Top Y Limit)8. Tilt to Lower Limit9. Get PTRZ Motor Encoder Count (Bottom Y Limit)10. Move to a rough Y level (probably below Y Center), and Center X.11. Roll CW to Limit12. Get PTRZ Motor Encoder Count (+Roll Limit)13. Roll CCW to Limit14. Get PTRZ Motor Encoder Count (-Roll Limit)15. Move to Center Roll

Calibrate Center Camera Roll and Tilt1. Capture Video Image2. Evaluate for Table edge intersection at frame edges.3. Output 2 Values: Y-Left and Y-Right

Delta of Y-Left – Y Right

+Delta- DeltaRoll 1

Encoder Count CW

Roll 1 Encoder Count CCW

Store Data for later Pixel to Encoder Roll/Tilt Co-efficients

0 Delta(or Minimized)

Delta of Y Values from

Target Y

Tilt 1 Encoder Count Up

Tilt 1 Encoder Count Down

TooHigh

TooLow

Target Y Reached

First Round Gets rough Encoder to Pixel Ratio, subsequent rounds should be able to calculate Target encoder Values (rather than move 1 encoder count at a time) And refine co-efficient accuracy. May need to start with more than 1 encoder count if more is required to register a pixel change.

Calibrate Center Camera Pan1. Pan to Left Limit - 12. Capture Video Image3. Evaluate for Left Table Corner (Left X Intersect)4. Pan to Right Limit - 15. Capture Video Image6. Evaluate for Right Table Corner (Right X Intersect)7. Output 2 values (Left X and Right X)

Store Data for later Pixel to Encoder Pan Co-efficients

Delta X-Pixel Left-Right

Pan Left 1 Encoder Count

Pan Right 1 Encoder

Count

+X-X

Re-Verify Tilt and Roll with Correct Center Pan Position

0 Delta(or Minimized)

Tilt/Roll Video Capture

19119

1

Left Y Pixels

Right Y Pixels

Left Pan Video Capture

305

Left X Pixels

Right Pan Video Capture

305

Right X Pixels

Step 2 Roll/Tilt Calibration

Step 3 Pan Calibration

Store Calibration Points in Database

Software approach

ImageProcessing

Requests to retreive preset PTR values

and requests to Store PTR values

for room

Interface to all devices. Service ran

on room server .

Macros of pre-defined

manufactures commands sent

from Device Services for

individual moves .

Controlling application . Socket interface to device services and Data

base Services . Main GUI

Feature Finding -high want- MatLab

that can convert *.m files to Java classes that can be called

from Client application

Remote Room

Calibration StudioICE

ICE

Macro

ICE Wrapper

PTR

DeviceServices

ICE Wrapper

Data BaseServices

Client Application

ICE Wrapper

Java Builder: .m file to .jar files-call java

class

Software approach

Data BaseServices

DeviceServices

Client Application

Requests to retreive preset PTR values

and requests to Store PTR values

for room

Macros of pre -defined

manufactures commands sent

from Device Services for

individual moves.

Controlling application . Socket interface to device services and Data

base Services. Main GUI

Feature Finding -high want- MatLab

that can convert *.m files to Java classes that can be called

from Client application

Remote RoomCalibration Studio

TCP

TCP

PTR

ImageProcessing

Macro

TCP

• Software:• Device Services runs on Room Service and waits for

a client to connect over ICE (Internet Communication Environment). One such device client is a java application derived from Kahuna called; Device Manager for the calibration studio, following this model, our own client application will control the sequencing of the image processing and PTR moves, our client application will also be a client for the Database Services that also runs on the Room Service that waits for database reads and writes from a client application.

• One a call is made into device services the call is translated into ‘macro’ for the device service. The current Device manager makes calls into devices services with macros that contain values. For example, the client application, Device Manager; also integrates control of other instruments in calibration studio such as the pattern generator for graphic overlays that aid the PTR alignment.

• PTR’s firmware is flashed on the PTR stages that contain manufactures instructions for the appropriate moves- this is not very flexible and cannot be changed very easily even though the PTRs can be remotely programmed. Any requirement that suggests changing existing PTR macros should be avoided to ensure backward compatibility for the installed base. The PTR macros were created to minimize the amount of data sent from the device service to the PTR controller.

• Each room has its own device list- we will simply add to the device list. The Device Services can communicate through TCP –this will enable our client application to use windows sockets and TCP as the protocol to interface our client app to the Device Service avoiding ICE, which is a 3rd party software layer, that is more complex and requires more administration to implement. The Device model that we will follow will exist of an I/O protocol, macros, and a command stock.

Calibration Studio

ICE

Client Application

ICE Wrapper

Java Builder: .m file to .jar files-call java

class

ImageProcessing

C/C++/

ICE Wrapper

Calibration Studio

Client Application

ImageProcessing/

Java or MATLAB

ICETCP

TCP