new d.u.s.t.: displays using swarming technology wireless senior...

1

D.U.S.T.: Displays Using Swarming Technology

Wireless Senior Capstone Design

Group Members:

Eric Love

Oliver Solano

Jens Taylor

Date: May 1, 2009

2

Table of Contents

I. Abstract ..................................................................................................... 4

II. Motivation ................................................................................................ 5

III. Design Goals ........................................................................................... 6

IV. Software Application ............................................................................. 7

1. Introduction to Message Types and Data Structures……………...8

2. Hardware Emulation Layer ……………………………………..…9

3. Swarm Algorithms……………………………………………….…11

4. Higher-Level Application…………………………………………..14

V. Hardware Design ................................................................................... 15

1. Hardware Design Introduction……………………………..….…...15

2. IrDA Protocol………………………………………………..….…...16

3. Board Design……………………………………………..…….……17

4. Sensing of 8eighboring Arrays Protocol…………………….…….20

5. Transmitting and Reading of I.P. Address and Commands…….22

6. 16 bit Timer/Counters 1 and 3…………………………..…….…...24

VI. System/Driver ....................................................................................... 25

VII. Design Problems Encountered and Solutions .................................. 28

1. Infrared board Implementation…………………………………….29

2. Infrared Signal Reflection……………………………………….….30

3. Infrared Reading Bytes……………………………………………..30

3

VIII. Future Considerations ..................................................................... 31

IX. Conclusion ............................................................................................ 32

References ....................................................................................................... 33

Appendix A: Microcontroller Register and Port Uses .................................... 34

Appendix B: Application Message Definitions .............................................. 35

Appendix C: Examples of Swarming Algorithms........................................... 37

Appendix D: Design Photos ............................................................................ 41

Appendix E: Project Code ............................................................................... 45

4

I. Abstract

Current display devices are used almost exclusively as distinct objects. Televisions, computer

monitors, and mobile phones are all meant to be used standalone, without any interaction with

other displays of the same or different type. Multi-device displays are generally only used to

create large screens in corporate or scientific settings, and even then mostly for show, not use.

Where usable solutions for consumers do exist, they are generally implemented in software with

no physical knowledge of position, orientation, or size; without such knowledge, the uses are

limited. Creating a physical device to allow for such knowledge enables displays to interact in

interesting, intuitive, and very useful ways.

The overall objective of the design was to implement a wireless network of displays capable

of knowing the aforementioned physical properties. Using four infrared sensors, one facing each

direction about the display, and a microcontroller containing the logic to manipulate them, such a

device was created: the sensor array.

Adding sensor arrays to a set of laptops allowed for unique interactions between the devices.

An application was created to scale an image over multiple displays when brought adjacent to

one-another. Using just position, a user could choose how large and in what orientation to scale

or stretch an image. While not necessarily useful for a series of large laptops, the ideal display

technology for use with this application would be electronic paper, a display technology

designed to mimic the appearance of ordinary ink on paper used in many electronic readers.

Several other applications were envisioned to increase the utility and usability of such an

electronic reading device. For instance, being able to transfer data wirelessly between devices

could be easily achieved using the sensor array. Viewing multiple pages of the same document

side-by-side is likely an immensely useful tool and is also easily achieved using the sensor array

design. For more active displays such as televisions or computer monitors, one could imagine

playing video over several displays in a simple, yet powerful way.

All applications of the sensor array, as well as the array itself, were designed to be as easy to

use as possible, utilizing automatic configuration, position based commands, and reliable data

transfer. The learning curve for first time users is minimal, making this technology an extremely

powerful contender for real-world implementation.

5

II. Motivation

All of civilization is built upon human interactions with one-another. Both work and play

focus on interacting with other individuals, whether it is business meetings, regular daily

proceedings at the office, team sports, or even playing video games. Technology, as it becomes

ever more mature, caters increasingly toward this ideal of bringing people together as well. Each

new release of Microsoft Windows, for instance, has included more and more features for

sharing photos and videos with friends, communicating with distant family, friends, or

colleagues and, of course, accessing the Internet. Mobile phones, which have little more to do

with than allowing people to communicate, have exploded in popularity in recent years.

Obviously, there is a large market for devices capable of bringing people together. But it is

not only important to allow people to interact; a key design goal in every device made for this

purpose is simplicity and ease of use. Allowing users to easily bring multiple devices together to

interact could increase both human-human interaction and the utility of the devices involved.

Imagine exchanging phone numbers by tapping phones together, or sharing a document in a

business meeting by simply placing two mobile devices next to each other. Large screen

televisions could be made of smaller, much more manageable blocks; using electronic reading

devices, multiple pages of a document could be compared side-by-side or used as a reference by

simply laying them near each other. The utility for allowing existing technologies to easily

communicate and share is nearly endless.

6

III. Design Goals

Successful completion of the design project was contingent upon completing a set of three

goals. First, a hardware system was to be designed to allow multiple rectangular displays of the

same size to communicate when placed next to one-another. It was required that each display be

capable of communicating with all four neighbors adjacent to itself, as shown in Fig. 1. Second,

the hardware was to be interfaced with a laptop computer to allow access to its functionality. The

interface, or device driver, would be required to communicate between programs on the

computer and the hardware's microcontroller. Third, applications were to be developed to

leverage the new hardware capabilities of the laptop. The applications should be easy to use and

unique to the newly designed hardware.

Fig. 1: Swarm Layout with Four Neighbors.

7

IV. Software Applications

Several application ideas were conceived to demo the constructed hardware using a laptop

computer. These ideas included scaling images and video over multiple screens, showing several

pages of a PDF document side-by-side, and effortlessly sharing documents and images with

other computers using only physical motions. Ultimately, a demo application for scaling images

over several screens was designed and implemented for use with any sized rectangular

configuration of screens.

The software application written for the laptops was designed with three separate logical

layers. First, the lowest software layer was designed as a driver and hardware emulation layer

providing functionality for communication with both the actual sensor array hardware and other

laptops in the swarm. Originally written as a temporary means of testing the higher levels of

software, the hardware emulation layer remained in use in the final project due to limitations in

the sensor array software. Second, a middle layer consisting of a series of algorithms provided

functionality related to the swarm layout and distributing messages accordingly. Third, the

highest layer consisted of the actual image scaling application, handling loading and scaling

images according to the swarm layout provided to it. Additional high-level applications could be

created to replace the image scaling with other features.

8

1. Introduction to Message Types and Data Structures

Six different message types were created for passing information around the swarm: the

incoming connection message, hello message, disconnect message, topology update message,

request update message, and display update message. A brief explanation of each message type

follows, with more detailed implementation information listed in Appendix B.

The incoming connection message is generated by the sensor array hardware upon receipt

of a valid connection message from another device. It signals the application that a verified

connection has been received, and that the application should now begin its own connection

initiation.

A hello message is a special message used only by the hardware emulation layer when

connecting to another computer using sockets. The message is used to indicate which physical

side – left, right, top, or bottom – the socket is connecting from.

A disconnect message is generated by the sensor array hardware upon noticing that a

neighboring device is no longer connected. It signals the application to remove the

corresponding device from the swarm.

A topology update message is generated by the swarm algorithmic part of the software, and

is used to indicate the swarm layout as seen by the sending device. These messages are sent

upon receipt of a new connection message to inform the newly connected device of the layout.

Swarm layout information is contained in a matrix-like structure, with an additional vector

indicating the current device’s position relative to the top-left corner.

A request update message is sent to request a display update, which contains display data to

be shown by the members of the swarm.

9

2. Hardware Emulation Layer

The hardware emulation layer of the software, written in the Python scripting language in

the files 'fakedriver.py' and 'DriverInterface.py' listed in Appendix D, uses a combination of

named pipes, TCP/IP sockets, and the driver interface provided by Linux to access the sensor

array hardware. As mentioned previously, the hardware emulation layer was originally written

as a temporary testing platform for the higher-level software functionality as shown in the

block diagram in Fig. 2. As such, it was designed to mimic the driver interface provided by

Linux to the higher level application using two named pipes, one for reading and one for

writing; these pipes were used to pass messages between the higher-level application and the

emulated hardware.

Fig. 2: Hardware Emulation and High Level Application Block Diagram.

The sensor array hardware was emulated as a set of four point-to-point connections using

TCP/IP sockets between the computer and its neighbors; one socket was used for each of the

four directions, with an additional socket being used as a connection reference on a fixed port

for each laptop. A special message type, the hello message, was created to indicate which side

an incoming socket connection was physically connecting from, since an all-software

implementation has no reference to real-world orientation. The sockets were then used to

forward all network messages from the higher-level application to the adjacent computers in

the swarm.

Two types of network messages could be created in the hardware emulation layer,

corresponding to the two types of messages created by the sensor array software. First, an

10

incoming connection message could be generated upon receipt of the hello message from an

incoming socket connection. This message would be sent to the higher-layer application to

indicate a new connection on a given side. Second, a disconnect message could be generated

upon discovery of a closed, or disconnected, socket from a neighbor. This message would be

sent to the higher-layer application to indicate that a neighbor has moved away, and is no

longer part of the swarm. It is important to note that these messages would only be sent to the

local application, not over the sockets to other devices; each sensor array (and emulator)

generates network messages only for its own device, leaving the higher-level application to

handle generating messages to be sent between devices.

Upon completion of the sensor array hardware and device driver, the hardware emulation

layer was to be replaced completely by the new hardware. Unfortunately, the message types

used by the application required use of variable length data payloads, a feature not

implemented in the hardware due to time constraints. A solution using both the hardware and

hardware emulation layer was found, shown in Fig. 3. The sensor array hardware replaced the

connection and disconnection detection functionality of the emulation layer, while the

emulation layer continued to use sockets to transmit the messages from the higher-level

application. The emulation layer was provided the IP address to connect to via the incoming

connection message generated by the sensor array.

Fig. 3: Final Block Diagram - Hardware, Hardware Emulation, and High Level Application.

11

3. Swarm Algorithms

The second software layer is comprised of algorithms used to determine the swarm layout

and handle message generation and passing. The algorithms are implemented throughout

several files - 'epapergtk.py', 'rect.py', and 'topology.py' - listed in Appendix D, and consists of

many functions. The algorithms will be discussed in two categories, one dealing with the

swarm layout, and the other dealing with messages. Examples of layout algorithms are given in

Appendix C.

Swarm layout is affected by three major algorithms: adding devices to the swarm,

removing devices from the swarm, and finding the largest block containing only connected

devices in the swarm layout. To add devices to the swarm, a topology update message must be

received either from a newly connected device or from an already connected neighbor.

Topology update messages contain the swarm layout as seen by the sending device, and are

generated automatically upon receipt of an incoming connection message; topology update

messages are additionally forwarded to all devices in the swarm for updating to the new layout.

The information contained in a topology update must be added to the receiving device's layout

accordingly, since it may contain some overlap with the current layout. This is done in the

'topology.py' file, lines 94 – 144, in a series of three steps. First, the reference point of the

incoming layout information is adjusted to reflect the local device's position in the swarm

instead of the sender's; this is easily achieved because the local device knows which direction

the message was received from, so it can simply add or subtract one in the appropriate axis to

reflect this knowledge. Second, the local layout's size is adjusted to contain all points in the

received layout. This is done by checking the reference point and size of both layouts, then

adding rows and columns to the layout until the reference point and dimensions match or are

larger in the original. Once the local layout is adjusted to overlap properly with the received

layout, the third step is to combine the layouts using a logical OR. This ensures that devices

can only be added, not removed, from the swarm using the topology update message, a

necessary condition to avoid infinite loops when sending two contradictory messages - one to

add a device, one to remove the same device due possibly to outdated information - around the

swarm.

12

Removing devices from the swarm is achieved using a separate message type, the

disconnect message, and a completely different algorithm found on lines 148 – 300 of the file

‘topology.py’. The disconnection algorithm consists of six major steps. First, as with adding

devices to the layout, the reference point of the incoming disconnect message data is adjusted

to local device’s position in the swarm. Second, the actual disconnect is performed; the

disconnecting device is simply removed from the layout by changing the appropriate position

in the topology matrix to a zero. The next four steps involve reducing the topology matrix to

the smallest size containing all the connected devices in the swarm. The third step is to check if

the row containing the newly disconnected device contains any connected devices. If it does

not, then that row and everything to the opposite side of it from the local device can be

removed from the layout. Fourth, the same operation is conducted using the column of the

disconnecting device. Fifth, the left-right and right-left diagonals containing the disconnecting

device are checked for all disconnected devices. If either diagonal contains no connected

devices, it and everything to the opposite side of it from the local device will be removed.

Finally, the topology matrix will be reduced to the minimum size necessary to contain all

connected devices.

Finding the largest block of connected devices in a swarm is achieved using the algorithm

found on lines 9 – 116 of the file ‘rect.py’. To find the largest block of connected devices, the

algorithm searches the topology matrix for unconnected devices. When an unconnected device

is found, the matrix is split in half based on the row of the device, and again split based on the

column of the device, removing it from the resulting matrices. The matrices are then tested

themselves for any unconnected devices, and the removal operation repeated. If a matrix is

found to contain only connected devices, it is stored as a potential solution. When the entire

search space has been exhausted by subdividing all matrices until they are smaller than the

largest potential solution, the best solution will be chosen based on two criteria: area and

width/height ratio. The larger area a solution has, the better it is considered. In the event that

two solutions are found with the same area, the solution matrix with dimensions closer to a

square or wider than tall are chosen.

There are only two major messaging algorithms: sending messages point-to-point and

broadcasting. First, point-to-point messages are only supported between two adjacent,

13

connected devices in the swarm; messages cannot be passed to arbitrary members of the

swarm. For point-to-point messages, all that need be done is send a message to the driver

indicating which side to send the message to – top, bottom, left or right.

Broadcasting messages is achieved by sending a message out all sides, or optionally

sending the message out all sides except the side the message was received from. Whenever a

device receives a broadcast message, it will proceed to forward the message out all of its own

connections. Using just this method, a potential infinite loop of messages could be created, as

messages are passed back and forth between computers in the swarm. To avoid this situation,

the receiving device will first check if the message has been previously received, then proceed

to forward it only if it is a new message. This ensures that all devices in the swarm receive the

message and that the broadcast will eventually end.

14

4. Higher-Level Application

The higher-level demo application consisted of a relatively simple image segmentation and

scaling application. Using the topology information and largest block of connected devices, the

application is able to segment and scale an image based on its position in the block. If the

device is a member of the largest block, it will display a portion of the image relative its

position in the block and the block’s size, as demonstrated in Fig. 4. If the device is not located

within the largest block, it will set itself to a blank image, awaiting the connection of more

devices to increase the block size including itself.

Fig. 4: Image Scaling Over Largest Block.

Control of which images are to be displayed is relegated to the master device, determined

as the most top-left device of the currently largest block of all connected devices. Changing the

image on the device is as easy as dragging and dropping an image file from the file viewer onto

the program’s main window. The master device can force a change in the image being

displayed on all devices in the swarm using a display update message broadcast to all

connected devices. Newly connecting devices to the swarm can additionally request the proper

image to be displayed after connection by sending a request update message to their neighbor,

which will return an authoritative reply based on the master’s initial display update.

The process of image scaling itself is performed using a bilinear interpolation method to

preserve the relative quality of the scaled image. After scaling the image to a larger size by

moving each pixel in the image by the appropriate shift, the holes left by the translated pixels

are filled using bilinear, or two-dimensional linear filtering.

15

V. Hardware Design

The hardware portion of the project primarily focuses on the components of the device and

the programming of the microcontroller for sensing, transmitting and receiving data. These areas

are important in order for the device to operate appropriately according to the application

specification and must able to allow data transmission and reception from the application

through a driver. Proper consideration was made into the specific components used for the

project, implementation of the components on a board, efficient protocols to be used for sensing

neighboring arrays, high speed transmission, and accurate reading of received data between

sensor arrays.

1. Hardware Design Introduction

The hardware which makes up the device can be designed many different ways based on

the application of the device. Early into determining the possible way of implementing the

application three different designs were considered. The first design composed of having four

RFID tags at each side that would primarily be used for sensing, transmitting and receiving

I.D. information from the neighboring sensor array and a primary RF transceiver which would

be in charge of transmitting files to be scaled, copied or viewed on multiple pages. The second

design composed of having four infrared sensors at each side that would accommodate all the

necessary functions of sensing which side it is connected to, transmitting, and receiving of

files, I.D., and any other sort of information which need to be called upon from the neighboring

array. The third design composed of using inductive coupling for sensing neighboring arrays

and possibly a form of communicating the neighboring arrays identification and an RF

transceiver as the transmitter and receiver for communication between other sensor arrays.

After further investigation it was found that the use of infrared transceivers would be best

economically, simpler in design, and consumes less power as opposed to the first design and

only simpler in design and implementation than inductive coupling. The use of RFID tags

would be more difficult to implement because the components to be used are much more

expensive than infrared transceivers and still requires another RF transceiver for transmitting

larger files. The use of inductive coupling would be the best economically, takes minimal

power consumption but would be more difficult to implement given that not much was known

16

about its design and application to existing devices. It was decided that infrared sensing would

be designed and demonstrated for the project because it was the best way to show proof of

concept given the amount of time to complete the project as opposed to using inductive

coupling where more research was needed in order to use its functionalities efficiently and

effectively.

2. IrDA Protocol

The IrDA protocol is the protocol which defines the standard for different layers that allow

communication between most infrared devices operating from 9600 kbps to 4Mbps. The first

component of the physical layer applies to all infrared transceivers, where they all operate

under half duplex. Communicating half duplex means that transmitting and receiving cannot be

done simultaneously.

There are three IrDA physical layers stated in the IrDA protocol of which are dependent on

the infrared’s operating data rate. The design is operating at less than 115 kbps which means

the first of the three physical layers is used. The first layer involves Return to Zero Inverted

modulation (RZI) for encoding data. The RZI modulation operates through logic “0” sent to the

infrared transmitter and not logic “1”. After a logic “0” has been sent to the transmitter a light

pulse is sent with a duration of 3/16 of the bit clock while if a logic ”1” has been sent through

the infrared will not send a light pulse at all. The receiving infrared will in turn output data

according to the light pulses sent to it as well. Reading data formatted this way may cause

many problems and would involve more programming since the bits received are only clearly

indicated when a logic “0” is sent and not a logic “1”. To solve this problem the protocol calls

for an encoder/decoder which will encode the data properly for infrared transmission and

decode the receiving data to better bit format. In order for the layer to operate according to this

standard an infrared transceiver, the Fast Infrared Transceiver Module TFDU6300 from Vishay

Semiconductors, and an infrared encoder/decoder, the Infrared Encoder/Decoder MCP2122

from Microchip, was chosen for proper operation.

17

3. Board Design

Each prototype board primarily consists of 4 infrared transceivers, 4 infrared

encoder/decoders, and a microcontroller development kit. These components are connected

to 16 pin wire wrap sockets with resistors and capacitors that serve as bypass capacitors and

current limiters. The connection between each wire wrap socket was connected with wire

wrap, 30 AWG wire, using a wire wrap gun. The infrared devices, although small in pitch,

was soldered onto with the wire wrap for connection onto the infrared encoder/decoder. After

all the sockets were wire wrapped and soldered some apparatus was needed to keep the

protoboard steady and mountable onto the back of a laptop. A ¼ inch wooden board was

bought and cut according to the size of the protoboard where the corners of the wooden board

were drilled into. Threaded rods were inserted into the holes and kept secure with a hex nut

and washer. Then the board was also drilled into and inserted into the threaded rod secured

by a nut and washer again. The microcontroller was kept secure using a separate threaded rod

and secured onto the wooden board. Last a connection was needed between the

microcontroller and the protoboard socket. The 25 pin male and female connectors were used

by soldering 22 AWG wire into the pins of the connectors.

18

A. Design Specifications

Fast Infrared Transceiver Module TFDU6300

• Supports data rates from 2.4 kbps to 4 Mbps.

• Supply voltage of 2.4 V to 3.6 V.

• Wavelength of 850 nm to 900 nm

• 15 degree transmit angle

Infrared Encoder/Decoder MCP2122

• Supports up to 115.2 kbaud operation

AT90USBKey

• Supply voltage of 3.3 V

• 6 ports with 8 pins each that may operate as input/output

• 3 Timer/Counters, 2 –16-bit counters, 1-8 bit counter

• 8 MHz system clock

Device operates at 31.25 kbps with a 3.3 V supply voltage and a 500 kHz bit clock.

TFDU4

Inf rared Transceiv er

Vcc2 IRED Anode1

IRED Cathode2

TXD3

RXD4

SD5

Vcc16

NC7

GND8

0

C1

4.7uF

0

C2

0.1uF

R1

10

U7

AT90USBKEY PORTF

F0

F2

F4

F6

F8

F1

F3

F5

F7

F9

0

TFDU5


Vcc2 IRED Anode1

IRED Cathode2

TXD3

RXD4

SD5

Vcc16

NC7

GND8

0

C3

4.7uF

0

C4

0.1uF

R2

10

TFDU6


Vcc2 IRED Anode1

IRED Cathode2

TXD3

RXD4

SD5

Vcc16

NC7

GND8

0

C5

4.7uF

0

C6

0.1uF

R3

10

TFDU7


Vcc2 IRED Anode1

IRED Cathode2

TXD3

RXD4

SD5

Vcc16

NC7

GND8

0

C7

4.7uF

0

C8

0.1uF

R4

10

0

0

0

U5

AT90USBKEY PORTB

B0

1

B2

2

B4

3

B6

4

B8

5

B1

6

B3

7

B5

8

B7

9

B9

10

0

U6

AT90USBKEY PORTE

E0

E2

E4

E6

E8

E1

E3

E5

E7

E9

Part Ref erence

Inf rared Encoder/Decoder

16X CLK1

TX2

RX3

VSS4

RESET5

RXIR6

TXIR7

VDD8

Part Ref erence


16X CLK1

TX2

RX3

VSS4

RESET5

RXIR6

TXIR7

VDD8

Part Ref erence


16X CLK1

TX2

RX3

VSS4

RESET5

RXIR6

TXIR7

VDD8

Part Ref erence


16X CLK1

TX2

RX3

VSS4

RESET5

RXIR6

TXIR7

VDD8

Fig. 5: Design Schematic.

19

B. Design Parts List

Table 1: Design Parts List.

Component Manufacturer Quantity Price

Break

One Array

Prototype

Four Arrays

Prototype

Microcontroller

(AT90USB1287)

Development Kit

Atmel 1 29.99 1 4

Infrared

Encoder/Decoder

Microchip

Technology

1 1.23

0.70

4 16

2500

Infrared Transceiver

Module

(Side View Surface

Mounting)

Vishay

Semiconductors

10 4.585 4 16

3300 0.70

AT90USB1287 AVR

Microcontroller

Atmel 1000 9.23 1 4

6 D 25 Female

Connector

Green Brook

Electronics

1 3.50 1 4

6 D 25 Male

Connector

Green Brook

Electronics

1 3.75 1 4

6” 8-32 Threaded

Rod

Loew’s 4 0.68 4 16

3” 8-32 Threaded

Rod

Loew’s 4 0.56 1 4

8-32 Hex Mach Nut Loew’s 9 0.8 8 32

Step Flashing

Aluminum

Loew’s 4 0.32 1 1

Permamount Tape RadioShack 1 1.99 1 2

Total 72.50 290.00

20

4. Sensing of 8eighboring Arrays Protocol

The sensing of neighboring arrays protocol is an integral function of the design project in

order for the arrays to know the topology it is contained in. It is important that there be an

infrared transceiver located at each side of the array for proper orientation to be defined. The

sensing protocol has been done by using the infrared transceivers to communicate repeatedly

over time trying to find a connection, establish a connection, and disconnect properly from a

connection.

The first task is for the transceiver to find a connection between arrays. This is done by

alternating sensing between each side starting from the right side, then the bottom, the left, and

then finally the top side. The infrared transceivers first transmit a connection_verify signal

after which it will wait for a response after completing its transmission. During this time the

microcontroller waits for an interrupt on the receiving end of the just recently transmitting

infrared. If a connection was established a connection_confirm message would have been

received at the transmitting side. These commands will notify the microcontroller and the

application which side it is currently connected to. With this type of protocol it is insured that

proper connection communication can be made between sides since it will only actually

connect after a message has been sent to it in response to its own message. It is also important

to note that data that is transmitted between each side can only communicate with its

counterpart side, right with left and top with bottom. After this sequence the microcontroller

proceeds to the next side and so on.

The second task is for the transceiver to establish a connection between each other. This is

done after a connection has been found on a side. After a connection has been found on a side a

connection between arrays/computers will be made over wireless based on its I.P. address. An

I.P. address is communicated between each array that is then passed over to the application for

connection between computers.

The last task is for the microcontroller to discern if a disconnection has been made between

transceivers. This task cannot be done by simply disconnecting based on if a transceiver is

receiving or not and must primarily be checked if data is no longer being communicated back

21

to the transceiver from the correct side after a number of times. Each message that is correctly

sent between already connected transceivers is an indicator that a connection is still live. The

transceivers continually check this connection every cycle of the function through transmission

and wait for response messages. Disconnection from a side indicated by the microcontroller is

done if a message has not been sent back to the side after 5 cycles the program runs.

These functions are important in enabling the microcontroller to correctly know which side

is connected or disconnected properly. This information is continually sent to the application

for its topology updates. The sensing of arrays is done repeatedly through transmitting and

receiving of messages. The microcontroller handles both these functions based on the infrared

capabilities and are the basis of all communication made between sensor arrays.

22

5. Transmitting and Reading of I.P. Address and Commands

The transmission and reading of I.P. addresses and commands are primarily set in the

microcontroller. When transmitting through infrared the microcontroller supplies the bit clock

the infrared encoder/decoder operates on. The bit clock set determines the data rate at which

bits are sent and how the encoder/decoder sends and receives signals to the infrared for proper

transmission and reception. Once the bit clock is set the microcontroller must properly output

data to be sent and read data that is input at the ports of the microcontroller. The transmission

of data is always carried out in a specific order dependent on the message that was or wasn’t

received. The messages are sent by byte which begins by sending a preamble, start_flag, the

side in which is sending the information from, the message, and then a stop flag. The

transmission of data follows this format because the receiving microcontroller needs to be

interrupted, define which side it is to read from, the message being sent and a stop flag to

indicate when all the data has been sent.

The transmitted data starts with a preamble that is defined as 0x00. The first byte that is

being transmitted is 8 logic “0” bits in order for the receiving microcontroller to interrupt on

the receiving pin. The preamble also must be logic “0” since the infrared transceiver only sends

signals if logic “0” is sent. Since the input ports are normally high receiving logic “0” will

interrupt the microcontroller when receiving is enabled and send will read off the pins at each

clock cycle determined the Timer/Counter 3 clock. Next the start_flag was used to determine

that the bytes to be received afterwards are the beginning of the data and the stop_flag

determines the end of data. The sides are sent in order for the receiving infrared to know which

side of a neighboring array sent a message so as to ignore message from non complementing

sides. Last the message is sent which may be a connection_verify, connection_confirm,

confirm_ipaddress or a transmit_ipaddress message. All messages sent are dependent on

responses received after a transmission and the bit reading reliability. Transmission and

reading of messages are highly sensitive to the Timer/Counters used throughout the

microcontroller.

23

Fig. 6: Sensing Task.

Table 2: Transmission Frame of Bytes if Command not Transmission I.P.

Byte 1 2 3 4 5

Data Preamble Start_Flag Tx_Side Command Stop_Flag

Table 3: Transmission Frame of Bytes if Command is Transmission I.P.

Byte 1 2 3 4 5-16 17-18 19

Data Preamble Start_Flag Tx_side Transmission I.P. Checksum Stop_Flag

24

I.P. Address

6. 16-bit Timer/Counters 1 and 3

The 16-bit Timer/Counter 1 and 3 are primarily used as clocks for the Infrared

Encoder/Decoder 16X clock, transmitting data at a set data rate, and for reading data on Port B

at the transmitter data rate. The 16-bit Timer/Counters 1 and 3 have a maximum allowable

clock cycle length of 16 bits and are continually compared to the Output Compare Register

(OCRnA) register after each clock cycle. Setting the Timer/Counter (TCNTn) to clock at a

desired frequency is dependent on registers OCRnA, TIMSKn, TIFRn, TCCRnA, and

TCCRnB registers. The registers TCCRnA and TCCRnB are primarily used to set the mode

which may be PWM, Fast PWM, or CTC mode, set the counter clock frequency and

functionality of the pin which the timer counter is located on. The TIMSKn register is

primarily used for enabling interrupts on the desired output compare register and when an

interrupt occurs the TIFRn register is set according to the interrupt which occurred at that time.

Last, the OCRnA register is used as a compare then interrupt register where the TCNTn

register is constantly compared to the OCRnA register. When the TCNTn register equals the

OCRnA register an interrupt occurs, which then calls the appropriate interrupt function on the

microcontroller.

25

VI. System/Driver

The principal objective of the USB driver is to integrate the hardware with the user

application. This is a very important step and crucial to the overall system, allowing the

microcontroller to properly communicate with the application and vice versa. Without the driver,

the microcontroller would not be able to receive instructions from the application, which it sends

to another computer via infrared. For the “communication link” between the hardware and

application to be accomplished, the driver uses certain protocols, which informs the computer of

the type of device it is communicating with, as USB devices are categorized into several classes.

When implementing a driver, it is important to know how the host can obtain information

about the device and also how the information is transferred between the device and host

computer. USB descriptors, written in the driver, describe to the host computer specific details

about the device. It describes what type of device it is connected to, the number of ways the

device can be configured and also list the number of endpoints along with descriptions of these

endpoints. There are four main descriptors that a driver of a USB must contain: a device

descriptor, configuration descriptors, interface descriptors and endpoint descriptors. A USB

device can only have one device descriptor, as this descriptor informs the host of the vendor and

product id and also how many configurations a device may have. The configuration descriptor

determines the amount of power used for the specified configuration and if the device is powered

via the USB bus or self-powered. The interface descriptor describes how the endpoints function

together as a group of that interface, while the endpoint descriptors define the type of transfer,

the direction of the transfer, and the maximum packet size of each endpoint. Most devices are

grouped into USB classes that have a pre-defined set of descriptors, which correlate to the

intended function of the hardware in use.

The class of the USB device defines how the device presents itself to the host computer.

These classes determine how data and control information are exchanged between software on

the host and particular endpoints on the USB device through pipes via descriptors. An endpoint

on a USB device is the final location of a communication transfer between the device and host

computer. When writing the driver, each endpoint is assigned a specific number and also the

direction of the data transfer it represents. These two details, along with the address of the USB

device, which is determined when the device is attached to the computer, allow for each endpoint

26

to be specifically called as required. The USB pipe is the connection between a device’s

endpoint and the user application on the host computer, allowing data to be transferred between

the endpoint and a memory buffer, (a device node on Linux). A USB device driver may define

as many pipes as needed for the various types of transfers it may need, as one pipe may support

only one type of transfer.

When sending information, the USB transfers data in structured packets determined by the

USB protocol for the specific data transfer type. There are four types of data transfers: control

transfers are used to configure a device when it is attached; bulk data transfers allow transfer of

information in blocks of bytes for large transfers while interrupt data transfers are used to send

interrupts, (one byte is sent to update the status of the device as in the Human Interface Device

class for a keyboard or mouse); and isochronous data transfers, which are used for streaming real

time transfers periodically. Due to the nature of the hardware and software designed for this

particular technology, control and bulk data transfers between the user application and

microcontroller are necessary transfer types.

The Communication Device Class (CDC) was chosen as the desired class to use and

implement with the designed software and hardware setup, allowing the host computer to

recognize the microcontroller as an USB device that can send and receive bulk data transfers, (an

array of characters). While creating the custom USB driver, it was recommended that an already

existing driver be utilized, but modified to meet the needs and requirements of the desired

hardware functions. The complexity of understanding all of the intricate details of how USB

drivers perform and are written would have taken an amount of time that did not fit into the

timeframe of the project itself. Therefore, the CDC_ACM driver was chosen as the best possible

option, which creates a virtual communication port with Linux, similar to that of a RS232 serial

port.

When the microcontroller is connected to the host computer, Linux creates a node, (memory

buffer or device file) which stores the desired data to be transferred to the microcontroller from

the host application and vice versa. As a standard, the CDC_ACM driver contains a set of

functions and tasks that allow proper data transmission and retrieval from the file Linux creates

on the host. Successful testing of the read and write functions were accomplished by creating a

27

python program which could open, read and write into the specified device node, ttyACM0,

located in the “/dev” folder.

The microcontroller was flashed with the CDC_ACM driver, along with added code that sent

messages to the device node, by selecting the data out endpoint, where the custom code was able

to open and read the node as if it were a file. To test the driver’s read function, the python code

was modified to also write to the device node and the microcontroller was flashed to read from

the device node on the host, by selecting the data in endpoint. If the message read from the

device node by the microcontroller matched a predetermined character array programmed into

the microcontroller, then a confirmation message was sent back to the host computer and stored

into the ttyACM0 node, where the python program could read from it. Understanding how to

read and write to the microcontroller was an essential step, as the host computer sends its IP

address to the microcontroller in the initialization of the user application. Also, after the IP is

passed down to the microcontroller and sent through IR to another microcontroller, it needs to be

transferred up to the corresponding host computer to determine which host computer sent it the

message. The data flow of the read and write functions are illustrated in Fig. 6, with the writing

to the microcontroller on top and reading from the microcontroller on bottom.

Fig 7: Block diagram of data transfer between Host Computer and USB device.

28

VII. Design Problems Encountered and Solutions.

1. Infrared Board Implementation

There were many problems that the group encountered during the semester involving the

hardware and microcontroller. The first problem was the implementing infrared transceiver to a

board for testing and use given that the transceiver size was small in size and pitch. The group

has never encountered a component of that size and had only two possible solutions to test and

integrate the device with the system. The first solution was to design and fabricate a PCB board

that would be able to fit the infrared transceiver’s footprint. The second solution was to

manually solder wire onto the infrared transceiver for connection to the microcontroller device

and infrared encoder/decoder. An attempt was made to design a PCB layout for implementing

the infrared transceiver where then a board would be fabricated at MERL as seen on Fig. A.

After designing the layout a redesign was needed to minimize real estate used in the design.

Then, a tank was unavailable to etch the board since a multi-layer design was used. A tank was

then designed to somewhat allow etching where in the end, the fabricated board became

unusable because the ferric chloride the board was placed in ate away too much of the copper

traces in the design. It was then that manual soldering became the only option.

Fig. 8: a.) PCB layout of device with multilayer displayed.

29

b.)Top layer of PCB layout. c.)Bottom layer of PCB layout.

2. Infrared Signal Reflection

The infrared transceiver also produced many problems during the semester where after

implementing multiple transceiver to an array unknown signals were being received the

transceiver. Testing was done at the oscilloscope to discern what may be affecting the

transceiver. After probing the infrared receiver triggered to the transmitter it was determined

that the transmitting side was receiving signals from its own array. To solve this problem,

blinders were implemented to the array which was composed of step flash aluminum covered

with electric tape to prevent conduction between wires. These blinders were formed into a

shape which both encased the infrared from receiving from the top and bottom but also from

the back side.

3. Microcontroller Reading Bytes

The microcontroller also had problems reading the bytes received from a neighboring

array. Although data was being sent out correctly messages were not being read by the

microcontroller correctly since connections were not made given the message. Changes were

made to the reading of the bytes and still the message was not reading correctly. Then code

was implemented to the program to display the values being read at the computer to fix the

issue. After many attempts of reading the message received through the old code another

function was implemented to use the Timer/Counter 3 as a clock which interrupts every clock

cycle of the data rate. This allowed for accurate reading of the bytes being sent from the

neighboring sensor array.

30

VIII. Future Considerations

The design allows an application to many different devices such as the TV, laptops, desktop

screens, tablets, e-readers, cell phones, etc. Implementing such a device to existing technology

can potentially evolve their current functions because of there lies an intelligent communication

method. The ability to watch TV on two screens allowing super widescreen viewing, e-reader

technology allowing large scale viewing, easy copying and reading capabilities and future

technologies such as OLEDs would benefit greatly due to the capabilities of the sensor array.

Some design changes maybe to work with inductive coupling as opposed to infrared sensors for

determining orientation of a device. Inductive coupling allows for a better sensing method since

an array must be close to each other. The infrared transceivers used have been found to reflect

signals all over the place causing unintended information to be received from the wrong sides. A

design involving inductive coupling would also be cheaper in value and consume less power than

infrared.

31

IX. Conclusion

The design was completed and functioned as intended when scaling images on multiple

screens. Copying to each device was also successful because the image to be scaled needs to be

stored at each computer. Many configurations worked accordingly such as two computers in any

configuration, three computers at any configuration and four computers in any configuration.

The design allowed for ease of use and automatic configuration which allowed for users to enjoy

the technology more. Many problems occurred during the design of the array when trying to

integrate the hardware with the application and the driver but solutions were found through great

debugging techniques and group troubleshooting. It takes time to understand the problem at hand

so group evaluation of the problem was the main reason the device functioned successfully. A

device such as this has so much potential and can give way for future technologies to be more

intelligent in their communication methods.

32

References

http://ww1.microchip.com/downloads/en/DeviceDoc/21894c.pdf Microchip, Infrared

Encoder/Decoder, MCP2122 datasheet

http://www.atmel.com/dyn/resources/prod_documents/doc7627.pdf Atmel Corp.,

AT90USB1287 development kit, AT90USBKey Hardware guide

http://www.atmel.com/dyn/resources/prod_documents/doc7593.pdf Atmel Corp., AVR

microcontroller, AT90USB1287 datasheet

http://www.vishay.com/docs/84763/tfdu6300.pdf Vishay Semiconductors, Fast Infrared

Transceiver Module (FIR, 4Mbit/s), TFDU6300, datasheet Rev. 1.8, 03-Jul-08

http://www.palantirisystems.com/publications/IntroductiontoIrDA.pdf SES Technology R&D

Group, Introduction to the IrDA Protocol, 09/03/1997

Peacock, C. “USB in a Nutshell,” April 6, 2007. Beyond Logic.

http://www.beyondlogic.org/usbnutshell/usb5.htm. March 25, 2009.

33

Appendix A: Microcontroller Register and Port Uses

Ports on AT90USBKEY

PORT A – Not Used

PORT B- Primarily set to input at pins 0 to 3 for reading incoming data from Infrared

Encoder/Decoder RX pin and output at pin 5 for Infrared Encoder/Decoder 16X clock.

Port B Pin 0 = Infrared right side RX

Port B Pin 1 = Infrared bottom side RX

Port B Pin 2= Infrared left side RX

Port B Pin 3= Infrared top side RX

Port C – Not set to input or output data. Primarily using Pin 2 with Timer Output Compare

function which is used as a system transmitting and receiving data rate clock.

Port D – Primarily set to output. Primarily used for turning on and off on board LED’s. Port D

Pins 4, 5, 6 and 7.

Port E – Primarily set to output for sending data to Infrared Encoder/Decoder TX pin for

transmission.

Port E Pin 0 = Infrared right side TX

Port E Pin 1 = Infrared bottom side TX

Port E Pin 6 = Infrared left side TX

Port E Pin 7 = Infrared top side TX

Port F – Primarily set to output turning on Bar LED on motherboard for indicating currently

connected sides.

34

Port F Pin 0 = LED for indicating connection on right side.

Port F Pin 1 = LED for indicating connection on bottom side.

Port F Pin 2 = LED for indicating connection on left side.

Port F Pin 3 = LED for indication connection on top side.

Registers on AT90USBKEY

General Purpose Input and Output Registers

GPIOR0 = General Purpose Input and Output Register 0. Primarily used for indicating sensor

array side that is connected.

GPIOR1= General Purpose Input and Output Register 1. Primarily used for holding command

value that was received from the transmitting sensor array.

GPIOR2= General Purpose Input and Output Register 2.

PCMSK0=Pin Change Interrupt Register

PCICR= Pin Change Interrupt Enable/Disable Register

PCICR=Pin Change Interrupt Flag Register

TCNTn=Timer Counter

OCRAn=Output Compare Register

35

Appendix B: Application Message Definitions

Header definition for passing data between devices

Type | From | Length | Data

2 bytes | 2 bytes | 4 bytes | variable

Types are defined as one of the following

'I' - Incoming connection. This is a special message generated by hardware indicating

an incoming connection. Length should be set to 0.

DRIVER user only.

'E' - Error message. This is a special message generated by the fakedriver.py if an

invalid connection is being written to. DEBUG use only. Length is zero.

'H' - Hello message. This is a special message generated by the fakedriver upon

connection request to indicate which socket belongs to which side of the device.

DRIVER use only. Length should be set to zero.

'D' - Disconnect message. Sent when disconnect occurs. May be generated by

hardware. Similar to topology update but only contains information for one

disconnected device.

'T' - Topology update message. Data will contain the topology as seen by connecting

36

device. Care must be taken in determining how to merge topology information.

Current device topology should be updated to incoming data.

'R' - Request update message. Sent to request a display update.

Can be sent with no data to request the filename of the file to be displayed, or with

the filename followed by NULL '\0' to request the actual file.

'U' - Update message. This is to update the content to be displayed. Either the filename,

or the filename and file data will be sent for display updating. Display should be

updated to reflect newly arrived content.

From is defined as a 2 byte (big endian) integer:

0 - Left side

1 - Top side

2 - Right side

3 - Bottom side

Length is big endian encoded integer representing the length of the data payload.

37

Appendix C: Examples of Swarming Algorithms

Example 1: Topology update with simple update message.

A device with the topology matrix represented by

[1 1]; (0, 1) Note: [1 1] is the topology matrix, (0, 1) is the current

device’s index within that matrix.

receives a topology update message from below containing just a single device. The topology

update received looks like

[1]; (0, 0).

In order to incorporate this information into the local topology matrix, the index must be

adjusted to reference the local device, which is one above the connecting device:

[1]; (0, 0) � [1]; (-1, 0)

A new row is then appended to the local topology to take into account the addition of a device

in a previously uncreated row.

[1 1]; (0, 1) � [1 1]; (0, 1)

[0 0]

Finally, the topologies are overlapped and logically ORed together:

[1 1] | � [1 1]

[0 0] | [1] � [0 1]

Thus, the final topology includes the device that connected to the bottom of the top-right

device.

Example 2: Topology update with non-trivial update message

A device with the topology matrix represented by

[1 1 1]; (1, 2)

38

[0 1 1]

[1 1 0]

receives a topology update message from the right. The topology update received looks like

[1 1 1]; (1, 0).

[1 1 0]

[1 0 0]

In order to incorporate this information into the local topology matrix, the index must be

adjusted to reference the local device, which is one to the right of the connecting device:

[1 1 1]; (1, 0) � [1]; (1, -1)

[1 1 0]

[1 0 0]

Two new columns are then appended to the local topology to take into account the addition of

two uncreated columns of devices.

[1 1 1]; (1, 2) � [1 1 1 0 0 ]; (1, 2)

[0 1 1] [0 1 1 0 0]

[1 1 0] [1 1 0 0 0]

Finally, the topologies are overlapped and logically ORed together:

[1 1 1 0 0] | [1 1 1] � [1 1 1 1 1]

[0 1 1 0 0] | [1 1 0] � [0 1 1 1 0]

[1 1 0 0 0] | [1 0 0] � [1 1 1 0 0]

Example 3: Disconnect update with simple disconnect message

A device with topology

[1 1 1]; (0, 0)

39

[1 1 0]

[1 0 0]

receives a disconnect message from the bottom with data (1, 0). Since this data came from a

bottom connection, the reference point needs to be updated by adding one to the x-axis count,

resulting in data of (2, 0). Removing the particular device by writing a zero into the matrix

results in

[1 1 1]; (0, 0).

[1 1 0]

[0 0 0]

Because the bottom row contains all zeros, it can be removed from the matrix along with

everything below it – in this case, no additional rows will be removed. The final matrix

becomes:

[1 1 1]; (0, 0)

[1 1 0]

Example 4: Calculating the largest block of all connected devices

A device would like to calculate the largest block of all ones in the topology

[1 1 1 1]; (1, 2)

[0 1 1 1]

[1 1 1 0]

Starting at the top-left row-wise, the first zero encountered is in the first column, second row.

The zero in question is removed by splitting the topology matrix first by the row it is in,

yielding two matrices,

[1 1 1 1] and [1 1 1 0]

and then splitting by the column it is in, yielding one matrix,

40

[1 1 1]

[1 1 1]

[1 1 0]

The first matrix is determined to contain all ones, so it is a likely candidate for the largest block

of connected devices. The second matrix contains a zero, and is therefore split by rows and

columns again. It is apparent, however, that only one matrix will be produced from the split:

[1 1 1]

Again, this matrix contains only ones. When comparing the area of the two possible solution

matrices, however, the first solution found was better and is therefore kept in preference to the

newly found solution. The last matrix contains a zero, and will be split into two matrices:

[1 1 1] and [1 1]

[1 1 1] [1 1]

[1 1]

Both of these matrices are possible solutions to the largest block problem, and both contain the

same amount of devices and more than the previously found best solution. To choose among

otherwise equal solutions, the dimensions are used. Widescreen is preferred to ‘tallscreen’, so

the first solution will be chosen. The final solution is:

[1 1 1]; (0, 1)

[1 1 1]

41

Appendix D: Design Photos

Fig. 9: Full Array Mounted on Laptop.

42

Fig. 10: Back of Array Connections.

Fig. 11: Top of Array.

43

Fig. 12: Top of Array Components.

44

Fig. 13: Close up of IR Transceiver and Encoder/Decoder.

Fig. 14: AT90USBKey Close Up

45

Appendix E: Project Code

Application Code:

The application can be run by following these steps:

1. Connect the microcontroller to one of the USB slots on the computer

2. Start the fakedriver.py program with the command ‘python fakedriver.py 1’

3. Start the application demo with the command ‘python epapergtk.py 1’

Dragging different images onto the main program window will change the image being

displayed if the current device is master. Putting other computers running the same program

next to each other will cause them to scale images accordingly.

Driver and Hardware Emulation - fakedriver.py

#!/usr/bin/env python

"""

Fake Driver for E-Paper frontend application.

This application provides a fifo interface to act like a device driver

for the application, while simulating the connection between devices

using sockets.

"""

import sys, socket, struct, os, time

import signal, select

import edefs

# Socks contains the sockets used to talk to other drivers

socks = [None]*5

# RF contains the read fifo for reading messages from the application

rf = None

# WF contains the write fifo for writing messages to the application

wf = None

46

# Actual driver connection (currently only used for incoming and

# disconnect messages)

driver = None

# Standard port number to use

PORT = 33151

def main(driverid):#, port, conlist):

global socks, rf, wf, driver, PORT

# Connect to the hardware driver

driver = open("/dev/ttyACM0", "r+")

# Send the driver our IP address

ip = os.popen("ifconfig ath0 | grep \"inet addr\" | awk '{print $2}' | sed -e s/.*://", "r").read().strip().split('.')

packed = "%02x%02x%02x%02x%02x%02x" % (int(ip[0]), int(ip[1]), int(ip[2]), int(ip[3]), (PORT & 0xFF00) >> 8, PORT & 0xFF)

driver.write('z' + packed)

driver.flush()

# Open connections to other devices

socks[4] = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

socks[4].bind(('', PORT))

socks[4].listen(5)

# Initialize FIFO driver interface

# Note: these will block until the other end connects

wf = open("tmp/edriver" + str(int(driverid) + 1), 'a')

rf = os.fdopen(os.open("tmp/edriver" + driverid, os.O_RDONLY | os.O_NONBLOCK), 'r')

# Program control

while True:

# Wait until there is something to read, either from

# the application or from the other drivers

readlist = [driver, rf] + filter(issocket, socks)

[r, w, e] = select.select(readlist, [], [])

for fp in r:

got_error = False

47

try:

fp.fileno()

except:

# Other side closed socket

got_error = True

if got_error:

for s in [0, 1, 2, 3]:

if issocket(socks[s]):

try:

socks[s].fileno()

except:

socks[s] = None

continue

# Got message from the application

if fp.fileno() == rf.fileno():

# Send data to appropriate device

handle_send()

# Got an incoming connection

elif fp.fileno() == socks[4].fileno():

handle_new_connection()

# Got message from driver

elif fp.fileno() == driver.fileno():

handle_driver()

# Got data from another device

else:

handle_data(fp.fileno())

# Send data to another device from application

def handle_send():

global rf, socks

# Read all the available messages and try to send

48

while True:

# Try to read the header. If it's not there, no more messages.

try:

header = rf.read(8)

if len(header) < 8:

raise IOError

except IOError:

break

# Check exit condition

if header[0:2] == edefs.GOODBYE:

# Shutdown because the pipe is about to break, application

# is exiting.

shutdown(signal.SIGPIPE, None)

(frm, length) = struct.unpack("!HI", header[2:8])

data = ''

while len(data) < length:

# Read the data

try:

data += rf.read(length - len(data))

except IOError:

# If data couldn't be read, wait at most 1 second and try again

select.select([rf], [], [], 1)

# Fix deadlock issue with multiple disconnects being sent

if header[0:2] == edefs.DISCONNECT:

time.sleep(0.5)

# If there is a device connected where we're trying to send, send data

if issocket(socks[edefs.FROM(frm)]):

socks[edefs.FROM(frm)].send(header + data)

else:

# Otherwise generate precautionary error message

global wf

header = edefs.ERROR + struct.pack("!H", edefs.FROM(frm)) + "\0\0\0\0"

49

wf.write(header)

wf.flush()

print "Warning, attempted write to unconnected socket"

# Handle an incoming connection from another device

def handle_new_connection():

global socks

[s, addr] = socks[4].accept()

header = s.recv(8)

if header[0:2] == edefs.HELLO:

[frm, length] = struct.unpack("!HI", header[2:8])

socks[frm] = s

# If data was sent for no reason, clear the buffer and print

# warning message

if length > 0:

#print "Warning, data received with HELLO message"

total_recv = 0

while total_recv < length:

total_recv = total_recv + len(s.recv(length - total_recv))

# Generate incoming connection message

incmes = edefs.INCOMING + header[2:4] + "\0\0\0\0"

wf.write(incmes)

wf.flush

else:

# Something went wrong

print "Error, incoming connection did not produce HELLO message, disconnecting"

s.close()

# Handle incoming message from the driver

def handle_driver():

global driver, wf

header = drivermsg(driver.readline().strip())

if header[0:2] == edefs.INCOMING:


print "Got driver connect message from", edefs.FROM_STR(frm)

50

if length > 0:

data = drivermsg(driver.readline().strip())

else:

print "Error: no IP address included!"

return

if data == '':

print "Error: invalid IP address received!"

return

IP = struct.unpack('!4BH', data[0:6])

port = int(IP[4])

IP = str(IP[0]) + '.' + str(IP[1]) + '.' + str(IP[2]) + '.' + str(IP[3])

# If nothing is already connected, open new connection

if not issocket(socks[frm]) and (frm == edefs.BOTTOM or frm == edefs.RIGHT):

#print "Opening socket to", edefs.FROM_STR(frm)

socks[frm] = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

socks[frm].connect((IP, port))

# Send Hello message stating where we're connecting from

helloheader = edefs.HELLO + struct.pack("!H", edefs.FROM(frm)) + "\0\0\0\0"

socks[frm].send(helloheader)

header = header[0:4] + '\0\0\0\0'

wf.write(header)

wf.flush()

elif header[0:2] == edefs.DISCONNECT:

#print "Got driver disconnect message"


# If anything else was received, read the data

if length > 0:

data = drivermsg(driver.readline().strip())

# Disconnect the socket if connected

if issocket(socks[frm]):

51

try:

socks[frm].close()

except:

pass

socks[frm] = None

# Send the disconnect message to the application

header = header[0:4] + "\0\0\0\4"

wf.write(header + data)

wf.flush()

# Handle incoming data from a socket. Send to application.

def handle_data(fileno):

global socks, wf

# Find the appropriate socket

frm = -1

for s in [0, 1, 2, 3]:

if issocket(socks[s]) and socks[s].fileno() == fileno:

frm = s

break

if frm < 0:

print "Got message from an unknown socket, now what?"

return False

header = socks[frm].recv(8)

if len(header) < 8:

# Other side shutdown, generate disconnect

#header = edefs.DISCONNECT + struct.pack("!H", frm) + "\0\0\0\4\0\0\0\0"

socks[frm].close()

socks[frm] = None

#wf.write(header)

else:

wf.write(header)

52

length = struct.unpack("!I", header[4:8])[0]

#print "RECV: Message", header[0], "from", edefs.FROM_STR(frm), "length", str(length)

# Read data from socket and write to application fifo

total_recv = 0

while total_recv < length:

data = socks[frm].recv(length - total_recv)

wf.write(data)

total_recv = total_recv + len(data)

# Flush the message from the buffer to send to application

wf.flush()

# Convert hex message from driver into binary message

def drivermsg(msg):

print msg

message = ""

try:

for x in range(0, len(msg), 2):

message += struct.pack("!B", int(msg[x:x+2], 16))

except:

message = ""

return message

# Return whether the input is a socket or not

def issocket(s):

return isinstance(s, socket.socket)

# Exit fakedriver, safely shutting down sockets and pipes

def shutdown(signum, frame):

global socks, rf, wf, driver

for sock in socks:

if issocket(sock):

try:

53

sock.close()

except:

pass

try:

if wf != None:

wf.close()

except:

pass

try:

if rf != None:

rf.close()

except:

pass

try:

if driver != None:

driver.close()

except:

pass

sys.exit(1)

if __name__ == "__main__":

if len(sys.argv) > 2:

PORT = int(sys.argv[2])

if len(sys.argv) < 2:

print "Usage:", sys.argv[0], "driverid [port]"

else:

# Set up signal handlers

signal.signal(signal.SIGINT, shutdown)

#signal.signal(signal.SIGKILL, shutdown)

signal.signal(signal.SIGPIPE, shutdown)

#main(sys.argv[1], int(sys.argv[2]), sys.argv[3:])

main(sys.argv[1])

Hardware and Driver Emulation – DriverInterface.py


54

"""

Connection interface for driver.

Opens a read and write fifo to connect to the fake driver.

"""

import struct, edefs

import os, select

class DriverInterface:

def __init__(self, driverid):

# driverid is used to choose the appropriate fifo

self.driverid = driverid

readid = str(int(driverid) + 1)

# Read interface must be read nonblocking because of a problem

# with select.select() and not returning properly

self.rf = os.fdopen(os.open("tmp/edriver" + readid, os.O_RDONLY | os.O_NONBLOCK), 'r')

self.wf = open("tmp/edriver" + driverid, 'a')

# Send a message to the driver

def send(self, to, mtype, data = "", flush = True):

#print "Data to", edefs.FROM_STR(to), "type", edefs.TYPE_STR(mtype), "length", str(len(data))

header = mtype + struct.pack("!HI", edefs.FROM(to), len(data))

self.wf.write(header + data)

if flush:

self.wf.flush()

# Separate flush command for broadcast messages, to avoid quickly writing

# and flushing several times for no reason

def flush(self):

self.wf.flush()

# Return fileno for use with call to select.select()

def fileno(self):

return self.rf.fileno()

55

# Read message(s) from the driver

def read(self):

messages = []

while True:

# Try to read the header, if it's not there, then no more

# messages are available

try:

header = self.rf.read(8)

if len(header) < 8:

raise IOError

except IOError:

break

mtype = header[0:2]

(frm, length) = struct.unpack("!HI", header[2:8])

data = ''

while len(data) < length:

try:

data += self.rf.read(length - len(data))

except IOError:

# If read didn't work, wait at most one second

# and try again

select.select([self.rf], [], [], 1)

messages.append((mtype, frm, data))

return messages

# Close the fifos for exiting the program

def close(self):

try:

self.rf.close()

except IOError:

pass

try:

56

# Send a goodbye message to the driver

self.wf.write(edefs.GOODBYE + struct.pack("!H", edefs.END) + "\0\0\0\0")

self.wf.flush()

self.wf.close()

except IOError:

pass

Global Definitions – edefs.py


"""

E-Paper application definitions and helper functions

"""

# Definitions for message types

INCOMING = "I\0"

ERROR = "E\0"

HELLO = "H\0"

GOODBYE = "G\0"

DISCONNECT = "D\0"

TOPOLOGY = "T\0"

REQUEST = "R\0"

UPDATE = "U\0"

# Definitions for message to/from

LEFT = 0

TOP = 1

RIGHT = 2

BOTTOM = 3

END = 4

ALL = 8

# Get the opposite side of the screen, ie.

# Left -> Right, Top -> Bottom, and vice versa.

def FROM(side):

return (side + 2) % 4

57

# Increment the side, wrapping around after bottom

def INCREMENT(side):

return (side + 1) % 4

# Return a list containing the sides to be iterated

# over. If ALL is an input argument, return all sides.

def ITERATOR(*args):

l = []

if ALL in args:

return [LEFT, TOP, RIGHT, BOTTOM]

for side in args:

l.append(side)

return l

# Return the string version of a side

def FROM_STR(side):

if side == LEFT:

return "LEFT"

elif side == RIGHT:

return "RIGHT"

elif side == TOP:

return "TOP"

elif side == BOTTOM:

return "BOTTOM"

else:

return "UNKNOWN"

# Return a string version of the message type

def TYPE_STR(t):

if t == INCOMING:

return "INCOMING CONNECTION"

elif t == ERROR:

return "ERROR"

elif t == HELLO:

return "HELLO"

58

elif t == GOODBYE:

return "GOODBYE"

elif t == DISCONNECT:

return "DISCONNECT UPDATE"

elif t == TOPOLOGY:

return "TOPOLOGY UPDATE"

elif t == REQUEST:

return "REQUEST UPDATE"

elif t == UPDATE:

return "DISPLAY UPDATE"

else:

return "UNKOWN"

Swarm Algorithms – rect.py


"""

Function to find the largest rectangle in a given topology

"""

import topology

def rect(t):

if not isinstance(t, topology.Topology):

return None

# Use TopologyLite object because it's smaller than Topology

tLite = topology.TopologyLite()

tLite.top = t.top

tLite.index = [0, 0]

tLite.size = t.size

# Special case, all ones

if allones(tLite.top):

return tLite

59

# Matrices contains topologies to be looked at

# Solution has the best currently found solution

matrices = [tLite]

solution = None

solnarea = 0

# Loop through all possible topologies

for tL in matrices:

next = False

# Loop through elements in topology, finding first zero

# and splitting matrix. If any split results in a complete

# rectangle, check it against the currently best found solution.

for i in range(tL.size[0]):

for j in range(tL.size[1]):

if tL.top[i][j] == 0:

next = True

# Generate possibilities and add to matrices

tlist = []

# If there's more than one row, split the matrix on rows.

if tL.size[0] > 1:

tlist.append(topology.TopologyLite())

tlist[-1].index = tL.index[:]

tlist[-1].size = [tL.size[0] - 1, tL.size[1]]

# Remove the first row, one matrix results

if i == 0:

tlist[-1].top = tL.top[1:]

tlist[-1].index[0] += 1

# Remove the last row, one matrix results

elif i == tL.size[0] - 1:

tlist[-1].top = tL.top[:-1]

# Remove a row somewhere in the middle, two matrices result

else:

tlist[-1].top = tL.top[:i]

tlist[-1].size[0] = i


tlist[-1].top = tL.top[i+1:]

60

tlist[-1].index = [i+1, tL.index[1]]

tlist[-1].size = [tL.size[0] - i+1, tL.size[1]]

# If there's more than one column, split the matrix on cols.

if tL.size[1] > 1:


tlist[-1].index = tL.index[:]

tlist[-1].size = [tL.size[0], tL.size[1] - 1]

a = []

b = []

# Remove the first column, one matrix results

if j == 0:

for row in tL.top:

a.append(row[1:])

tlist[-1].index[1] += 1

tlist[-1].top = a

# Remove the last column, one matrix results

elif j == tL.size[1] - 1:

for row in tL.top:

a.append(row[:-1])

tlist[-1].top = a

# Remove a column in the middle, two matrices result

else:

for row in tL.top:

a.append(row[:j])

b.append(row[j+1:])

tlist[-1].top = a

tlist[-1].size[1] = j


tlist[-1].top = b

tlist[-1].index = [tL.index[0], j+1]

tlist[-1].size = [tL.index[0], tL.index[1] - j+1]

# Look at the matrices we just generated and check if they

# are solutions

for tmp in tlist:

area = tmp.size[0] * tmp.size[1]

# If it's a solution, check if the area is larger.

61

# If the area is equal, take the wider of the two

if allones(tmp.top):

if area > solnarea:

solution = tmp

solnarea = area

elif area == solnarea:

ratio1 = abs(1 - float(tmp.size[0]) / tmp.size[1])

ratio2 = abs(1 - float(solution.size[0]) / solution.size[1])

if ratio1 < ratio2:

solution = tmp

# If it's not a solution, check if the area is greater than the

# best solution so far. If so, Add it to the list of matrices to check.

elif area > solnarea:

matrices.append(tmp)

# Go to the next matrix

if next:

break

# Go to the next matrix

if next:

break

return solution

# Helper function to print a matrix in a legible format

def printmatrix(matrix):

if not isinstance(matrix, list) or not isinstance(matrix[0], list):

print "InputError: Input not a matrix",

print matrix

return

for i in range(len(matrix)):

for j in range(len(matrix[0])):

print matrix[i][j],

print

62

# Check if a matrix consists of all ones

def allones(matrix):

for row in matrix:

for element in row:

if element == 0:

return False

return True

Swarm Algorithms – topology.py


"""

Topology Class definition

"""

import struct, edefs

"""

TopologyLite class contains the same data as

Topology, but without the overhead of all the member

functions.

"""

class TopologyLite:

def __init__(self):

self.top = [[1]]

self.index = [0, 0]

self.size = [1, 1]

"""

Topology class details the topology of connected devices

and has easy-to-use functions to adding and removing

more devices.

"""

class Topology:

# top - the actual topology matrix. Implemented as a list

63

# of rows

# index - the position of the current device, as measured from

# the top left corner

# size - the size of the topology

def __init__(self):

self.top = [[1]]

self.index = [0, 0]

self.size = [1, 1]

# Add a row to the end of the topology

def append_row(self):

self.top += [[0]*self.size[1]]

self.size[0] += 1

# Add a row to the beginning of the topology

def prepend_row(self):

self.top = [[0]*self.size[1]] + self.top

self.size[0] += 1

self.index[0] += 1

# Add a column to the end of the topology

def append_col(self):

for i in range(self.size[0]):

self.top[i] += [0]

self.size[1] += 1

# Add a column to the beginning of the topology

def prepend_col(self):


self.top[i] = [0] + self.top[i]

self.size[1] += 1

self.index[1] += 1

# Set a row, m, to row

def setrow(self, m, row):

if len(row) != self.size[1]:

64

return

self.top[m] = row

# Set a column, n, to col

def setcol(self, n, col):

if len(col) != self.size[0]:

return


self.top[i][n] = col[i]

# Set point (m, n) to val

def setmn(self, m, n, val):

self.top[m][n] = val

# Check if a device is connected in the given direction

def connected(self, direction):

if direction == edefs.LEFT:

if self.index[1] > 0:

return self.top[self.index[0]][self.index[1] - 1]

elif direction == edefs.RIGHT:

if self.index[1] < self.size[1] - 1:

return self.top[self.index[0]][self.index[1] + 1]

elif direction == edefs.TOP:

if self.index[0] > 0:

return self.top[self.index[0] - 1][self.index[1]]

elif direction == edefs.BOTTOM:

if self.index[0] < self.size[0] - 1:

return self.top[self.index[0] + 1][self.index[1]]

return 0

# Merge a topology into self, return if self changed

def merge(self, frm, packed):

(index, unpacked) = self.unpack(

new d.u.s.t.: displays using swarming technology wireless senior...

Documents