sensor recycling and reuse

International Journal of Sensor and Related Networks (IJSRN)

Volume 1,Issue1 , February 2013

http://ijsrn.info/article/IJSRNV1I102.pdf

Sensor Recycling and Reuse Ioannis Deliyannis

#

#Department of Audio and Visual Arts, Ionian University

Plateia Tsirigoti 7, Corfu, Greece [email protected]

Abstract— The concepts of sensor recycling and re-use offer a

new developmental methodology for both centralised and

network-based interactive multimedia systems and multimedia

art applications. This work formalises the proposed

methodology, researches common issues that appear when reuse

or recycling of sensing devices is introduced and discusses its

application within physical interactive systems. It becomes

apparent that the high system development costs typically

introduced in the visual arts domain can clearly be reduced via

the use of alternative recycled and reused sensing devices. A

number of interactive new-media art systems case studies are

presented to demonstrate its flexibility, economy and ecological

advantages. This work aims to render sensor re-use a design

choice that offers an alternative and inexpensive approach from

the theoretical, engineering and artistic perspectives in various

sensor-driven interactive multimedia systems.

Keywords— interaction; sensors; recycled sensors; sensor reuse;

inexpensive sensors; interactive sensor-based applications;

interactive multimedia art.

I. INTRODUCTION

This research lies within the area of sensor

development for interactive multimedia systems

and focuses on the development of sensing devices

used to interact with physical user-input. Typical

candidate systems include installation-art and

computer-based multimedia applications that

capture user input, process, manipulate and trigger

interactive responses. Under complex systems, the

use of commercial sensing solutions can

significantly increase the cost of development,

while the use of proprietary communication

interfaces between sensing components and the

main application system may limit the flexibility to

employ open multimedia authoring systems. On the

other hand various commercial [1] or open source

[2-4] software and hardware-based environments

are available today enabling alternative sensing

devices to be de-composed and/or be re-

programmed in order to implement the desired end-

system sensing functionality. From our research [5]

and practical experience [6] we know that it is

possible to develop sensors via recycling and reuse

of computer-based input devices and network

technologies [7]. The process of sensing

replacement and reuse methodology offers a highly

desirable developmental solution that via the

analysis of practical and theoretical complexities

that affect the performance and the quality of

experience for the user manages to propose

alternative sensing configurations. The proposed

method introduces a classification mechanism

employed for each sensing mechanism in order to

identify its performance and hence its suitability for

the task in hand. Typical data collected for each

device include the sampling resolution, rate,

frequency, data rate, communication interface and

further processing requirements in order for the

sensing output to become accessible by the main

application algorithm. It is therefore important to

research the literature for appropriate frameworks

and methodologies in order to identify related

approaches and methods that may be employed

constructively to address the current research

problem.

A. LITERATURE RESEARCH

New media art systems are classified under

information systems [8]. This implies that their

functions (input, processing, output) can be

specified using standard engineering techniques and

methods [9-11]. As we are only concerned with the

interaction forefront of the end-system or

application, our research focuses on interaction-

based classification. Interactive systems have

evolved over the years and their main characteristic

is the increase of their functional complexity [9, 12].

This is reflected to the user by the availability of

advanced interactive system functions. Similarly to

experimental multimedia systems [12], the

development of self-adapting [13] and component-

8




based [14] systems is limited to software and

theoretical adaptation of existing processes. As end-

system functions are deterministic and precisely

specified by computer coding, a reversed

engineering approach similar to that introduced by

Taylor is followed in this work: “if the analyst

accepts that the analysis will only be valid for a

particular target system then the specification of the

system can be used to infer the behaviour of the

software that interacts with it” [15]. This can be

used to reduce represent system complexity using

the actual functional components (hardware and

software) that are active in end-system, a strategy

that may be recursively applied to identify only the

subset of functional system components for a given

sensing specification.

In the area of interactive art, despite the fact that

creators and End-User Software Engineers (EUSE)

face similar interdisciplinary problems [16], limited

bibliography and few real-life examples may be

accessed publically. This is attributed to the fact

that most artists and developers do not reveal their

techniques openly and each participating group of

experts may employ inherently distinct professional

methodologies during the design and development

the process. Artists make extended use of more

creative and non-formalised processes (sketches,

storyboards, diagrams, often employing other

systems as examples for reference). Engineers

choose systematic approaches through the use of

CASE tools [11] in order to produce precise

specifications at early stages of development and

predict accurately completion time, cost and risks.

Research in interactive multimedia technologies

moves towards the area of sense-enhancement.

According to Hansen, interactive multimedia art is

widely recognised as an «enhanced» creative tool

that appeals to wider audiences more than

traditional forms of Art [17]. Artists active in the

area of new media arts appreciate the capabilities

offered by multimedia technologies and

demonstrate increased interest in the development

of multimedia artwork enabling the transfer of

emotions, experiences, feelings and messages to the

audience. From the scientific forefront this is

clearly aided by the introduction of multimedia

frameworks (MPEG-21). Within certain interactive-

art installations the dividing line between virtual art

and real life becomes blurred [18]. As a

consequence to that, standards for human

interchange with virtual worlds (MPEG-V) are

being established [19-24]. The role of authoring

tools is quite important as they abstract the media-

handling processes, offering automate repetitive

programming tools. Artists focus on interactive

programming in their attempt to compose unique

adaptive artworks featuring advanced sensory

experiences [20] that appeal to a wider audience

ranges. It is apparent that interactive multimedia

research benefits as well as a whole from the

application of technologies in highly demanding

projects which often feature increased processing,

communication, interfacing, interaction and sense-

enhancing requirements [12, 25].

Belucci et. al introduced in 2012 a framework for

rapid prototyping of physical interaction, which

consists of a hardware-based abstraction toolkit

[26]. Other researchers utilise generalised

engineering approaches, tools and techniques for

complex systems specification that present high-

order methodologies in order developers to grasp,

analyse and address specific problems efficiently

[27]. Alternative approaches study partial system

failure options where they recognise that when risk

reduction is part of the system mechanics increases

system efficiency [28], and this is a direction that

clearly may aid under a sensor replacement

approach. Various interesting practical approaches

need to be pointed out as they include the

development of attention-aware systems [29],

application data to application logic linking [30],

content-context sensing for mobile applications [31],

semantic interpretation [32] and inexpensive

developmental approaches [33]. In the case study

forefront typical paradigms involve human-body

interaction [34], robotics [35] and examples where

reprogramming and reuse of existing sensing

devices such as the Wii-controller (Nintendo

Wiimote) are used to develop systems that offer

gesture recognition [36]. From various models that

deal with complex systems, PSM are considered as

the most appropriate for our purpose: “Phantom

9




System Models (PSM), a modeling methodology

inspired by philosophical and conceptual thinking

from the arts, and is driven and supported by

systems engineering theory, methodology, and

practice” [37].

Chapter II examines the problem in hand from

the perspective of the developer and the artist and

proposes the new methodology. Chapter III presents

a number of case studies where via the use of

examples the applicability of the method is tested.

Chapter IV concludes the work by discussing

further research directions.

II. INTRODUCING THE RECYCLED SENSOR REPLACEMENT

METHODOLOGY

The lack of specification methods that focus on

sensor-based new media art systems is evident from

our literature review. It is clear that in order to

capture the specification essence for any given

system, the proposed methodology needs to address

and capture software, hardware and quality of

experience issues. New media arts systems

specification is a complex task due to a number of

interrelated issues: the complexity of the hardware

components output, sensor-based data interface

linking to the main application software

environment and the overall configuration as a

system. Therefore the problem of sensor

replacement may clearly be addressed as a complex

system beyond its relation to the sensing process,

may also affect all other factors: end-system

performance, quality of representation,

communication, software and hardware issues.

System complexity may expressed algebraically

[38], experimentally [39], through models [40] or

discretely [41]. We choose to express it abstractly

using the notion of “dimensions” in order to

describe using a universal notion the functional

complexities. For this purpose we consider the end-

system as one or more applications (A1 to An) and a

number of sensing inputs (S1 to Sn).

The dimensionality of the representation relates

directly to sensing complexity. A one-dimensional

representation is used to represent a single value

that a sensor S1 detects and the routine

communicates to the main application A1. The

value may be multi-typed ranging from a Boolean

value to a numerical value such as distance,

velocity, acceleration, direction, humidity or

temperature. When two one-dimensional sensors

are combined in order the sensing component or the

software to produce the required data for the

evaluation process, the complexity increases to two-

dimensions. A sensor may also produce two-

dimensional output types. For example one may

consider the typical matrix data type produced by a

camera that captures a two-dimensional image.

Using the sensing information as the basic building

block for each sensing device one may then

combine various sensors into systems that require

higher-order sensing complexity. A system with Nd

sensing dimensions requires N0 sensors according

to the following equation:

The above representation enables the

development of an open-ended methodology in

terms of functional and technical characteristics.

The developer can use it to measure and contrast a

system’s sensing complexity to that of other

systems, or as a sensor-selection method enabling

the design of an efficient system from scratch. In

the case of complex systems with n-dimensions of

sensing, it is possible to model the complexity

algorithmically [42]. Automation of the comparison

process can be implemented using the methodology

in the form of a computer program or database

containing the necessary information. This may in

turn be used to run specific queries that output

possible design options using multiple designer-

based criteria.

The issue of temporal sensor-based dimensions

does not always increase complexity, as the whole

process is deterministic. This does not imply that

time cannot affect the sensing quality of the process

and developers are urged to take great care in order

to specify the minimum temporal requirements for

each sensing process. For example, when a frame

differencing motion-detection algorithm is

employed, the picture elements of two images

10




captured at time-states t0 and t1 are numerically

compared in order to detect movement, by detecting

the changes between the images taken at t1 and t0.

Under this scenario, temporal system behaviour is

part of the detection process which is completed

with the comparison of the image elements. If the

developer does not set a minimum frame-rate

requirement for the application in hand, the sensing

complexity of cameras with a lower frame-rate is

equal to those with better performance. In other

words, when temporal information is essential for

the process, it is treated as another “dimension” to

the sensor complexity representation.

Network issues are also categorised under

temporal-based dimensions. In that respect, network

efficiency is calculated across the communications

chain between the sensor and the controlling

application and its suitability is assessed based on

its end-to-end performance.

A. Generalisation of the Recycled Sensor Replacement (RSR) Methodology

The plethora of digital to analogue and digital to

digital sensing devices that are commonly available

today and combine sensing and data interfacing

features is constantly increasing. Mobile phones,

cameras, keyboards, mice, joysticks, scanners,

hands-free microphones, track pads, barcode

readers and many more allow creative flexibility to

interactive multimedia system developers and

artists who wish to cover the interaction

requirements of their projects using efficient and

inexpensive hardware that may be recycled

specifically for this purpose. The key issue here is

not only to cover the functional requirements but to

also provide an pleasant experience that suits the

system’s aesthetics. Therefore often there is a need

for artists to cooperate with developers [11], in

order to overcome practical and technological

difficulties [43] and view the process from the

artist’s perspective [44]. In some instances,

developers may require to merge software

components and allow networked sensing platforms

to communicate and exchange data in order to

achieve the desired effect [45]. It is common for

projects that combine novelty with new

experimental multimedia technologies and content

to require non-conventional engineering approaches.

These difficulties are addressed by the proposed

methodology as it is designed in a modular way that

deals with each component separately, and the end-

system encapsulates other sub-systems, as each of

which serves a highly important role in the process

[46]. This work proposes a software/hardware

selection process based on the project needs and

system compatibilities that the developer may use

as a guide in order to obtain the intended system-

functionality. This is clearly a multi-dimensional

problem that is examined further in order to reduce

the problem’s complexity and allow effortless

component comparison to be implemented. The

propose Recycled Sensor Replacement (RSR)

methodology described below presents developers

with a method that allows identification of the best

combination of software and hardware for their

intended purpose. This is implemented directly by

categorising the parameters of importance for a

candidate system and by excluding inappropriate or

less efficient components automatically. To state

this through a generalised example from the system

developer’s perspective, the selection of appropriate

software and hardware system components from the

list of all available components is filtered using the

following query:

find the least number of authoring environments

that:

i. can be combined in order to present the

new-media-art content to specification

and

ii. can handle the interaction mechanisms and

iii. the developer knows or can learn how to

link and program.

Then for each of those systems that require

sensor-based interaction mechanisms:

a. find the appropriate sensors that

meet the minimum physical

requirements (resolution, refresh-

rate, data transfer rate over the

interface etc.) and

b. present them in an order that places

first those that interface with the

11




system with least and less important

complications.

Clearly system designers must refine further the

above query in order to suit their purposes. For

example in the case one does not need to combine

different authoring environments, queries (i) and (ii)

can be ignored. Or in the case where there exist

multiple repositories covering a large percentage of

the requirements under a system, this may be

entered in the sequence as a fourth (iv) query,

enabling further refinement for the environment

through re-programming and refinement of existing

code segments [3]. In the case where the complex

solution of intercommunication between separate

systems, network-based communication

components will have to be utilised in order to

establish a communication protocol across the

applications, this should be considered in both (a)

and (b) cases. Finally, one may notice that the

sensing complexity is not part of the selection

procedure. This is intentional as conversely to the

algorithmic computer science definition, complex

sensing solutions are not always considered worse

or underperforming, as long as they meet the

minimum physical requirements set in (a) for a

candidate system. Case study III.B presents a

characteristic example.

As one may notice from the above definition, the

RSR methodology functions between the interactive

units (entities) of the system, by matching and

linking the communicated data across the building

components of the system. This may be shown

using a diagram presented in Figure 1, where the

application receives input from either routine A or

B. A sensing entity (routine A or B) combines

software, hardware and data interface components

and evaluation routines necessary to collect, process

and broadcast the data and interfaces with the main

application algorithm.

Fig. 1 Replacement of sensing mechanisms for the same application

Inherently this implies that should the issue of

exchanging an entity with another arises, this may

implemented by switching the input from one

sensor evaluation routine to another. Various

developments may be triggered by this modular

organisation for both hardware and software

components. First each entity encapsulates all the

components required for it to function properly.

Only the output data need to be exchanged with the

main application algorithm, permitting an out-of-

the-box experience for those that experiment with

ready-made libraries, a process that leads to both

software and hardware recycling. Another

advantage is that such organisation may later on

enable automatic testing of components without

human intervention. Furthermore, functional and

tested entities may be exchanged in order to test

their suitability in a variety of inputs.

The interdisciplinary value of the proposed

methodology needs to be mentioned at this point, as

its use may be extended to a wide area of

applications. Interest is evident in multiple

forefronts that include creative [47], academic [48]

and military applications such as the Adaptable

Sensor System (ADAPT) developed by DARPA in

the US [49]. Related engineering processes may be

traced in the literature [50, 51] that specify the

12




increased interaction needs between participating

developing parties. Some view the problem from

the user perspective and specify systems by

considering the human factors introduced in the

design and development process [50]. Clearly as

sensing problems become more complex for

example with the introduction of “Experimental

Multimedia” [12], it becomes apparent that there is

a wider need to clearly specify the sensor

mechanics and factors that can affect the

developmental process from multiple perspectives:

the artist, the developer and the audience.

III. SENSOR REPLACEMENT AND REUSE CASE STUDIES

Today, various computer-based interactive

multimedia applications utilise recycled sensing

technologies due to their wide availability and

minimal cost. The applicability of the method

ranges from the single sensor-replacement and

recycling level to the development of complex

systems evaluating combined sensing output. At the

base level, various examples may be mentioned.

Take for example the case of an interactive drinking

vending system designed to dispense the drink

when this is below certain temperature level.

Various sensor configurations may be consider to

perform such a task, each characterised by different

properties and scenarios: one may embed a

resistance temperature detector (RTD) at the fluid’s

exit and calculate the temperature decay in time,

embed an infrared thermometer at the dispenser and

provide dynamic measurements or develop mugs

from heat-sensitive colour-changing thermal

material and use colour detection to identify when

the fluid is at the appropriate temperature. This

indicates that there are more than one solutions to

the sensing requirements and the appropriate

solution must be selected according to the system’s

sensing requirements.

Typical projects undertaken under the course

Experimental Multimedia at the Department of

Audio and Visual Arts, Ionian University, Greece

have in the past employed a plethora of alternative

solutions to enable interaction: wireless mouse

devices were used to detect motion, mice devices

have been disassembled and its functional rollers

were used to measure distance and acceleration.

Keyboards were used to detect 3D object shapes by

detecting the set of keys that were pressed and

standard web cameras were transformed to input

devices using either colour-detection, motion

(frame-differencing) and light-source detection

through the use of infrared-lights and night-vision

in order to be used as sensing input devices. It is

important therefore to view a number of case

studies presented in increasing complexity and

discuss the findings.

A. DEVELOPMENT OF AN AUDIOVISUAL MUSICAL

INSTRUMENT

This case study aimed to develop an audiovisual

instrument that may be used to compose music and

imagery at the same time. The original hardware

specification included a laptop computer and a

typical piano-based keyboard connected through a

midi interface. Playing music through the musical

keyboard would sound the audio and trigger

selected images with specific transparency settings

to appear compose visuals on screen. As a result,

different performances would result in different

audiovisual effects which could then be randomised

further by dynamic image association to the keys

pressed.

In our attempt enable users to experiment with

the musical instrument without the necessity for

musical hardware and midi interface, the use of the

Recycled Sensor Replacement methodology

revealed that it was possible to replace the piano-

based keyboard with the standard laptop keyboard

in order to trigger the appropriate note

combinations. Each key was programmed to

function as an individual note, emulating the piano-

based keyboard functionality, and upon a key press

was programmed to action both the sound and the

visual, as shown in Figure 2. Programming was

implemented through processing [4].

13




Fig. 2 The coding environment and visual output of the audiovisual

instrument that utilises the keyboard as a musical input device

B. REPLACING INTERACTIVE SENSING MECHANISMS FOR

USER-GAME INTERACTION

Another case study is used in order to consider

replacement of digital to analog input. The game

was implemented on a laptop and the project

required students to alter the code, aesthetics and

interaction design of an existing keyboard-

controlled 2D shoot-em-up game based on the

Adobe Director multimedia authoring environment.

The final aesthetic changes are displayed in Figure

3.

Fig. 3 2D computer game controlled through audio and visual user input

By employing the Recycled Sensor Replacement

methodology we detected at least two sensing

devices that could be used to interact with the game

beyond the standard keyboard: the track pad, the

microphone and the built-in camera. As a result, the

internal microphone was used as an input device

that detects audio level, programmed to trigger the

«firing» function when a specific threshold is

reached. For spaceship movement the local

coordinates originally controlled via the key presses

function were later replaced with direct access to

the track pad-controlled cursor coordinates,

enabling the implementation a much more

responsive interaction mechanism. Following this

implementation and with the requirement to refine

more the system, the “Adobe Director” multimedia

authoring environment was extended with the

“cXtraVideoCapture” allowing colour detection and

positioning of an appropriately coloured object in

front of the camera. Colour detection algorithms are

used to identify the location of an appropriately

coloured object (a plastic water bottle cap was used

in our case) to control the location and position of

the spaceship accordingly. As x,y coordinates were

needed in order to identify the position of the ship,

the sensing algorithm was adapted to find the

geometrical center of the coloured object and return

those coordinates back to the main application

algorithm.

The dimensionality of this problem may be

calculated separately for each sensing component.

The «fire» function is clearly considered one-

dimensional as it may be represented using a

Boolean value. The x,y location function is two-

dimensional as it utilises image array information

used to describe object location in relation to the

camera viewport. Provided that a developer would

like to achieve similar functionality but without the

use of a camera, an alternative way to replace the

original keyboard-based functionality via the use of

a mouse pointing device or a track pad. This offers

a clearly different input device enabling faster

response and accuracy rates, similar dimensionality

and varying cost options that depends on the

availability of equipment selected.

C. SENSING, AUDIO AND VISUAL PRESENTATION WITHIN

MULTIPLE MULTIMEDIA ENVIRONMENTS

Complex case studies introduce the need for

sensing under multiple environments, functional

component synchronisation and coordination. The

interactive installation entitled “Invisible places –

immense white” [6], Figure 4, lies within the field

of biometric-based art applications [52].

14




Fig. 4 The 3D representation of the installation “Invisible places – immense

white”

This life-size installation presented in Strasbourg

in 2008 consists of a room-based installation (figure

5) featuring five projecting screens that display a

video. Visitors interact with the video by wearing a

mental-activity sensing device that transforms the

activity to image, which is composited with the

video, rendered and visualised via the VVVV open

source software.

Fig. 5 The actual installation setting for “Invisible places – immense white”

The issue here was the need for synchronisation

of static and sensor-generated content across

different authoring platforms. The Recycled Sensor

Replacement (RSR) methodology indicated that

from multiple possible configurations, only few

could handle the interaction mechanisms and only

one of those environments (VVVV) was known to

developers. In the sensor forefront various high and

low-level approaches were proposed in order to

identify a safe and precise way to identify mental

activity level: from heartbeat and blood-pressure

measurement equipment to BioSensors, which were

ultimately chosen as other methods did measure

only the effect of mental activity and not the

activity itself. On the system forefront, computer-

controlled video playback was synchronised with

computer-generated composite visual output of the

sensed data. The multimedia stream was

synchronised via retiming as the static content

reproduction sequence was accurately timed and the

interactive system was programmed to synchronise

its output at specific key frames. The 10.2 surround

audio written for this particular interactive

installation was embedded directly with two video

streams, and a single computer using two multi-

VGA output video cards and two audio cards was

used to reproduce and synchronise the video and

audio streams.

Previous examples re-engineered the sensing

mechanisms through the use of human interaction

devices. However the proposed methodology may

be applied beyond the single sensor-replacement

scenario to more complex case studies. Networked

environments featuring self-adjusting sensing

systems can be deployed that are capable to recover

from partial failure [28] and dynamic changes in

their spatial configuration. A typical scenario

currently being developed as an experimental

multimedia project introduces the development of a

soundscape, aiming “to engage the spectator in the

navigation of semantic and sensual space that has

its own quasi-mythical structure” [53]. The project

sensing requirements are clearly be covered by the

proposed methodology as various sensors randomly

positioned in the geographical area can be used to

communicate the experience virtually in another

part of the world. The solution proposed for this

research project includes randomly positioned wind,

humidity, heat, distance, range, object detection,

ultra sonic range measurement devices, IR distance,

vibration and sound sensors, where their location,

orientation and sensing capabilities are known to

the controlling system application and they

15




communicate via wireless Arduino-based

networking modules [54].

IV. CONCLUSIONS

Although the proposed methodology was

presented under a popular form of artistic

expression influential to society [48], that of

interactive multimedia art, it can with be directly

used to cover the requirements of other sensor-

based domains of interest [55]. This work has

introduced the concept of sensor recycling and

reuse in the development of interactive multimedia

systems. The introduction of the Recycled Sensor

Replacement (RSR) methodology enables the

formalisation of the process in order to minimise

development time and enable the use of appropriate

hardware and software components in order to

reduce end-system complexity and guarantee

sensing efficiency. A small number of related case

studies were presented and discussed while many

more examples are traced in the literature. From the

perspective of the artist, the findings are introduced

in the creative forefront, while for developers in the

design offering implementation efficiency. In the

audio and visual art-based interactive systems

presented, replacement of existing hardware-based

sensing mechanisms was implemented via

recycling/reuse of human interaction sensing

devices. Their sensing capabilities were evaluated

and selected sensing features were employed to

cover specific interaction requirements. As re-

engineering of the sensing process relies heavily on

algorithmic solutions, this approach also increased

the software-based sensing workload, a process that

introduces software recycling. However this issue

requires further investigation as various approaches

in the literature can be employed to cover

reusability requirements [10, 11].

Further research directions include the

establishment of a developmental framework that

enables artists to appreciate the limitations of

technology and expand its capabilities in order to

cover their presentation requirements. This

development, combined with the introduction of

new virtual reality multimedia standards will

certainly enable the creation of a powerful sense-

enhancing medium enabling artistic expression and

exploration through interactive multimedia

technologies.

REFERENCES

[1] Adobe. (2013). Adobe Director. Available:

http://www.adobe.com/products/director/

[2] Arduino. (2013). http://arduino.cc.

[3] L. Jaccheri, "Open Software and Art: A

Tutorial," in Entertainment Computing –

ICEC 2011. vol. 6972, J. Anacleto, S. Fels,

N. Graham, B. Kapralos, M. Saif El-Nasr,

and K. Stanley, Eds., ed: Springer Berlin

Heidelberg, 2011, pp. 468-471.

[4] Processing. (2013, 22 November).

http://www.processing.org.

[5] I. Deliyannis, Ed., Interactive Multimedia.

Rijeka, Croatia: InTech, 2012, p.^pp. Pages.

[6] M. Strapatsakis, "Invisible Places -

Immense White," ed. Strasbourg November

2008, 2008, pp. Sensor-Based Interactive

Video Installation.

[7] G. Shea, "Art, design, education and

research in pursuit of interactive

experiences," in Proceedings of the 7th

ACM conference on Designing interactive

systems, Cape Town, South Africa, 2008.

[8] B. J. Oates, "New frontiers for information

systems research: computer art as an

information system," European Journal of

Information Systems, vol. 15, 2006.

[9] E. Nardelli, "A classification framework for

interactive digital artworks," in

International ICST Conference on User

Centric Media, Palma, Mallorca, 2010.

[10] A. Trifonova, S. U. Ahmed, and L. Jaccheri,

"SArt: Towards Innovation at the

Intersection of Software Engineering and

Art

Information Systems Development," C. Barry, M.

Lang, W. Wojtkowski, K. Conboy, and G.

Wojtkowski, Eds., ed: Springer US, 2009,

pp. 809-827.

[11] A. Trifonova, L. Jaccheri, and K. Bergaust,

"Software engineering issues in interactive

16

http://www.adobe.com/products/director/

http://arduino.cc/

http://www.processing.org/




installation art," Int. J. Arts and Technology,

vol. 1, pp. 43-65, 2008.

[12] I. Deliyannis, "From Interactive to

Experimental Multimedia," in Interactive

Multimedia, I. Deliyannis, Ed., ed Rijeka,

Croatia: Intech, 2012, pp. 3-12.

[13] D. Weyns, M. U. Iftikhar, D. G. d. l. Iglesia,

and T. Ahmad, "A survey of formal

methods in self-adaptive systems,"

presented at the Proceedings of the Fifth

International C* Conference on Computer

Science and Software Engineering,

Montreal, Quebec, Canada, 2012.

[14] V. Kumar, A. Sharma, R. Kumar, and P. S.

Grover, "Quality aspects for component-

based systems: A metrics based approach,"

Software: Practice and Experience, vol. 42,

pp. 1531-1548, 2012.

[15] R. Taylor, "Verification of Hardware

Interaction Properties of Software," in

Abstract State Machines, Alloy, B, VDM,

and Z. vol. 7316, J. Derrick, J. Fitzgerald, S.

Gnesi, S. Khurshid, M. Leuschel, S. Reeves,

and E. Riccobene, Eds., ed: Springer Berlin


[16] A. J. Ko, R. Abraham, L. Beckwith, A.

Blackwell, M. Burnett, M. Erwig, C.

Scaffidi, J. Lawrance, H. Lieberman, B.

Myers, M. B. Rosson, G. Rothermel, M.

Shaw, and S. Wiedenbeck, "The state of the

art in end-user software engineering," ACM

Comput. Surv., vol. 43, pp. 1-44, 2011.

[17] M. B. N. Hansen, New Philosophy for New

Media: MIT Press, 2004.

[18] M. Chayko, "What is Real in the Age of

Virtual Reality? “Retraining” Frame

Analysis for a Technological World,"

Symbolic Interaction, vol. 16, pp. 171-181,

1993.

[19] M. Waltl, B. Rainer, C. Timmerer, and H.

Hellwagner, "Enhancing the user experience

with the sensory effect media player and

ambientlib," presented at the Proceedings of

the 18th international conference on

Advances in Multimedia Modeling,

Klagenfurt, Austria, 2012.


Hellwagner, "A toolset for the authoring,

simulation, and rendering of sensory

experiences," presented at the Proceedings

of the 20th ACM international conference

on Multimedia, Nara, Japan, 2012.


Hellwagner, "An end-to-end tool chain for

Sensory Experience based on MPEG-V,"

Signal Processing: Image Communication,

2012.

[22] C. Timmerer, M. Waltl, B. Rainer, and H.

Hellwagner, "Assessing the quality of

sensory experience for multimedia

presentations," Image Commun., vol. 27, pp.

909-916, 2012.


Hellwagner, "Sensory Experience for

Videos on the Web," presented at the

Proceedings of the 2011 Workshop on

Multimedia on the Web, 2011.

[24] M. Waltl, "The next dimension of video

experience: Sensory effects," presented at

the Proceedings of the 2011 IEEE

International Symposium on a World of

Wireless, Mobile and Multimedia Networks,

2011.

[25] K. J. Randall Packer, Multimedia: From

Wagner to Virtual Reality. New York:

Norton, 2001.

[26] A. Belluci, A. Malizia, and I. Aedo,

"Towards a framework for the rapid

prototyping of physical interaction," in

Fourth International Workshop on

Physicality, BCS HCI 2012 Conference,

University of Birmingham, UK, 2012.

[27] J. E. Bartolomei, D. E. Hastings, R. de

Neufville, and D. H. Rhodes, "Engineering

Systems Multiple-Domain Matrix: An

organizing framework for modeling large-

scale complex systems," Systems

Engineering, vol. 15, pp. 41-61, 2012.

[28] N. J. Slegers, R. T. Kadish, G. E. Payton, J.

Thomas, M. D. Griffin, and D. Dumbacher,

"Learning from failure in systems

17




engineering: A panel discussion," Systems

Engineering, vol. 15, pp. 74-82, 2012.

[29] C. Roda and J. Thomas, "Attention aware

systems: Theories, applications, and

research agenda," Computers in Human

Behavior, vol. 22, pp. 557-587, 2006.

[30] B. Hartmann, L. Abdulla, M. Mittal, and S.

R. Klemmer, "Authoring sensor-based

interactions by demonstration with direct

manipulation and pattern recognition,"

presented at the Proceedings of the SIGCHI

Conference on Human Factors in

Computing Systems, San Jose, California,

USA, 2007.

[31] P. Korpipää and J. Mäntyjärvi, "An

Ontology for Mobile Device Sensor-Based

Context Awareness," in Modeling and

Using Context. vol. 2680, P. Blackburn, C.

Ghidini, R. Turner, and F. Giunchiglia, Eds.,

ed: Springer Berlin Heidelberg, 2003, pp.

451-458.

[32] K. Whitehouse, F. Zhao, and J. Liu,

"Semantic Streams: A Framework for

Composable Semantic Interpretation of

Sensor Data," in Wireless Sensor Networks.

vol. 3868, K. Römer, H. Karl, and F.

Mattern, Eds., ed: Springer Berlin


[33] J. Y. Han, "Low-cost multi-touch sensing

through frustrated total internal reflection,"

presented at the Proceedings of the 18th

annual ACM symposium on User interface

software and technology, Seattle, WA, USA,

2005.

[34] B. P. Castro, L. Velho, and D. Kosminsky,

"INTEGRARTE: digital art using body

interaction," presented at the Proceedings of

the Eighth Annual Symposium on

Computational Aesthetics in Graphics,

Visualization, and Imaging, Annecy, France,

2012.

[35] A. P. D. Pobil, "Plenary lecture 2: a

framework for physical interaction and its

execution based on multiple sensors,"

presented at the 12th WSEAS international

conference on Robotics, Control and

Manufacturing Technology, Rovaniemi,

Finland, 2012.

[36] T. Schlömer, B. Poppinga, N. Henze, and S.

Boll, "Gesture recognition with a Wii

controller," presented at the Proceedings of

the 2nd international conference on

Tangible and embedded interaction, Bonn,

Germany, 2008.

[37] Y. Y. Haimes, "Modeling complex systems

of systems with Phantom System Models,"

Systems Engineering, vol. 15, pp. 333-346,

2012.

[38] N. Mastronardi, G. H. Golub, S.

Chandrasekaran, M. Moonen, P. V. Dooren,

and S. V. Huffel, "Numerical Linear

Algebra in Signal Processing

Applications," EURASIP Journal on

Advances in Signal Processing, vol. 2007, p.

3, 2007.

[39] A. Kansal, E. Yuen, W. J. Kaiser, G. J.

Pottie, and M. B. Srivastava, "Sensing

uncertainty reduction using low complexity

actuation," in Proceedings of the 3rd

international symposium on Information

processing in sensor networks, 2004.

[40] H. F. Durrant-Whyte, "Sensor Models and

Multisensor Integration," The International

Journal of Robotics Research, vol. 7, pp.

97-113, 1988.

[41] K. Rudie and J. C. Willems, "The

computational complexity of decentralized

discrete-eventcontrol problems," IEEE

Transactions on Automatic Control, vol. 40,

pp. 1313-1319, July 1995 1995.

[42] M. J. Beal, H. Attias, and N. Jojic, Audio-

Video Sensor Fusion with Probabilistic

Graphical Models. Heidelberg: Springer

Berlin, 2002.

[43] A. Kerne, "Concept-context-design: a

creative model for the development of

interactivity," in Proceedings of the 4th

conference on Creativity & cognition,

Loughborough, UK, 2002, pp. 192-199.

[44] R. Nakatsu, "Image/Speech Processing that

Adopts an Artistic Approach Toward

Integration of Art and Technology," in IEEE

18




International Conference on Acoustics,

Speech, and Signal Processing (ICASSP'97),

1997, p. 207.

[45] T. Mens, "A State-of-the-Art Survey on

Software Merging," IEEE Transactions on

Software Engineerin, vol. 28, pp. 449-462,

May 2002 2002.

[46] C. T. Foo, "Systems engineering

perspectives on the 2500-year-old Art of

War by Sun Tzu," Systems Engineering, vol.

11, pp. 277-286, 2008.

[47] S. Ahmed, C. Camerano, L. Fortuna, M.

Frasca, and L. Jaccheri, "Information

Technology and Art: Concepts and State of

the Practice," in Handbook of Multimedia

for Digital Entertainment and Arts, B. Furht,

Ed., ed: Springer US, 2009, pp. 567-592.

[48] K. Fleischmann and C. Hutchison, "Creative

exchange: an evolving model of

multidisciplinary collaboration," Journal of

Learning Design, vol. 5, pp. 23-31, 2012.

[49] M. Rich. (2012). Adaptable Sensor System

(ADAPT). Available:

http://www.darpa.mil/Our_Work/STO/Progr

ams/ADAPTable_Sensor_System_(ADAPT

).aspx

[50] A. Biswas, "Managing art-technology

research collaborations," Int. J. Arts and

Technology, vol. 1, pp. 66-89, 2008.

[51] A. Biswas and J. Singh, "Software

Engineering Challenges in New Media

Applications," presented at the Software

Engineering Applications SEA 2006, Dallas,

TX, USA, 2006.

[52] E. Edmonds, D. Everitt, M. Macaulay, and

G. Turner, "On physiological computing

with an application in interactive art,"

Interacting with Computers, vol. 16, pp.

897-915, 2004.

[53] T. Tsiridou, I. Zannos, and M. Strapatsakis,

"An interactive windscape," Thechnoetic

Arts, vol. 9, pp. 153-162, 2012.

[54] Website. (2013). Trossen Robotics.

Available:

http://www.trossenrobotics.com/c/arduino-

sensors.aspx

[55] W. Khan, Y. Xiang, M. Aalsalem, and Q.

Arshad, "Mobile Phone Sensing Systems: A

Survey," Communications Surveys &

Tutorials, IEEE, vol. PP, pp. 1-26, 2012.

19

http://www.darpa.mil/Our_Work/STO/Programs/ADAPTable_Sensor_System_(ADAPT).aspx



http://www.trossenrobotics.com/c/arduino-sensors.aspx

http://www.trossenrobotics.com/c/arduino-sensors.aspx

sensor recycling and reuse

Documents