table of contents - infotech oulu

INFOTECH OULU Annual Report 2010 3

TABLE OF CONTENTS

Organization .................................................................................................................................................... 4Foreword ......................................................................................................................................................... 5Infotech Oulu Research Groups ...................................................................................................................... 6

Circuits and Systems (CAS-Oulu) ................................................................................................................8Intelligent Systems Group (ISG) .................................................................................................................17Machine Vision Group (MVG) ....................................................................................................................29MediaTeam Oulu ........................................................................................................................................40Wireless Communication Systems (WICS) ................................................................................................50Biomedical Engineering Group (BME) .......................................................................................................56Electronics Materials, Packaging and Reliability Techniques (EMPART) ...................................................58Human Interaction with Advanced Mobile Services and Intelligent Environments (INTERACT) ................60

Infotech Oulu Doctoral Program .................................................................................................................... 62International Research Co-operation 2006– 2010 ......................................................................................... 66Infotech Oulu vuonna 2010 ........................................................................................................................... 68

4 INFOTECH OULU Annual Report 2010

ORGANIZATION

Governing Board 2007–2010• Professor Jorma Lammasniemi, Scientifi c Director,

VTT, Chairman• Professor Helena Aksela, Department of Physical Sci-

ences• Professor Kalervo Hiltunen, Biocenter Oulu• Professor Kauko Leiviskä, Dean of the Faculty of Tech-

nology• Professor Matti Pietikäinen, Scientifi c Director of Info-

tech Oulu• Dr. Susanna Pirttikangas, representative of young re-

searchers• Professor Jouni Pursiainen, Dean of the Faculty of Sci-

ence

The Governing Board. Front row: Lammasniemi, Pietikäin-en and Pursiainen. Back row: Leiviskä, Pirttikangas, Ak-sela and Hiltunen.

Governing Board 2011–2013• Professor Jussi Paakkari Vice President, R&D, ICT,

VTT, Chairman• Professor Helena Aksela, Department of Physical Sci-

ences• Professor Harri Haapasalo, Dean (research), Faculty of

Technology • Professor Kalervo Hiltunen, Biocenter Oulu• Professor Jari Oksanen, Dean (research), Faculty of Sci-

ence• Professor Matti Pietikäinen, Scientifi c Director of Info-

tech Oulu• Dr. Susanna Pirttikangas, representative of young re-

searchers

Doctoral Program Board 2010–2013• Professor Markku Juntti, University of Oulu, Depart-

ment of Electrical and Information Engineering, Direc-tor of the Doctoral Program, Chairman

• Research Professor Pentti Karioja, VTT• Academy Professor Juha Kostamovaara, University of

Oulu, Department of Electrical and Information Engi-neering

• Professor Kari Kuutti, University of Oulu, Department of Information Processing Science

• Professor Matti Latva-aho, University of Oulu, Depart-ment of Electrical and Information Engineering

• Professor Juha Röning, University of Oulu, Department of Electrical and Information Engineering

Advisory Board• Timo Kenakkala, City of Oulu, Chairman• Martti Artama, Metso Automation• Ilkka Heikkilä, Polar Electro• Pekka Heinonen, Nokia• Panu Jalas, Fairchild Semiconductor• Jukka Klemettilä, Oulu Innovation• Erkki Oja, Aalto University• Jussi Paakkari, VTT, Oulu• Asko Pietarila, Tekes, Oulu• Matti Pietikäinen, University of Oulu, Infotech Oulu• Leila Risteli, University of Oulu, Research Services• Veikko Seppänen, University of Oulu• Kauko Suhonen, Powernet• Ari Takanen, Codenomicon

The Advisory Board of Infotech Oulu.


FOREWORD

Scientifi c Director Matti Pietikäinen, Doctoral Program Director Markku Juntti and Coordinator Tapio Repo, Infotech Oulu

mkp(at)ee.oulu.fi ,markku.juntti(at)ee.oulu.fi , tre(at)ee.oulu.fi

The principal goal of Infotech Oulu is to create an environ-ment for the development of world-class research groups by promoting long-term research, researcher training, and international cooperation. An independent international panel evaluates the research groups that apply for mem-bership of Infotech Oulu, using criteria similar to those used by the Academy of Finland. With such regular ex-ternal evaluation and with our internal quality control, we can help our research groups succeed and give them the valuable Infotech label. Our new four year period started in 2010. We have fi ve groups as full members, and three groups as associate members. An important part of our operations is the Infotech Oulu Doctoral Program, which supports all our goals by providing funding for doctoral student positions and for lectures given by international experts that visit our research groups.

The University of Oulu is currently experiencing organiza-tional changes, especially with regard to doctoral studies. The University of Oulu will in the future have only one graduate school called the University of Oulu Graduate School (UniOGS). As part of transition on a national level, what used to be called graduate schools, like the former In-fotech Oulu Graduate School, are now called doctoral pro-grams. The Infotech Oulu Doctoral Program general meet-ing elected a new director, Professor Markku Juntti, start-ing in 2010. UniOGS will govern all doctoral programs and also the international master programs of University of Oulu in the future. This represents a relatively signifi -cant change, because the role of faculties and departments will diminish and the tasks of doctoral programs may in-crease. The Infotech Oulu Doctoral Program will naturally be involved in this process, and also actively contribute to this development. Our key target is to maintain and de-velop the best practices and the high quality of doctoral education within the Infotech Oulu Doctoral Program.

Infotech Oulu has achieved its best concrete results through the doctoral program. The present 25 student po-sitions fi nanced by the Ministry of Education and Culture are funded to the end of year 2013. In addition, an annual EUR 107 500 was granted by the Academy of Finland for courses and other costs for the same time period. The re-search student positions are vital for our researcher train-ing. They represent almost one million Euros annually, and about one third of the external long term research funding of our research groups. The number of doctoral disserta-

tions within Infotech has grown signifi cantly during the last few years. In 2010, the output was 17 dissertations.

One of the principal goals of Infotech Oulu is to promote international collaboration. Most of our groups have al-ready extensive collaboration with overseas partners. Both bilateral cooperation between research groups, and partici-pation in EU or other international research programs are important. Infotech Oulu and its doctoral program have promoted international activities by providing fi nancial support for organizing workshops, and arranging courses and lectures given by visiting experts. We have also spon-sored researcher mobility. In 2010, Infotech Oulu research groups were awarded funding from the University of Oulu for three positions for recruiting international experts from abroad. The personnel of our research groups has become more international year by year. At the end of 2010, 21.2% of our whole staff and 27.8% of the doctoral students were from abroad.

Direct annual support from the University to Infotech Oulu was once again EUR 400 000 after a temporary increase in the previous year. The research groups faced remarkable fi nancial diffi culties resulting from the fact that almost all the research funding left from the previous years could no longer be used. However, our research is still highly ranked by the external funders. The total external annual research funding was about EUR 15.5 million. Our most signifi cant funding source is Tekes (the Finnish Funding Agency for Technology and Innovation). We have been pleased to no-tice that the greatest increase during the last few years has been with funding from the Academy of Finland, even if the year 2010 was not so good in that sense.

The scientifi c output has been continuously increasing, and many of our papers are frequently cited. A citation analysis we made shows that our most cited professors compare well to the Academy Professors and the Finnish Centre of Excellence leaders in the ICT area with the number of ci-tations in papers published between 2003–2008. Starting from the beginning of 2010, two important positions were granted to our research group directors: Professor Heli Jantunen became one of the 11 members of the Research Council for Natural Sciences and Engineering in the Acad-emy of Finland, Professor Olli Silvén was selected to be the Vice President (Provost for Education) of the Univer-sity of Oulu. In addition, Professor Juha Kostamovaara continues as an Academy Professor.


The research of Infotech Oulu is carried out in groups. The research groups mainly operate in those laboratories and departments where their directors are working. The selec-tion of Infotech Oulu research groups is for a four-year period, and it is based on external scientifi c evaluation of the research presented by the applicants. For the period 2010–2013, fi ve groups were selected as full members, and three groups as associate members. Some of them are joint groups at the University of Oulu and VTT Techni-cal Research Centre. The evaluators are internationally renowned scientists in information technology: Professor Hans Zappe from the University of Freiburg, Germany, Professor Willy Sansen from Katholieke Universiteit Leu-ven, Belgium, Professor Anja Skrivervik from EPFL, Lau-sanne, Switzerland and Professor Tuomas Sandholm from Carnegie Mellon University, Pittsburgh, USA.

The following groups were selected to Infotech Oulu as full members:• Circuits and Systems Group (CAS - Oulu)• Intelligent Systems Group (ISG)• Machine Vision Group (MVG)• MediaTeam Oulu• Wireless Communication Systems Group (WICS)and as associate members:• Biomedical Engineering Group (BME)• Electronic Materials, Packaging and Reliability Tech-

niques (EMPART)• Human Interaction with Advanced Mobile Services and

Intelligent Environments (INTERACT)The associate members do not receive direct funding from Infotech Oulu, but the doctoral students in these groups can apply for doctoral program positions.

Information technology is designated as one of the four focus research areas of the University of Oulu, including machine vision and ubiquitous computing technologies, wireless communications, high-speed electronics and pho-tonics, biomedical engineering, and information systems and software. The main research areas of Infotech Oulu are• electronics• communications engineering• computer science and information engineering

FundingThe University of Oulu provided special funding of EUR 400 000 for Infotech Oulu in 2010. About 75% of this was directed to the support of the full member groups. For the year 2010, EUR 726 500 was granted by the Ministry of Education and Culture to the Infotech Oulu Doctoral Pro-gram for its 25 student positions and a coordinator posi-tion. In addition, the Academy of Finland granted EUR 430 000 for the years 2010–2013. From this sum, EUR 107 500 was for the year 2010. This funding was for ar-ranging courses given by visiting experts and for cover-

ing traveling costs of doctoral students when taking part in conferences or working abroad.

The research is mainly funded externally. The total sum of annual external funding for the full members was EUR 11.8 million in 2010. Total external research funding, in-cluding that for the associate members, was EUR 15.5 mil-lion. Only about a quarter of the research work is fi nanced through internal funding in the form of salaries of profes-sors, research assistants, etc., excluding their estimated time used for teaching. The main source of fi nancial sup-port is Tekes, the Finnish Funding Agency for Technology and Innovation, representing over one third of our external funding.

The following table shows the total external funding of the research groups in 2010, including the associate mem-bers. Also the funding for the full members is presented in parenthesis. All the funding of the full member groups is channeled through the university, whereas the total fund-ing includes also funding channeled through VTT. The sums cannot be compared directly to the previous year, because the groups are no longer the same. When looking at the funding on the research group level, it can be seen that there has been a small increase in the total sum.

Source EUR

Academy of Finland 3 380 000 (2 385 000)

Ministry of Education and Culture (doc-toral programs)

1 130 000 (725 000)

Tekes4 879 000

(3 788 000)

other domestic public 367 000 (367 000)

domestic private 1 874 000(1 487 000)

international 3 856 000(3 042 000)

total 15 486 000(11 794 000)

PersonnelThe number of personnel in the full member research groups of Infotech Oulu was 287 at the end of 2010. The total number of personnel, including the associate mem-bers, was 376. When this is compared to the previous year on the research group level, there has been a small increase, like in external funding. Personnel from abroad represents 21.2% of the total staff, and 27.8% of the doc-toral students. The table below shows the total number of personnel in different positions. The personnel in the full member groups is in parenthesis. The number of profes-sors also includes acting professors, FidiPro professors and VTT’s research professors. The number of the uni-

INFOTECH OULU RESEARCH GROUPS


versity nominated professors participating in the activities of the groups is about 25. The category “others” includes Master’s level students, secretaries and other support staff. Almost 95% of the personnel work at the University of Oulu, and almost all the others at VTT. Two of the groups have an associate director from VTT.

professors 36 (23)

other senior and postdoctoral researchers 63 (42)

doctoral students 184 (129)

others 93 (93)

total 376 (287)

person years 251 (202)


Background and MissionThe Circuits and Systems group consists of about 30 re-searchers working at the Electronics Laboratory of the De-partment of Electrical and Information Engineering at the University of Oulu. Its main activity is in the fi eld of elec-tronic and optoelectronic circuit and system design. The primary implementations are based on CMOS and BiC-MOS IC technologies. The main interest of the group is devoted to certain novel circuit topologies and functional units, although the group is also interested in applications, especially in the fi eld of telecommunications and electron-ic/optoelectronic measurements.

The main research fi elds are:• time-to-digital converters and timing circuits • generation and detection of powerful and high-speed

electrical and optical pulses/transients, breakdown phe-nomena in semiconductors in general

• development of pulsed time-of-fl ight laser range fi nding technologies, especially for industrial applications

• radio telecommunications including linearization of power amplifi ers, AD/DA conversion and baseband blocks, frequency synthesis

The group has created a well-functioning partnership with some international research units working in the same or in a complementary fi eld. The group is mainly funded by the Academy of Finland, Tekes and industry.

Scientifi c ProgressIn the following, some details and results of the work of the group are given in selected important research fi elds.

Time-to-Digital Converters and Timing Cir-cuitsA 6-Channel Time-to-Digital Converter CMOS Chip Based on Multi-Phase Delay Line Interpo-lationA 6-channel time-to-digital converter (TDC) circuit has been developed which is especially targeted at pulsed time-of-fl ight laser range-fi nding studies. In this circuit, time-to-digital conversion is based on a counter and stabi-lized delay line interpolation, which combined, provides a ms-level linear dynamic range, stable ps-level LSB reso-lution and good temperature stability. The TDC utilizes autocalibration, i.e. the resolution of the TDC is locked to the cycle time of the external reference crystal. No ad-ditional calibration cycles are thus needed during the mea-surement.

In its main operation mode, the TDC digitizes the arrival time of the rising edge of the start pulse, and the rising and falling edges of a maximum of three stop pulses in one stop channel, as shown in Figure 1 below. These results enable the TDC to calculate not only the pulse positions with respect to the start signal (TSP1 - TSP3), but also the widths of the stop pulses (Tw1 - Tw3), which may be pro-portional to the amplitudes of the detected analog receiver signals. In pulsed time-of-fl ight laser ranging, these sepa-rate stops might represent echoes from the case window, the effect of rain at the distance where the optics is most sensitive and from the wanted object, for example. Pulse width measurement enables one to compensate for the tim-ing walk error induced by varying echo amplitudes. The 6-channel TDC is based on a TDC core that is shown in detail in Figure 1.

Figure 1. Timing diagram and architecture of the 6-channel CMOS TDC.

The fi rst coarse interpolation level creates many small, even length time samples inside each reference clock cycle. The interpolator recycles the reference clock rising edge in a short (a small number of cascaded delay ele-ments) delay line many times during each reference clock period. After N recycling rounds a new, jitter-free refer-

CIRCUITS AND SYSTEMS (CAS-OULU)

Professor Juha Kostamovaara and Professor Timo Rahkonen,Electronics Laboratory, Department of Electrical and Information Engineering, University of Oulu

juha.kostamovaara(at)ee.oulu.fi , timo.rahkonen(at)ee.oulu.fi http://www.infotech.oulu.fi /cas


ence edge is transmitted to the delay line, and the recycling begins again. The recycling is realized by connecting the end of the delay line to the fi rst element with a multiplexer, as shown in the fi rst fi gure. This multiplexer is controlled with a counter, which counts the recycling rounds. A small number of delay elements improves the integral nonlinear-ity of the delay line. The reference clock frequency can be decreased simultaneously by increasing the number of recycling rounds N. The cascaded delay elements produce time samples with an even time margin (τ1), when the re-cycling signal propagates in the delay line. The incoming timing signals (start and stop) record the states of the delay line, and control (enable and disable) the counter, which counts the rounds of the recycling delay line between the timing signals. This interpolation structure with a counter makes long range time interval measurement possible with the resolution of delay element delay τ1.

Time interval measurement resolution of one delay element unit delay (τ1 ≈ 320 ps @ 0.35 μm CMOS) can be achieved with the interpolation structure mentioned above. In order to achieve higher resolution, sub-gate-delay interpolation structures have to be used. The second interpolation level (fi ne interpolator) interpolates the location of the timing signal within the result of the fi rst interpolation level. The region of the result of the fi rst level (τ1) is interpolated with ~10 ps resolution by utilizing a parallel load capacitor-scaled delay line structure. The synchronized result from the coarse interpolator (sync-signal) creates 8 parallel time samples with τ2 ≈ 10 ps resolution. Simultaneously the de-layed, asynchronous timing signal (start or stop) creates 8 time samples with an 8τ2 ≈ 80 ps time difference. The co-incidence of these time samples is registered, and it defi nes the second interpolation level result.

The characteristics and the measured performance of the developed TDC core is presented in Table 1. The block diagram and layout of the TDC realization are shown in Figure 2.

Table 1. Performance of the CMOS TDC.

Description Specifi ed Value

Technology 0.35 μm CMOS

Resolution LSB ≈ 10 ps

Measurement precision ≈ 8 ps

Measurement range +/-83.5 ms, equals +/- 12.5 km

Nonlinearity < ±10 ps (± 1.5 mm) range < 1 m

Nonlinearity < ±2 ps (± 0.3 mm) range >1 m

Temperature driftfrom start to stop pulseIn pulse width measurement

-0.5 ps/°C (9 mm @ -40 .. 85 °C)-3.0 ps/°C (50 mm @ -40 .. 85 °C)

External reference 3 MHz - 20 MHz

Internal frequency ≈ 200 MHz typical

Conversion time after last stop ≈ 50 ns

SPI clock frequency max 100 MHz

Timing signal inputs LVCMOS compatible

Other IO-signals LVCMOS compatible

Supply voltage 3.3 V typical (3.0 - 3.6 V)

Power consumption ≈ 85 mWTemperature range -40 °C .. 85 °CCircuit size 2.4 mm x 3.7 mmPackage QFN 36

Figure 2. Block diagram and layout of the 6 channel CMOS TDC circuit realization.

A Sub-ps-level Resolution Time-to-Digital Con-verter CMOS Based on Time Domain Succes-sive Approximation InterpolationA time-to-digital converter (TDC) with sub-ps resolution and μs-level dynamic range is under development. The sub-ps resolution is achieved with interpolation based on a cyclic time domain successive approximation (CTD-SA) method utilizing a pair of digital-to-time converters (DTC), between which the propagation delay difference is implemented by digitally controlling the unit load capaci-tors of their delay cells and also by digitally controlling the discharge current of the load capacitances. The research work focuses on improving the dynamic range and linear-ity of the DTC.

To increase the linear dynamic range of the DTC, the out-put inverter has been replaced with a comparator. Instead of using the process parameters dependent threshold volt-age of an inverter to detect the discharging of the load ca-pacitance CL, a comparator with a stable reference voltage Vref is used. According to the simulations, a linear dynamic range of 12-13 bits, i.e. several ns, with sub-ps resolution can be achieved with the comparator, because the com-parator is much less sensitive to the slew rate of the input signal than the CMOS inverter. To further improve the res-olution of the DTC, and to reduce the number of load ca-pacitances, also the discharge current of the current starved inverter is digitally controlled. The overall operating point of the DTC is set by the bias current k⋅Iref, where Iref is a reference current, and k is the common control word set by a stabilizing loop to reach the desired dynamic range and resolution. The bias voltage of the previous design has been replaced by a current DAC (digital-to-analogue con-verter) to have digital control over the discharge current of the inverter. Furthermore, the discharge current can be scaled by a factor 2q during conversion. This has the same effect as the scaling of capacitances, and this method is used after the minimum capacitance value has been reach in the conversion process. The load capacitance can be controlled either linearly, n⋅CL, if we want to adjust the propagation delay of the DTC in general time interval generation, or as powers of two, 2n⋅CL, as required by the binary search algorithm of successive approximation, see Figure 3.


Figure 3. Conceptual schematic diagram of the new DTC with sub-ps resolution and ns-level dynamic range.

A Time-to-Digital Converter Board Achieving 1 ps Measurement Precision A time-to-digital converter based on time-to-amplitude interpolation was designed and tested. The converter was implemented with discrete components. The measurement range is about 0 - 328 μs with a state-of-the-art single shot precision of 1.5 ps. The time interval is measured by counting full clock cycles with a counter implemented in an FPGA circuit and the fractional time residue is then measured with time-to-amplitude converters that provide the picosecond level precision. The block diagram of the TDC is presented in Figure 4, and the actual realization in Figure 5.

Figure 4. Simplifi ed block diagram of the TDC based on time-to-amplitude interpolation.

Figure 5. The TDC board (10 cm * 10 cm).

The interpolators consist of a constant current source, a fast switch, a Miller integrator and offset compensation circuitry. The interpolator charges a capacitor with a con-stant current during the fractional time residue. The out-put voltage of the interpolator is then AD converted and

combined with the result of the full cycle counter. With a reference clock of 200 MHz, the measurement range of a single time-to-amplitude interpolator is 5 ns. All the bias voltages are produced by DA-converters that can be con-trolled via software. The nonlinearities of the interpolators reduce the single shot precision to about 1.5 picoseconds. The integral nonlinearities of the TACs are shown in Fig-ure 6. When this systematic INL error is compensated, single-shot precision can be improved to ~1 ps.

Figure 6. INL of the time-to-amplitude converters.

Figure 7 shows the distribution of measurement results when a constant delay of 396 ns was measured. The stan-dard deviation of the INL compensated results corresponds to a single-shot precision of 0.9 ps. This result is believed to represent the state-of-the-art among published time-to-digital converters.

Figure 7. Distribution of time interval measurement re-sults.

An Integrated Laser Radar ChipA CMOS laser radar chip, including an optical receiver channel and a time to digital converter (TDC) integrated onto the same die, and realized using a 0.13 μm CMOS technology has been tested as the part of a pulsed time-of-fl ight (TOF) laser rangefi nder. The CMOS laser radar chip is based on a receiver channel using a slew rate measure-ment for the walk error compensation.

The linearity of the laser radar system was measured by sweeping a target from 1 meter to 21 meters with a 0.5 me-ter step and with a one meter step using a white paper and a diamond grade refl ective sheet as targets, respectively.


The dynamic range of more than 1:2000 could be achieved in this measurement. Figure 8a shows the accuracy of the measured results as a function of the distance with the white paper (black line) and the diamond grade refl ective sheet (grey line) used as targets. The accuracy of the sys-tem is better than +/- 5 mm from 4 m to 21 m. Peaks at the distance of approximately 2 m are due to disturbance from the comparator of the discrete start channel.

The precision of the laser radar chip was measured over the whole dynamic range 1:10000 by using an optical den-sity fi lter with two different threshold values of the com-parator used for walk error compensation (the threshold value of a stop channel comparator multiplied by two and three). Figure 8b shows the measured single-shot precision standard deviation value as a function of the signal ampli-tude of the received pulse with those two different thresh-old values 2*(SNR=10) and 3*(SNR=10). These curves include the jitter of the TDC, the jitter of the receiver chan-nels (stop and walk error compensation channels) and the inaccuracy of the look-up table used for walk error com-pensation. The solid curve shows the single-shot precision standard deviation value of the stop channel, including of the jitter of the TDC and the receiver channel.

Figure 8. Linearity and precision of the laser radar receiver CMOS IC.

A Single Photon Avalanche Diode (SPAD) for Raman SpectroscopySeveral time gated single photon avalanche diode (SPAD) structures aimed for use in Raman spectroscopy were fab-ricated and tested in 0.35 μm HVCMOS technology. A

SPAD can be implemented as a reverse biased p-n junc-tion, and with a lightly doped guard ring can be used to prevent an unwanted surface breakdown. When the diode is biased above the breakdown voltage, even a single pho-ton can induce the breakdown and can then be detected by conventional digital circuitry. In addition, a time-gate function was realized in the SPAD; this is because the out-put of the SPAD is read only during the short laser pulse illumination to suppress the fl uorescence background, typ-ically dominating in the Raman spectra. This arrangement reduces also the dark count rate of the SPAD, resulting in a better SNR.

Figure 9 shows the layout of the SPAD die (a), and the cross section of a single SPAD (b). The diameters of the tested circular SPADs were 10 μm and 20 μm. Figure 10 shows the dark count rates (DCR) of 10 μm and 20 μm (a) as a function of excess bias voltage (voltage above a breakdown). The dark count rates can be decreased by us-ing time gating. The measured DCR of the 20 μm SPAD with a time gate window of 500 ps was measured to be 0.45 Hz, when a pulse rate of 120 kHz and an excess bias of 2.3 V were used. Experimental data of the reverse I-V characteristic is shown in Figure 10b, indicating a break-down voltage of approximately 18.9 V.

Figure 9. Layout and cross section of the CMOS SPAD.

Figure 10. Dark count rate and breakdown characteristics of the CMOS SPAD.

(a)

(b)

(a)

(b)

(a)

(b)


Generation of Electrical/Optical TransientsHigh-energy Picosecond Laser Diode Pulse GenerationIn our recent work, we proposed a new laser diode struc-ture that, based on its inherent properties, can produce with quite a simple driving scheme a ten-fold improvement in the achievable optical energy of a single picosecond opti-cal pulse, compared to a standard laser diode confi gura-tion working in gain switching mode. The main idea is to utilize the structure with an extremely large active layer width (da) to optical confi nement factor ratio (Гa) (da/Гa >> 1 μm). Such a large value of the “equivalent spot size” (=da/Гa) is rather unusual since it would in principle mean a higher steady state value for the lasing threshold cur-rent. The typical value of the equivalent spot size (da/Гa) is ~1 μm for high power CW operated lasers, for example.

However, as we have shown, from the dynamic behavior point of view, the use of the specialized construction with da/Гa >> 1 μm results in an increased time delay between threshold point and the beginning of the laser generation (“enhanced gain switching”). So we can accumulate more carriers in the active layer before beginning of the genera-tion. As a result, higher pulse energy will be generated.

It is not however a trivial task to experimentally realize a structure with such a high value for da/Гa. We have shown that in the cases of bulk or Quantum Well (QW) edge emit-ting lasers, this can be, in principle, achieved in strongly asymmetric waveguide structures which also have the advantage of supporting only a single fundamental trans-verse mode.

As an example of the performance that is available from this device, recent results obtained using a bulk GaAs/GaAlAs laser diode chip are shown in Figure 11. This test chip had no refl ection coatings on its facets, and thus the total peak output power available in a single pulse was roughly twice that seen in Figure 11. As a result, a pulse with a peak power of >20 W (stripe width 120 μm) and width of less than 150 ps should be available in fundamen-tal transverse mode from this bulk laser diode operating at about 850 nm with a pulse drive current of approx. 10 A.

Figure 11. Measured driving current and optical output pulses (only from the other edge) of the suggested DH bulk laser diode based on “enhanced gain-switching”.

A Unique GaAs Avalanche Switch: Effi ciency, Stability and Reliability A drastic reduction in the residual voltage (from ~100 V to a few volts) and a signifi cant (factor of ~ two) increase in the dV/dt switching rate is demonstrated experimentally in the superfast (~200 ps) avalanche switching of a GaAs bipolar junction transistor (BJT) with an increased emitter area, see Figure 12. This result is not a trivial one, as only a small number of conductive channels of a few microns in diameter participate in the transient independently of the emitter size, while the remaining (passive) part of the structure supplies the switching channels with the currents circulating inside the chip, which makes the impact ioniza-tion in the fi laments more powerful.

Figure 12. Switching transient in specially designed and fabricated GaAs avalanche transistors with “small” and a “large” emitter areas [3].

Excellent agreement was found between the experiment and a specially developed quasi-3-D model, which provid-ed a convincing physical explanation for drastic improve-ment in the switching effi ciency, stability and reliability of a “large” area device with an increased emitter area, and also much lower power dissipation in the structure.

The transient thermal task in a GaAs avalanche BJT was achieved for the fi rst time, and it was shown that only conic spreading of the “collapsing” fi eld domain front (Figure 13) inside n+ - substrate prevents overheating of the switching channels, while increase in the switching ef-fi ciency for the device of increased area limits the highest local temperature inside the structure to as a low value as ~400 K (Figure 14).

Figure 13. Penetration of the conic ionization front into an n+ subcollector reduces drastically the peak power dissipa-tion.


Figure 14. Temperature profi les along the switching chan-nels during the switching transient (a) in the transistor of “small” (b) and “large” (c) areas.

Another problem which is critically important for device reliability is an avalanche breakdown on the mesa-surface of the device, which may cause current instability lead-ing to device destruction when the transistor is biased near the bulk breakdown voltage. We have found a new mechanism which suppresses the avalanche multiplication on a negatively beveled mesa surface (intrinsic of a GaAs avalanche BJT) and permits the bulk breakdown voltage to be achieved. This important phenomenon we termed Ava-lanche-assisted carrier trapping.

Avalanche-assisted Carrier Trapping on the Sur-face of a GaAs Avalanche BJT (ABJT)This mechanism [11,12] consists in electron trapping on the deep surface acceptors, whose negative charge broad-ens the space-charge region on the surface, thus reducing the peak of the electric fi eld and suppressing the breakdown process. Once started, the surface avalanche breakdown initiates negative feedback, which “softens” radically the breakdown I-V characteristics (Figure 15) and allows the voltage to be increased up to a volume breakdown value with a gradual increase in the surface current up to an ac-ceptable level.

Figure 15. “Soft” surface breakdown of the collector-base junction of a GaAs avalanche BJT. Curves 1 and 1’ cor-respond to two samples: the highest and lowest currents observed in the experiment. Curves 2-4 simulated at differ-ent scenarios for initial charging of the surface traps along the mesa perimeter.

This mechanism allows using a GaAs avalanche BJT in the practically important case where technological condi-tions do not allow a guard ring or passivation layer to be formed on a negatively beveled mesa-surface.

3-D Phenomena in a Si ABJTCurrently Si ABJT’s are the commercial components most frequently used for nanosecond pumping of pulsed laser diodes. Surprisingly, we have only recently explained a comprehensive physical description of its operation at high currents using 1-D and 2-D approaches. Very lately, we have found additionally a trade-off between the switch-ing effi ciency of a Si ABJT in short-pulsing (~2 ns) mode, and its reliability with long (close to 10 ns), high-current (~100 A) pulses. This trade-off is determined in turn by a competition between two 3-D effects: one is the turn-on spread along emitter-base interface, and the other one is a competition between different parts of the perimeter with inhomogeneities according to “the winner takes all” prin-ciple. This work started a few years ago and now continues. We suggested earlier, and have lately shown [13] that turn-on spread, takes place indeed in the very beginning of the switching transient, while the second mechanism, which is more important for the reliability with long pulses, appears to be more complicated and its investigation requires fur-ther efforts. This activity is critically important for correct design of the transistor chips (and used circuitry) depend-ing on the required switching parameters.

THz Emission from a GaAs BJT Structure during its Avalanche SwitchingAnother very promising application for the avalanche switching mode in GaAs BJT is the generation of pulsed broad-band terahertz emission. Periodical nucleation and annihilation of ultra-narrow, powerfully ionizing “col-lapsing” domains is believed to cause the THz emission observed in our experiments. The task is very challeng-ing and should be divided into several steps, most urgent and important of which is the correct characterization of the spectral power density in a broad spectral range. It requires development and characterization of a large number of band-pass spectral fi lters in the sub-THz range, investigation of different transistor structures with differ-ent chip designs, etc. A solution to those tasks is under-way (see an example for one of the emitting transistors shown in Figure 16). In this particular case, a room-tem-perature quasi-optical detector based on a Schottky diode was used, while in general we use also specially designed high-speed NbN and MoRe (low noise) bolometers and a Golay cell. The main applications of our future emitters we see in new generation 2-D and 3-D mm-wave and sub-mm wave imaging; the work in this direction has already started [14,15].


Figure 16. An example of a Schottky diode response on the emission of a GaAs ABJT passed through different narrow-band, band-pass fi lters. One can see that different spectral modes exhibit different waveforms.

Circuit Analysis Techniques and ResultsDistortion Contribution Simulation Algorithm VoHBDr. Janne Aikio has been developing a general-purpose distortion contribution analysis called VoHB (Volterra on Harmonic Balance), and in 2009 it appeared as a func-tional, multi-device, model-independent prototype within AWR-Aplac’s analog simulator. During 2010, its capabili-ties have been extended, and it has been verifi ed against many test circuits, whose distortion contributions have been previously hand-analyzed in the literature. Two jour-nal publications are under preparation, and an example study of the ways found for improving the linearity of cur-rent-steering IF DA converters was reported in the Nor-chip 2010 conference.

The new capabilities of VoHB include support for a multi-tone and high-order analysis (e.g. a Doherty type RF power amplifi er was simulated with 9th-order modeling), and adaptive building of polynomial models. In the latter, DC bias circuits are, for example, recognized and modeled with linear models, or VCCS elements (voltage-controlled current or charge sources) that have several, seriously cor-relating input signals are also recognized and character-ized separately with somewhat perturbed test signals to break the correlation between the original control signals.

Furthermore, the polynomial models fi tted by VoHB can be used for a traditional AC Volterra analysis, and this has been employed for a very fast harmonic load pull analy-sis. The implemented analysis can perform 5000 analyses (performed by Volterra analysis, but corresponding to a two-tone harmonic balance simulation) in two seconds. Such a high speed facilitates much more careful analysis of optimum harmonic matching, for example, and opti-mizing the linearity over a broad bandwidth range.

Design Methods for Switching Amplifi ersLessons from a previously integrated GaAs RF PA were reported by Hietakangas in [10] in 2010. The research on switching amplifi ers was continued. First, a rather exhaus-tive design space exploration was made for an inverse class E amplifi er to see how sensitive switching amplifi ers are to mistuning. It was found that with a broadband re-sistive load, the effi ciency maximum is rather broad (and hence less sensitive to dimensioning errors), but if the load itself is seen through a narrowband impedance matching network, the matching network seems to have a signifi cant effect on the broadness of the optimum. This was a new fi nding, as the vast majority of publications just assume a broadband resistive load.

The research continued by studying the effects of non-lin-ear output capacitance in a class E power amplifi er, also in EER type applications. It was found that a non-linear output capacitance increases the peak voltage, and hence the voltage stress of the switch, and it also causes severe Vdd-PM distortion in EER type applications. It was also found that the very often suggested 50% duty cycle is not the only possibility, but both the level of the harmonics and the voltage stress can be reduced by extending the off-time of the switch.

By the end of the year, the study moved to the input side of a class E amplifi er, and supply-modulated switching amplifi ers were found to constitute a very diffi cult load: the fundamental tone input impedance varies very strongly with the supply voltage, and due to the resonant output network, the input impedance shows a negative real part. The output resonator also causes heavy 2nd harmonic injec-tion from drain to gate.

A researcher exchange carried out with Trondheim Techni-cal University in 2009 has resulted in three joint publica-tions.

Biomedical ApplicationsOlli Kursu began work in the Infotech Oulu Graduate School in 2010, and his area is – together with Dr. Mikko Vähäsöyrinki’s group at Dept. of Physics – the measure-ment of nerve signals of small animals. During 2010, he fi nalized and reported a servo positioning setup, where a small 3D piezo actuator setup is used to compensate for the movements of an insect to be monitored. Since that, he has been designing an integrated multi-channel nerve signal recorder for similar applications.

BIST (Built in Testing) of AD/DA Converters and Converter PairsIn the doctoral thesis by Esa Korhonen, ways of testing converters without expensive precision instruments were studied. A novel calculation algorithm for the ADC dif-ferential non-linearity (DNL) and integral non-linearity (INL) estimation was proposed. The algorithm assumes that two stimuli with constant offset between them are applied to the ADC under test, and that the code density histograms for both stimuli are recorded. The probability


density function (PDF) of the stimulus is then solved us-ing simple calculations so that the DNL and INL of the ADC can be estimated without a priori known stimuli. If a DAC is used to generate the stimulus to ADC, all inputs and outputs are digital, and the new algorithm can be used to characterize the INL and DNL of the DAC as well. This all-digital test confi guration enables a simultaneous testing of both converters thanks to the new algorithm. One inter-esting application of the proposed test technique is the pro-duction testing of mixed-signal microcontrollers where the proposed calculation algorithm runs in the microcontroller and the only additional test hardware is the offset genera-tor connected between the converters. The accuracy of the proposed method is limited by that of the offset between the stimuli. Therefore, an accurate and small offset genera-tor was designed. The proposed on-chip offset generator is built using only four resistors and switches. It occupies 122x22 μm2 in a 130 nm CMOS process, and accuracy is appropriate for the INL testing of 12-bit converters from rail-to-rail. In Figure 17, the INL measurement results us-ing the offset generator and 12-bit converters are shown.

Figure 17. Measured INL of a 12-bit DAC (upper) and ADC (lower) converter with the proposed method.

Linearization Studies and ExperimentsSupply modulated RF transmittersTimo Rautio summarized his results on supply-modulated RF transmitters in his Licentiate thesis in summer 2010. During the autumn, the group received a donation of a complete setup for driving envelope-tracking PAs, and this has been prepared for use in new, more broadband experiments. The group was also asked to beta-test a com-mercial PA characterization tool developed by a measuring instrument manufacturer, and this was conducted during autumn.

Digital Error Correction on A/D and D/A Convert-ers2010 was quite successful in the study of A/D and D/A converters. A previously started study on the use of par-tial histogram to linearize a segmented A/D converter was reported in the Master’s thesis by M. Kangas. M. Neitola wrote three journal papers, all accepted on different as-pects of (a well-matured area of) multi-bit sigma-delta A/D and D/A converters.

First, a generalized data weighted algorithm (DWA) for shaping the mismatch noise (INL) in multi-bit ADCs was developed and reported in [9] – it is also published as a downloadable Simulink model. The algorithm applies equally well for low-pass and band-pass converters, and performs fi rst-order mismatch noise shaping. Further, it was noticed that adding one additional unit element in the multi-bit thermometer-coded DAC (this was called incre-mental DWA or IDWA) cleans up spurious mismatch tones from the signal band, and this triggered additional studies and publications.

The literature study of different mismatch noise shaping algorithms showed that the reported results are not always repeatable, especially when a small set of random mis-match errors is used. This led to the development of a more general and exhaustive test set for characterizing the per-formance of different DEM (dynamic element matching) algorithms. In short, a fi xed set of different shaped INL re-sponses is used (this avoids the need for long Monte-Carlo type analysis to gain statistical signifi cance), and the per-formance is checked by a 2-D sweep over amplitude and signal frequency. The results are shown as “fi ngerprints” in Figure 18: here a) are low-pass and b) band-pass AD converters, and the oversampling ratios are 16, 32, and 64 from left to right. It is seen that the highest spurious tones (black area) move towards lower signal amplitudes when OSR is increased. The developed fi ngerprints provide a repeatable, exhaustive, and yet only moderately time-con-suming method for comparing different DEM techniques. Also, a new, unit element utilization fi ngerprint was de-veloped to test how sensitive the DEM algorithms are for repeating patterns.

Figure 18. Spurious tone fi ngerprints in a) low-pass and b) band-pass SD ADC. Black means high spurious tones due to mismatch noise.


The discovery of an IDWA structure triggered two stud-ies on the origins of spurious tones in ΣΔ converters in general. A theory that relates the signal amplitude and INL shape of a multi-bit DWA DAC to the frequency of the spurious tones has been accepted for publication, and will appear during 2011. A more general study relating the type of quantizer and the frequencies of the spurious tones is under development.

Personnel

professors, doctors 8

doctoral students 19

others 4

total 31

person years 20

External Funding

Source EUR

Academy of Finland 536 000

Ministry of Education and Culture 125 000

Tekes 165 000

domestic private 27 000

international 139 000

total 992 000

Doctoral ThesesLoikkanen M (2010) Design and compensation of high perfor-mance class AB amplifi ers. Acta Universitatis Ouluensis C356.Korhonen E (2010) On-chip testing of A/D and D/A converters. Acta Universitatis Ouluensis C365.

Selected PublicationsKozmin K, Johansson J & Kostamovaara J (2010) A low propa-gation delay dispersion comparator for a level-crossing AD con-verter. Analog Integrated Circuits and Signal Processing 62(1): 51-61. 1

Korhonen E, Carsten W & Kostamovaara J (2010) Combining the standard histogram method and a stimulus identifi cation al-gorithm for A/D converter INL testing with a low-quality sine wave stimulus. IEEE Transactions on Circuits and Systems I 56(6): 1166-1174. 2

Vainshtein S, Yuferev V, Kostamovaara J, Kulagina M & Moil-anen H (2010) Signifi cant effect of emitter area on the effi ciency, stability and reliability of picosecond switching in a GaAs bipo-lar transistor structure. IEEE Transactions on Electron Devices 57(4): 733-741. 3

Hallman LW, Ryvkin B, Haring K, Ranta S, Leinonen T & Kostamovaara J (2010) Asymmetric waveguide laser diode op-erated in gain switching mode demonstrates high power optical pulse generation. Electronics Letters 46(1): 65-66. 4

Nissinen I & Kostamovaara J (2010) Jitter characteristics of an on-chip voltage reference-locked time-to-digital converter. Ana-log Integrated Circuits and Signal Processing 64(3): 271-280. 5

Korhonen E & Kostamovaara J (2010) On-chip offset generator for accurate integral non-linearity testing of A/D converters and D/A-A/D converter pairs. Analog Integrated Circuits and Signal Processing (OnlineFirst). DOI: 10.1007/s10470-010-9496-2, 9 p. 6

Kostamovaara J (2010) Interview. Electronics Letters 46(1):4. 7

Neitola M, Rahkonen T & Raappana J (2010) A qualifi cation ap-proach to DAC mismatch shaping methods. IEEE Trans. on Circ. Syst.-II 57(11): 858-862. 8

Neitola M & Rahkonen T (2010) A generalized data weighted averaging algorithm. IEEE Trans. on Circ. Syst.-II 57(2): 115-119. 9

Hietakangas S, Typpö J & Rahkonen T (2010) Design of inte-grated 1.6 GHz, 2W tuned RF power amplifi er. Analog Integrat-ed Circuits and Signal Processing 64(3). 10

Duan G, Vainshtein S & Kostamovaara J (2011) Peculiarities of surface breakdown in GaAs bipolar junction structures. condi-tionally (mandatory changes) accepted for publication in IEEE Trans. Electr. Dev. MS#TED-2010-12-0876-R. 11

Duan G, Vainshtein S & Kostamovaara J (2010) Physical inter-pretation of “soft” surface breakdown typical of GaAs avalanche transistors. Annual Journ. of Electronics (ISSN 1313-1842), Book 1, 22-25. 12

Duan G, Vainshtein S & Kostamovaara J (2010) Self-organiz-ing of avalanche transistor operating area in accordance with pa-rameters of external circuit. Annual Journ. of Electronics (ISSN 1313-1842), Book 1, 26-29. 13

Vainshtein S & Kostamovaara J & Yuferev V (2011) Nanosecond pulses for sub-terahertz imaging from avalanching GaAs bipolar transistors. Chapter 26 in Terahertz and Mid Infrared Radiation/Generation, Detection and Applications Series: NATO Science for Peace and Security Series B: Physics and Bio-physics. 14

Vainshtein S, Yuferev V, Kostamovaara J & Palankovski V (2010) Collapsing fi eld domains in avalanche GaAs transistor: peculiar phenomenon and prospective applications. Annual Journ. of Electronics (ISSN 1313-1842), Book 1, 11-16. Plenary talk at 19th international conference ET 2010. 15


INTELLIGENT SYSTEMS GROUP (ISG)

Professor Juha Röning and Professor Jukka Riekki, Computer Science and Engineering Laboratory, Department of Electrical and Information Engineering, University of Oulu

juha.roning(at)ee.oulu.fi , jukka.riekki(at)ee.oulu.fi http://www.ee.oulu.fi /research/isg

Background and MissionThe main scientifi c objective of the basic research con-ducted in the Intelligent Systems Group is to generate new applicable knowledge on intelligent systems, and to gener-ate positive societal impacts by applying this knowledge using scientifi cally plausible methods and state-of-the-art technology. We conduct research on spatial computing, collective intelligence, sensor networks, optimization of industrial processes, mobile robotics and cybernetics, hu-man-computer interaction (HCI), human-robot interaction (HRI), computer networks, and security in complex infor-mation processing systems.

Our strategic research themes, whereby we aim to con-duct world class basic research, are: 1) Safety and security on all levels of intelligent systems, 2) Data mining with a special focus on optimization of industrial processes and well-being, 3) Human-environment interaction with a spe-cial focus on novel physical user interfaces, robot-envi-ronment and human-robot interaction, 4) Mobile robotics and cybernetics with a special focus on spatial computing, adaptation, and fi eld robotics, and 5) Sensor networks with a special focus on understanding behaviours and patterns in our everyday environment.

We have conducted basic research related to these research themes for over ten years. Our team consists of three pro-fessors, fi ve postdoctoral researchers and 21 doctoral stu-dents. The annual external funding of the group is more than two million EUR in addition to the basic university funding. There have been 14 completed doctoral degrees from the group. From the research of the group, seven spin-off companies have been established so far: Codenomicon, Clarifi ed Networks, Hearth Signal, Nose Laboratory, Neli-lab, Atomia and Probot.

We co-operate with many international and domestic partners. In applied research, we are active in European projects. In addition, several joint projects are funded by the Finnish Funding Agency for Technology and Innova-tion (Tekes) and industry. We are a research partner in the Cooperative Traffi c ICT SHOK, the Devices and Interop-erability Ecosystem (DIEM) ICT SHOK and the Cloud Software Program ICT SHOK where we are responsible for the cloud security theme. In the II City project, we col-laborate with the University of Lapland and Sonic Studio from Piteå, Sweden, and in the Pervasive Service Com-puting project with Shanghai Jiao Tong University from China. These projects are funded by the European Union (Interreg IVA North) and the Academy of Finland (the MO-TIVE program). Also we have started new collaboration with Tokyo University of Agriculture and Technology in the project Interactive Context-aware System for Energy

Effi cient Living funded by the Academy of Finland. Prof. Jukka Riekki visited the Huazhong University of Science and Technology (HUST) in Wuhan, China and Shanghai University for one month in winter 2010 and gave invited talks during the visit.

We are active in the scientifi c community. For example, Prof. Juha Röning co-chaired numerous international workshops in the software security area, including the 6th Crisis Management Workshop (CRIM 2010) in Oulu, Finland. He acted as a member of the SAFECode Interna-tional Board of Advisors, 2010 and as a chief judge in the European Land Robot Trial (Elrob 2010) in Hammelburg in May. Elrob is the biggest outdoor robot event in Europe, and participants are research facilities and companies that represent the state-of-the-art in Europe in this research area. Prof. Jukka Riekki co-chaired the Third Asia-Europe Workshop on Ubiquitous Computing 2010 (AEWUC’10) and the Pervasive 2010 Poster Session, both organized in May 2010 in Helsinki, Finland, and the UBI Challenge Workshop at Ubicomp 2010 organized in September 2010 in Copenhagen, Denmark. Several members of the group were also on the committees of international conferences.

Scientifi c Progress

Research on Prototyping: from a Smart En-vironment towards Remote Distributed Intel-ligenceVerifi cation of the developed methods and models in pro-totypes is an important part of our research. To support this activity, we develop software and hardware architec-tures for smart environments. In addition to verifi cation, prototypes speed up the commercialization of the research results. In prototyping, we have set and tackled the follow-ing objectives:

Developing Hardware And Software for Prototypes And Commercial Devices: The goal is to make designing of embedded systems faster and easier, while preserving the commercial applicability of the resulting device. This has been approached via the Embedded Object Concept (EOC). EOC is a concept that utilizes common object-ori-ented methods used in software by applying them to com-bined Lego-like software-hardware entities. This approach enables people without a comprehensive knowledge in electronics design to create new embedded systems. For experts, it shortens the design time of new embedded sys-tems. The conceptual idea of embedded objects has been successfully implemented with the Atomi II framework, which provides the so called Atomi modules.


The EOC research has proceeded by making the concept easier to grasp for new users. This has happened by devel-oping the documentation, tutorials and example cases that are to be presented to local companies and research facili-ties in workshops and lectures.

Research on Nanoresolution Tools for Interdisciplinary Applications: The project “Nanoresolution tools for inter-disciplinary applications” (NRT) develops novel research, manipulation, and manufacturing methods for micro- and nanotechnology components and instruments. This means ever smaller objects, which are placed on surfaces not measurable with existing tools. A network of contacts and collaborators is used to select the objectives where com-mercial instruments are not yet available. One basis of the selection is a study of the commercial potential and value. Commercial potential affects the cost and selection of the components to be used thereby enabling the commerciali-zation process. In order to improve the regional effect of the project, market studies are carried out.

The tools developed in NRT are based on a Scalable Mod-ular Control Architecture (SMCA) that is being developed in the project, and the previously mentioned embedded objects. SMCA is a generalized modular architecture for both the device hardware and the control software on a PC. The architecture is extensible, scalable and portable, and it enables reuse of modules. It is hierarchically lay-ered hybrid architecture to implement research equipment. SMCA enables swift changing of actuators, sensors and tools with minimal effort, thus being an ideal frame for various applications.

Implementation of SMCA is included in the framework within the project. The framework aims to improve the quality of the developed tools and enable fast prototyping. To achieve these goals, the framework includes reusable components that perform the tasks needed in all developed measurement tools such as data management, data visuali-zation and confi guration of devices. It also supports run-time specialization of measurement tasks to enable the use of the same hardware to perform different measurement tasks easily. Future work with SMCA will concentrate on applying the architecture in new areas, and the dissemina-tion of the technologies for the local companies and re-search facilities.

Research on Mobile RoboticsAt the end of 2010, the Academy of Finland granted fund-ing for the Evolutionary Active Materials (EAM) project. EAM is a joint effort of the Computer Science and En-gineering laboratory (CSE) and the Microelectronics and Materials Physics laboratories. The aim of the EAM project is to develop novel evolutionary computation (EC) based design methods for active and versatile materials and structures, and realise the fi rst components through a novel holistic design process utilising constantly increas-ing computation power, the development of multi-physics simulators, and EC techniques such as genetic algorithms. One objective of the project is to accelerate the paradigm

shift from the conventional design process for active ma-terials towards a new goal driven holistic design process using the full potential of new materials, material com-binations, and nonlinear dynamics of materials combined with complex geometries. The results of the project will be demonstrated fi rst in the design of energy effi cient piezo-electric actuators.

Global pose estimation is of fundamental importance in various mobile robot applications where the accurate pose of the robot in a given environment is needed in order to successfully perform application specifi c tasks. Today, nu-merous techniques exist for indoor pose estimation. The most common techniques are based on range sensors and computer vision and these techniques have been success-fully applied in various environments and mobile robot applications. Despite the fact that good solutions already exist for indoor pose estimation, there is room for new techniques which can make the pose estimation more ac-curate when combined with other techniques, more robust against possible changes in environmental conditions, and perhaps provide more cost effi cient solutions for pose esti-mation. We proposed a global indoor pose estimation tech-nique utilizing the ambient magnetic fi eld. The technique is based on the well known observation that magnetic fi eld fl uctuations commonly exist inside buildings. These fl uctu-ations arise from both natural and man-made sources, such as steel and reinforced concrete structures, electric power systems, electric and electronic appliances, and industrial devices. Assuming the anomalies of the magnetic fi eld in-side a building are nearly static and they have suffi cient local variability, the anomalies provide a unique magnetic landscape that can be utilized in global pose estimation. Our experiments suggest that the ambient magnetic fi eld may remain suffi ciently stable for longer periods of time giving support for pose estimation techniques utilizing the local fl uctuations of the magnetic fi eld.

Our work on indoor positioning techniques making use of the ambient magnetic fi eld continued in 2010. A new magnetic fi eld mapping instrument has been developed. The new mapping instrument can be used to collect large amounts of data about the three-dimensional structure of the magnetic fi eld.

A key issue in magnetic fi eld localization is the building of a map of the surrounding magnetic fi elds. In contrast to the Tuli project, where the mapping is performed manually, in the robotics domain this is referred to as a simultaneous lo-calization and mapping (SLAM) problem. During the year 2010, we published two papers focusing on experimental studies in the magnetic fi eld SLAM and on theoretical is-sues relating to near-optimal SLAM exploration. Results from our experimental studies (see Figure 1) showed that indoor magnetic fi elds contain enough spatial variation for accurate mapping. In near-optimal SLAM explora-tion studies, we developed new methods to autonomously model continuous spatial processes while having uncertain localization information. These methods can be utilized in time-effi cient magnetic fi eld SLAM.


Figure 1. Magnetic fi eld SLAM in the CSE lobby. The up-per left image presents robot’s trajectory using only raw odometry information from the robot’s wheel encoders. The upper right image presents a corrected trajectory where we have used rao-blackwellized particle fi lters in magnetic fi eld SLAM.

As part of our SLAM work, we have proposed a sub-mod-ular sensing quality function which extends studies from discrete sensor placement into an autonomous sampling scheme where sensing sites must be visited frequently. This is benefi cial in a SLAM context where the sensing sites themselves bear uncertainties. Also in time critical applications, the modelling accuracy has to be balanced with the sensing time. Our SLAM studies are inspired by our research on indoor mobile robot localization, utiliz-ing ambient magnetic fi elds which can be modelled by three orthogonal GPs providing a fl exible framework for localization and SLAM. We have proved that for applica-tions where sensing sites must be visited frequently, mu-tual information provides near-optimal solutions. We have extended this into metric probability spaces where sensing sites are treated as random variables. We have shown that with particle fi lter discretization, this framework can attain near-optimality.

Research on Human-Environment Interac-tionOur current work on human-environment interaction fo-cuses on physical user interfaces and human-robot interac-tion. In physical user interfaces, mobile terminals are used as physical objects rather than as traditional I/O devices. Our current studies concentrate on touch-based and ges-ture-based interaction: users interact with the local envi-ronment by touching objects with their mobile terminals and moving their terminals in the air. The touch-based user interfaces use NFC technology (i.e. RFID technology for

mobile phones): an act of touching brings an RFID reader near an RFID tag and hence the data in the tag is read and delivered to the system. The objects that can be touched are advertised in the environment by RFID icons, graphi-cal icons resembling the icons of the graphical user inter-faces of computers and other user terminals. Gestures are recognized from the sensor data produced by mobile ter-minals’ acceleration sensors.

In 2010, we continued research on physical user interfaces in the DIEM ICT SHOK together with researchers from VTT, the University of Tampere, and the Tampere Univer-sity of Technology. The DIEM ICT SHOK UI work pack-age studies multimodal interaction for interactive spaces. Our main research area in this workspace is interaction using NFC technology. During 2010, our work package developed and tested the Ambient Conference Assistant, an application offering information to conference par-ticipants, supporting interaction between the participants, and between the participants and organizers as well. Our group developed for this application the Interactive Poster service (Figure 2). This service provides a physical, touch-based user interface for interacting with a poster. With their mobile phones, visitors and poster presenters touch graphical icons on the poster and on a control panel placed next to the poster. For example, by touching one of the icons, a visitor can play and control videos related to the poster in a display nearby. The visitor can also “pick” mul-timedia fi les from the poster, listen to the poster abstract, comment on the poster, and get the author’s business card. This application was deployed and tested in two confer-ences held in Finland during 2010: Pervasive 2010 and Mindtrek 2010.

Figure 2. The control panel for the Interactive Poster at Pervasive 2010.

We also developed an application supporting three to fi ve-year-old children in their efforts in learning to read. This application was developed and tested in collaboration with the Department of Education at the University of Oulu. We equipped the nametags of children in a kindergarten with RFID tags and star icons advertising the positions of the tags. When a child touched a star on a nametag with an NFC phone, the phone said aloud the name in that name-tag. The aim was that the children learn to associate the written name with the pronunciation of the name. In the second mode, the phone presented a name and the task


was to touch the corresponding nametag. The pilot study showed improvement in the ability to recognize letters; suggesting that this type of application might be useful in the early stages of learning to read

In the II City project we developed two novel applications in collaboration with Sonic Studio from Piteå, Sweden, and the University of Lapland. The Audio Guide directs people to selected targets using audio instructions (without speech); freeing the people to be present and interact with their local environment. The City Sound Generator is an artistic application representing the city’s state as audio. It embodies playful interaction, breaking some known us-ability principles to enhance its artistic impact. With this application we are studying new, playful ways of commu-nicating information to the local community. The applica-tion produces music from the local temperature, amount of people near the display and the season. Users can change the values by melting the cubes with their hands and by turning the record and this way experience what different temperatures, amounts of people, and seasons sound like (Figure 3).

Figure 3. The City Sound Generator user interface for a large touch display.

Research on Sensor NetworksThe research on sensor networks is targeted at understand-ing behaviours and patterns in our everyday environment. We are studying sensor data processing platforms, sensor data processing algorithms, lightweight knowledge repre-sentations suitable for sensor networks, and reasoning. In 2010, we continued developing the sensor data process-ing platform in the Cooperative Traffi c ICT SHOK. This research was conducted in the Sensor Data Fusion and Applications (SDFA) project together with the University of Jyväskylä, the Tampere University of Technology, the Information Processing Department of the University of Oulu, and VTT. We completed a second prototype that utilizes data from built-in acceleration sensors and GPS modules of mobile phones to analyze the travel mode of a person, and the road condition when the phone is carried in a car (Figure 4). The data uploaded from the mobiles is stored in a database and processed in real time on a server, which also exposes an interface to deliver the results to clients. The server integrates several modules developed by the consortium, including map matching of GPS co-ordinates to digital map data, travel mode detection and road condition analysis. Traffi c data is also streamed to the server from the national DigiTraffi c web service. A cli-ent prototype was developed to visualize the processing results in real time on a map in a web browser.

Figure 4. An SDFA prototype collecting data with mobile phones attached to vehicles and analyzing road conditions from the collected data.

In the UBI program, we continued the development of a platform on collecting, processing and storing sensor data from a heterogeneous wireless sensor network. The fi rst prototype for measuring the energy consumption in house-holds, using wireless sensor nodes over IEEE 802.15.4 networks with 6LoWPAN protocol stack, was completed in early 2010 together with MediaTeam Oulu. In total, 11 households in Oulu participated in the fi rst pilot testing and the initial results confi rm the usefulness of the system.

In the Pervasive Service Computing project, we continue the development of service-oriented infrastructure unob-trusively supporting people in their daily activities. Sen-sor networks are an important part of this infrastructure; providing measurements about the environment and the people. This work involves collaboration with MediaTeam Oulu and Shanghai Jiaotong University. Several proto-types were developed to demonstrate the capabilities of dynamic service composition and coordination. Moreover, we are moving towards deploying service compositions in the Cloud. In 2010, we concentrated on developing infra-structure for context-based service creation and composi-tion. This infrastructure facilitates the building of services by utilizing rules and context information that is obtained through processing data produced by sensor networks.

Research on Human-Robot InteractionTwo ongoing projects, the Academy funded AFHRI (Af-fective-Human-Robot interaction) and the European Re-gional Development Fund (ERDF) funded Minotaurus combines the theoretical and practical aspects of creating service robotics of the future. Both projects co-operated with the Machine Vision Group (MVG). Human-Machine communication and robot’s operation in real world condi-tions are major topics. Key aspects are putting research results into practice, and integrating them as a part of the


operation of the robot. The main challenges are represen-tation of information and adapting a set of algorithms to the current state in the surrounding environment. Human friendliness of the outward appearance of the robot is driv-ing the design of the robot.

Interaction with a machine has been studied from various perspectives. The combination of machine vision, speech recognition and synthesizing, touch and touchless interac-tions, along with the robot’s operation in the environment requires a software platform that processes, distributes and stores information effi ciently. Real-time Linux based op-erating system services along with general purpose repre-sentation for information (called Markers) have been de-veloped to support integration of the algorithms. Marker- representation is used for representing the output of sensor processing algorithms, for representing a model of envi-ronment and obstacles around the robot, and information related to the robot’s tasks.

In the Minotaurus project, speech and natural language processing have been studied to be able build a natural interface between a robot and human. At the fi rst stage, the research has been concentrating on speech recognition where a small vocabulary model, based on a CMU Sphinx engine, is used to recognize greetings, commands, and questions. Furthermore, a simple language understanding model, based on predefi ned rules and regular expressions, is implemented to be able detect current speech action to form responses in human-robot dialogue. Responses are based on the robot’s internal knowledge as well as external knowledge from the web. Besides speech recognition and dialogue, speaker orientation is estimated using multiple microphones and phase correlation analysis to be able to detect a current active speaker and her orientation. In the future, work will be concentrating on large-scale speech recognition and language processing based on statistical machine learning to be able to implement more natural communication techniques between a human and robot without relying on a set of fi xed rules.

The project includes also instrumentation of a mobile robot that can operate among people. The basic mobile platform for the robot is Segway RMP200, a two parallel wheeled balancing robot (Figure 5). It is equipped with onboard computers, a display, and several sensors like cameras,

lasers and inclinometers. One of the main sensors is the Microsoft Kinect, a cheap depth and colour camera with orientation sensors. It is used for creating 3D model of the surrounding environment, and detecting human pose.

Research on Data MiningData mining research challenges are divided into three mutually supportive categories; the research on algorithms producing knowledge, the software running the algorithms and the knowledge bases storing the acquired knowledge. When put together, these three categories form a strong combination which can be applied to virtually any phe-nomenon where data can be processed into knowledge.

The research on algorithms has been focused on advancing methods for time series analysis, variance modelling and novelty detection. Special interest was directed towards establishing data driven methods for these areas, where the exact shape and nature of the observed data can be used to characterize the phenomena under study. In software research, the implementation of the fi rst version of new software architecture for running the algorithms reached a successful conclusion. The architecture presents the algo-rithms as information generating logical devices to which the information measuring physical devices connect. A suitable combination of logical and physical devices can thereafter be used to form a data mining software applica-tion. In knowledge base research, the general ideas behind the role of knowledge bases were established, and the im-plementation of the fi rst version of a concrete knowledge base was carried through to a successful end.

During 2010, the combination of the three data mining focus areas (algorithms, software and knowledge bases) proved its effi ciency. Prototypes were implemented uti-lizing all the focus areas while the interfaces developed enabled modifi cations in the three areas individually. For example, inside the prototypes, the algorithms in use could be modifi ed without extra work in software implementa-tion or effect on the knowledge base structure.

The actual research has focused on four projects, XPRESS (2007–2010), MIDAS (2008–2010), PISKET (2009–2011) and prob2E (2010–2012).

Computer Vision / Augmented Reality: In the online car door assembly guidance system presented in Figure 6, computer vision and machine learning methods are ap-plied to track the pose of the car door and to recognize if particular door parts are installed. Model-based tracking where a CAD model of the door, along with edge-based image data is applied to estimate the door position and ori-entation, as well as, to segment different door parts. Sup-port Vector Machines (SVM) are trained to recognize the existence of door parts with a view to detecting the correct work phases. In SVM, a kernel for structured data is ap-plied to image patches of local interest points. These meth-ods are capable of detecting installed car door parts auto-matically for monitoring assembly and to recognize work phases completed correctly. In Figure 6, the car door frame and part segmentation images can be seen before and after the worker has placed the third part during the assembly scenario. The eight rectangles on the door frame pictures Figure 5. A test setup for the Human-Robot Interaction.


are the segmented part areas. The colour of the rectangle is red if the part has not been recognized and green if the part has been successfully identifi ed. The blue dots on the part area images are the detected interest points used in the part recognition. The recognition prototype system was successfully demonstrated to project partners.

Figure 6. Car door frame and part segmentation images before and after the worker has placed the third part during the assembly scenario.

Sensor signal analysis with synchronized video: Human motion recognition using wearable sensors is a growing fi eld. Commercial applications are starting to emerge in gaming, health, sports and consumer electronics. The proc-ess of developing a human motion recognition system, and designing the recognition intelligence, includes collecting and labelling the data. Labels presenting the ground truth about the classes of the data sequences can be used to train the recognition system and evaluate the recognition results it produces.

In 2010, ISG completed the MIDAS-project (Methods for Innovative Data Analysis Solutions, funded by the Eu-ropean Structural Fund Programmes, Polar Electro and Ruukki) in which we examined how the labelling proc-ess could be improved. Video based labelling methods are widely used, because the videos provide easily perceivable meta information. In the MIDAS-project, the video anal-ysis approach is developed further by synchronizing the video with the inertial data that is labelled. The technical feasibility of the developed synchronized video analysis approach for human inertial data was proved by instantiat-ing it as a labelling tool. In an experiment, it was found to be completely appropriate for labelling more complex human inertial data. The synchronized video analysis ap-proach showed better results in labelling quality compared with other labelling approaches. Interviews and focus groups pointed out that data being labelled with high ac-curacy using a video based approach is considered to be valuable. On the other hand, the video based approaches are still criticized for the amount of effort. The labelling tool developed is now utilized in other research projects in the university and in industrial R&D by Polar Electro.

Gesture/Activity Recognition: In assembly guidance, the wearable inertial measurement units can be used in two different ways: using gestures as control commands, and monitoring the work tasks performed by the worker. In

some situations, gesture recognition is a good option for handling human-computer interaction because it enables natural interaction and no input devices, such as a keyboard and a mouse, are needed. In this scenario, the gestures are recognized by matching the shapes of wrist movement to predefi ned time series templates (Figure 7). The second approach uses wearable sensors for activity monitoring to guide workers in real-time. The recognition is done using acceleration and angle speed information, and the results have been promising. Both approaches were successfully demonstrated to project partners in summer 2010, and the results were also presented at international conferences.

Figure 7. Gestures to control a user interface and the cor-responding tri-axial acceleration.

The XPRESS project (2007–2011), fi nanced from the 6th Framework Programme of the European Commission, has entered its fi nal integration phase. The overall goal of the project is to develop a new kind of intelligent manufac-turing system composed of hierarchically organized au-tonomous units called manufactrons. Two ISG members participated in a week-long integration meeting at the premises of the Fraunhofer Institute in Stuttgart, Germany. The objective of the meeting was to bring together the contributions of several project partners into a functional demonstration system that covers every level of the manu-factron hierarchy. ISG contributed two major components to the system: a human-machine interface and a knowl-edge system.

The human-machine interface (HMI) gives guidance and feedback to a factory worker executing a manufacturing task such as welding or manual assembly. It integrates the results of several areas of research: gesture and activity recognition, machine vision, augmented reality and user interfaces. Besides a presentation of the task execution protocol and a post-execution quality summary, a work-er using the HMI receives real-time instructions adapted to monitoring information representing the progress and current status of the work. In an assembly context, for in-stance, the interface can show the worker which part to pick up next and where to attach it. The capabilities of the HMI were demonstrated at the integration meeting, and


later in the year at a meeting in Oulu with a representative of Centro Ricerche Fiat, an XPRESS partner representing the automotive industry.

All manufactrons in the XPRESS framework receive their assignments in the form of task description documents (TDD). Once received, a TDD must fi rst be transformed into a method description document (MDD), which specifi es to the manufactron how the task should be executed. This operation is called task-to-method transformation (T2MT) and performing it is the responsibility of the knowledge system (KS). To solve the T2MT problem, previous results on software architectures and knowledge bases were com-bined with new research on knowledge representation and retrieval. Metadata annotations and a specially devised three-tier query scheme are used to match methods with tasks. The resulting KS architecture was used to implement T2MT in two XPRESS cases, developed in collaboration with two German XPRESS partners: the Augsburg-based company KUKA and the Karlsruhe University of Applied Sciences. A report on this work was presented at an IEEE international conference.

Work on the new application platform of the ISG, the De-vice-Based Software Architecture (DBSA), also contin-ued, although the focus shifted in 2010 from extending the architecture with new features to adapting it to the require-ments of different application domains. Both the human-machine interface and the knowledge system developed in the XPRESS project were built on modifi ed versions of the DBSA platform. The results of this work show the ar-chitecture to be a solid foundation for various data mining, signal processing and knowledge management applica-tions. DBSA was introduced to the academic community in a paper presented at an ACM international conference.

The future of living and housing: During 2010 a new project for researching the future of living has been pre-pared. The project title is “Interactive Context-aware Sys-tem for Energy Effi cient Living” (INCA) and it is funded

by the Academy of Finland. The goal for the project is to develop an interactive and context-aware feedback and control system for rationalizing energy effi cient living. The system itself collects personalized data from the en-vironment and offers information about consuming habits for the individuals. With the help of the information, it is possible to advise people on more energy effi cient living and to give recommendations for avoiding unnecessary energy consumption.

The research in the INCA project is performed jointly with Associate Professor Kaori Fujinami from the Tokyo University of Agriculture and Technology. From the data mining perspective, the group will be concentrating on de-veloping methods to discover instant and long-term habits and anomalies in water and electricity consumptions asso-ciated with the users’ context information such as identity, location, and activities, automatically learned and detected from the user’s behavioural patterns measured by visual and ambient sensors attached to an intelligent home envi-ronment. Figure 8 presents a context-aware system to be developed in the project for energy effi cient living.

Activation for physical activity and health: In the CallUp&Go (Communal, ICT based Activation for Physi-cal Activity and Health funded by Tekes) the goal is ICT-based activation on the physical activity and health among conscription aged men in Finland. In other words, the aim is to persuade young men to be physically more active, and the persuasion is done through gamifi cation. A gamifi ed computer-platform will combine health information, game technology, tailoring and activity monitors. In the project, different combinations of sensor and game technologies, social media and their interfaces will be studied and an interactive social media prototype will be developed. The prototype consists of a video game, ICT systems, content services, and performance and activity monitors, both ex-isting ones and those that are still to be constructed. The developed game is not going to be a typical video game, to proceed in the game subjects need to exercise. The novelty

Figure 8. The context-aware system for energy effi cient living.


in this is that unlike in current exercise-based games, like Wii Sports, Wii Fit, Blobo Fun & Fit, EyeToy Play Sports, subjects do not exercise in front of TV or monitor, instead they need to go out and exercise, see Figure 9. This way the subjects gain points in the game based on their physical ac-tivity and they can proceed to the next level. The ICT plat-form will be set up using multidisciplinary collaboration by consulting local enterprises with related top expertise. The ISG’s role is to construct a new innovative computer-platform for the project and develop methods to monitor physical activity. The game developed in CallUp&Go will be tested by a limited number of real users. CallUp&Go is a part of a bigger project called MOPO.

Figure 9. The game: the player earns points based on his physical activity. After collecting enough points he proceeds to the next level.

Data mining methods for steel industry applications: The Intelligent Systems Group has long experience in advanc-ing data mining methodologies in the steel industry, and is a member of the Centre for Advanced Steels Research - CASR, which is one of the interdisciplinary umbrella or-ganizations of the University of Oulu. In 2010, advanced data analysis solutions for the needs of the steel industry were developed in three active projects.

In 2010, ISG fi nished a long lasting project in which a generic system for the semi-automatic maintenance of statistical prediction models was developed. The system developed makes use of production data to keep up-to-date applications that employ statistical predictive models to control, optimize and plan the production. The system ana-lyzes the production data and monitors the performance of models. When a need for model update actions is detected, the system serves as a tool and graphical user interface which process engineers can easily use to generate a new model candidate, validate it and deliver it to the applica-tions that optimize, plan and control the production proc-ess. The model candidates and earlier models are stored in a model archive and the system works as a diagnostic tool that helps to compare different models and to select the best model for production use. The system developed is currently applied to keep up-to-date prediction models utilized in the planning of steel plate products and product variations.

Currently, the system developed provides support for the modelling of distribution properties of a continuously measured response using neural networks or generalized linear models. The component-wise implementation al-lows for extending the algorithm library to support other model types and maintenance paradigms. The system can be a cost-effi cient method of keeping prediction models up-to-date in a wide variety of industrial applications.

The quality of steel plates can be improved by optimiz-ing the rolling pass schedule. An accurate prediction for rolling temperatures and loads of the rolling is necessary. The statistical prediction of rolling loads and temperatures is demanding and application of advanced functional data analysis methods is needed. In the PISKET project (Im-proving Pass Scheduling Calculation Taking into Account Flatness) prediction models are developed to predict the rolling temperatures and loads. In 2010, a model for pre-dicting the rolling temperatures has been developed. The major challenge has been in the selection of a feature set that describes the deformation history of the earlier pass schedule, but is invariant to the pass number. The models developed will be utilized in the Ruukki steel plate mill.

Production effi ciency with probability predictions: In 2010, ISG has launched a new project (prob2E) to develop further a probability prediction approach to develop and utilize statistical models. The aim of the project is to de-velop strategies for applying disqualifi cation probability predictions in the optimization of production processes. Statistical methods that focus on accurate modelling of the dependence of deviation and tail distribution shape from input variables are employed as the data analysis tools. The expected result of the project is know-how on developing probability prediction models and on applying them in the optimization of the manufacturing industry processes. The main hypothesis is that decision making that is based on predicted disqualifi cation risk decreases disqualifi cations, and decreases variation in the product quality character-istics. The hypothesis and related economic benefi ts are validated by applying the developed probability predic-tion procedure into three different manufacturing industry processes (cheese manufacturing, steel strip rolling and the tempering of steel bars). The project is funded by Tekes, Ruukki, Ovako, and Valio.

Research mobility: Dr. Perttu Laurinen was on a one year research visit (March 2009 - March 2010) at the Idiap Re-search Institute, affi liated with EPFL, in Switzerland. The purpose of the visit was work on developing new methods for labelling and identifying shapes of interest from time series data. The integration of multiple data sources (like video, audio and accelerometer signals) was of especial interest. The results can be used to recognize and label events from a numerical time series with the help of syn-chronized audiovisual recording. The visit was part of the MIDAS-project.

Eija Haapalainen was on a one year research visit (Asla- Fulbright scholarship) at Carnegie Mellon University and has been working on a study that aims to measure the cog-nitive load of a person, based on physiological measure-ments. In the study, a large number of different biosignals are examined, and the best measure or set of measures for assessing the cognitive load is searched for. The signals being measured include pupil size, heart rate, skin tem-perature and conductance as well as electrical brain activ-ity. These signals are recorded while the test subject solves elementary cognitive tasks of different levels of diffi culty. The prospective results of the study have broad applica-tions in systems reasoning about human attention. The goal of the project is to develop methods for assessing the


round-the-clock activity level of a person and to measure energy expenditure based on this information.

Research on Software SecurityWithin the Intelligent Systems Group, the Oulu University Secure Programming Group (OUSPG) has continued re-search in security and safety in intelligent systems. Secu-rity and safety challenges in intelligent systems are three-fold: increasing complexity leads to unforeseeable failure modes, quality is not the priority and awareness is lacking. We have approached the challenges from these three direc-tions in our research.

Complexity - Model Inference and Pattern Recognition: We work under the premises of unmanageable growth in software and system complexity and emergent behaviour (unanticipated, not designed) having a major role in any modern non-trivial system. We have worked on natural science approaches to understanding artifi cial information processing systems. We have developed and applied mod-el inference and pattern recognition to both content and causality of signalling between different parts of systems.

Quality - Building Security In: Software quality problems, wide impact vulnerabilities, phishing, botnets, criminal enterprise have proven that software and system security is not just an add-on despite past focus of the security in-dustry. Instead, security, trust, dependability and privacy have to be considered over the whole life-cycle of the sys-tem and software development from requirements all the way to operations and maintenance. This is furthermore emphasized by the fact that large intelligent systems are emergent and do not follow traditional development life-cycle. Building security in not only makes us safer and secure, but also improves overall system quality and de-velopment effi ciency. Security and safety are transformed from inhibitors to enablers. We have developed and ap-plied black-box testing methods to set quantitative robust-ness criteria. International recognition of the Secure De-velopment Life Cycle has provided us with a way to map our research on different security aspects.

Awareness - Vulnerability Life Cycle: Intelligent systems are born with security fl aws and vulnerabilities, new ones are introduced, old ones are eliminated (Eronen 2009, Askola 2008). Any deployment of system components comes in generations that have different sets of vulner-abilities. Technical, social, political and economic factors all affect this process. We have developed and applied processes to handling the vulnerability life-cycle. This work has been adopted in critical infrastructure protection. Awareness in vulnerabilities and processes to handle them all increase the survivability of emergent intelligent sys-tems for developers, users and society.

These research goals are reached through a number of re-search activities.

Secure Software Development Lifecycle in Cloud Comput-ing as part of the Cloud Software project we approach all three goals by researching practical ways of building se-curity into complex Cloud Computing services, from the design phase to actual operational use.

Situational awareness of modern information networks and applications in the MOVERTI project where we ap-proach information systems from all three viewpoints, and have developed methods for combining measurements from network traffi c, vulnerability feeds and scans, and socio-economic networks to gain situational awareness of modern information systems.

Identifi cation of protocol gene: This research, PROTOS-GENOME, approaches the problems of complexity and quality by developing tools and techniques for reverse-en-gineering and identifi cation of protocols based on using protocol genes - the basic building blocks of protocols. The approach is to use techniques developed for bioinfor-matics and artifi cial intelligence. Samples of protocols and fi le formats are used to infer structure from the data. This structural information can then be used to effectively cre-ate large numbers of test cases for this protocol. In 2010, the project further developed the existing methodology resulting in improvements in effi cacy and discovering a number of vulnerabilities in web browsers.

Hash function security research is a part of cryptography, where hash functions and their properties are studied. Hash functions play a major role in many modern communica-tion protocols, and are at the moment a very hot topic since NIST (National Institute for Standards in Technology) proposed a competition for a new and more secure hash function standard, SHA-3. This competition vets the one best algorithm from 64 different propositions. At OUSPG, the study of hash functions has led to some practical re-sults, but mostly concerning theoretical advances in the fi eld of hash function security. The results gained form this research and the cryptographic expertise are then applied to the practical information security work in OUSPG.

Exploitation of ResultsThe results of our research were applied to real-world problems in many projects, often in collaboration with in-dustrial and other partners. Some examples of exploitation are described below.

The Intelligent Systems Group utilizes a robotics laborato-ry and a pressure-sensitive fl oor (EMFi material) installed in our laboratory as part of a smart living room. Other equipment includes a home theatre, two degree-of-freedom active cameras, four mobile robots and one manipulator, a WLAN network, and various mobile devices. Our aim is to gradually build a versatile infrastructure that offers vari-ous generic services for pervasive applications. Naturally, this kind of environment enables realistic experiments that lead to a better understanding of such applications.

The generic system for the semi-automatic maintenance of statistical prediction models is now in use for product planning at the Ruukki steel plate mill. The system enables us to keep up-to-date these models which are important for ensuring that the products fulfi l the customer needs. As new products and production methods are continuously developed and novel process settings are taken into use, data from previously unseen conditions occur, and thus there is a constant need to update the prediction models.


Another major reason that causes a need for model updates is process drift. The system developed generates semi-automatically new model versions when the newest data makes it possible to improve the prediction accuracy. Each model version is a dynamic link library (DLL) that can be directly plugged in to the on-line applications that actually utilize the models. The system will give economic benefi ts because improved prediction accuracy in the product plan-ning decreases production costs and the costs needed to maintain the good prediction accuracy will now be small.

The embedded objects implemented according to the Ato-mi II Framework specifi cation are called Atomi objects. Several different Atomi objects have been created for real life tests. The Atomi objects have been used in many projects, and they have proved to be very usable.

The Atomi objects are being used both in pure research projects, and in projects that aim ultimately at commercial products. The Atomi objects have been applied to telep-resence robots, nanoscale manipulation and measurement technology, a hand held medical device, and several robot applications (Figures 10 and 11).

Figure 10. The Atomi modules have been used in several research projects. They have made prototyping of new in-novations easy and fast.

The SMCA and the framework has been declared open source, and the source is freely available at http://www.tnt.oulu.fi /oswiki/. This enables their use in both academic and commercial research and development. Many applications have been built with the developed techniques, and most of them have been disseminated in different journals and conferences. Currently new applications are being devel-oped.

The latest work on both SMCA and Atomi objects has concentrated on improving the documentation, wiki pages and tutorials for making their use easier. This work aims to spread the technologies for the use of local companies and research facilities in order to enable them to gain benefi t in research and development work and prototyping. At the same time this aims towards a large scale use test in order to improve the technologies further.

During the reporting year, the group continued utilizing outdoor robotic systems. Development and utilization of Mörri, a multipurpose, high performance robot platform continued. More focus was put on perception in natural conditions, representation of detections, knowledge, and environment model of operating environment. The soft-ware architecture further developed the earlier work on Property Service Architecture and the Marker concept as general purpose representation was further developed.

Future GoalsWe will continue to strengthen our long term research and researcher training. We will also continuously seek opportunities for the exploitation of our research results by collaborating with partners from industry and other re-search institutions on national and international research programs and projects. The group is a founding member of the European Robotic Network of Excellence (EURON). The group is a contract member of EURON II which was approved for the EU’s FP6 as a Network of Excellence.

Figure 11. Micromanipulation platform includes a PC, a probe, an actuator device and a controller device.


We will strengthen our international research co-operation. Within the last years, the group has created collaboration projects with Japanese researchers. In 2010, we continued the collaboration and developed it further through a mem-orandum of understanding with Keio University. Our aim is to continue and strengthen the collaboration in 2011. We will continue the collaboration with Sonic Studio from Sweden and Shanghai Jiao-Tong University from China as well. With the University of Tianjin, China we have a joint project in which methods and a system will be developed for vision-based navigation of Autonomous Ground Vehi-cles, which utilize an omni-directional camera system as the vision sensor. The aim is to provide a robust platform that can be utilized in both indoor and outdoor AGV (Au-tonomous Ground Vehicles) applications. This co-opera-tion will continue.

In the USA, we will continue to co-operate with the Hu-man-Computer Interaction Institute in Carnegie Mellon University. A doctoral student from ISG is currently on a one year research visit to the institute and is co-operating with Assistant Professor Anind K. Dey. The research is on human modelling in the area of human machine interac-tion. Shorter research visits to European partners in EU-funded projects are also planned.

In 2011, the aim is to utilize more widely the know-how from sensor technology and data mining. New applica-tion areas will be studied including rehabilitation, exercise motivation and energy effi ciency in households, and the benefi ts of our expertise will be highlighted to the actors in the areas.

Personnel



others 29

total 58

person years 30

External Funding

Source EUR



Tekes 872 000



total 2 171 000

Selected PublicationsDavidyuk O, Georgantas N, Issarny V & Riekki J (2010) MEDU-SA: Middleware for end-user composition of ubiquitous applica-tions handbook of research on ambient intelligence and smart environments: trends and perspectives. IGI Global (Eds. Mastro-giovanni F & Chong NY).Davidyuk O, Sanchez I & Riekki J (2010) CADEAU: Supporting autonomic and user-controlled application composition in ubiq-uitous environments pervasive computing and communications design and deployment: technologies, trends, and applications. IGI Global (Ed. Malatras A).Gilman E, Sanchez I, Saloranta T & Riekki J (2010) Reasoning for smart space application: comparing three reasoning engines CLIPS, Jess and Win-Prolog. Proc. 10th IEEE International Con-ference on Computer and Information Technology (CIT 2010).Haapalainen E, Kim S, Forlizzi J & Dey A (2010) Psycho-physi-ological measures for assessing cognitive load. 12th ACM Inter-national Conference on Ubiquitous Computing (Ubicomp 2010), September 26-29, 301-310.Halunen K, Kortelainen J & Kortelainen T (2010) Combinato-rial multicollision attacks on generalized iterated hash functions. Eighth Australasian Information Security Conference (AISC 2010), ACS, Brisbane, Australia, 105:86-93 (Eds. Boyd C & Susilo W).Halunen K & Röning J (2010) Preimage attacks against variants of very smooth hash advances in information and computer secu-rity. Springer Berlin / Heidelberg, 6434: 251-266 (Eds. Echizen I, Kunihiro, N & Sasaki R).Hosio S, Jurmu M, Kukka H, Riekki J & Ojala T (2010) Support-ing distributed private and public user interfaces in urban envi-ronments. Proc. 11th Workshop on Mobile Computing Systems and Applications (HotMobile 2010), Annapolis, MD, USA, 25-30. (Best Presentation Award).Hosio S, Kukka H & Riekki J (2010) Social surroundings: bridg-ing the virtual and physical divide. IEEE MultiMedia 17(2): 26-33.Suutala J, Fujinami K & Röning J (2010) Persons tracking with Gaussian process joint particle fi ltering. IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2010), Kittilä, Finland, August 29 - September 1, 160-165.Aubert J, Schaberreiter T, Incoul C, Khadraoui D & Gateau B (2010) Risk-based methodology for real-time security monitor-ing of interdependent services in critical infrastructures. IEEE Computer Society, 262-267.Juutilainen I & Röning J (2010) How to compare interpretatively different models for the conditional variance. Journal of Applied Statistics 37(5): 983-998.Kantola J, Perttunen M, Leppänen T, Collin J & Riekki J (2010) Context awareness for GPS-enabled phones. Proceedings of ITM2010, Institute of Navigation, January 25-27.Kemppainen A, Haverinen J, Vallivaara I & Röning J (2010) Near-optimal SLAM exploration in Gaussian processes. 2010 IEEE International Conference on Multisensor Fusion and Inte-gration for Intelligent Systems, 7-13.Kortelainen J, Halunen, K & Kortelainen T (2010) Multicolli-sion attacks and generalized iterated hash functions. Journal of Mathematical Cryptology.Kätevä J, Laurinen P, Rautio T, Suutala J, Tuovinen L, Röning J (2010) DBSA - A device-based software architecture for data mining. Proceedings of the 2010 ACM Symposium on Applied Computing (SAC 2010), 2273-2280.


Leppänen T, Perttunen M, Riekki J & Kaipio P (2010) Sensor network architecture for cooperative traffi c applications. Pro-ceedings of 6th International Conference on Wireless and Mobile Communications, September 20-25, 400-403.Mikkola H, Rahikkala A, Kumpulainen K & Riekki J (2010) Ac-tiveAquarium – virtual fi sh refl ecting your physical activity. Pro-ceedings of the 20th Annual Conference of the European Teacher Education Network (ETEN) (Eds. Kumpulainen K & Toom A), 7-18.Thepvilojanapong N, Konomi S, Yura J, Iwamoto T, Pirttikan-gas S, Ishida Y, Iwai M, Tobe Y, Yokoyama H, Nakazawa J & Tokuda H (2010) Aquiba: An energy-effi cient mobile sensing system for collaborative human probes. DASFAA 2010, Part II (Ed. Kitakawa H), Springer-Verlag, Berlin / Heidelberg, April, LNCS 5982: 420-423.Pietikäinen P, Karjalainen K, Röning J & Eronen J (2010) Socio-technical security assessment of a VoIP system. IEEE Computer Society, Los Alamitos, CA, USA, 141-147.Siirtola P, Laurinen P, Koskimäki H & Röning J (2010) Rec-ognizing user interface control gestures from acceleration data using time series templates. The 7th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2010), Funchal, Madeira-Portugal, June 15-18, 176-182. Pirttikangas S, Tobe Y & Thepvilojanapong N (2010) Smart environments for occupancy sensing and services. Chapter in Handbook of Ambient Intelligence and Smart Environments (AISE) (Eds. Nakashima H, Aghajan H & Augusto JC), Springer, 825-849. Riekki J, Sanchez I & Pyykkönen M (2010) Remote control for pervasive services. International Journal of Autonomous and Adaptive Communications Systems 3(1): 39-58.Stenlund L, Riski K, Seppä J, Pudas M, Vähäsöyrinki M, Tuhkanen V & Röning J (2010) Traceable characterization of a bending millimetre scale cantilever for nanoforce sensing. Meas-urement Science and Technology 21(7).Su X, Fucci D & Riekki J (2010) A framework to enable two-lay-er inference for ambient intelligence. International Symposium on Ambient Intelligence (ISAmI 2010) (Eds. Augusto JC et al), Advances in Soft Computing 72, Springer Berlin / Heidelberg, 29-36.Su X & Riekki J (2010) Transferring ontologies between mobile devices and knowledge-based systems. Proc. 8th IEEE/IFIP In-ternational Conference on Embedded and Ubiquitous Computing (EUC 2010), December 11-13, Hongkong, China, 127-135.Su X & Riekki J (2010) Bridging the gap between semantic web and networked sensors: A position paper. 3nd International Workshop on Semantic Sensor Networks (SSN 2010), with 2010 International Semantic Web Conference (ISWC), November 7-11, Shanghai, China.

Kortelainen T, Kortelainen J & Halunen K (2010) Variants of multicollision attacks on iterated hash functions, Inscrypt 2010, Springer-Verlag.Tamminen S, Juutilainen I & Röning J (2010) Modelling of Charpy V test rejection probability. Ironmaking & Steelmaking 37(1): 35-40.Tamminen S, Juutilainen I & Röning J (2010) Quantile regres-sion model for impact toughness estimation. 10th Industrial Con-ference on Data Mining (ICDM 2010), Berlin, Germany, July 12-14, 263-276.Tokola T, Halunen K & Röning J (2010) Benefi ts and arrange-ments of Bachelor’s thesis in information engineering. Koli Call-ing 2009.Tuovinen L, Talus T, Koponen E, Laurinen P & Röning J (2010) A task-to-method transformation system for self-confi guration in manufacturing. Proceedings of the 8th IEEE International Con-ference on Industrial Informatics (INDIN 2010), 245-252.Turunen T, Pyssysalo T & Röning J (2010) Mobile AR require-ments for location based social networks. The International Jour-nal of Virtual Reality, 67-78.Vallivaara I, Haverinen J, Kemppainen A & Röning J (2010) Simultaneous localization and mapping using ambient mag-netic fi eld. 2010 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, Salt Lake City, Utah, USA, 14-19.Huikari V, Koskimäki H, Siirtola P & Röning J (2010) User-in-dependent activity recognition for industrial assembly lines - fea-ture vs. instance selection. Proc. 5th International Conference on Pervasive Computing and Applications (ICPCA 2010), Maribor, Slovenia, December 1-3.Zhou J & Riekki J (2010) Context-aware pervasive service com-position. IEEE Proceedings of 1st Int. Conference on Intelligent Systems, Modelling and Simulation, ISMS2010, January 27-29, Liverpool, UK, 437-442.Zhou J, Gilman E, Palola J, Riekki J, Ylianttila M & Sun J (2010) Context-aware pervasive service composition and its implemen-tation. Personal and Ubiquitous Computing, 1-13.Zhou J, Gilman E, Riekki J, Rautiainen M & Ylianttila M (2010) Ontology-driven pervasive service composition for everyday life. ISoLA, Part I (Eds. Margaria T & Steffen B), LNCS 6415, Springer-Verlag, 375–389.Zhou J, Gilman E, Ylianttila M & Riekki J (2010) Pervasive serv-ice computing: visions and challenges. Proc. 10th IEEE Int. Con-ference on Computer and Information Technology (CIT2010), June 29 - July 01, Bradford, UK, 1335-1339.Collin J, Kaipio P, Kutila M, Leppänen T, Markkula J, Maz-helis O, Perttunen M, Pirttikangas S & Riekki J (2010) Älyä liikenteeseen. Prosessori (11): 22-25.


MACHINE VISION GROUP (MVG)

Professor Matti Pietikäinen, Professor Janne Heikkilä and Professor Olli SilvénComputer Science and Engineering Laboratory,

Department of Electrical and Information Engineering, University of Oulumkp(at)ee.oulu.fi , jth(at)ee.oulu.fi , olli(at)ee.oulu.fi

http://www.cse.oulu.fi /MVG

Background and MissionThe Machine Vision Group (MVG) is renowned world-wide for its expertise in computer vision. The group has a solid record, which now spans over 29 years, of scientifi c merits on both basic and applied research on computer vi-sion. The mission of the group is to always pursue actual research challenges and improve the state-of-the-art meth-ods.

MVG works as a single well-focused research group, and intensively collaborates with other groups of similar status in Europe, the USA, China and Japan. Within the Seventh Framework Programme FP7, the group currently partici-pates in a project consortium of Trusted Biometrics un-der Spoofi ng Attacks (TABULA RASA). Another project, Mobile Biometry (MOBIO), which was also coordinated by the IDIAP Research Institute, Switzerland, ended in December 2010.

The research areas of the group range from generic com-puter vision methodologies to machine vision applications and vision systems engineering. The main areas of research are computer vision methods, human-centered vision sys-tems and vision systems engineering. The results of the group’s research have been widely exploited in industry, and contract research forms a part of its activities.

Highlights and Events in 2010During 2010, the group produced a large number of signif-icant scientifi c contributions and published its key results through major conferences and journals. The group has es-tablished its position among the most signifi cant research teams in the fi eld of machine vision. One of the indicators of this is that many of its papers are frequently cited. Its landmark paper “Multiresolution gray-scale and rotation invariant texture classifi cation with local binary patterns”, which was published in the IEEE Transactions on Pat-tern Analysis and Machine Intelligence (PAMI) journal in 2002, has now well over 600 citations according to the ISI Web of Knowledge. It is the most cited Finnish paper published since 2002 in electrical & electronic engineer-ing and all computer science categories, and the 6th most cited paper of the prestigious PAMI journal.

In addition, visibility outside academic forums has been achieved. The group has a tradition of presenting regularly its research projects and activities in different media. For example, a story on the human-robot interaction research, conducted jointly by the Machine Vision Group and the

Intelligent Systems Group, appeared on Prisma Studio sci-ence program on Finnish TV channel YLE TV1 on March 23, 2010.

The group hosts regularly visits by respected and re-nowned scientists from abroad. In 2010, the group had the pleasure of hosting Professor Maja Pantic from the Impe-rial College London, UK, Professor Nicu Sebe from the University of Trento, Italy, Professor Jan Flusser from the Academy of Sciences of the Czech Republic, Dr. Matthew Blaschko from the University of Oxford, UK, Dr. Peter Sturm from INRIA Rhône-Alpes, France, and Dr. Christo-pher Mei from the University of Oxford, UK. In addition, several domestic visitors from both public and private sec-tors were briefed about our research activities.

The group has established active collaboration with some of the world’s leading institutions and top scientists. It has had collaboration with the University of Maryland (USA) from the early 1980´s. More recent partners include the Chinese Academy of Sciences, the Academy of Sciences of the Czech Republic, the Czech Technical University in Prague, the École Polytechnique Fédérale de Lausanne (EPFL), the University of Oxford, the University of To-kyo and the consortium of the European Mobile Biometry (MOBIO) project. In 2010, MVG established contacts with Eurecom (France), the University of Turin (Italy) and the consortium of the new TABULA RASA project.

The group fosters international mobility to and from our unit. Two of our researchers made 1-3 month research vis-its to foreign institutions during the reporting period. The group has attracted visiting postdoctoral researchers and postgraduate students from abroad, who are affi liated with us for periods of a couple of weeks up to several years. Two such short visits were made to MVG in 2010 and two long visits began. In addition, MVG recruited two doctoral students from abroad.

The group and its members are active in the scientifi c community. For example, in 2010 Prof. Pietikäinen was invited to join the editorial board of the Image and Vision Computing journal. Dr. Abdenour Hadid co-organized the fi rst Algerian summer school on biometrics, which was held in Algiers, Algeria on May 23-27, 2010. He also lec-tured a tutorial on image and video descriptors at the In-ternational Conference on Image Processing Theory, Tools and Applications (IPTA 2010) which was held in Paris, on July 7-10, 2010.

The professors of the group were committee members of several major conferences and many researchers of the


group served as reviewers for various journal and con-ference articles. Dr. Guoying Zhao and Prof. Matti Pi-etikäinen served as a co-editors for a book entitled “Ma-chine Learning for Vision-Based Motion Analysis: Theory and Techniques”, Springer, 2011, together with Dr. Liang Wang and Dr. Li Cheng. Dr. Guoying Zhao was also serv-ing as a guest editor of the special issue New Advances in Video-based Gait Analysis and Applications: Challenges and Solutions on IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics, together with Dr. Liang Wang, Dr. Nasir Rajpoot and Prof. Mark Nixon. Dr. Esa Rahtu served as a member of the Board of the Pattern Rec-ognition Society of Finland.

The Pattern Recognition Society of Finland ranked Timo Ahonen’s PhD thesis “Face and texture image analysis with quantized fi lter response statistics” as the best Finn-ish PhD thesis made on pattern recognition in 2008-2009. This thesis was also awarded as the best PhD thesis of the University of Oulu in 2009.

Scientifi c ProgressThe current main areas of the research are: 1) Computer vision methods, 2) Human-centered vision systems, and 3) Vision systems engineering.

Computer Vision MethodsThe group has a long and highly successful research tradi-tion in two important generic areas of computer vision: texture analysis and geometric computer vision. In the last few years, the research in computer vision methods has been broadened to cover also two new areas: computation-al imaging and object detection and recognition. The aim in all these areas is to create a methodological foundation for development of new vision-based technologies and in-novations.

Texture AnalysisTexture is an important characteristic of many types of im-ages and can play a key role in a wide variety of applica-tions of computer vision. A textured area in an image can be characterized by a non-uniform or varying spatial dis-tribution of intensity or color. The variation refl ects some changes in the scene being imaged. The MVG has long traditions in texture analysis research and ranks among the world leaders in this area. The Local Binary Pattern (LBP) texture operator has been highly successful in numerous applications around the world and has inspired plenty of new research on related methods, such as the Local Phase Quantization (LPQ) and Weber Law Descriptor (WLD), also proposed by researchers from MVG. The writing of a book entitled “Computer Vision Using Local Binary Pat-terns”, authored by M. Pietikäinen, A. Hadid, G. Zhao and T. Ahonen, was in its fi nal stages in 2010. The book will be published by Springer in 2011.

In the reporting year, a novel learning framework for im-age descriptor design was proposed. The Fisher separation criterion (FSC) is used to learn the most reliable and ro-bust dominant pattern types considering intra-class simi-

larity and inter-class distance. Image structures are thus described by a new FSC-based learning (FBL) encoding method. LBP was taken as an input to this framework, but it can be easily generalized for other purposes by utilizing different histogram descriptors. FBL-LBP differs from the previous LBP approaches since the FBL framework learns robust dominant types of each class instead of using fi xed pattern types. The proposed approach outperformed other comparative methods in the classifi cation experiments on three commonly used texture databases.

The FCS-based learning framework.

We have also continued our research on dynamic texture segmentation and recognition of actions utilizing spatio-temporal texture descriptors. The results of this research will be published in 2011 - 2012.

We presented a novel blur tolerant decorrelation scheme for the local phase quantization (LPQ) texture descriptor. As opposed to previous methods, the new model can be applied with virtually any kind of blur, regardless of the point spread function. Furthermore, the new model takes into account the changes in inter-pixel correlation origi-nating from the blur. The implementation does not suffer from multiple solutions like the decorrelation scheme used in the original LPQ, but still retains the same run-time computational complexity. The experimental evaluation has indicated considerable improvements in the perfor-mance of the LPQ descriptors in texture classifi cation with blurred images and only negligible loss of accuracy with sharp images.

For video analysis, a novel descriptor called volume lo-cal phase quantization (VLPQ) was developed. VLPQ is an extension to the LPQ descriptor used for 2D image analysis, and the method is based on binary encoding of the quantized phase information of the local 3D Fourier transform. The VLPQ descriptor is highly tolerant to im-age blur in the case of centrally symmetric spatial blur-ring caused by, for example, misfocused optics or camera motion. The method has been used for dynamic texture classifi cation, and the results achieved are promising. The VLPQ descriptor is not only shown to be blur insensitive, but effi cient also in the case of sharp video frames.

Texture is a key feature in the visual diagnosis of variety of medical specimens. Manual inspection of specimens with


a light microscope is still to date the standard procedure. In collaboration with Dr. Johan Lundin’s research group from Institute for Molecular Medicine Finland (FIMM), we have been developing computer-aided diagnosis tools to introduce ways of focusing and speeding-up this quite labor intensive manual workload of pathologists. The LBP texture descriptor was successfully used to automatically classify cancerous tissues into stromal and parenchymal tissue compartments using the Support Vector Machine (SVM) classifi er. Other applications include parasite de-tection and patient outcome prediction based on machine vision tools. In addition to research on analysis algorithms, our collaboration includes developing of a mobile diag-nostic device, based on mobile microscopy, which could be used particularly in developing countries.

Computational ImagingComputational imaging is a relatively new term and it re-fers to the convergence of the camera and the computer. It provides more fl exibility to image acquisition and en-ables more advanced techniques to be employed that go beyond the capabilities of traditional photography. Images can be processed in many ways, for example, to synthesize new views from a set of input frames. As the actual im-age is reconstructed in the computer, computational cam-eras can also utilize unconventional optical solutions for image acquisition. Due to its wide range of applications computational imaging (also referred to as computational photography) has become a major fi eld of research within both the computer vision and the computer graphics com-munities.

In our research, we have investigated means for utilizing computer vision in various problems of computational imaging. One important problem is transparency, which complicates object segmentation and produces artifacts in free viewpoint video. The current methods for alpha matte estimation rely heavily on user input and smooth image statistics. Following the success of multi-view stereo meth-ods, we developed a multi-view alpha matting method that requires no user input and makes no assumption on image statistics. It analyzes the 3D structure of the scene to fi nd several samples of the same physical point as well as the background color. It is then able to formulate constraints on the components and estimate the true foreground color and level of transparency.

Stereo matching is a fundamental problem in free view-point rendering, as well as in computer vision, but it still remains an open problem. Recent patch based multi-view reconstruction approaches are able to accurately recon-struct a well textured scene. However, non-textured sur-faces are not reconstructed by such methods, thus result-ing in a semi-dense depth map for each image. In order to complement these methods, we have developed a hole-fi ll-ing algorithm that produces a fully dense depth map from a semi-dense one. It uses planar models to fi t a surface to the homogenous regions, taking into account the recon-structed geometry on its boundary.

Depth map reconstruction using the hole-fi lling algorithm.

Knowledge of the scene type can provide important in-formation for image acquisition. In our recent work, we have studied different approaches that can be used in rec-ognizing landscape scenes. The experiments indicate that a global texture-based approach competes with or out-performs other more complex methods in the landscape image recognition problem. Especially the computational cost of the method relying on Local Binary Pattern (LBP) representation is minimal compared to the GIST and bag-of-words methods investigated in the study.

Automatic image quality evaluation can be used to pro-vide immediate feedback about the image quality to the photographer. In our research, we have developed a novel framework for no-reference image quality evaluation. The method uses seven image degradation features that are extracted and fed to a classifi er that decides whether the image has good or bad quality based on properties such as blur, under or over exposure, saturation, and lack of meaningful information. The method was designed to be computationally effi cient in order to enable real-time per-formance in embedded devices.

Flash-no-fl ash imaging is a branch of computational imag-ing that aims to combine the natural color contents of am-bient light images with the details available in all the input images. Aligning images taken with fl ash and without fl ash is a challenging problem, where contents of image pairs can differ quite radically. Flash removes natural shadows and creates new ones altering the gradient directions and strengths, and also the colors may change radically. Sur-faces may refl ect the fl ash light resulting in anomalies compared to ambient light scenes. Many conventional ap-proaches are unsuitable for reliable fl ash-no-fl ash image registration due to these changes.

We have developed a new image aligning method utiliz-ing mutual information driven point matching refi nement for the fl ash-no-fl ash image pair registration. Initial point matches obtained with block wise phase correlation are refi ned using iterative location adjustment with increase in mutual information as the driving force. The approach generally outperforms the tested reference methods in terms of registration accuracy for this special case. Visual inspection also confi rms that the proposed method suc-ceeds to register fl ash-no-fl ash image pairs in cases where other state-of-art approaches fail. We are further interested in the fusion of regular low ambient light images with in-frared images taken with and without fl ash.


Flash-no-fl ash image registration.

We recently considered the problem of visibility improve-ment in outdoor environments. The main cause of impair-ments in visibility is often represented by atmospheric scat-tering, which derives from the presence in the air of aero-sol particles, and results in what is commonly perceived as fog, haze, as well as rain or snow. Several current methods try to remove the haze layer from a single image by using a physical model of atmospheric scattering and solving an inverse problem. However, the quality of the de-hazed im-ages strongly depends on the accuracy of the parameters that must be estimated: scene depth, and airlight color. The latter typically gives rise to color artifacts that manifest lo-cally in the processed image. We have developed a novel method to robustly estimate the airlight color from a single image and improve the de-hazing results, eliminating arti-facts that are still visible in state-of-the-methods.

Object Detection and RecognitionObject recognition and detection refer to the tasks of clas-sifying and localizing a given object from an image or video sequence. Since the problem in its general form is very diffi cult to tackle, it is usually constrained to a par-ticular aspect with restricted object class and detection tolerance. For example, one can focus on detecting the bounding boxes containing all persons in the given image. In another example, one could be interested in the pixel level segmentations of cars in aerial images acquired over a city. Our work in MVG mainly concentrates on detecting salient image segments and humans.

Generally, current object detectors scan the image on several scales and examine all sub-windows using a pre-trained class specifi c classifi er. Depending on the classi-fi er, this might be a computationally very expensive ap-proach, and it can still be extended only to a small part of the possible image sub-windows. Furthermore, the process has to be repeated for each object class to be detected. An attractive alternative would be to fi rst use some light pro-cessing to identify a set of salient focus points, and then concentrate the computationally intensive methods only to these locations. It is also likely that the human visual sys-tem proceeds in this way.

In our research, we have developed methods for locat-ing salient image segments in a computationally effi cient manner. The new approach is based on measuring the lo-cal feature contrasts in a novel probabilistic framework. Compared to other state of the art saliency detector, our method offers more reliable segmentations which are also in line with the human annotators. Furthermore, the meth-od is also applicable to video sequences, where the issue of computational complexity of class specifi c detectors is often prohibitively large.

Salient object segmentation using the local feature con-trast method.

For human detection, we have presented a new descriptor, which is mainly targeted at still images. It is referred to as isotropic granularity-tunable gradients partition (IGGP), which is extended from granularity-tunable gradients par-tition (GGP) descriptors. The isotropic representation is achieved by aligning the features with different orientation channels according to their principal angles. The benefi ts of this extension are twofold: fi rstly, the noise introduced by the small partitions in the original GGP descriptors is eliminated and the performance can be essentially im-proved; secondly, the integral image based fast computa-tion is applied and more than 20 times speedup has been achieved.

A novel descriptor for human detection in video sequences has been developed. It is referred to as the spatial-temporal granularity-tunable gradients partition (STGGP), which is an extension of granularity-tunable gradients partition (GGP) from the still image domain to the spatial-temporal domain. Specifi cally, the moving human body is consid-ered as a 3-dimensional entity in the spatial-temporal do-main. Then, in 3D Hough space, we defi ne the generalized plane as a primitive to parse the structure of this 3D entity. The advantage of the generalized plane is that it can toler-ate imperfect planes with a certain level of uncertainty in rotation and translation. The robustness to the uncertainty is controlled quantitatively by the granularity parameters defi ned explicitly in the generalized plane.

In the HIMA project that was fi nished in 2010, a compara-tive study of recent human detection methods was carried out. The objective was to fi nd the best available human detection methods that could be used in the safety applica-tions of non-road mobile machinery. The study suggested that when considering both accuracy and speed, a good choice would be the well-known HOG based approach, although new approaches such as IGGP can achieve bet-ter accuracy, but at the cost of much higher computational complexity.

Human detection methods published in the literature are usually based on analyzing monocular images or image sequences. Stereo images, on the other hand, provide also the depth information that can be used as complementary measurements for human detection. The analysis frame-work developed in the HIMA project was based on the fu-sion of several single view human detectors and 3D fi lter-ing of the candidate objects. The framework uses the idea of pixel-wise human probabilities, which are obtained by several separate detection trials following binomial dis-tribution. The framework has been extensively evaluated


with image sequences of authentic work machine environ-ments, and it has proven to be feasible.

Geometric Computer VisionGeometry is an important aspect of computer vision. The laws of geometry and optics describe how the three-dimen-sional world is imaged on the camera sensor and, hence, an understanding of imaging geometry is important for the development of automatic image analysis methods. The basis for this understanding is the geometry of multiple views, which we have studied from both theoretical and computational viewpoints. In this area, we have a strong research background in certain basic problems, such as geometric camera calibration and image matching, and our research in these areas spans a period of several years. Recently, we have increased the emphasis on research in image-based modeling and rendering. This work directly builds upon our previous expertise. For example, our ear-lier approach to quasi-dense image matching could be par-ticularly useful in image-based modeling. The motivation for studying multi-view reconstruction and visualization of 3D scenes lies in the increasing number of applications where such techniques are becoming useful and practical.

One of our recent developments in image-based modeling is the extension of the quasi-dense match propagation al-gorithm to the case of three views. The results of this work show that using the three-view constraint and the best-fi rst propagation strategy together during the correspondence growing improves the accuracy of matching and reduces the occurrence of outliers. In particular, compared with two-view stereo, the new method is more robust for re-peating texture. Further, the approach seems particularly promising because it is able to provide reliable and ac-curate matches with a single growth stage and without additional outlier fi ltering or iterative refi nement for the matched patches, as required in some other patch-based multi-view stereo methods.

In addition, besides studying conventional passive multi-view stereo reconstruction methods, we have recently ex-plored possibilities for utilizing modern active depth cam-eras in computer vision applications. For example, range sensors that are capable of operating at a video frame rate, such as time-of-fl ight cameras, could be useful in human-computer interaction.

Multi-view reconstruction: A depth map (bottom left) is com-puted from the three views above using our quasi-dense matching method. A quasi-dense point cloud obtained by fusing several depth maps is illustrated on the right.

Human-Centered Vision SystemsIn future ubiquitous environments, computing will move into the background, being omnipresent and invisible to the user. This will also lead to a paradigm shift in hu-man-computer interaction (HCI) from traditional com-puter-centered to human-centered systems. We expect that computer vision will play a key role in such intelligent systems, enabling, for example, natural human-computer interaction, or identifying humans and their behavior in smart environments.

Face Recognition and BiometricsAutomatic face analysis has become a very active topic in computer vision research as it is useful in several applica-tions, like biometric identifi cation, visual surveillance, hu-man-machine interaction, video conferencing and content-based image retrieval. Face analysis is challenging due to the fact that a face is a dynamic and non-rigid object which is diffi cult to handle. Its appearance varies due to changes in pose, expression, illumination and other factors such as age and make-up. In 2010, our biometric related research focused on novel face descriptors, face analysis using fa-cial dynamics, face recognition from degraded images, face detection and verifi cation in mobile devices, soft bio-metrics (e.g. gender/age/ethnicity classifi cation), face rec-ognition under spoofi ng attacks and speaker recognition using visual information.

In the period 2008-2010, MVG participated in the Mo-bile Biometry (MOBIO) project funded by the European Commission, aiming at researching new mobile services secured by biometric authentication means. The LBP method developed in MVG played an important role in the project. Starting from November 2010, we are con-tinuing the biometry theme within a new FP7 consortium called TABULA RASA (starting in November 2010 and lasting 42 months). TABULA RASA looks at the vulner-abilities of existing biometric systems to spoofi ng attacks to a wide range of biometrics including face, voice, gait, fi ngerprints, retina, iris, vein, electro-physiological signals (EEG and ECG) etc. We have started investigating whether skin refl ectance properties could be used to detect possible spoofi ng attacks against face based biometric systems.

Examples of occluded face images from different source.


The presence of facial occlusions (e.g. due to sunglasses, hats, scarves, beards etc) is unfortunately quite common in real-world applications, especially when individuals are not cooperative with the system such as in video surveil-lance scenarios. In 2010, we investigated, in cooperation with Eurecom institute (France), new approaches to im-prove the recognition of faces occluded by sunglasses and scarves. We propose an effi cient approach which consists of fi rst detecting the presence of a scarf/sunglasses and then processing the non-occluded facial regions only. The occlusion detection problem is approached using Gabor wavelets, PCA and support vector machines (SVM), while the recognition of the non-occluded facial part is performed using block-based local binary patterns. Experiments on the AR face database showed that the proposed method yields signifi cant performance improvements compared to existing work on recognizing partially occluded and also non-occluded faces.

Blur is often present in real-world images and signifi cantly affects the performance of face recognition systems. Basi-cally, there are two main approaches for face recognition under blur. The fi rst direction consists of deblurring the face images via deconvolution with an aim of recovering sharp images ready for processing using any conventional face recognition algorithm. Another way of handling the recognition of blurred faces consists of deriving blur-in-variant features directly from the blurred images without having to recover the sharp images which, anyway, are not necessarily needed for recognition. In 2010, we continued our joint research with the University of Tokyo for im-proving the recognition of blurred faces and proposed a new approach which inherits and combines the advantages of the two categories of methods. The idea consists of fi rst reducing the amount of blur in the images via deblurring and then extracting blur-tolerant descriptors (using LPQ) for recognition. We assessed our analysis on real blurred face images (FRGC 1.0 database) and also on face imag-es artifi cially degraded by focus blur (FERET database), demonstrating signifi cant performance enhancement com-pared to the state-of-the-art.

In 2010, speaker identifi cation using visual information has also been investigated. Visual information from cap-tured video is indeed important for speaker identifi cation under noisy conditions. Combination of LBP dynamic tex-ture and Edge Map structural features were proposed to take both motion and appearance into account, providing the description ability for spatiotemporal development in speech. Spatiotemporal dynamic texture features of local binary patterns extracted from localized mouth regions are used for describing motion information in utterances, which can capture the spatial and temporal transition char-acteristics. Structural edge map features are extracted from the image frames for representing appearance character-istics. A combination of dynamic texture and structural features takes both motion and appearance together into account, providing the description ability for spatiotem-poral development in speech. In the experiments on the BANCA and XM2VTS databases, the proposed method obtained promising recognition results compared to the other features.

Recognition of Facial Expressions and Emo-tionsA goal of facial expression recognition is to determine the emotional state of the face regardless of its identity. The face can express emotion sooner than people verbalize or even realize their feelings, and research in social psychol-ogy has shown that facial expressions form the major mo-dality in human communication. So facial expression is one of the most powerful, natural and immediate means for human beings to communicate their emotions and in-tentions.

In facial expression recognition, both Region-of-Inter-est (ROI) features and geometric features are helpful for representation. We have investigated an effective compo-nent-based approach to facial expression recognition by combining ROI and geometric features together. Spatio-temporal local texture features were computed from geo-metric facial areas centered at 38 detected fi ducial interest points (determined by ASM) and the generated histograms were concatenated to describe facial expressions. Further-more, since not all features are important to facial expres-sion recognition, the problem of how to select the most discriminative features for facial components is addressed by using AdaBoost for boosting the strong appearance and motion features.

Component-based spatiotemporal appearance features.

Moreover, it is widely agreed that emotion is a multimodal procedure, which can be expressed mainly by facial ex-pression, but also by head and body movements, gestures, speech and some physical characteristics like heart rate and blood pressure. Our recent plan is to collaborate with Prof. Tapio Seppänen’s Biomedical Engineering Group to combine signals from different sensors, and develop ro-bust systems for emotion recognition. Investigation into micro-expression analysis and distinguishing spontaneous facial expressions from acted expressions will also be in-cluded.

Visual Speech Analysis and SynthesisIt is well known that human speech perception is a mul-timodal process. Visual observation of the lips, teeth and tongue offers important information about the place of pronunciation articulation. Techniques for interpreting speech using visual information only (namely, lip-reading) have a wide range of applications in the real world. For in-


stance, such a system can be used to understand someone’s speaking in a highly noisy environment (e.g., among a large crowd or in a moving vehicle). It can also be used to improve the quality of the lives of people with hearing im-pairments. We propose a novel graph embedding method for the problem of automatic lip-reading. To characterize the temporal connections among video frames of the same utterance, a new distance metric is defi ned on a pair of frames, and graphs are constructed to represent the video dynamics based on the distances between frames. Audio information is used to assist in calculating such distances. For each utterance, a subspace of the visual feature space is learned from a well-defi ned intrinsic and penalty graph within a graph-embedding framework. Video dynamics are found to be well preserved along some dimensions of the subspace. Discriminatory cues are then decoded from curves of the projected visual features to classify different utterances.

Overview of the proposed method.

Video-realistic speech animation plays an important role in the area of affective human-robot interactions. The goal is to synthesize a visually realistic face that can talk just as we do. In this way, it can provide a natural platform for a human user and a robot to communicate with each other. For instance, when there are interactions between human users and machines, instead of using text or audio only to communicate, an animated talking head synchronized with the audio may attract more attention from users and make such applications more engaging. A visually realistic talking head could make users feel comfortable and natu-ral, and hence improve the quality of the human-machine interactions. Moreover, such synthesis techniques may be

used to generate visual cues for audio clips so as to help hearing-impaired people understand machine responses better.

The key for synthesizing a talking head is how to real-ize a visually realistic talking mouth since most of the dy-namic shape and texture changes on the face appear in the mouth area. In our work, video synthesis is achieved by fi rst learning generative models from the recorded speech videos and then using the learned models to generate vid-eos for novel utterances. A generative model considers the whole utterance contained in a video as a continuous process and represents it using a set of trigonometric func-tions embedded within a path graph. To synthesize a video for a novel utterance, the utterance is fi rst compared with the existing ones to fi nd the phoneme combinations that best approximate the utterance, and the image frames are then synthesized using the learned model.

An example of graph representation of a speech video.

Tracking and Recognition in Camera NetworksAn ever increasing number of cameras are being utilized in surveillance to monitor activities in locations like air-ports, hospitals and offi ce buildings. One major issue of multiple camera surveillance is to detect and track mov-ing objects in each view of a camera network and main-tain their identities while moving across the views. Our research has focused on the building blocks of a complete camera network tracking system, including object detec-tion, tracking and re-identifi cation nodes, and a communi-cation infrastructure which binds all the nodes together to form a fl exible sensor network.

Tracking objects across non-overlapping camera views is one of the main challenges in object tracking. During the year 2010, we developed a real-time system for tracking and re-recognition of moving objects across views, and published the underlying algorithm in the International Conference on Pattern Recognition (ICPR). The algorithm uses adaptive boosting (AdaBoost), binary classifi cation trees and k-means clustering to build classifi ers for fea-ture-based classifi cation of object tracking sequences. The features (color, texture, shape, etc.) extracted from the single-view tracks are clustered and then learned to obtain boosted tree classifi ers that provide reliable object re-rec-ognition even on previously unseen views of the object.


Image clusters for a single tracking sequence. The k-means clustering of image features to three clusters results in similar temporal divisions frequently.

We also presented a new solution to the problem of person re-identifi cation. Person re-identifi cation means to match observations of the same person across different time and possibly different cameras. The appearance based person re-identifi cation must deal with several challenges such as variations of illumination conditions, poses and occlu-sions. Our proposed method is inspired from the spirit of self-similarity. Self-similarity is an attractive property in visual recognition. Instead of comparing image descriptors between two images directly, the self-similarity measures how similar they are to a neighborhood of themselves. The self-similarities of image patterns within the image are modeled in two different ways in the proposed Global Color Context (GCC) method. The spatial distributions of self-similarities with regard to color words are combined to characterize the appearance of pedestrians. Very prom-ising results are obtained on the public ETHZ database compared with the state-of-art.

An overview of the proposed method. Local color features are extracted densely and clustered to form a color code-book (a). Then, the assignments of color features to color words in the codebook are explored. Color features from the same visual color word are marked with the same color in (b). For each color word in the codebook, the spatial oc-currence distributions of color self-similarities are learned (c).

Affective Human-Robot InteractionMobile robots equipped with cameras and other sensors provide an excellent environment for investigating human-machine interfaces, because communication with home-aid robots developed for service tasks in homes, nursing homes, retirement homes and offi ces, for example, must be easy and natural. An intelligent robot should be able to detect and identify the user in order to personalize its ser-vices and guarantee security, it should recognize the user’s emotions to allow for affective interaction, and it should be able to communicate easily with the user and understand given commands by recognizing speech and gestures. The

machine must also learn to change its behavior according to the user and his/her emotional state.

Together with the Intelligent Systems Group, we have col-laborative research on Affective Human-Robot Interaction (HRI), supported by the Academy of Finland (2009-2012) and the European Regional Development Fund (2010-2012). The research is divided into three parts dealing with machine vision methodology for HRI, robot embodiment and learning in HRI, and experimental validation of affec-tive HRI. The basic research of the MVG in 2010 focused on developing methodologies for a videorealistic talking face to be used in human-robot communication, and for recognizing human gestures.

The experimental system being developed for HRI re-search consists of 1) a Segway Robotic Mobility Platform (RMP 100) equipped with computers, cameras, micro-phones, magnetic fi eld sensors, an avatar display etc., 2) a ubiquitous CSE laboratory environment containing a network of cameras and other sensors embedded in the en-vironment. During the reporting year, a large collection of algorithms developed in the MVG were modifi ed for our new Machine vision algorithm library, from which they can be easily used in HRI project demonstrations. The real-time demonstrations made in 2010 include face and people detection and tracking, determining gender, facial expression and identity of the tracked faces, and applying simple gesture recognition for each tracked person.

A real-time demonstration for human-robot interaction.

Camera-Based Interfaces for Mobile DevicesDesigning comfortable user interfaces for mobile phones is a challenging problem, given the limited amount of in-teraction hardware of the device. However, multiple built-in cameras and the small size of handhelds are under-ex-ploited assets for creating novel applications that are ideal for pocket size devices, but may not make much sense with personal or laptop computers. Studies into alternatives for mobile user interaction have, therefore, become a very ac-tive research area in recent years.

A key advantage of using cameras as an input modality is that it enables recognizing the 3D context in real-time, and at the same time provides for single-handed operations in which the users’ actions are interpreted without touching the screen or keypad. For example, the user’s position and gaze can be measured, in order to display true 3D objects even on a typical 2D screen.

(a) (b) (c)


In our research on camera-based mobile user interfaces, we have developed a prototype for a virtual 3D user inter-face based on face tracking. The determining of the user’s position and gaze is analyzed in real time, enabling the device to display true three-dimensional objects even on a typical 2D LCD screen. In the developed interface, the user-motion and camera input enable interactivity by pro-viding real-time feedback on the context, while simulta-neously guiding the user actions. The solution lies in the extraction of motion and features from sequential video frames, and this can be augmented with the information provided by integrating the data from different sensors such as accelerometers, gyroscopes and magnetometers. The implementations include a parallel pipeline that uses the mobile Graphical Processing Unit (GPU) integrated on the platforms, reducing the computation and power needs of the application.

A virtual 3D user interface.

In the research area of interactive camera-based applica-tions for mobile devices, we have studied algorithms for implementing visual tracking of unknown objects. The user can select the object to be tracked on the touch screen of the phone. To improve the robustness of tracking, on-line learning and adaptive techniques have been consid-ered for object detection. In the algorithms special atten-tion is paid to their computational effi ciency.

On mobile platforms, we have also studied the use of mul-timodal sensor information for several user interaction scenarios: face detection based backlight and key lock con-trol, face detection based automatic image capturing mode detection, and automatic activation of a ”point and fi nd” application. Recently, we have also applied measurement fusion techniques for controlling an interactive panorama image constructor and motion based image browsing.

Vision Systems EngineeringVision systems engineering research provides guidelines for identifying attractive computing approaches, archi-tectures, and algorithms for useful commercial systems. In practice, solutions from low-level image processing to even equipment installation and operating procedures are considered simultaneously. The roots of this expertise are in our industrial visual inspection studies in which we met extreme computational requirements already in the early 1980’s, and we have contributed to the designs of several

industrial systems. We have also applied our expertise to applications intended for smart environments and embed-ded platforms.

Visual inspection research of automated wood strength grading for sawn timber using machine vision has contin-ued. Earlier we have developed a solution that employs real-time feature extraction, classifi cation, and the Finite Element Method (FEM) combined into an adaptive learn-ing scheme. In new developments, one key area was to determine the correlation between features extracted from images and the actual strength of the board in question. Performance of different classifi ers was also studied us-ing only knot based features. Good results were achieved for Finnish Pine in which knots are common and are in many cases one of the most important reasons for reduced strength qualities.

We have continued developing our unusual event detec-tion system. The aim is to create cheap systems that are easy to install unnoticeably. To accomplish this, we have been testing the system with very thin compound eye op-tics. Compound eye optics introduce a new challenging problem for the system, the image quality is considerably lower than with conventional optics. The work in the fu-ture will focus on improving the image quality and thor-ough testing.

Unusual event detection using compound eye imaging.

The dramatic increase in the number of mobile consumer devices such as digital cameras, mobile phones and laptops has been made possible by the advances in the energy effi -ciency of electronics. However, new applications are con-stantly being invented, and old ones are updated, and this has led to an ever-increasing demand for computational power, which results in higher power consumption of de-vices. As battery technologies advance only very slowly, improving the effi ciency of computations has become the only viable alternative: more calculations need to be done with less power.

Our research on energy effi ciency concentrates on this problem from the perspective of signal processing, which is required in practically all mobile devices. Improving the energy effi ciency of signal processing systems cannot be accomplished solely in software or hardware, but re-quires consideration of them both, as well as the interface. Besides the work on improving the energy effi ciency of signal processing, our group is also interested in hardware-


software development tool chains with a focus on design automation. This work is done in close cooperation with national and international partners.

The work on design automation and multiprocessing for DSP has continued in 2010. The topic of quasi-static scheduling for Reconfi gurable Video Coding reached a new milestone as the fi rst implementation of automated quasi-static scheduling was completed, and is to be pub-lished at IEEE ICASSP 2011. The work on Transport Trig-gered Architecture processors has produced, for example, an implementation of a customized, programmable pro-cessor for computing Local Binary Patterns.

We also carried out a performance study on a GPU to ful-fi ll the real-time requirements of the LTE MIMO-OFDM detector. The limited size of the fast on-chip memory re-sources and the required conditional code execution were found to be the limiting factors for the current GPU imple-mentations. An interesting future solution would be a GPU that is designed specifi cally for baseband solutions.

In the InterSync project, we have studied energy effi cient signal processing methods for condition monitoring of steel wire ropes used in cranes. Various fi ltering methods and algorithms for detecting faults in the magnetic mea-surement signal were developed. Promising results were achieved using Wavelet and Hilbert transforms to detect momentary abnormalities. In addition, we have imple-mented methods on the transport triggered architecture (TTA) signal processor in order to achieve effi ciency needed by energy autonomous wireless sensor nodes. The work is now progressing towards pilot installations.

Exploitation of ResultsMany researchers have adopted and further developed our methodologies. Our research results are used in a wide variety of different applications around the world. For example, the Local Binary Pattern (LBP) methodology is used in numerous image analysis tasks and applications, such as biomedical image analysis, biometrics, industrial inspection, remote sensing and video analysis.

The MVG has actively encouraged and supported the birth of research group spin-outs. This gives an opportunity for young researchers to start their own teams and groups. Side results are the spin-out enterprises. According to our experience their roots are especially in the strands of “free academic research”.

There are currently fi ve research based spin-outs founded directly on the computer vision area. The number of the spin-outs could be extended up to thirteen when taking into account the infl uence of the MVG´s thirty-year history and the spin-out companies’ from the spin-out research groups in the area of computer science and engineering.

Future GoalsResearch on human-centered vision systems and affec-tive computing will be further extended. For example, we have started collaboration with Prof. Tapio Seppänen’s

Biomedical Engineering Group in the area of multimodal emotion recognition for affective computing, combining vision with speech and biosignal data.

The research on biomedical imaging is also planned to be extended. MVG is a partner in a new Oulu BioImag-ing OBI network (obi.oulu.fi ), which has been accepted as an associated partner to Euro-BioImaging. The mission of Euro-BioImaging is to create a coordinated and harmo-nized plan for biomedical imaging infrastructure deploy-ment in Europe.

Among our research staff, as many as 40% are currently from abroad. Due to our excellent international reputa-tion, we increasingly attract visitors from abroad to join us for some time, and many of our researchers are willing to make research visits to leading groups abroad. Such bilat-eral collaboration will bring new fresh ideas and expertise to our research.

Close interaction between basic and applied research has always been a major strength of our group. The scientifi c output of the group has been increasing signifi cantly in recent years. With this we expect to have much new po-tential for producing novel innovations and exploitation of research results in collaboration with companies and other partners.

Prof. Matthew Turk from the University of California, Santa Barbara, has been selected to receive the 2011–2012 Fulbright-Nokia Distinguished Chair in Information and Communications Technologies. With this funding, he will stay two months at MVG. This will offer great opportuni-ties for future collaboration.

Personnel



others 16

total 49

person years 31

External Funding

Source EUR



Tekes 99 000



total 1 007 000


Doctoral ThesesKannala J (2010) Models and methods for geometric computer vision. Acta Universitatis Ouluensis C 353.

Selected PublicationsBoutellier J, Bhattacharyya SS & Silvén O (2010) A low-over-head scheduling methodology for fi ne-grained acceleration of signal processing systems. J. Signal Processing Systems 60(3): 333-343.Boutellier J, Lucarz C, Lafond S, Martin Gomez V & Mattavelli M (2011) Quasi-static scheduling of CAL actor networks for reconfi gurable video coding. J. Signal Processing Systems, in press.Boutellier J, Lucarz C, Martin Gomez V, Mattavelli M & Silvén O (2011) Multiprocessor scheduling of datafl ow programs within the reconfi gurable video coding framework. Algorithm-Architec-ture Matching for Signal and Image Processing, 237-252.Cai Y & Pietikäinen M (2010) Person re-identifi cation based on global color context. Proc. Int’l Workshop Visual Surveillance, 10 p.Cai Y, Huang K, Tan T & Pietikäinen M (2010) Recovering the topology of multiple cameras by fi nding continuous paths in a trellis. Proc. Int’l. Conf. Pattern Recognition, 3541-3544.Cai Y, Takala V & Pietikäinen M (2010) Matching groups of people by covariance descriptor. Proc. Int’l Conf. Pattern Rec-ognition, 2444-2447.Chen J, Shan S, He C, Zhao G, Pietikäinen M, Chen X & Gao W (2010) WLD: A robust local image descriptor. IEEE Trans. Pat-tern Analysis and Machine Intelligence 32(9): 1705-1720.Guo Y, Zhao G & Pietikäinen M (2010) Descriptor learning based on Fisher separation criterion for texture classifi cation. Proc. Asian Conf. Computer Vision, 1491-1500.Hadid A, Nishiyama M & Sato Y (2010) Recognition of blurred faces via facial deblurring combined with blur-tolerant descrip-tors. Proc. Int’l Conf. Pattern Recognition, 1160-1163.Heikkilä J, Ojansivu V & Rahtu E (2010) Improved blur insensi-tivity for decorrelated local phase quantization. Proc. Int’l Conf. Pattern Recognition, 818-821.Heimonen T & Heikkilä J (2010) A human detection framework for heavy machinery. Proc. Int’l Conf. Pattern Recognition, 416-419.Huang X, Zhao G, Pietikäinen M & Zheng W (2010) Dynamic facial expression recognition using boosted component-based spatiotemporal features and multi-classifi er fusion. In: Advanced Concepts for Intelligent Vision Systems, ACIVS 2010 Proceed-ings, LNCS 6475: 312-322.Huttunen S & Heikkilä J (2010) Multi-object tracking based on soft assignment of detection responses. Proc. Int’l Conf. Com-puter Vision Theory and Applications, 1: 296-301.Janhunen J, Silven O & Juntti M (2010) Programmable processor implementations of K-best list sphere detector for MIMO receiv-er signal processing. Signal Processing 90(1): 313-323.Kellokumpu V, Zhao G & Pietikäinen M (2011) Recognition of human actions using texture descriptors. Machine Vision and Ap-plications, in press.Koskenkorva P, Kannala J & Brandt SS (2010) Quasi-dense wide baseline matching for three views. Proc. Int’l Conf. Pattern Rec-ognition, 806-809.Kämäräinen J, Hadid A & Pietikäinen M (2011) Facial feature representation. In: Handbook of Face Recognition, 2nd ed. (Eds. S.Z. Li & A.K. Jain), Springer-Verlag, in press.Laksameethanasan D & Brandt SS (2010) A Bayesian recon-

struction method with marginalized uncertainty model for cam-era motion in micro-rotation imaging. IEEE Trans. Biomedical Engineering 57(7): 1719-1728.Lei Z, Liao S, Pietikäinen M & Li SZ (2011) Face recognition by exploring information jointly in space, scale and orientation. IEEE Trans. Image Processing 20(1): 247-256.Liao S, Zhao G, Kellokumpu V, Pietikäinen M & Li SZ (2010) Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 6.Liu Y & Heikkilä J (2010) Isotropic granularity-tunable gradi-ents partition (IGGP) descriptors for human detection. Proc. Brit-ish Machine Vision Conf., 63.1-11.Liu Y, Shan S, Chen X, Heikkilä J, Gao W & Pietikäinen M (2010) Spatial-temporal granularity-tunable gradients partition (STGGP) descriptors for human detection. In: Computer Vision, ECCV 2010 Proceedings, LNCS 6311: 327-340.Martinkauppi B, Hadid A & Pietikäinen M (2011) Skin color in face analysis. In: Handbook of Face Recognition, 2nd ed. (Eds. S.Z. Li & A.K. Jain), Springer-Verlag, in press.Nishiyama M, Hadid A, Takeshima H, Shotton J, Kozakaya T & Yamaguchi O (2011) Facial deblur inference using subspace analysis for recognition of blurred faces. IEEE Trans. Pattern Analysis and Machine Intelligence 33(4): 838-845.Pedone M, Nikkanen J, Lepistö L, Kaikumaa T & Heikkilä J (2010) Evaluation of denoising methods with RAW images and perceptual measures. Proc. Int’l Conf. Computer Vision Theory and Applications, 168-173.Pietikäinen M (2010) Local binary patterns. Scholarpedia, 5(3): 9775.Rahtu E, Kannala J, Salo M & Heikkilä J (2010) Segmenting salient objects from images and videos. In: Computer Vision, ECCV 2010 Proceedings, LNCS 6315: 366-379.Rintaluoma T & Silven O (2010) SIMD performance in software based mobile video coding Proc. Int’l Conf. Embedded Com-puter Systems, 79-85.Starck T, Remes J, Nikkinen J, Tervonen O & Kiviniemi V (2011) Correction of low-frequency physiological noise from the resting state BOLD fMRI-Effect on ICA default mode analysis at 1.5T. J. Neuroscience Methods, November (epub).Takala V, Cai Y & Pietikäinen M (2010) Boosting clusters of samples for sequence matching in camera networks. Proc. Int’l Conf. Pattern Recognition, 400-403.Wang L, Zhao G, Rajpoot N & Nixon M (2010) Editorial: Spe-cial Issue on New Advances in Video-Based Gait Analysis and Applications: Challenges and Solutions. IEEE Trans. Systems, Man, and Cybernetics-Part B: Cybernetics 40(4): 982-985.Wang L, Zhao G, Cheng L & Pietikäinen M (eds.) (2011) Ma-chine Learning for Vision-Based Motion Analysis: Theory and Techniques. Springer-Verlag London.Xie S, Shan S, Chen X & Chen J (2010) Fusing local patterns of Gabor magnitude and phase for face recognition. IEEE Trans. Image Processing 19(5): 1349-1361.Zhao G, Huang X, Gizatdinova Y & Pietikäinen M (2010) Com-bining dynamic texture and structural features for speaker iden-tifi cation. ACM Multimedia 2010 Workshop on Multimedia in Forensics, Security and Intelligence, 93-98.Zhou Z, Zhao G & Pietikäinen M (2010) Synthesizing a talking mouth. Proc. Indian Conf. Computer Vision, Graphics and Image Processing, 216-223.Zhou Z, Zhao G & Pietikäinen M (2011) Towards a practical lip-reading system Proc. IEEE Conf. Computer Vision and Pattern Recognition, 137-144.


MEDIATEAM OULU

Professor Timo Ojala and Professor Mika YlianttilaComputer Science and Engineering Laboratory,

Department of Electrical and Information Engineering, University of Oulutimo.ojala(at)ee.oulu.fi , mika.ylianttila(at)ee.oulu.fi

http://www.mediateam.oulu.fi

Background and MissionMediaTeam Oulu (MediaTeam), founded in 1997, is a re-search group of about 35 people at the Computer Science and Engineering Laboratory of the Department of Electri-cal and Information Engineering at the University of Oulu. MediaTeam conducts leading edge research with the ob-jective of making a visible and lasting impact on society. This involves technology-led research in selected areas, unique deployment of a pervasive computing infrastruc-ture at downtown Oulu and development of “proof of con-cept” prototypes that are subjected to empirical evaluation in the hands of real users in authentic setting.

The visible results of our past and ongoing research in-clude, for example, the panOULU WLAN network and the UBI-hotspots at downtown Oulu.

MediaTeam is highly multidisciplinary and international in its activities, collaborating with domestic and inter-national research partners from different disciplines and contributing to the international academic community. For example, Professor Ojala serves as the chair of the steer-ing committee of the international conference on Mobile and Ubiquitous Multimedia (MUM) and co-chaired the UBI Challenge Workshop at Ubicomp 2010 in Copenha-gen, Denmark. MediaTeam was awarded its fi rst FiDiPro (Finnish Distinguished Professor) post in March 2010, when Professor Vassilis Kostakos from the University of Madeira was appointed as a FiDiPro Fellow. Further, Pro-fessor Henning Schulzrinne from Columbia University, one of the leading internet researchers in the world, kicked off his sabbatical with a three-month research visit to our research group in summer 2010.

Scientifi c ProgressMediaTeam’s main research areas in 2010 were urban computing, multimedia and pervasive service computing, P2P networks and wireless sensor networks.

Urban ComputingUrban computing is an emerging interdisciplinary research fi eld which considers public spaces as potential sites for the development of ubiquitous computing. Urban comput-ing is driven by two important and related trends, urban-ization and rapid deployment of computing infrastructure in urban spaces. However, while urban spaces offer the greatest opportunities and strongest demands for ubiqui-tous computing, there is no fundamental theory, knowl-edge base, principled methods nor tools for designing and building ubiquitous systems as integral elements of the urban landscape.

The UBI (UrBan Interactions) research program, compris-ing of multiple projects (UbiCity, UbiGo, RealUbi, UBI Anthropos, Urban Flows and Networks) funded by the Finnish Funding Agency for Technology and Innovation (Tekes), the ERDF, the Academy of Finland, the City of Oulu and industry, continued in 2010. The new three-year Urban Flows and Networks project starting in January 2010 is funded by Tekes as part of the Finnish Distinguished Professor (FiDiPro) program, inviting Professor Vassilis Kostakos from the University of Madeira to Oulu as a Fi-DiPro Fellow. In 2010 the multidisciplinary research con-sortium coordinated by MediaTeam comprised of groups from fi ve different universities in Finland, Japan and Por-tugal, not just computer scientists and engineers, but also architects, cultural anthropologists, artists and designers. The Finnish members were the Aalto University School of Art and Design (Professor Lily Diaz), the Oulu University of Applied Sciences School of Engineering (project man-ager Veijo Korhonen), and from the University of Oulu the Intelligent Systems Group (Professor Jukka Riekki), the Department of Architecture (Dr. Aulikki Herneoja) and the Cultural Anthropology (Professor Taina Kinnunen). The international members were the Ubiquitous Networking Laboratory from the Tokyo Denki University (Professor Yoshito Tobe) and the Madeira Interactive Technologies Institute at the University of Madeira (Professor and FiDi-Pro Fellow Vassilis Kostakos).

The objective of the UBI program is to introduce a visible change in society by building a functional prototype of an open ubiquitous city in Oulu. From the user community’s point of view, such a city appears as a smart urban space providing rich interaction between the physical, virtual and social spaces. From the R&D community’s point of view, the city appears as an open horizontal testbed stimu-lating innovation, research and development of new ser-vices and applications. The testbed enables urban comput-ing research in an authentic urban setting with real users and with suffi cient scale and time span. Such studies are important because real world systems are culturally situ-ated, and cannot be reliably evaluated with lab studies de-tached from the real world context. By deploying a system for a suffi ciently long time “in the wild”, we can establish the technical and cultural readiness and the critical mass of real users, both of which are needed for determining whether the system can be deemed ‘(un)successful’.

To realize the open ubiquitous city, the UBI program en-gages an iterative cycle of deploying a new open pervasive computing infrastructure at downtown Oulu, technology-led research and application-led research. The technology-led research produces new technology and knowledge, which together with the new computing infrastructure


creates fresh opportunities for application-led research. It produces novel proof of concept prototypes, which are empirically evaluated with real users in authentic urban settings. This provides valuable feedback and creates new requirements on the technology-led research and the com-puting infrastructure.

In 2010, the key activities in the infrastructure deploy-ment included the installation of the panOULU WSN network, the expansion of the panOULU BT network, the installation of the UBI-projectors, and the maintenance of the UBI-displays and the core services of the panOULU WLAN.

The panOULU WSN is an IP-based wireless sensor net-work comprising of 13 edge routers (Sensinode Nano-Router Ethernet 2.0 Reference Platform) across the city (Figure 1a). The ERs are equipped with an IEEE 802.15.4 radio on the 868 MHz band and the 6LoWPAN protocol stack. 12 ERs are installed inside the panOULU WLAN mesh APs (Figure 1b-c), so that the mesh AP provides en-closure, power and IPv6 connectivity.

It should be noted that a mesh AP having suffi cient free space for housing a small ER was not planned, but was just a lucky coincidence. One ER is placed inside an UBI-hotspot. An ER has about 500 m line-of-sight range with 1 mW transmission power. The fi rst two use cases for the panOULU WSN are automated power metering (see the UBI-AMI pilot) and environmental monitoring using low-power sensors.

Figure 1. (a) Locations of panOULU WSN ERs around downtown Oulu; (b) panOULU WSN ER and its placement inside panOULU WLAN mesh AP; (c) panOULU APs in-stalled at traffi c lights.

The panOULU BT is a network of Bluetooth APs (each equipped with three BT radios) scattered across the city center. In 2010, 18 APs were installed at traffi c lights (Fig-ure 1c) and they use the panOULU WLAN for Internet access. An additional 12 AP’s are placed inside the UBI-hotspots. All APs sniff bypassing BT radios and the real-time traces are used for modeling pedestrian and vehicular fl ows and networks. Further, the 12 APs inside the UBI-hotspots are used for distributing multimedia content to mobile devices over no-cost BT connectivity.

Two high performance UBI-projectors were installed at the City Theatre in October 2010 (Figure 2). Both units com-prise of a Sanyo PDG-DET100L projector (7500 ANSI lu-mens) placed inside a weather proof Tempest Blizzard en-closure, a MacMini Server, an amplifi er, loudspeakers and an IP camera for providing a video feed of the projections for monitoring in the web. Further, a mobile projector unit comprising of a Viewsonic Pro 8500 DLP projector (5000 ANSI lumens), a Macbook Pro laptop and an aggregator has been acquired.

Figure 2. UBI-projectors at the City Theatre.

The heterogeneous computing resources of our ubiqui-tous city constitute a large distributed system which is ar-ranged and virtualized by the UBI-middleware. In 2010, our middleware related activities focused on distributed resource management, and creating open and standardized API’s for the computing resources. We developed a web-based framework for managing the screen real estate of interactive public displays using declarative XML based descriptions of virtual screens, and a fi nite state machine expressed with the State Chart Extensive Markup Lan-guage (SCXML) and implemented with the Apache Com-mons SCXML engine. The framework includes an algo-rithm for transforming the declarative virtual screens into a sequence of steps required for transiting between layout states. The framework separates the layout description from application logic which allows rapid confi guration of the interaction model upon application deployment. The upcoming UBI Open Urban Service Network API com-prises of a set of open API’s which will allow the gen-eral public, the open source community and the SME’s to develop their own applications and services atop the UBI infrastructure.

The application-led research is driven by the domain-spe-cifi c problems of urban computing, which the UBI pro-

(a)

(c)(b)


gram addresses by developing two types of “proof of con-cept” prototypes, demos and pilots. A demo is a short-lived small-scale evaluation of new technology, service concept or user interface paradigm. It is often implemented using rapid application development, and empirically evaluated in the form of a small user test either in a laboratory or real-life setting. Successful functionalities from the demos are integrated into pilots, which are longitudinal larger-scale prototypes of the future ubiquitous city evaluated by real users in an authentic urban setting. The evaluation quantifi es the benefi ts brought to the chosen application domain, and generates feedback and requirements on the technology-led research and the computing infrastructure to be taken into account in the next iterative design cycle.

In 2010, the UBI program produced fi ve research demos (UBI Ghostbusters, Ubinion, UBI-hotspot Video Guide, UFN Trails, UFN Heatmap) and two industrial demos by the SMEs supported by the UbiGo project. UBI Ghost-busters is a distributed pervasive game, where the objec-tive is to hunt, with a mobile phone, ghosts appearing on UBI-hotspots (Figure 3a). Ubinion allows (young) UBI-hotspot users to playfully express their opinions, includ-ing automatic dissemination via Facebook and Twitter. UBI-hotspot Video Guide allows UBI-hotspot users to make video calls, which are managed and streamed us-ing a RED5 Flash streaming server, and support real-time collaborative editing using Google Wave. UFN Trails es-tablishes geospatial trails of particular devices from the wireless traces collected by the panOULU WLAN and pa-nOULU BT networks. UFN Heatmap builds “heat maps” from the wireless traces to illustrate relative levels of ac-tivity around the city (Figure 3b).

Figure 3. (a) Test user playing UBI Ghostbusters at down-town Oulu; (b) Illustration of UFN Heatmap.

In 2010, the UBI program conducted two pilots, UBI Pilot 2010 and UBI-AMI Pilot 2010 (reported in conjunction with wireless sensor networks). The UBI Pilot 2010 corre-sponded to the longitudinal evaluation of the UBI-hotspot version 2.0 (Figure 4), which was released at the 2nd Open Ubiquitous City Seminar on May 31, 2010. The new ver-sion of the hotspot included a revised interaction model in-cluding a subtle interaction phase, a new login mechanism based on Bluetooth ID and PIN, personalization of the hot-spot in terms of visual appearance and Facebook integra-tion, and a new ”widget-based” service wizard and menu. The UBI-portal 2.0 contained a number of new services: a service directory, City of Oulu’s page, public transport, fast food in Oulu, UBI Mosquitos game, 65 Degrees North news, and the fi rst commercial third party services. We ar-ranged in June-August 2010 weekly “UBI walks” that the general public could join to learn to use the UBI-hotspots. Research data was collected with various methods, includ-ing logging of quantitative usage, questionnaires, obser-vations and interviews with hotspot users, and an online survey to Oulu businesses.

Figure 4. The UBI-hotspot version 2.0.

A common criticism targeted at many studies on interac-tive public displays is that their evaluation usually takes place in artifi cial lab environments and for short periods of time. Our hotspots represent the largest longitudinal study on the usage of interactive public displays by real users in authentic urban setting to date. It has revealed a num-ber of factors that lab studies do not necessarily take into account: user sampling, location, curiosity, novelty, and weather. These substantial differences have led us to ques-tion the validity of lab studies in this domain. However, the external validity of our research comes with a substantial cost and a number of challenges. The greatest challenge in our deployment has been maintenance, as a substantial amount of our resources has been used to make sure that the system behaves as expected and to troubleshoot faults. Another challenge is the economic viability, ensuring that the hotspots become self-suffi cient. To cover the opera-tional expenses of the hotspots, we have generated reve-nue from commercial use, earning about EUR 105000 in 2010. However, the commercial usage has confl icted with

(a)

(b)


research objectives. A given capacity sold for example to digital signage has to be delivered, which constrains the screen layout and the interaction model. Thirdly, the disci-pline as a whole lacks standardized metrics for evaluating the success of such systems. In other words, despite the availability of huge volumes of quantitative and qualitative data on the usage of the hotspots, it is challenging to mea-sure their actual impact on the community. Therefore, we applied for and were granted funding by the Academy of Finland on a new multidisciplinary four-year project titled “UBI Metrics - Multidisciplinary Framework for Evaluat-ing Ubicomp Systems in Real-World Urban Settings”.

The dissemination of the results of the UBI program took place in different forms. We organized several interna-tional events and challenges discussed below. In 2010, we published two journal articles, one book chapter and 12 conference papers, one of which received the Best Pre-sentation Award in ACM HotMobile 2010. The fi rst UBI doctorate was born in May 2010 when Jürgen Scheible from the School of Art and Design of the Aalto Univer-sity successfully defended his dissertation “Empowering Mobile Art Practice: A Recontextualization of Mobile and Ubiquitous Computing”. Four M.Sc. theses and two B.Sc. theses were completed in 2010. We gave presentations at 19 international events and three national events and ap-peared 36 times in popular media in 2010.

Multimedia and Pervasive Service Comput-ingThe research on multimedia and pervasive service com-puting covers topics such as content processing, descrip-tion, composition, and delivery techniques; service archi-tectures; service provisioning and context-aware services. These research themes have emerged to be highly essential for enabling large-scale and interconnected multimedia systems and services, and have attracted industrial collab-oration in several joint research projects. It is necessary for novel internet services to deliver topical and relevant con-tent for end-users, while guaranteeing the quality of the service and experience. This includes also effi cient search techniques for the contemporary multimedia services. De-veloping techniques for processing content descriptive data (metadata) elements enables scalable and interoper-able service development and deployment.

The Collaborative aggregated multimedia for digital home (CAM4Home) project was a pan-European industrial proj-ect in the ITEA 2 framework program in 2007-2010. The objective of the CAM4Home project was to create a meta-data-enabled content delivery framework to allow end us-ers and commercial content providers to create and deliver rich multimedia experiences for networked communities. These multimedia experiences were based on a novel con-cept of collaborative aggregated multimedia that was run on the CAM4Home service platform and metadata frame-work. In the project, researchers in MediaTeam designed and developed a content annotation service that creates semantic CAM objects from multimedia fi les utilizing computational content analysis techniques. MediaTeam also developed a web-based end user service that allows users to upload multimedia fi les and produce semantic

CAM objects to the interconnected platform services and applications for various multimedia delivery, distribu-tion and consumption tasks (Figure 5). MediaTeam was also involved in the design and development of the CAM metamodel and metadata framework that enabled interop-erability between platform services to support diverse scenarios for multimedia creation, delivery, distribution, modifi cation and consumption. In the project, MediaTeam published 14 conference papers, one magazine article and three book chapters, had one research visit, and organized three international workshops. The CAM4Home project was coordinated internationally by Thomson Video Net-works, and the Finnish partner consortium was led by VTT. The CAM4Home received the silver ITEA Achieve-ment Award 2010 at the ITEA2/ARTEMIS Co-Summit 2010 event. The project was placed second in the compari-son of the ITEA2/ARTEMIS projects.

Figure 5. Web application for content annotation that de-tects person sequences from the uploaded media.

The Adaptive Content Delivery Clusters (ACDC) project is a pan-European industrial project in the ITEA 2 frame-work program that started in 2010. The goal of the ACDC project is to research, develop and demonstrate an adaptive content delivery cluster, intelligent user-aware applica-tions and new business models. The project aims at novel user-aware multimedia and entertainment TV applications (IPTV, web/internet TV and mobile TV), on demand video and entertainment, personal video recording and targeted advertising services on a variety of networks and termi-


nals (set-top-boxes, PC and mobile) with adaptation, per-sonalization and anticipation features based on semantic knowledge. As a technological enabler for user-aware multimedia and entertainment applications, the project will research, develop and demonstrate a distributed com-puting infrastructure for large scale digital content and se-mantic knowledge processing, storage and delivery. Over the distributed computing infrastructure, the project will focus on software tools and a service platform to provide a development framework for user-aware applications. MediaTeam contributes by researching and developing semantic knowledge processing technologies to support business and application scenarios for user-aware applica-tions. During the fi rst year of the project, MediaTeam has produced two M.Sc. theses and four conference publica-tions. The ACDC project is coordinated internationally by Thomson Video Networks while the Finnish consortium is led by VTT.

Next Media is a new research program of the ICT SHOK that started in 2010. It aims to meet people’s insatiable need for engaging and activating media experiences by means of new business models, concepts and technology. Production and consumption of media is under radical transformation. Digitalization of production, distribution and consumption of media, as well as the growing pen-etration of broadband access and mobile internet allow in-creasingly rich media content to be distributed to a variety of terminals. This endeavor takes advantage of the current transition towards co-creation, interactivity and indepen-dence of time and place. The diversity of terminals and devices in consumers’ everyday life is increasing, and the signifi cance of mobility is emphasized. The company led program consists of four work packages whose research themes cover all aspects of media production, from media content access to working processes and business models, as well as user experience. All major Finnish research orga-nizations and universities are participating in the program. MediaTeam participates in the Multichannel Multimarket Media Services work package, exploring disruptive tech-nologies for modern media services that need scalability across multiple markets.

The Pervasive Service Computing (PSC) project funded by the Academy of Finland continued in 2010. The PSC project is a collaboration project between the Intelligent Systems Group and MediaTeam of the University of Oulu and the Embedded and Pervasive Computer Center of the Shanghai Jiao Tong University in China. The project’s key research hypothesis is that the time a user spends access-ing and controlling particular IT based services is the most important quantitative parameter in measuring how easy to use these services are. The objective is to establish a so-called PSC model that shortens this time and allows the users to focus on their everyday activities either alone or in a community. The project defi nes PSC scenarios to de-velop the PSC model and to design, execute and test proto-types. The project suggests a new service-centric solution, which enables encapsulating computational resources into web services and building versatile web service sets that support the users in achieving their goals. The general goal of the research is to produce knowledge and solutions

that enable people to focus on living their lives. In other words, it will no longer be necessary for people to search from their immediate surroundings and information net-works for services that support their activities, to guide the services they fi nd and to combine the services into the functionalities they require. Instead, by applying the PSC model, service-centric solutions supporting peoples’ ac-tivities without distracting them too much can be realized. In 2010, we have focused on building the theoretical con-cepts and verifying them with simulations and prototypes. Two project workshops have been organized in China to-gether with the Shanghai Jiao Tong University, who will visit us in Oulu in May 2011.

The Digital Watermarking of Speech and Holograms project, funded by the Academy of Finland, concluded in 2010. The project focused on digital rights management technologies, especially digital watermarking of speech signals and holographic data. Holograms are widely used for protecting products or trademarks against forgery. In most cases, the authenticity check has to be done visually, or with some specialized equipment such as laser readers. Digital watermarking is a method of embedding a secret sequence of bits in the host media such that it is hard to perceive or remove. This signature, encoding the owner-ship and/or distributor id, can be extracted from the host signal when a proper secret key and the necessary algo-rithms are available. The essential technical challenges in watermarking include invisibility, robustness and capacity. A wide range of algorithms have been proposed for wa-termarking, especially for watermarking of digital images. Generally, however, the resilience of this information in a physical printout, like binary images, holograms or color images has been less studied.

We have been successful in developing robust methods that make possible the extraction of an embedded mes-sage from a print-out of a digital image with a scanner and camera phone, and from binary images with a camera phone. In these, especially in the DA/AD transform, dis-tortions caused by user interaction and air interface have been considered. When a watermark is read from a holo-gram, the reading process should be robust against lighting and refl ections as the hologram surface is very refl ective. The hologram also differs from printed images in that the colors depend on the viewing angle and vary across the hologram. Figure 6 illustrates the problem of watermark-ing holograms. In 2010, we have developed a method for reading a watermark from a hologram and the detailed results will be published in early 2011. In the future, we hope to deepen the understanding of especially mobile phone readable watermarks in printed images and holo-grams as well as integrate technologies with augmented reality applications.

Figure 6. Watermarking of a hologram.


Further, the project developed novel methods of classi-fying emotional speech samples on the basis of prosodic features. Classifi cation of the emotional content of short Finnish emotional [a:] vowel speech samples was per-formed with a new method using prosodic features derived from vocal source and traditional intonation contour pa-rameters. A multiple kNN classifi er based decision level fusion classifi cation architecture was proposed for multi-modal speech prosody and vocal source expert fusion. The sum fusion rule and the sequential forward fl oating search (SFFS) algorithm were used to produce leveraged expert classifi ers. Automatic discrimination tests in fi ve emotion-al classes demonstrated that signifi cantly higher than ran-dom level emotional content classifi cation performance is achievable using both prosodic and vocal source features. The fusion classifi cation approach is further shown to be capable of emotional content discrimination in the vowel domain approaching the performance level of the human reference.

In 2010, as a result of ten years of research on content-based video retrieval, MediaTeam released Kuukkeli-TV (http://kuukkelitv.virtues.fi ), a novel content-based search engine for indexing and retrieving television programs (Figure 7). Kuukkeli-TV allows users to search TV pro-grams broadcasted in the Finnish DVB networks based on Finnish closed captions. The service matches query terms against metadata that has been extracted from the DVB stream, and returns a list of matching programs. The goal is to obtain knowledge of the impact and usage of novel content-based technology for accessing rich multimedia data. The users of the openly available service are encour-aged to give feedback about the service in a short question-naire and give votes on the quality of the hits. The service is unique so that it allows users to view matching content within the television program through multimedia quotes and allows them to search for a particular program’s web stream from the internet. The service is the fi rst of its kind in Finland, and one of the fi rst publicly available services in the world that utilizes multimedia content data effec-tively in locating and browsing timely video broadcasts.

P2P NetworksWe have been witnessing a turning point in networking, so that peer-to-peer (P2P) based systems have challenged the traditional client/server (C/S) based systems. In the past ten years, P2P networks have rapidly evolved from the early unstructured systems towards structured systems, where the nodes are organized in a controlled manner, ac-cording to certain rules and topologies, such as distributed hash tables (DHT). P2P networks have also expanded from the traditional content sharing to a broader range of application areas, such as interpersonal communication (e.g. Skype) and social content distribution (e.g. Spotify), to mention but a few. At the same time, the mobile internet continues its growth due to the emergence of more capable mobile devices and wireless access technologies. P2P net-working is thus becoming an interesting option for mobile devices, as well. However, mobile devices and networks constitute a challenging environment for P2P systems.

The key project in this research area was the Decentralized Inter-Service Communications (DECICOM) project that was funded by Tekes and industry was concluded in 2010. DECICOM was a so-called parallel project of two univer-sities (University of Oulu and Aalto University School of Science and Technology) and three companies (Ericsson, Exfo NetHawk and Nokia). In addition, the consortium included two SME companies (Futurice and Icecom) as supporting members participating in the meetings of busi-ness and technology expert groups. The primary focus in our work was on the utilization of P2P technologies for communication purposes, referred to as decentralized in-terservice and interpersonal communications. The charac-teristics of interservice and interpersonal communications differ from the traditional resource sharing applications, leading to different kinds of traffi c patterns and usage of resources. To enable experimental evaluations, we devel-oped a prototype architecture consisting of both real-life and simulated protocol prototypes, as well as experimental applications on top of them. The prototype architecture is based on the forthcoming Peer-to-Peer SIP (P2PSIP) stan-dard of the Internet Engineering Task Force (IETF).

Figure 7. Kuukkeli-TV allows users to search for television programs with textual queries of Finnish closed captions, and view excerpts of the matching programs.


Using the prototype architecture, we gathered a compre-hensive set of results concerning the performance, robust-ness and effi ciency of P2P systems in different communi-cation scenarios, in both mobile and fi xed environments. We also evaluated the user experience, and linked the tech-nical results with business analysis to establish guidelines for developing better solutions for P2P-based interservice and interpersonal communications. To summarize, we pointed out that further optimization for the current and emerging P2P communication technologies are needed on the protocol and algorithm levels, especially when they are utilized in mobile systems. We showed that runtime opti-mization of the P2P overlay is needed to ensure a high lev-el of robustness, performance and effi ciency of the system. Additional load balancing mechanisms are also necessary components in structured P2P systems due to the inabil-ity of the existing P2P algorithms to equally balance the load generated by the network management and the pay-load traffi c. In systems including mobile nodes, fair load balancing requires advanced load monitoring capabilities that take into account the energy status of the participating nodes. Further, if a P2P network includes group manage-ment functionalities, organizing the overlay hierarchically according to the group topologies is the preferred solution for improving the performance, effi ciency and security of the communication. Finally, structured P2P networks require advanced query mechanisms for providing a key-word search and other types of complex queries. By fol-lowing these guidelines, the performance, robustness and effi ciency of P2P-based interpersonal communication sys-tems can be signifi cantly improved.

In 2010, our research produced Zhongong Ou’s doctoral dissertation “Structured peer-to-peer networks: Hierarchi-cal architecture and performance evaluation”, 4 journal articles and 2 conference articles. In addition, the DECI-COM project contributed to three doctoral dissertations at the Aalto University, as well as several M.Sc. and B.Sc. theses both at the University of Oulu and at Aalto Uni-versity. International collaboration included two research visits to Columbia University in the USA. An important activity was standardization in the IETF, where the DECI-COM project partners published several RFCs and internet drafts related to the project. Research continues in the ICT SHOK Future Internet program that MediaTeam will join in 2011 for the third phase of the program.

Wireless Sensor NetworksThe UBI-AMI Pilot 2010 conducted by the UBI (UrBan Interactions) research program focused on the real-time metering of energy consumption at homes using multi-hop IP-based wireless sensor networks. The architecture of the UBI-AMI system comprises of three functional building blocks (Figure 8a). Sensors in multiple 6LoWPAN sensor networks measure the loads and turn on/off devices con-nected to the sensors. Each network has a router collect-ing packets from multiple sensors over a multi-hop mesh topology, i.e. a sensor is able to forward packets of other sensors. The central UBI-AMI server receives packets from the routers, stores the data in a database and creates different representations of the data. The user can browse

the measurement data with a web browser and subscribe to RSS feeds summarizing the data. Further, the user can control the sensors and subscribe to email/SMS alerts based on load measurements.

A socket sensor (Figure 8b) measures the loads of individ-ual devices or groups of devices equipped with pluggable power cords. A mains sensor (Figure 8c) measures the load at the root of the load tree of a building, including all non-pluggable devices such as electric heating, water heaters and lighting. The sensors are furnished with a Radiocrafts RC2301 radio module which is equipped with an IEEE 802.15.4 radio on the 2.4 GHz band. The radio module is fl ashed with a Sensinode NanoStack 2.0 protocol stack, which implements the IETF 6LoWPAN specifi cation for transmitting IPv6 packets over IEEE 802.15.4 devices. Ef-fectively, the protocol stack provides the sensors with half-duplex multi-hop IPv6 connectivity. The sensors transmit measurement data every 10 seconds and receive possible turn on/off commands in a response message. We built the WSN router on an affordable Linksys WRT54GS WLAN router by adding a Radiocrafts RC2301 radio module to a free serial port connected to an external antenna con-nector. The Linksys router was fl ashed with customized Kamikaze OpenWRT Linux fi rmware capable of routing packets between the 6LoWPAN network and the inter-net. The UBI-AMI server was realized on top of the open source GSN (Global Sensor Network) middleware in Java. The UBI-AMI user interface (UI) was implemented as a website on top of the open source GlassFish application server as Java Server Pages, providing various views such as power and energy consumption. The user can also con-fi gure alarms received either via email or SMS to a mobile phone, and turn on/off individual socket sensors and the attached appliances.

Figure 8. (a) UBI-AMI architecture; (b) Socket sensor; (c) Mains sensor.

(a)

(b) (c)


The UBI-AMI system was assessed by longitudinal user evaluation in seven households in the Oulu region in 2010. While the households used the UBI-AMI system for 11 months, we only used the quantitative data collected in August- December 2010. By omitting the fi rst six months we wished to eliminate any bias contributed by the curi-osity and novelty value of a new system introduced in a household. Qualitative data was collected through ques-tionnaires and interviews with the test users. The results confi rm that visibility of both the aggregate load and ap-pliance level loads in a home is very useful functionality that has an impact on how people use electric appliances. The UBI-AMI system shows that IP-based wireless sensor networks and the web provide a very potent technology platform for implementing such functionality in a cost-ef-fective, reliable and standardized manner, in contrast to the many proprietary solutions on the market.

The Academy Fellow project Cross-layer Optimization of Wireless Sensor Networks led by Dr. Junzhao Sun has in-vestigated high-level cross-layer optimization techniques for wireless sensor networks in a top-down fashion in order to bridge the gap between the technologies for modeling, processing and networking. Modeling techniques for ap-plications, data, systems, and the environment have been studied. Processing techniques for query optimization and multi-sensor fusion have been investigated. The research has also focused on the networking mechanisms for self organization and routing of sensor data. In 2010, the re-search has focused on SQL query language extensions for data collection and information retrieval in WSNs. To que-ry the sensor database, four extensions to traditional SQL query language have been proposed, including timing, aggregations, QoS, and semantics. In each extension, the unique features of a database for wireless sensor networks have been taken into consideration. The proposals are gen-eral, and thus enable a database view with query based interaction applicable to most sensor network applications and tasks. Re-programmability refers to the capacity of be-ing able to deploy applications dynamically to WSNs that have been deployed in the fi eld, without the need for man-ual intervention. This capacity brings great fl exibility and adaptability to WSNs, and thus has attracted increasing research interest. In 2010, the research produced a survey on existing OS-based approaches and code dissemination protocols for re-programming WSNs, and identifi ed re-lated research issues and trends. In the emerging pervasive social computing paradigm, an interesting topic is how to blog with automatically association of user behavior. In 2010, we developed Mlogger, an automatic blogging sys-tem that can detect, recognize and track user behavior for automatic association with new blog entries. In the system, Sun SPOTs were used for sensing raw behavioral data. The Mlogger processes the raw data and infers high-level user behavioral information such as “what the user is doing, and where, when, and with whom”. Associated with the inferred information, a new entry about user behaviors can be created and published automatically.

Events in 2010MediaTeam organized and contributed to a number of events in 2010.

We organized the 2nd Open Ubiquitous City Seminar held in Oulu, Finland on May 31. The high profi le seminar fea-tured many distinguished international experts address-ing various aspects of ubiquitous and urban computing. The seminar was capped with a panel, which gave the 140 strong audience an ample opportunity to interact with the speakers. (http://www.ubioulu.fi /en/2nd-Open-Ubiquitous-City-Seminar)

We organized the 1st International UBI Summer School 2010 held in Oulu, Finland, on May 31 - June 4. It com-prised of six parallel workshops instructed by leading in-ternational experts on ubiquitous and urban computing: “Real World Context-Aware Systems” (Professor Anind Dey, Carnegie Mellon University, USA), “Urban Infor-matics and Sustainable Cities” (Professor Marcus Foth and Dr. Jaz Choi, Queensland University of Technology, Australia; Patrick Hofmann, Google, Australia), “Urban Social Networks Analysis” (Professor Vassilis Kostakos, University of Madeira, Portugal), “Creating and Sharing Artistic Experiences with Ubiquitous Technology” (Dr. Jürgen Scheible, Aalto University, Finland), “IP-based Wireless Sensor Networks” (Chief Nerd Zach Shelby,

Figure 9. (a) Instructors of the 1st international UBI Sum-mer School 2010 posing with their souvenir ‘kuksas’.(b) Recipients of the best poster awards with Professor Timo Ojala.

(a)

(b)


Sensinode, Finland) and “Interactive Textures – rethink-ing materiality” (Professor Mikael Wiberg, Umeå Univer-sity, Sweden). The summer school had enrollments by 72 students from 20 countries via an open international call. All students attended a number of joint events, including a poster session where students presented their background and ongoing research, an opening plenary where each workshop was introduced and the closing plenary where each workshop presented their results. Each workshop had its own curriculum and activities, which included theoreti-cal presentations by the instructor and practical projects conducted in groups of 3-5 students. The summer school also included Adam Greenfi eld’s walkshop in downtown Oulu. In addition to researcher training, the summer school also promoted the upcoming UBI Challenge. (http://www.ubioulu.fi /en/UBI-summer-school-2010)

We organized the inaugural UBI Oulu Forum in Oulu on August 30. The forum invites researchers, practitioners and the general public to get together for engaging discus-sions on the broad and multidisciplinary fi elds of ubiqui-tous and pervasive computing. The stage is set by stimu-lating and maybe even provocative position statements by senior scholars, who anchor the ensuing discussion by serving as a panel for Q&A with the participants. Desig-nated scribes record the discussion and present an immedi-ate summary at the end. The theme of the fi rst forum was real world relevance and the speakers were Professor Hen-ning Schulzrinne from the Columbia University, Professor Vassilis Kostakos from the University of Madeira and Pro-fessor Timo Ojala from the University of Oulu. There were about 30 people in the audience, mostly junior academics. (http://www.ubioulu.fi /en/UBI-Oulu-Forum)

We organized the UBI Challenge Workshop 2010 (UCW 2010) at Ubicomp 2010, in Copenhagen, Denmark, on September 26. The workshop promoted the upcoming UBI Challenge and solicited original contributions within the broad scope of real world urban computing. The full-day workshop had two sessions. The morning session comprised of presentations of the four accepted papers, an overview of the recent activities in the open ubiqui-tous Oulu and an introduction of the UBI Challenge. The presentations provided the stimulus for the afternoon ses-sion, which focused on prototyping of new urban comput-ing applications and services using different prototyping methods and starting assumptions. The workshop had 11 participants. (http://www.ubioulu.fi /en/UBI-Challenge-Workshop-2010)

We have organized two challenges to make our open urban computing testbed available to the greater R&D commu-nity and to stimulate the innovation of new services.

The national UBI Challenge 2010 (UBI-haaste 2010) was opened in February 2010, challenging both individuals and organizations to innovate and implement new services to UBI-hotspots. A EUR 4000 grant was awarded to each proposal selected for implementation by a local expert jury. Further, the service deemed most successful among those deployed on the UBI-hotspots would receive an ad-ditional grant of EUR 2000 and a high end smart phone.

Participation was stimulated by a raffl e of a high end smart phone between all entries. By the submission deadline in May 2010, we received nine valid proposals, from which the jury selected three for implementation: Battleship (game proposed by two local exchange students), Diversus Oulu (interactive multimedia art piece proposed by a local freelancer artist couple) and UBI Mixer (interactive mu-sic mixing application proposed by an SME). Eventually, Battleship and Diversus Oulu were successfully deployed on the UBI-hotspots. (http://www.ubioulu.fi /UBI-haaste)

The 1st International Open Ubiquitous City Challenge 2010-2011 (”UBI Challenge” for short) was prepared to-gether with a number of leading international experts on ubiquitous and urban computing. It challenges the global R&D community to design, implement, deploy end evalu-ate novel applications and services in a real-world set-ting in the City of Oulu, Finland. The motivation of the international challenge is to stimulate global research col-laboration on urban informatics in a very concrete man-ner, to provide the international R&D community with an opportunity to transfer ideas from labs into real-world urban environment, to make our testbed available to the international R&D community, and to support developing metrics for evaluating urban computing infrastructure and applications in a real-world setting. Participation was en-couraged by advertising that up to fi ve proposals would be invited as fi nalists for deployment in Oulu, receiving a grant of up to EUR 10000 and a full paper in the MUM 2011 proceedings (subject to regular peer review by se-lected members of the challenge jury). The program of the MUM 2011 conference, to be held in Beijing, China, in December 2011, will have a special session dedicated to the challenge, including presentations by the fi nalists and the presentation of the awards to the winner(s). 11 writ-ten proposals were submitted by November 2010 deadline, three from Finland, six from Europe and two from outside Europe. The international jury invited four proposals to the fi nal. All four proposals were submitted by European university teams, who will arrive in Oulu at the beginning of June 2011 to fi nalize the implementation and deploy-ment of the service by the beginning of July. Empirical evidence will be collected in July-August for reporting in September. Further, the jury’s Oulu-based members will meet and assess the fi nalists in-situ. Each fi nalist has been assigned a dedicated liaison researcher to serve as the pri-mary technical contact point. As the fi rst task, the liaison researchers provided the fi nalists with detailed technical, content related and cultural assessments of the propos-als in the light of our own knowledge and experiences of the open ubiquitous Oulu. (http://www.ubioulu.fi /en/UBI-challenge)

We contributed to the organization of the Oulu Open Hack on November 11-12. The 24-hour event solicited hacks in three categories: open, QML and UBI. Two “lightweight” UBI-hotspots were taken to the hack site at the premises of the Center for Internet Excellence. Two hacks were submitted to the UBI category, an adaptation of the Heia-Heia service and an adaptation of the FourSquare service. (http://ouluopenhack.posterous.com)


Personnel



others 14

total 35

person years 27

External Funding

Source EUR



Tekes 610 000

other domestic public 82 000



total 1 425 000

Doctoral ThesesOu Z (2010) Structured peer-to-peer networks: Hierarchical ar-chitecture and performance evaluation. Acta Universitatis Oulu-ensis C 361.

Selected PublicationsBilasco IM , Amir S, Blandin P, Djeraba C, Laitakari J, Martinet J, Gracia EM, Pakkala D, Rautiainen M, Ylianttila M, Zhou J (2010) Semantics for intelligent delivery of multimedia content. Proc. 25th ACM Symposium on Applied Computing, Sierre, Switzerland, 1367-1374.Heikkinen J, Jansson-Verkasalo E, Toivanen J, Suominen K, Väyrynen E, Moilanen I & Seppänen T (2010) Perception of ba-sic emotions from speech prosody in adolescents with Asperger syndrome. Logopedics Phoniatrics Vocology 35(3): 113-120.Heikkinen T, Lindén T, Ojala T, Kukka H, Jurmu M & Hosio S (2010) Lessons learned from the deployment and maintenance of UBI-hotspots. Proc. 4th International Conf. on Multimedia and Ubiquitous Engineering (MUE 2010), Cebu, Philippines, 1-6.Hosio S, Jurmu M, Kukka H, Riekki J & Ojala T (2010) Support-ing distributed private and public user interfaces in urban envi-ronments. Proc. 11th Workshop on Mobile Computing Systems and Applications (HotMobile 2010), Annapolis, MD, USA, 25-30. (Best Presentation Award)Hosio S, Kukka H & Riekki J (2010) Social Surroundings: Bridging the virtual and physical divide. IEEE Multimedia 17(2): 26-33.Kassinen O, Harjula E & Ylianttila M (2010) Analysis of mes-saging load in a P2P overlay network under churn. Proc. 2010 IEEE Global Telecommunications Conference (GLOBECOM 2010), Miami, FL, USA, 1-5.Kassinen O, Harjula E, Koskela T & Ylianttila M (2010) Guide-

lines for the implementation of cross-platform mobile middle-ware. International Journal of Software Engineering and Its Ap-plications 4(3): 43-57.Keskinarkaus A, Pramila A & Seppänen T (2010) Image water-marking with a directed periodic pattern to embed multibit mes-sages resilient to print-scan and compound attacks. Journal of Systems and Software 83(10): 1715-1725.Koskela T, Julkunen J, Kassinen O & Ylianttila M (2010) User evaluation of a community-centric music voting service. Journal of Digital Content Technology and its Application 4(1): 69-78.Koskela T, Järvinen S, Liu M & Ylianttila M (2010) User ex-perience in added value location-based mobile music service. Proc. 8th IEEE International Conference on Web Services (ICWS 2010), Miami, FL, USA, 465-472.Koskela T, Kassinen O, Harjula E, Pellikka J & Ylianttila M (2010) Case study on a community-centric mobile service en-vironment. International Journal of Web Applications 2(3):187-206.Koskela T, Kassinen O, Ou Z & Ylianttila M (2010) Improving community management performance with two-level hierarchi-cal DHT overlays. Journal of Internet Technology 11(2): 167-181.Lindén T, Heikkinen T, Ojala T, Kukka H & Jurmu M (2010) Web-based framework for spatiotemporal screen real estate man-agement of interactive public displays. Proc. 19th Int’l World Wide Web Conf. (WWW 2010), Raleigh, NC, USA, 1277-1280.Ojala T (2010) Case studies on context-aware mobile multime-dia services. Journal on Digital Information Management 8(1): 3-14.Ojala T, Kukka H, Heikkinen T, Lindén T, Jurmu M, Kruger F, Sasin S, Hosio S & Närhi P (2010) Open urban computing test-bed. Proc. 6th International Conference on Testbeds and Research Infrastructures for the Development of Networks & Communi-ties (TridentCom 2010), Berlin, Germany, 1277-1288.Ojala T, Kukka H, Lindén T, Heikkinen T, Jurmu M, Hosio S & Kruger F (2010) UBI-hotspot 1.0: Large-scale long-term deploy-ment of interactive public displays in a city center. Proc. Fifth International Conference on Internet and Web Applications and Services (ICIW 2010), Barcelona, Spain, 285-294.Ojala T, Valkama V, Kukka H, Heikkinen T, Lindén T, Jurmu M, Kruger F & Hosio S (2010) UBI-hotspots: Sustainable ecosys-tem infrastructure for real world urban computing research and business. Proc. 2nd International Conference on Management of Emergent Digital EcoSystems (ACM MEDES 2010), Bangkok, Thailand, 196-202.Ou Z, Harjula E, Kassinen O & Ylianttila M (2010) Performance evaluation of a Kademlia-based communication-oriented P2P system under churn. Computer Networks 54(5): 689-705.Sun J, Zhou J & Pihlajaniemi T (2010) Mlogger: An automatic blogging system by mobile sensing user behaviors. Proc. 7th In-ternational Conference on Ubiquitous Intelligence and Comput-ing (UIC 2010), Xi’an, China, 650-664.Tiensyrjä J, Ojala T, Hakanen T & Salmi O (2010) panOULU Conqueror: Pervasive location-aware multiplayer game for city-wide wireless network. Proc. 3rd International Conference on Fun and Games (FNG 2010), Leuven, Belgium, 157-165.Zhou J, Gilman E, Palola J, Riekki J, Ylianttila M & Sun J (2010) Context-aware pervasive service composition and its implemen-tation. Personal and Ubiquitous computing 15(3): 291-303.Zhou J, Moisanen J, Rautiainen M & Ylianttila M (2010) Design and development of digital movie guide service for approach-ing community coordinated multimedia. Proc. Fifth International Conference on Digital Information Management (ICDIM 2010), Thunder Bay, Canada, 159-162.


WIRELESS COMMUNICATION SYSTEMS (WICS)

Professor Matti Latva-aho and Professor Markku Juntti, Centre for Wireless Communications, University of Oulu matti.latva-aho(at)ee.oulu.fi , markku.juntti(at)ee.oulu.fi

http://www.infotech.oulu.fi /wics

Background and MissionThe Wireless Communication Systems Group (WICS) consists of the Telecommunication Laboratory and the Centre for Wireless Communications (CWC) at the Uni-versity of Oulu. The core competence of WICS is in the overall wireless systems design for commercially impor-tant applications. WICS, with a staff of 120, has an an-nual budget of approximately 6 million euros, consisting to a large extent of external funding received for research projects. The broad variety of its customers include Nokia Siemens Networks, Nokia, the Finnish Funding Agency for Technology and Innovation, the European Commis-sion, the European Defence Agency, the European Space Agency, the Finnish Defence Forces, Nethawk, Xilinx and Elektrobit.

WICS has a two-fold mission. On the one hand, WICS wants to excel in academic performance by producing world class graduates, as well as publications. On the other hand, the group was established to bridge the grand canyon of academic results and industry needs. Thus WICS seeks to help its customers by creating new tech-nology and by giving support to industry in their aspira-tions. WICS’s track record in designing both commercial and military communication systems that are operational today is unique and exceptional at both the national and global levels.

Wireless technologies are envisioned to be mature enough within the next 10–15 years to be among the key enablers in solving some of the essential global challenges, includ-ing: climate change, population growth and emerging economies, ageing and well-being, dependence on ICT, ubiquitous internet and scarcity of energy.

New technical solutions are needed to answer to the afore-mentioned challenges. Amongst the major challenges for communication technologies to be employed in future sys-tems are the following: support for higher data through-puts, improved spectral and energy effi ciency, transparent interoperability on heterogeneous networks, mobile wire-less driven internet, robust and trustworthy communica-tions.

The close relation between the global challenges and wire-less technology challenges is more than evident. The major challenge is to investigate how modern wireless technolo-gies can be applied and developed further to address some of the serious global problems. To solve some of today’s challenges the approach in WICS is cross-layer design and optimization for future inter-connected networks driven by mobile wireless technologies. Because the future internet will be a network of networks effi ciently taking into ac-

count all the above requirements and even more, we will concentrate on a unifi ed framework to optimize, tailor, and harmonize the joint operation of various networks with different requirements and deployment. This research is targeted at developing cross-layer optimized fundamental technologies for future wireless networks and then apply-ing them in various application areas. This fundamental technology and competence are developed based on the constraints posed by the challenges and opportunities. The proposed cross-layer approach for future networks requires the mastering of several fi elds of wireless com-munications: the physical layer, radio channel characteri-zation, the data link and network layers, all the way to the application layer of the open system interconnect (OSI) model and moreover, the overall wireless systems design across all layers.

Another, possibly even greater challenge is to develop ap-plication and system level knowledge to solve optimiza-tion problems for inter-connected systems, such as com-munication & control networks for smart energy grids. This area requires an understanding of electricity genera-tion and distribution, energy markets, control engineering, measurements technologies, communication technologies, data fusion etc. The potential scenarios to be studied in-clude buildings (residential houses, offi ce environment, factories), group of households forming a so-called micro-grid and electric vehicle based approach leading to mo-bile electricity. Each of the scenarios pose a multitude of interesting research problems suitable for a person with a background in wireless communications. For example, the future cellular networks will be based on distributed network management principles which can be readily ap-plied to power grids management. Many of the underlying theoretical tools such as game theory and other optimiza-tion methods can be applied to power grid usage optimiza-tion. As power grids are even more vital for modern so-ciety than communication networks, the requirements for the control and communication networks for power grids are more demanding than mobile communication systems, for example. Novel robust solutions need to be developed. The overall approach to be taken in this project is to in-vestigate inter-systems networking, i.e., how the future internet, telecommunications and energy sectors can be co-designed to result in robust solutions for the needs of customers, businesses and societies in the future.

Future networks will employ new topologies, such as multi-hop wireless networks, mobile ad hoc networks, and the integration of wireline and wireless networks. It fur-ther calls for advanced routing, scheduling, radio resource management, and radio link control solutions capable of using the spectrum effi ciently and adapting to diverse traf-


fi c behaviour. A fundamental problem in designing such complex systems is to derive network control mecha-nisms comprising of fl ow control, routing, scheduling, and physical resource management. The control mechanisms must ensure network stability under a large set of service demands and, simultaneously, provide a certain degree of fairness between concurrent sessions. This necessitates holistic network optimization across the OSI model lay-ers. Network protocols for layered architectures have his-torically been obtained mainly based on an engineering intuition, and many of the recent cross-layer designs are conducted in the same heuristic manner. Despite the recent progress, there is little understanding of how to divide the network protocol into layers in a systematic way. A new approach, based on a rigorous mathematical theory, is re-quired for optimizing the future network architectures and technologies. The research will also address the technolo-gies needed in realization of the network nodes, devices, and the computational paradigms in an energy-saving and cost effi cient manner.

One of the major challenges for mobile wireless internet operation is that all content related to the operating envi-ronment becomes highly dynamic. Under such a setting, server based information acquisition will become very in-effi cient and user co-operation and information exchange will become more attractive, leading to, for example, “publish and subscribe” -style internet architectures. Fur-thermore, due to the time-varying environment, the net-work must be selected and optimized dynamically. Envi-ronment-aware intelligent radio networks are often called cognitive radio networks. In order to be able to develop a functioning cognitive radio network, one must mas-ter radio channel phenomena, physical layer techniques, MAC and link layer techniques, and overall wireless net-work optimization, as well as the intelligence necessary to self-optimize and manage the whole system. This type of cross-layer optimization is one of the key elements in the development of future cognitive radio networks.

As outcomes, the Wireless Communication Systems Group aims at providing:• theoretical breakthroughs for understanding the capacity

of future wireless networks;• alternatives for mobile wireless driven internet architec-

tures;• cross-layer optimized protocols and transmission meth-

ods for inter-systems networks;• methods for intelligent and context aware network man-

agement for various applications;and fi nally, understanding on the co-design of future merg-ing inter-connected systems e.g. internet, wireless net-works and energy grids.

Research GroupsWICS’ mission statement and research strategy have re-sulted in an organisation where research is divided into basic research oriented groups and application oriented groups. Currently we have the following groups (and group level teams):

Communication Signal Processing (CSP) The requirements of wireless systems change steadily. Future systems must offer higher system capacity, higher data rates and smaller latency than current systems to en-able the increasing diversity of services offered by differ-ent service providers. New network topologies and smart ubiquitous environments are needed to satisfy the require-ments of future services. The implementation of these sys-tems requires cross layer designs and optimizations close to realization. Devices operating in future wireless net-works must be reconfi gurable and more fl exible than cur-rent designs. To fulfi l these requirements joint system level and device level, and joint signal processing algorithm and architecture research are needed.

Goal: The Communication Signal Processing Group (CSP) implements the strategy of CWC in the area of sig-nal processing and radio engineering research for wireless systems. The group’s research goals are to allow more ef-fi cient use of network devices, decrease the power/energy consumption of wireless devices and allow the develop-ment of smaller and lighter wireless devices, as well as improve understanding on the fundamental limits of wire-less systems. Application areas of CSP group’s research results are mainly cellular systems, sensor networks and smart wireless devices. Interest groups for the application areas include device manufacturers, operators and semi-conductor vendors.

Focus areas: the group’s focus areas are information theo-retic foundations and their impact on wireless system de-sign, signal processing system and device level algorithms, implementation architectures and effi cient implementa-tions, as well as radio engineering and channel modelling. Currently, future technologies under investigation include multiuser multi-antenna technologies for relay based and mesh networks, sensor network and wide area network in-tegration, effi cient computation platforms, and implemen-tation design processes.

Teams: the CSP group is led by Prof. Markku Juntti, and currently consists of fi ve teams, each concentrating on their specifi c expertise areas:• Algorithms and Architectures for Wireless Systems• Iterative Information Processing• Positioning and Mesh Networks• Radio Engineering• Robust Signal Processing

Radio Access Technologies (RAT) The telecommunications industry is growing fast. The in-crease both in the number of users and data rates poses serious challenges to the energy effi ciency and overall ar-chitecture of wireless networks. At the same time, an ever increasing share of internet access has become wireless, which sets new requirements on the structures and man-agement policies for the future internet. Cellular network topologies are being diversifi ed. A full coverage backbone network can be supplemented with various short range lo-cal connectivity options (e.g., device-to-device links, sen-sor networks, relays, femtocells) to improve spectral plus energy effi ciency and coverage, and to reduce costs. Intel-


ligent context aware network control is needed to man-age these dynamic and heterogeneous environments. A thorough understanding of radio access networks and their impact on other layers and segments of the network is im-perative, and therefore a mission of this research group.

Goal: The Radio Access Technologies Group (RAT) im-plements the strategy of CWC in the broad area of future access network development. The group’s research goal is to gain a fundamental knowledge on how to optimize wireless networks for different settings: • macrocellular, femtocells and relay based topologies• opportunistic spectrum use concepts• self-organizing networks• future internet driven access networks• energy and cost effective networksand to continue to offer support to local industry in Inter-national Mobile Telecommunications - Advanced (IMT-A) (and beyond) and its further development within the group’s expertise areas. The application areas of RAT group’s research results are mainly broadband cellular sys-tems, heterogeneous wireless networks and smart wireless devices. Interest groups for the application areas include network and device manufacturers, operators and semi-conductor vendors

Focus areas: The group’s focus areas are cognitive radio networks, energy effi cient networking technologies, fem-tocell concepts for capacity boosting, local connectivity in mobile cellular systems, fundamental networking research (e.g., cross layer optimization and network coding), mobile wireless driven future internet architecture, the evolution of mobile cellular networks, position assisted network(s) management and experimental networking research using WARP platforms.

Teams: The RAT group is led by Prof. Matti Latva-aho and currently consists of 4 teams each concentrating on their specifi c expertise areas: • Mobile Cellular Systems• Radio Access Networks• Cognitive Radio Systems• Broadband Short-range Access

Internetworking The growing interest in new services with higher quality and increasing usage in the internet is one of the major challenges in the near future. Different networks are be-coming more and more interdependent and joint optimi-zation of such networks should provide more effi cient solutions, resulting in better performance and lower cost of operation. Recent advances in development of low-cost wireless internet coupled sensor platforms also open up opportunities for novel wireless sensor network applica-tions, with increasing security threats that deserve ever increasing attention. Finding solutions to these problems requires an approach that jointly optimizes several net-working core functionalities and interconnects both theo-retical modelling of the system and limitations set by the functional devices in the network.

Goal: The Internetworking Group fulfi ls the mission of CWC in the area of wireless communication network re-

search by developing new methodologies and concepts around the evolving new wireless internet. The research goals of the group involve fi nding new solutions for opti-mizing network utility through layered network architec-ture. The results will be used to defi ne new common air interfaces for 4G cellular together with ad hoc and sensor networks. Application areas of Internetworking Group’s research results are mainly cellular networks and wireless data networks with varying geographical coverage. The interest groups for the application areas include network operators and network system and security designers.

Focus areas: The group’s focus areas are those key is-sue in networking necessary to complement the existing solutions on the physical layer for building up the new generation of wireless communications. The research will include: network optimization, opportunistic network cod-ing as an alternative to the conventional routing concept, intercell interference aware resource management, game theory based solutions, introduction of active network ele-ments (mobile agent traffi c optimization), topology con-trol and security issues in wireless networks.

Teams: The Internetworking Group is led by Prof. Savo Glisic and currently consists of 2 teams each concentrating on their specifi c expertise areas • Networking• Secure and Trustworthy Wireless Internet

Wireless Systems (WS) The world is changing and ICT is expected to penetrate new areas as an enabler of new services and applications. It is therefore a necessity to start looking at cross-dicipli-nary approaches to research. This calls for the identifi ca-tion of opportunities, interpretation of weak signals and thus new openings for ICT.

Goal: Wireless Systems (WS) implements the strategy of CWC by identifying new opportunities and performing re-search in these new areas, and especially identifying new system development opportunities as well as continuing work on pre-identifi ed areas. CWC has extremely broad expertise in wireless systems research as our results in se-curity and defence related systems development shows. The goal is to carry out research in such a fashion that it supports the needs of our customers in their product devel-opment or system specifi cation work.

Focus areas: Security and defence related communication, navigation and radar systems have been on our research agenda since the inception of CWC. Medical ICT and Short Range Communication (SRC) have a history span-ning more than 10 years. In the SRC area especially sen-sor networking has attracted a great deal of research effort in recent years. New emerging areas include smart energy grids and solutions for emerging markets. In smart energy grids, our research efforts are concentrated on identifying novel architectures for smart distributed control and data distribution, including modifi cations to existing communi-cation systems such as LTE or LTE-A.

Teams: WS group is led by Ari Pouttu and currently con-sists of 3 teams each concentrating on their specifi c exper-tise areas:


• Security and Defence Systems • SRC and Medical ICT• Transfer for Emerging Technologies

Scientifi c Progress2010 was a successful year in CWC for its implementa-tion and demonstration activities. On January 11th, the world’s fi rst telephone call over a cognitive radio network was made between the University of Oulu’s Rector, Lauri Lajunen and the CWC Board Chairman Juha Hulkko using CWC’s cognitive radio network demonstration system, as shown in Figure 1. In September, the same demo system won the best demo award of the ACM MOBICOM 2010 conference. There were almost 30 demos from various universities and companies, such as Rice University, Uni-versity of Illinois at Chicago, Stanford University, Uni-versity of California (Santa Barbara), ETH Zurich, Nokia Research Centre, INRIA, Thales etc.

The cognitive radio networking demo is built on WARP platforms from Rice University (see Figure 2). The plat-forms are very fl exible software defi ned radio platforms where every layer of the protocol stack is programmable. Most of the development work has been carried out on the MAC layer as it is the single most important part of cogni-tive radio networking in its traditional meaning, which is the effi cient usage of the spectrum. The developed novel MAC protocol is designed for mobile ad hoc networks and is capable of using different frequencies for each link in the network. As there is no centralized frequency management, the system has a built-in interference avoid-ance capability. The physical layer of the demo system is traditional OFDM with spectrum sensing capability. The network layer uses OLSRv2 routing protocol. The layers are interconnected by a special control interface, which enables transmitting measured performance information between the layers. The control interface also enables fi ne-tuning of the operating parameters of the different layers on the fl y with a cognitive engine.

The on-going effort with a demonstrator is a spectrum us-age database and how it can be utilized in spectrum man-agement. In this scheme each node of the network sends its measured spectrum information to a spectrum manager, from which spectrum usage strategies can be made.

Figure 2. The WARP Wireless Open-Access Research Platform.

Demonstration of a VOIP Call through a Cog-nitive Radio NetworkThe demonstration is about making a voice over internet protocol (VOIP) call over a combination of a cognitive radio network, LAN and Wi-Fi network. A laptop that is connected to a secondary user (SU) node which is in the cognitive radio network (CRN) forms one end of the call connection, and the other end of the call connection is formed by mobile equipment connected to a VOIP net-work. The CRN operates in the presence of primary users (PUs) which is using a different MAC protocol compared with the secondary users (SUs). The frequency channels used by the SU might be used by a PU randomly at any time, and this might cause disruption to the PU data com-munication. To avoid this disturbance the SU uses a simple

Figure 1. World’s fi rst phone call over a cognitive radio network.


spectrum sensing technique to detect the existence of a PU and avoid the corresponding frequency channel used by the PU. The SUs use the opportunistic MAC for a mobile ad hoc network (OMAN) MAC protocol as shown in Fig-ure 3, where in the MAC frame is divided into a TDMA part using a common control channel (CCH) and a ran-dom access part. The TDMA slots are used for broadcast transmissions whereas the random access slot is used to exchange a RTS CTS message to determine an opportun-istic channel (OTCH), and eventually send the data over the selected OTCH. The PUs however use distributed time division multiple access (DTDMA). The demonstration will consist of 6 nodes, 4 of which will be SUs, and 2 will be PUs. The SUs exchange control packets over a common control channel (CCH) which is free from interference by the PU. The data transmission happens over an opportun-istic traffi c channel (OTCH) which is in general one of the licensed channels used by PUs and this is selected as a result of spectrum sensing by the communicating SU pairs. All the SUs use optimized link state routing protocol (OLSR) on the network layer, and hence communication between SUs, which are several hops away, can be real-ized.

Figure 3. Flow diagram of the OMAN.

Each demonstrator node is built on a Linux enriched wire-less open access research platform (LE-WARP); this is an improvised form of the WARP platform from Rice Uni-versity, and the following modifi cations were made to WARP.• Porting of the Linux kernel to the WARP hardware• Using the one remaining processor as the network layer

handler• Adding hardware and software drivers to facilitate inter

processor communicationIn the process of modifying, we had change the system as-sembly of the FPGA design. The changes that facilitate the above modifi cations are refl ected in Figure 4. LE-WARP added a fully functional network layer with IPv4 stack and transport, application layer to WARP. The open source Linux enables utilization of the OLSR routing protocol or any other routing protocol designed for Linux to be used on WARP.

Figure 4. The system assembly of LE-WARP.

The CRN demonstrator comprises of the following mod-ules.• Spectrum Sensing at PHY layer (SS)• Cognitive Engine module (CE)• Time synchronization module (TS)• Opportunistic MAC with Network layer information

(OMAN)• Linux OS as the Network layer handler with OLSR and

full IPv4 TCP/IP protocol stack running on it• PUs using DTDMA protocol.• Logging unit used to set up a GUI for interactive usage.The graphical user interface can show the network topol-ogy and its changes, and the whole data transmission proc-ess from a projector display. By clicking on nodes, we can show information like• Whether a node is a PU or SU• Current state of the node and the packet types received

and transmitted• Neighbouring radio environment information as seen by

the node considered• Routing table information• Throughput• Information on critical PHY and MAC parameters of the

node considered• Slot structure by the PU• MAC frame intervals of PU and SU• Current channel used by the PU and SULE-WARP design, with its improved performance and added features, has enabled us to achieve our goal of real-izing a self confi gurable and self aware wireless ad hoc network demonstrator system. The observations and re-sults achieved can be used as inference in the ongoing projects in the research community across academic insti-tutes and industries striving to achieve a similar objective. The award winning performances at different international conferences say a lot about the potential of using the LE-WARP design. It can be further used to set up a wireless ad hoc network test bed to verify theoretical algorithms at different layers, and thus establish the practical limitations of theoretical results.


Personnel



others 34

total 118

person years 94

External Funding

Source EUR



Tekes 2 042 000

other domestic public 285 000


international 2 098 000

total 6 199 000

Doctoral ThesesVartiainen J (2010) Concentrated signal extraction using con-secutive mean excision algorithms. Acta Universitatis Ouluensis C 368. Haapola J (2010) Evaluating medium access control protocols for wireless sensor networks. Acta Universitatis Ouluensis C 350. Ylioinas J (2010) Iterative detection, decoding, and channel esti-mation in MIMO-OFDM. Acta Universitatis Ouluensis C 358.

Selected PublicationsKhan Z, Lehtomäki J, Codreanu M & Latva-aho M (2010) Throughput-effi cient dynamic coalition formation in cognitive radio networks. EURASIP Journal on Wireless Communications and Networking 2010, doi:10.1155/2010/653913, 13 p.Khan Z, Glisic S, Da Silva L & Lehtomäki Janne (2011) Mod-eling the dynamics of coalition formation games for cooperative spectrum sharing in an interference channel. IEEE Transactions on Computational Intelligence and AI in Games 3(1): 17-30.Viittala H, Nahar BN, Hämäläinen M & Iinatti J (2010) Medical applications adapting ultra wideband - a system study. Interna-tional Journal of Ultra Wideband Communications and Systems - Special Issue on Applications of Ultra Wideband Systems in Biomedicine 1(4): 237-247.Taparugssanagorn A, Pomalaza-Raez C, Isola A, Tesi R, Hämäläinen M & Iinatti J (2010) UWB channel modelling for wireless body area networks in a hospital. International Journal of Ultra Wideband Communications and Systems 1(2010).Weeraddana C, Codreanu M, Latva-aho M & Ephremides A (2010) On the effect of self-interference cancelation in multiHop wireless networks. EURASIP Journal on Wireless Communica-tions and Networking, doi:10.1155/2010/513952, 10 p.Myllymäki S, Huttunen A, Palukuru V & Jantunen H, Berg M, Salonen E (2010) Capacitive recognition of the user’ s hand grip position in mobile handsets. Progress In Electromagnetics Re-search B (22): 203-220.Ketonen J, Juntti M & Cavallaro JR (2010) Performance-com-plexity comparison of receivers for a LTE MIMO-OFDM sys-tem. IEEE Transactions on Signal Processing 58(6): 3360-3372.Pantelidou A & Ephremides A (2010) A cross-layer view of optimal scheduling. IEEE Transactions of Information Theory 56(11).Vehkaperä M & Juntti M (2010) On the performance of space-time coded and spatially multiplexed MIMO systems with linear receivers. IEEE Transactions on Communications 58(2): 642-651.Niemelä V, Rabbachin A, Attaphongse T, Hämäläinen M, Iinatti J (2010) Simulations of some UWB WBAN receivers in a hospital environment. Sports And Wellbeing - Multidisciplinary research symposium: Technological and nutritional aspects.


BIOMEDICAL ENGINEERING GROUP (BME)

Professor Tapio Seppänen, Computer Science and Engineering Laboratory, Faculty of Technology, and Professor Timo Jämsä, Department of Medical Technology, Faculty of Medicine,

University of Oulutapio.seppanen(at)ee.oulu.fi , timo.jamsa(at)oulu.fi

http://www.cse.oulu.fi /bme

The mission is to carry out basic, applied and translational research in biomedical engineering.

Cardiovascular and Respiratory System Re-searchRespiratory Sinus Arrhythmia Component Extraction by Combining LMS and PCA Derived Respiration. Respira-tory sinus arrhythmia (RSA) is a phenomenon where the heart rate changes synchronously with respiration. We de-vised a method to produce ECG derived surrogate respira-tion that can be used as a reference signal in an LMS adap-tive fi lter to extract the RSA component from cardiovas-cular signals. A surrogate respiration signal is obtained by principal component analysis applied to QRS complexes. A method minimizing residual energy is used to select an optimal PC component for the surrogate. Results indicate that one PC is adequate as respiration reference for the adaptive fi lter and that the ECG-based surrogate produces comparable results to the actual respiration signal.

Continuous Assessment of Nasal Airfl ow Resistance by Adaptive Modelling. A method to assess nasal airfl ow resistance was introduced that provides a continuous re-sistance value, and applies a novel minimally obtrusive measurement technique. The continuous pressure record-ing is produced with a nasopharyngeal catheter inserted 8 cm deep along the fl oor of one nasal cavity and the fl ow recording is produced with respiratory effort bands. A least-mean-square (LMS) extension for the resistance model by Broms is developed that dynamically adapts to the time-varying characteristics of the nasal functioning, and produces the continuous resistance values. Experi-mental results are shown that demonstrate the uniqueness and applicability of the new technique in assessing the dif-ferences between normal and symptomatic patients, and the effect of nasal treatment on patients.

Central Nervous System ResearchElectroencephalogram-based depth of anaesthesia meas-urement. Increasing concentrations of anaesthetics in the blood induce a continuum of neurophysiological changes, which refl ect on the EEG. A novel method was developed to estimate the phase of the continuum using the feature data extracted from the EEG. In the method, the feature data calculated from EEG sequences corresponding to continuously deepening anaesthesia are considered to form a one-dimensional nonlinear manifold in the multi-dimensional feature space. Utilizing a recently proposed algorithm, Isomap, the dimensionality of the feature data is reduced to achieve a one-dimensional embedding, rep-resenting this manifold, and thereby the continuum of

neurophysiological changes during induction of anaesthe-sia. The proposed method provides a novel approach to assessing neurophysiological changes during anaesthesia and offers potential for the development of more advanced systems for the depth of anaesthesia monitoring.

Usually, general anaesthesia is induced with a combina-tion of drugs. In addition to the hypnotic agent, such as propofol, opioids are often used due to their synergistic hypnotic and analgesic properties. A novel technology for separating the anaesthetic effects of propofol and an ultra-short-acting opioid, remifentanil, using the spectral features of EEG was proposed. Including the detection of the presence of opioids to the estimated effect of propo-fol signifi cantly improves the determination of the clinical state of the patient.

Musculoskeletal System ResearchIn our osteoporotic fracture risk study, we have shown that the risk of trochanteric fractures could be discrimi-nated, based on the standard clinical bone mineral den-sity (BMD) based criterion, whereas cervical fracture cases would remain under-diagnosed if solely using this criterion. Instead, geometrical risk factors are able to dis-criminate cervical fracture cases even among individuals with high BMD. We also showed that dual-energy digital radiography (DEDR) can be applied for the assessment of bone density. In future, combining BMD measurements using DEDR with structural and geometrical information available on digital radiographs could enable a more com-prehensive assessment of bone.

Using the fi nite element method, we analysed the principal strain distributions, and computed the strain ratio for strain patterns over the cervical and trochanteric regions. Using this novel methodology, we were able to predict the hip fracture type with an accuracy of 85%.

Using the accelerometer-based technology, we have now developed a novel index DIS (Daily Impact Score) for as-sessing the osteogenic response of daily physical activity. A patent application was fi led on the new method. In fu-ture, we will test if a similar index can be developed for as-sessing the response of exercise to other health outcomes as well, like osteoarthritis, glucose tolerance etc. We per-formed a joint bone exercise study in Sendai, Japan, that is now fi nished and the results have been published. Using our physical activity measurement technology, we were able to show that even a short exercise session during the working day has positive effects for bone, if the intensity of exercise is high.


Personnel



total 25

person years 18

External Funding

Source EUR



Tekes 111 000



total 582 000

Selected PublicationsAhola R, Korpelainen R, Vainionpää A & Jämsä T (2010) Daily impact score in long-term acceleration measurements of high-impact exercise. J Biomech 43(10): 1960-1964. Finnilä M, Zioupos P, Herlin M, Miettinen HM, Simanainen U, Håkansson H, Tuukkanen J, Viluksela M & Jämsä T (2010) Ef-fects of 2,3,7,8-tetrachlorodibenzo-p-dioxin exposure on bone material properties. J Biomech 43(6): 1097-1103.Heikkinen J, Jansson-Verkasalo E, Toivanen J, Suominen K, Väyrynen E, Moilanen I & Seppänen T (2010) Perception of ba-sic emotions from speech prosody in adolescents with Asperger syndrome Logopedics Phoniatrics Vocology, 35(3): 113-120. Herlin M, Kalantari F, Stern N, Sans D, Larsson S, Viluksela M, Tuomisto JT, Tuomisto J, Tuukkanen J, Jämsä T, Lind PM & Håkansson H (2010) Quantitative characterization of changes in bone geometry, mineral density and biomechanical properties in two rat strains with different Ah-receptor structure after long-term exposure to 2,3,7,8-tetrachlorodibenzo-p-dioxin. Toxicol-ogy 273(1-3): 1-11.Hyytinen-Oinas MK, Ylitalo K, Karsikas M, Seppänen T, Raa-tikainen MJ, Uusimaa P, Huikuri HV & Perkiömäki JS (2010) Electrocardiographic abnormalities and ventricular tachyarrhyth-mias after myocardial infarction. Scandinavian Cardiovascular Journal, May 28, Epub ahead: 1-9.Isojärvi H, Keinänen-Kiukaanniemi S, Kallio M, Kaikkonen K, Jämsä T, Korpelainen R (2010) Serum triglycerides, fi tness and physical activity are associated to peripheral nervous sys-tem function in overweight adults. Med Sci Sports Exerc 42(7): 1241-1245. Karjalainen T, Göransson H, Viinikainen A, Jämsä T & Ryhänen J (2010) Nickel-titanium wire as a fl exor tendon suture material. An ex vivo study. J Hand Surg Eur 35(6): 469-744.Kenttä T, Karsikas M, Junttila J, Perkiömäki J, Seppänen T, Ki-viniemi A, Nieminen T, Lehtimäki T, Nikus K, Lehtinen R, Viik J, Kähönen M & Huikuri H (2010) QRS-T morphology measured from exercise electrocardiogram as a predictor of cardiac mortal-

ity. Europace, in print. On-line doi: 10.1093/europace/euq461.Kiviniemi A, Tiinanen S, Hautala A, Seppänen T, Norton K, Frances M, Nolan R, Huikuri H, Tulppo M & Shoemaker J (2010) Low-frequency oscillations in R-R interval and blood pressure across the continuum of cardiovascular risk. Autonomic Neuroscience: Basic and Clinical 158(1-2): 92-99.Kiviniemi A, Tiinanen S, Huikuri H, Hautala A, Seppänen T, Mäkikallio T & Tulppo M (2010) Frequency of slow oscillations in arterial pressure and R-R intervals during muscle metaborefl ex activation. Autonomic Neuroscience 152(1-2): 88-95.Koivumäki J, Thevenot J, Pulkkinen P, Salmi JA, Kuhn V, Lo-chmüller EM, Link TM, Eckstein F & Jämsä T (2010) Associa-tion of strain distribution with the occurrence of cervical vs. tro-chanteric hip fractures. Med Biol Eng Comput 48: 711-717.Korpelainen R, Keinänen-Kiukaanniemi S, Nieminen P, Heikkin-en J, Väänänen K & Korpelainen J (2010) Long-term outcomes of exercise: follow-up of a randomized trial in older women with osteopenia. Arch Intern Med. 170(17): 1548-1556.Kortelainen J, Väyrynen E, Seppänen T (2010) Isomap ap-proach to EEG-based assessment of neurophysiological changes during anesthesia. IEEE Transactions on Neural Sys-tems and Rehabilitation Engineering, In press. DOI: 10.1109/TNSRE.2010.2098420. Niu KJ, Heikkinen R, Guo H, Korpelainen R, Uchimaru J, Vainionpää A, Sato K, Sakai A, Salo S, Komatsu S, Jämsä T & Nagatomi R (2010) Effect of offi ce-based high-impact exercise on bone mineral density in healthy premenopausal women: The Sendai Bone Health Concept Study. J Bone Miner Metab 28(5): 568-577.Pulkkinen P, Partanen J, Jalovaara P & Jämsä T (2010) BMD T-score discriminates trochanteric fractures from unfractured controls, whereas geometry discriminates cervical fracture cases from unfractured controls of similar BMD. Osteopor Int 21(7): 1269-1276. Rantalainen T, Nikander R, Heinonen A, Multanen J, Häkkinen A, Jämsä T, Kiviranta I, Linnamo V, Komi PV & Sievänen H (2010) Neuromuscular performance and body mass as indeces of bone loading in premenopausal and postmenopausal women. Bone 46: 964-969.Rantanen A, Laukka S, Lehtihalmes M & Seppänen T (2010) Heart rate variability (HRV) refl ecting from oral reports of nega-tive experience. Procedia Social and Behavioral Sciences, 5: 483-487.Seppänen T, Koskinen M, Seppänen TM & Alho O-P (2010) Ad-dendum to ‘Continuous assessment of nasal airfl ow resistance by adaptive modeling’ technical repeatability. Physiol Meas 31: 1547-1551.Seydou F & Seppänen T (2010) Recovery of potential singulari-ties in ultrasound imaging. International Review of Physics, Oc-tober.Tahvanainen PS, Lammentausta E, Pulkkinen P, Tervonen O, Jämsä T & Nieminen MT (2010) Dual-energy digital radiog-raphy for the assessment of bone mineral density. Acta Radiol 51(5): 543-548.Thevenot J, Pulkkinen P, Kuhn V, Eckstein F & Jämsä T (2010) Bilateral asymmetries in hip related to experimental fracture type. Calcif Tissue Int 87: 203-210.Tölli H, Kujala S, Levonen K, Jämsä T & Jalovaara P (2010) Bi-oglass as a carrier for reindeer bone protein extract in the healing of rat femur defect. J Mater Sci Mater Med 21(5): 1677-1684.Väyrynen E, Toivanen J & Seppänen T (2010) Classifi cation of emotion in spoken Finnish using vowel-length segments: In-creasing reliability with a fusion technique. Speech Communica-tion, doi:10.1016/j.specom.2010.09.007.


The EMPART research group is a multidisciplinary re-search unit. Its main activities lie in two focus areas of the University of Oulu, concentrating both on the electron-ics materials and devices, and nanotechnology. The group is a key player in the Micro- and Nanotechnology Centre of the University of Oulu, where our overall target is to integrate nanostructures enabling novel functionality of electronic, telecommunication, bio/medical and environ-mental devices.

The group comprises of specialists in microelectronics, materials, mechanical engineering, measuring techniques, and also in chemistry and physics. The personnel consists of six professors, eight post-doctoral researchers and 24 doctoral students.

The group in 2010 was funded by the University of Oulu, Tekes, EU, Academy of Finland, Nordic Innovation Cen-tre, Scientifi c Advisory Board for Defense and by domes-tic and foreign industry. Global research co-operation is a characteristic feature of the EMPART group, having key roles in several EU and other international projects.

In accordance with the long term research targets, we have continued the integration of interdisciplinary topics to-wards future advanced device and component implemen-tations. In addition, a wide range of application areas uti-lizing the generic materials knowledge of the group have had great importance from the application point of view.

Value-chain approach provides solutions capable of creat-ing new business opportunities.

The group co-operated in 2010 with other Infotech Oulu research groups, especially with the Optoelectronics and Measurement Unit, being also a partner in PrintoCent (Printed Electronics and Optical Measurements Innova-tion Center) having an established printed electronics laboratory. National and international research co-op-eration included common projects and publications, and

student, researcher and lecturer exchange. The group also acknowledges Finnish and foreign industrial partners for their active participation in research projects.

Materials, components and technologies developed by the group are widely applied in the national electronics industry, especially in wireless telecommunication, sen-sors/actuators and hybrid microelectronics technology. LTCC micromodules and printed electronics devices are important examples of present exploitation, together with recent scientifi c achievements of nanotechnology with applications. Novel materials, as well as the progress in fabrication have been utilized in antennas, gas sensors, ce-ramic/polymer integrations, fi lters, micro-pumps, lens and mirror positioning systems, energy harvesters, etc.

The EMPART group achieved remarkable results in the history of Infotech Oulu when considering the amount of scientifi c refereed journal publications. The total amount was around 50 with increasing impact factors.

Personnel



total 38

person years 11

External Funding

Source EUR



Tekes 980 000



total 2 520 000

Doctoral ThesesPalukuru V (2010) Electrically tunable microwave devices using BST-LTCC thick fi lms. Acta Universitatis Ouluensis C 366.

ELECTRONICS MATERIALS, PACKAGING AND RELIABILITY TECHNIQUES (EMPART)

Professor Heli Jantunen, Microelectronics and Materials Physics Laboratories,Department of Electrical and Information Engineering, University of Oulu

Research Professor Pentti Karioja, VTT Ouluheja(at)ee.oulu.fi , pentti.karioja(at)vtt.fi

http://www.infotech.oulu.fi /empart


Plekh M (2010) Ferroelectric performance for nanometer scaled devices. Acta Universitatis Ouluensis C 372.

Selected PublicationsBernas H, Eränen K, Simakova I, Leino A-R, Kordás K, Myllyoja J, Mäki-Arvela P, Salmi T & Murzin DY (2010) Deoxygenation of dodecanoic acid under inert atmosphere. Fuel 89, 2033-2039.Dombovari A, Halonen N, Sapi A, Szabo M, Toth G, Mäklin J, Kordás K, Juuti J, Jantunen H, Kukovecz A & Konya Z (2010) Moderate anisotropy in the electrical conductivity of bulk MW-CNT/epoxy composites. Carbon 48, 1918-1925. Gracia-Espino E, Sala G, Pino F, Halonen N, Luomahaara J, Mäklin J, Tóth G, Kordás K, Jantunen H, Terrones M, Helistö P, Seppä H, Ajayan PM & Vajtai R (2010) Electrical transport and fi eld-effect transistors using inkjet-printed SWCNT fi lms having different functional side groups. ACS Nano 4, 3318-3324.Halonen N, Rautio A, Leino A-R, Kyllönen T, Tóth G, Lappa-lainen J, Kordás K, Huuhtanen M, Keiski RL, Sápi A, Szabó M, Kukovecz A, Kónya Z, Kiricsi I, Ajayan PM & Vajtai R (2010) Three-dimensional carbon nanotube scaffolds as particulate fi l-ters and catalyst support membranes. ACS Nano / DOI: 10.1021/nn100150x.Hannu J, Juuti J & Jantunen H (2010) Recent patents on piezo-electric energy harvester transducer structures. Recent Patents on Electrical Engineering 3, 19-24. Hannu J, Saikkonen T, Häkkinen J, Karttunen J & Moilanen M (2010) Enabling remote testing: embedded test controller and mixed-signal test architecture. Journal of Electronic Testing 26, 641-658, / DOI 10.1007/s10836-010-5175-6. Huttunen A, Myllymäki S, Komulainen M & Jantunen H (2010) Capacitive sensor arrangement to detect external load on a mo-bile terminal antenna. Progress In Electromagnetics Research Letters 15, 13-18. Juuti J, Teirikangas M, Sonoda K & Jantunen H (2010) Ther-moplastic 0-3 ceramic-polymer composites with adjustable mag-netic and dielectric characteristics for radio frequency applica-tions. International Journal of Applied Ceramic Technology 7, 452-460. Koskela P, Teirikangas M, Alastalo A, Forsman J, Juuti J, Tapper U, Auvinen A, Seppä H, Jantunen H & Jokiniemi J (2010) Syn-thesis of cobalt nanoparticles to enhance magnetic permeability of metal-polymer composites. Advanced Powder Technology, doi:10.1016/j.apt.2010.09.010.Kukkola J, Rautio A, Sala G, Pino F, Tóth G, Leino A-R, Mäklin J, Jantunen H, Uusimäki A, Kordás K, Gracia E, Terrones M, Shchukarev A & Mikkola J-P (2010) Electrical transport through single-wall carbon nanotube-anodic aluminum oxide-aluminum heterostructures. Nanotechnology 21, 035707. Kukovecz A, Molnár D, Kordás K, Gingl Z, Moilanen H, Mingesz R, Kónya Z, Mäklin J, Halonen N, Tóth G, Haspel H, Heszler P, Mohl M, Sápi A, Roth S, Vajtai R, Ajayan PM, Pouil-lon Y, Rubio A & Kiricsi I (2010). Carbon nanotube based sen-sors and fl uctuation enhanced sensing. Phys. Status Solidi (C) 7, 1217-1221, / DOI 10.1002/pssc.200982973.Lappalainen J, Puustinen J, Hiltunen J & Lantto V (2010) Pulsed-laser deposited amorphous-like PZT thin-fi lms: microstructure and optical properties. Journal of the European Ceramic Society 30, 497-502.Mohl M, Pusztai PJ, Kukovecz A, Konya Z, Kukkola J, Kordás K, Vajtai R & Ajayan PM, Low-temperature large-scale synthe-sis and electrical testing of ultralong copper nanowires. Lang-muir 26(21): 16496-16502. Molnar D, Heszler P, Mingesz R, Gingl Z, Kukovecz A, Konya Z, Haspel H, Mohl M, Sapi A, Kiricsi I, Kordás K, Mäklin J,

Halonen N, Toth G, Moilanen H, Roth S, Vajtai R, Ajayan PM, Pouillon Y & Rubio A (2010) Increasing chemical selectivity of carbon nanotube-based sensors by fl uctuation enhanced sensing. Fluctuation and Noise Letters 9(3): 277-287. Myllymäki S (2010) Dual-dome micromechanical resonator fab-ricated with reactive ion etching and plasma-enhanced chemical vapor deposition techniques. Journal of Micro/Nanolithography, MEMS, and MOEMS 9, 023012.Myllymäki S, Huttunen A, Palukuru VK, Jantunen H, Berg M & Salonen E (2010) Capacitive recognition of the user’s hand grip position in mobile handsets. Progress In Electromagnetics Research B 22, 203-220.Nelo M, Sowpati AK, Palukuru VK, Juuti J & Jantunen H (2010) Formulation of screen printable cobalt nanoparticle ink for high frequency applications. Progress In Electromagnetics Research 110, 253-266.Palosaari J, Juuti J, Heinonen E, Moilanen V-P, & Jantunen H (2010) Non-uniform electric fi eld in poling of structurally grad-ed monolithic piezoactuator. Journal of Electroceramics, DOI 10.1007/s10832-010-9609-4.Palukuru VK, Sonoda K, Surendran R & Jantunen H (2010) BST_COC composite based rectangular DI-electric resonator antenna (DRA) for 2.4 wlan wrist applications. Progress In Elec-tromagnetics Research C 16, 195-205.Peräntie J, Hagberg J, Uusimäki A & Jantunen H (2010) Elec-tric-fi eld-induced dielectric and temperature changes in a <011> - oriented Pb(Mg1/3Nb2/3)O3-PbTiO3 single crystal. Phys. Rev. B 82, 134119, 8 p. Plekh M, Narkilahti J, Levoska J & Tyunina M (2010) Polydo-main confi guration in epitaxial Pb0.5Sr0.5TiO3/La0.5Sr0.5CoO3 het-erostructures. Applied Physics Letters 97, 202909 (1-3).Puustinen J, Lappalainen J, Hiltunen J, Lantto V & Jantunen H (2010) Orientation dependend dielectric characteristics of nano-crystalline Pb(ZrxTi1-x)O3 fi lms with inter digital electrodes. Fer-roelectrics 405, 227-235. Sebastian MT & Jantunen H (2010) Polymer-ceramic compos-ites of 0-3 connectivity for circuits in electronics: a review. Inter-national Journal of Applied Ceramic Technology 7, 415-434.Sobocinski M, Zwierz R, Juuti J, Jantunen H & Golonka L (2010) Electrical and electromechanical characteristics of LTCC embed-ded piezoelectric bulk actuators. Advances in Applied Ceramics 109(3): 135-138. Sonoda K, Juuti J, Moriya Y & Jantunen H (2010) Modifi cation of the dielectric properties of 0-3 ceramic-polymer composites by introducing surface active agents onto the ceramic fi ller sur-face. Composite Structures 92, 1052-1058.Sowpati AK, Palukuru VK, Pynttäri V, Mäkinen R, Kartikeyan MV & Jantunen H (2010) Performance of printable antennas with different conductor thicknesses. Progress In Electromagnet-ics Research Letters 13, 59-65.Teirikangas M, Juuti J & Jantunen H (2010) Layered dielectric-magnetic composite structures for Rf-applications. Composite Structures 93, 179-183.Tyunina M, Levoska J, Nuzhnyy D & Kamba S (2010) Enhanced relaxor behavior in epitaxial PbMg1/3Nb2/3O3 fi lms. Physical Re-view B 81, 132105 (1-4).Tyunina M, Narkilahti J, Plekh M, Oja R, Nieminen RM, De-jneka A & Trepakov V (2010) Evidence for strain-induced fer-roelectric order in epitaxial thin-fi lm KTaO3. Physical Review Letters 104, 227601.Valant M, Dunne L, Axelsson A-K, Alford NM, Manos G, Perän-tie J, Hagberg J, Jantunen H & Dabkowski A (2010) Electro-caloric effect in a ferroelectric Pb(Zn1/3Nb2/3)O3-PbTiO3 single crystal. Physical Review B 81, 214110.


HUMAN INTERACTION WITH ADVANCED MOBILE SERVICES AND INTELLIGENT ENVIRONMENTS (INTERACT)

Professor Kari Kuutti, Department of Information Processing Science, University of OuluResearch Professor Heikki Ailisto, VTT Oulu

kari.kuutti(at)oulu.fi , heikki.ailisto(at)vtt.fi http://www.infotech.oulu.fi /interact

Background and MissionThe research group consists of researchers from the De-partment of Information Processing Science, and VTT, Advanced Interactive Systems Research Field.

Currently, Human Computer Interaction (HCI) research is facing ever increasing challenges, with advanced mo-bile services and intelligent environments moving from research laboratories towards everyday use. A smoothly operating and intuitive user interface will be a fundamen-tal element of all successful applications, and an important element in the competition. Mobile services may still bear some resemblance to traditional applications, but new, in-telligent, adaptive environments will differ considerably with respect to user interfaces, and the usefulness of old design knowledge will indeed be limited.

There are therefore a number of diffi cult but interesting research problems on different levels:• The emergent use of non-traditional interaction modali-

ties, such as voice, gestures and gaze, sets new require-ments on HCI design, and opens up totally new, unex-plored possibilities.

• There is a need for advanced technological architectures that can provide the necessary HCI services to the user and adapt to situations.

• We do not know how to effi ciently utilize small, differ-ently sized interfaces in the realisation of advanced mo-bile services.

• We lack effi cient design methods for combinations of mobile devices and traditional services, not to mention methods for designing interfaces for intelligent environ-ments.

• Although there is a body of knowledge for user-cen-tred HCI design, we do not know which parts of that knowledge are directly applicable and what else may be needed.

The user experience in mobile networks and intelligent environments will be less controlled than with traditional applications. This is due to the fragmentation of service provision, which will be composed from parts offered by different providers. Ensuring that this collection will not have a detrimental effect on the user experience is a to-tally novel design challenge. Moreover, the increase in the number of stakeholders in service provision will make the organising of HCI design even more complex.

Human interfaces and experience with advanced mobile services and intelligent environments will be one of the major design challenges when the systems are moved from research laboratories towards everyday use. The INTER-

ACT research group is addressing this problem area on a number of levels which support each other. On the level of basic research, the group is modelling the use situation of advanced services and intelligent environments, and correspondingly envisioning and constructing novel tech-nology architectures needed to provide the optimal user experience in those environments, in a controlled way. The group is also developing and experimenting with nov-el forms of enhanced multimodal interaction for devices and environments. Moreover, the group is developing new methods and techniques for design interaction in intel-ligent environments, and constructing test environments where both the interaction techniques and design methods can be tested and validated. Finally, the group is develop-ing methods for the evaluation and improvement of HCI design processes in product development organisations.

Scientifi c ProgressDuring 2010 the main scientifi c results were:• The research on usability in open source development

(N. Iivari and her group)• The development of user-centred methods for designing

with children (N. Iivari and her group)• Research on visualizing personal mobile communica-

tion patterns (M. Isomursu and the VTT team).• The two-stage model on long-term user experience (PhD

thesis by S. Luojus)

Exploitation of ResultsDissemination of results - The group has been publishing actively, and additionally the members of the group have been giving talks, presentations and demonstrations on a number of occasions, both in Finland and abroad, in both academic and industry led events. Several short visits from other groups have also been hosted.

Industrial cooperation - The cooperation in the SECS project has continued. A number of Master’s theses have been supervised in cooperation with industry.

Personnel



total 27

person years (university 50%, VTT 50%) 20


External Funding

Source EUR





total 590 000

Doctoral ThesesLuojus S (2010) From a momentary experience to a lasting one. The concept of and research on expanded user experience of mo-bile devices. Acta Universitatis Ouluensis A 559.

Selected PublicationsErvasti M, Kinnula M & Isomursu M (2010) User experiences with mobile supervision of school attendance. International Jour-nal on Advances in Life Sciences 2(1-2): 29-41.Häikiö J, Wallin A & Isomursu M (2010) Digitally-enhanced services for the elderly. International Journal of Services Sci-ences. Special issue on innovative information technology in the service sector, 232-249. Iivari N (2010) Discursive construction of ‘user innovations’ in the open source software development context. Information and Organization 20(2): 111-132.Iivari N (2010) Culturally compatible usability work - an inter-pretive case study on the relationship between usability work and its cultural context in software product development organiza-tions. Journal of Organizational and End User Computing 22(3): 40-65.Kuutti K (2010) Where are the Ionians of user experience re-search? Proc. 6th Nordic Conference on Human-Computer Inter-action (NordiCHI 2010), 715-718.Könönen V, Mäntyjärvi J, Similä H, Pärkkä J & Ermes M (2010) Automatic feature selection for context recognition in mobile de-vices. Pervasive and Mobile Computing 6(2): 181-197. Mazzone E, Iivari N, Tikkanen R, Read J & Beale R (2010) Con-sidering context, content, management, and engagement in de-sign activities with children. Proceedings of the 9th International Conference on Interaction Design and Children, 108-117.Rajanen M & Iivari N (2010) Traditional usability costs and ben-efi ts - fi tting them into open source software development. Pro-ceedings of 18th European Conference on Information Systems (ECIS 2010), June 7-9, Pretoria, South Africa.


INFOTECH OULU DOCTORAL PROGRAM

Infotech Oulu Doctoral Program (formerly Infotech Oulu Graduate School) operates in three areas, corresponding to the major research fi elds of Infotech Oulu. These are electronics, communications engineering, and computer science and information engineering.

The Electronics section consists of electronic circuit and system design, microelectronics, electronics manufactur-ing technology, physical electronics, electronic and opto-electronic measurement technology, and testing and dis-turbance techniques in electronics.

Communications engineering covers telecommunication systems from the architectures and implementations of transceivers to telecommunication networks, systems and services. The main research themes include broadband wireless access, short range communications and sensor networks.

Computer science and information engineering empha-sizes information processing methods, ubiquitous systems and human-computer interaction. The topics include ma-chine vision, biosignal analysis, data mining, intelligent robots, software security, mobile computing, urban com-puting and ubiquitous internet.

During 2010, the graduate school system was nationally re-organized. The University of Oulu now has only one umbrella Graduate School consisting of several Doctoral Programs. At the end of 2010, the Infotech Oulu Graduate School offi cially changed its name to the Infotech Oulu Doctoral Program.

The Doctoral Program BoardEach of the main research areas is represented on the In-fotech Oulu Doctoral Program Board, as is VTT. Profes-sor Markku Juntti is the Director of the program and the Chair of the Board. The representatives of the main areas on the Board are Professors Juha Kostamovaara (elec-tronics), Matti Latva-aho (communications engineering), Juha Röning and Kari Kuutti. (both computer science and information engineering). Professor Pentti Karioja is the representative of VTT.

FundingIn 2010, the Infotech Oulu Doctoral Program had 25 doc-toral student posts and a position for a coordinator funded by the Ministry of Education and Culture. The posts are for doctoral students working in the research groups that are full or associate members of Infotech Oulu. This fund-ing has been confi rmed until the end of 2013 for the stu-dents. The coordinator position from the Ministry of Edu-cation and Culture has funding until the end of 2011. The posts altogether represented annually EUR 726 500 in sal-ary costs. Additional funding has been obtained from the Academy of Finland for arranging doctoral courses and for other costs. The Academy granted EUR 430 000 for the years 2010–2013. EUR 107 500 were designated for the year 2010. Together with the annual direct fi nancial sup-port of EUR 20 000 from Infotech Oulu, the total budget of the doctoral program was EUR 854 000. In addition, Infotech Oulu supports its doctoral program by fi nancing international workshops and researcher visits that contain doctoral education, through the work of the staff in the research groups.

Students and Doctoral ThesesThe Infotech Oulu Doctoral Program had 184 students at the end of 2010. This includes all the doctoral students in our research area working in the full and associate member groups of Infotech Oulu.

The output was 17 doctoral theses. The funding from Info-tech Oulu Doctoral Program positions was used for eight of them: Antti Siirtola, Juha Heiskanen, Mikko Loikkanen, Jari Ylioinas, Maxim Plekh, Zhonghong Ou, Vamsi Pa-lukuru, and Johanna Vartiainen defended their theses in 2010. The last three theses were funded only for a few months through our short-term posts that are used just for completing doctoral theses. The theses can be found in electronic format on the web from http://www.infotech.oulu.fi /dissertations.

Teaching ActivitiesStrong research contacts with other universities and re-search institutes are utilized in arranging lecturers for the

The Infotech Oulu Doctoral Program student meeting was held at VTT in April with a poster session.


courses. In each of the three major areas covered by the doctoral program, several lectures (Infotech Oulu Lecture Series) and intensive courses are held annually. Also the Infotech Oulu Distinguished Lecture Series on Ubiquitous Technology was continued with a view to boosting co-operation in this research area. These all provide a valu-able extension to the other doctoral courses in information technology provided by the university. The breadth of the courses below is expressed in ECTS credits.

The Infotech Oulu Lecture SeriesTo gain two credits, a graduate student must follow 20 hours of lectures and make a written summary of one lec-ture. The following lectures were held in 2010.

Electronics• Professor Anita Lloyd Spetz, Linköping University,

Sweden - Gas sensor research at Linköping University• Dr. David Stoppa, Fondazione Bruno Kessler (FBK),

Trento, Italy - CMOS single photon detection devices and techniques

• Dr. Michael Levinshtein, Ioffe Physico-Technical Insti-tute, Russian Academy of Sciences, St. Petersburg, Rus-sia - Silicon carbide devices: physical issues

Communications Engineering • Professor Mikko Valkama, Tampere University of Tech-

nology - Digitally-enhanced RF for future radio devices • Dr. Lorenzo Mucchi, University of Florence, Italy - Se-

curity in cognitive networks: a novel approach for phys-ical layer cryptography

• Dr. Redha M. Radaydeh, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Ara-bia - Adaptive schemes with interference management in narrowband and wideband wireless systems

• Professor Mark Reed, Australian National University and National ICT Australia (NICTA) - Femtocells: in-terference avoidance and cancellation mechanisms

Computer Science and Information Engineering• The 7th International Crisis Management Workshop

(CRIM2010), 1) lecturers: • Karl Cristian Posch, Graz University of Technology,

Austria - How to attract interest to a boring topic• Professor Bart Preneel, Katholieke Universiteit Leu-

ven, Belgium - Identity management and privacy• Professor Gerald Quirchmayr, University of Vienna,

Austria - Business process security and continuity management

• Dr. Luke Hebbes, IT Security Consultant, Senior Lecturer - Disappearing cryptography

• Professor Antti Valmari, Department of Software Sys-tems, Tampere University of Technology - Recent re-sults on DFA minimization and other block splitting algorithms

• Dr. Timo Ahonen, Nokia Research Center, Palo Alto - Mobile computational photography with Frankencam-era

• Professor Jan Flusser, Institute of Information Theory and Automation, Prague, Czech Republic - Advances in moment invariants

Infotech Oulu Distinguished Lecture Series on Ubiquitous Technology• Professor Henning Schulzrinne, Columbia University,

New York, USA - Future internet• 1st International UBI Summer School 2010, lectures in

2nd Open Ubiquitous City Seminar 2)

• Professor Marcus Foth, Queensland University of Technology, Australia - Urban informatics and sus-tainable cities

• Professor Mikael Wiberg, Umeå University, Sweden - Interactive textures – rethinking materiality

• Professor Anind Dey, Carnegie Mellon University, USA - Real world context-aware systems

• Zach Shelby, Head of Research, Sensinode, Finland - IP-based wireless sensor networks

• Dr. Jürgen Scheible, Aalto University, Finland - Creat-ing and sharing artistic experiences with ubiquitous technology

• Adam Greenfi eld, Do projects - Elements of a net-worked urbanism

• Professor Vassilis Kostakos, University of Madeira, Portugal - Urban social networks analysis

• Professor Maja Pantic, Imperial College London, UK and University of Twente, the Netherlands - Toward hu-man-centred computing

Intensive Courses and WorkshopsHow to Get a PhD: Methods and Practical Hints I–II (2010–2011), 3+5 credits• Research Professor Aarne Mämmelä, VTT - Doctoral

studies at the University of Oulu; literature reviews: ex-isting knowledge from data bases; fi nal result: a scien-tifi c publication; research methods: analytical approach; research methods: systems approach; history of research methods; summary of the course

• Dr. Lisa Lena Opas-Hänninen, University of Oulu - Ac-ademic writing

• Professor Riitta Keiski, University of Oulu - Good prac-tices in researcher education; research ethics

• Professor Erkki Oja, Aalto University, School of Science and Technology - Experiences of a senior researcher

• Dr. Kari Leppälä, Provisec Ltd - How to do scientifi c research on engineering and technology

• Dr. Anthony Heape, University of Oulu - Scientifi c com-munication skills: how to tell (and listen to) a story

• Professor Timo Koivumäki, VTT and University of Oulu - Quantitative questionnaire study

• Professor Olli Silvén, University of Oulu - Peer review process: the task of a referee

• Professor Jaakko Kujala, University of Oulu - Designing qualitative approach in research

• Research Professor Aarne Mämmelä, VTT and Professor Olli Silvén, University of Oulu - Discussion session

Electronics• Dr. Charles E Free, University of Surrey, UK - Introduc-

tion to microwave design techniques, 4 credits• Professor Andrea Baschirotto, University of Milano-Bi-

cocca, Milan, Italy - CMOS opamp design, 5 credits; analog fi lters for telecommunications, 5 credits


• Professor Anita Lloyd Spetz, Linköping University, Sweden - Gas sensor structures

• Professor Zuo-Guang Ye, Simon Fraser University, Canada - Relaxor frerroelectrics & related materials, 4 credits

• Professor Arthur Hill, University of Salford, UK - Cryo-technology and applications, 2 credits

• Professor Göran Salerud, Linköping University, Sweden - Biomedical optics, 4 credits 3)

• Infotech Oulu Workshop 2010 on Optoelectronics De-vices and Instrumentation X: Optical imaging and mea-surements in life sciences, 5 credits, lecturers: • Professor Chia-Liang Cheng, National Dong Hwa

University, Taiwan, - Nanodiamond: spectroscopy, surface functionalization and bio applications

• Professor Steven Jacques, Oregon Health & Science University, Portland, USA - Laser-tissue interactions

• Professor Janis Spigulis, University of Latvia - Opti-cal technologies for skin assessment (photoplethys-mography, laser-Doppler, fl uorescence, multi-spec-tral)

• Professor Lihong Wang, Washington University in St. Louis, USA - Photoacoustic tomography: Break-ing through the optical diffusion limit

• International Workshop Nanocarbon Photonics and Op-toelectronics (NPO), Koli 4)

• Dr. David Stoppa, Fondazione Bruno Kessler (FBK), Trento, Italy - Single photon geiger detectors in CMOS technologies with their applications, 2 credits

Communications Engineering • Associate Professor Elza Erkip, Polytechnic Institute

of New York University, USA and FiDiPro Professor Behnaam Aazhang, Rice University, USA - Joint source-channel coding in cooperative wireless networks, 3 cred-its

• Professor Luiz DaSilva (Stokes Professor in Telecom-munications), Trinity College Dublin, Ireland - Game theory for wireless networks

• Professor Sergei A.Nazarov, Russian Academy of Sci-ences, St. Petersburg, Russia - Computational spectral theory

• Associate Professor Emil Jovanov, The University of Alabama in Huntsville, USA - Body sensor networks, 2 credits

• Associate Professor Wendi Heinzelman, University of Rochester, USA - Recent results and current challenges in wireless sensor networks, 2 credits

• Dr. Sastri Kota, SoHum Consultants - Hybrid / integrat-ed satellite - terrestrial networks: architectures, emerg-ing standards and applications, 7 credits

• Professor Evsey Morozov - Russian Academy of Sci-ences, Russia - Stochastic modeling of communication systems (part of QoS in networks & queuing theory course, 4 + 8 credits)

Computer Science and Information Engineering• Dr. Matthew Blaschko, University of Oxford, UK -

Structured learning and applications in computer vision• Dr. Christopher Mei, University of Oxford, UK - Vision-

based simultaneous localisation and mapping• 1st International UBI Summer School 2010, 2 credits,

and 2nd Open Ubiquitous City seminar, lecturers: Pro-fessor Anind Dey, Carnegie Mellon University, USA; Professor Marcus Foth and Dr. Jaz Choi, Queensland University of Technology, Australia; Patrick Hofmann, Google Australia; Professor Vassilis Kostakos, Uni-versity of Madeira, Portugal; Jürgen Scheible, Aalto University, Finland; Zach Shelby, Head of Research, Sensinode, Finland, Professor Mikael Wiberg, Umeå University, Sweden, Adam Greenfi eld, Do projects and Professor Timo Ojala, University of Oulu 2)

Speaker panel in Q&A session with the audience in the 2nd Open Ubiquitous City Seminar. From left: Professor Anind Dey, Professor Marcus Foth, Zach Shelby, Dr. Jürgen Scheible, Professor Vassilis Kostakos, Adam Greenfi eld, Professor Mikael Wiberg, and Professor Timo Ojala as moderator.


• Professor Henning Schulzrinne, Columbia University, USA - Advanced internet services, 6 credits 6)

• PhD Student Workshop on Software Security 5), visit-ing keynote speakers: Juhani Anttila, Q&R Research Helsinki, Sokratis A. Katsikas, University of Pireaus; Professor Günther Pernul, University of Regensburg, Germany; Professor Karl Posch, Graz University of Technology, Austria; Professor Bart Preneel, Katholieke Universiteit Leuven, Belgium and Christopher C. Wills, CARIS Research Ltd and Kingston University London, UK

• Professor Nicu Sebe, University of Trento, Italy - Hu-man-centered computing: challenges and perspectives, 3 credits

Professor Nicu Sebe.

Co-operationThe following external organizations have provided co-fi -nancing or other support for the courses and workshops:• EU IST Program, Internal Security ICT Agency -

HALTIK, the Finnish Defence Forces, the Border Guard of Finland, Crisis Management Initiative - CMI 1)

• Nokia Research Center, OSKE Ubiquitous Computing Cluster Programme 2)

• International Graduate School in Biomedical Engineer-ing and Medical Physics, Graduate School of Modern Optics and Photonics 3)

• University of Eastern Finland, Graduate School of Mod-ern Optics and Photonics, OSKE Nanotechnology Clus-ter Programme, Federation of Finnish Learned Societ-ies, Ministry of Education and Science of the Russian Federation 4)

• EU IST-programme, the Finnish Defence Forces, Inter-national Federation of Information Processing Science - IFIP, Gartner Group, Nokia, TeliaSonera, Cisco 5)

• Graduate School in Electronics, Telecommunications and Automation, GETA 6)

Course InformationInformation about the courses is distributed through our web-pages and by email. The web address for the doctoral program is http://www.infotech.oulu.fi /GraduateSchool, where students can also register to the doctoral student mailing list.


INTERNATIONAL RESEARCH CO-OPERATION 2006– 2010

Full Member GroupsStanzbiegetechnik, Austria (ISG)Upper Austria University of Applied Sciences, Hagenberg, Austria (CAS)Vienna University of Technology, Austria (CAS)Magwel N.V., Belgium (CAS)State University of Campinas, Brazil (WICS)Carleton University, Ottawa, Canada (CAS)Kaben Research Inc., Ottawa, Canada (CAS)Beijing University of Post and Telecommunications, China (MTO, WICS)Bejing Jiaotong University, China (WICS)Chinese Academy of Sciences, Institute of Computing Technology, Beijing, China (MVG)Chinese Academy of Sciences, Institute of Automation, Beijing, China (MVG)Huazhong University of Science and Technology (HUST), China (ISG)Shanghai Jiaotong University, China (ISG)Shanghai Jushri Wireless Technologies, Inc, China (WICS)Shanghai Research Center for Wireless Communications, China (WICS)Southeast University, Nanjing, China (MVG, WICS)Tongji University, China (WICS)Tsinghua University, China (WICS)University of Electronic Science and Technology of China, China (WICS)University of Science and Technology Beijing, China (MVG)Wuhan University, China (MVG)Academy of Sciences of the Czech Republic, Prague, the Czech Republic (MVG)Czech Technical University, Prague, the Czech Republic (MVG)Brno University of Technology, the Czech Republic (MVG)Škoda Auto a.s., Mladá Boleslav, the Czech Republic (CAS)Aalborg University, Denmark (WICS)Swantec, Lyngby, Denmark (ISG)ELIKO, Tallinn, Estonia (ISG)Tallinn University of Technology, Estonia (ISG, MVG)Cea List, France (ISG)EADS Defence and Security Systems SAS, France (ISG)EADS Secure Networks SAS, France (ISG)École Nationale Supérieure de Techniques Avancées (ENSTA), Paris, France (WICS)EURECOM, France (MVG)INRIA, Grenoble, France (MVG)INRIA, Paris, France (MTO)INRIA, Rennes, France (ISG)Institut Eurécom, France (WICS)Institut Telecom Sudparis, France (MTO)Mandriva, France (ISG)Montpellier 2 University, France (CAS)MORPHO, France (MVG)Multimedia University, France (CAS)Technax Industrie, Genas, France (ISG)Trusted Logic, France (ISG)University of Avignon, France (MVG)Airbus Deutschland, Hamburg, Germany (ISG)Brötje Automation, Wiefelstede, Germany (ISG)Fraunhofer IFF, Magdeburg, Germany (ISG)Fraunhofer IPA, Stuttgart, Germany (ISG)Harms & Wende, Hamburg, Germany (ISG)Infi neon, Germany (CAS)Karlsruhe Institute of Technology, Germany (ISG, WICS)

Karlsruhe University of Applied Sciences, Germany (ISG)KUKA Schweißanlagen, Ausburg, Germany (ISG)LMU Munich, Germany (MTO)SICK A.G., Freiburg, Germany (CAS)Steinbeis-Europa-Zentrum, Germany (ISG)University of Cologne, Germany (CAS)University of Freiburg, Germany (MVG)University of Hannover, Germany (WICS)University of Wuppertal, Germany (CAS)National Technical University of Athens, Greece (CAS)Gamax, Budapest, Hungary (ISG)University of Debrecen, Hungary (MTO)Center for Science, Society and Citizenship, Italy (MVG)Centro Ricerche Fiat, Torino, Italy (ISG)Italtel, Italy (ISG)Politecnico di Torino, Italy (MVG, WICS)Università della Calabria, Italy (WICS)University of Bologna, Italy (WICS)University of Cagliari, Italy (MVG)University of Florence, Italy (WICS)University of Genoa, Italy (ISG)University of Rome (Tor Vergata), Italy (WICS)Hiroshima University, Japan (WICS)Japan Advanced Institute of Science and Technology, Japan (WICS)Keio University, Tokyo, Japan (ISG)Nagoya University, Japan (ISG)Osaka University, Japan (WICS)Tokyo Denki University, Japan (ISG, MTO)Tokyo University of Agriculture and Technology, Japan (ISG)University of Tokyo, Japan (MVG)Waseda University, Tokyo, Japan (ISG, MTO)Yokohama National University, Japan (WICS)Public Research Centre Henri Tudor, Luxembourg (ISG, MTO)AWL-Techniek, Harderwijk, the Netherlands (ISG)ERCOM, the Netherlands (ISG)Gemalto, the Netherlands (ISG)NXP Semiconductors B.V., the Netherlands (CAS)University of Auckland, New Zealand (MVG)Polish Welding Centre of Excellence, Gliwice, Poland (ISG)Critical Software, Coimbra, Portugal (ISG)IdMind, Lisboa, Portugal (ISG)University of Madeira, Portugal (MTO)University of Porto, Portugal (ISG, MVG)A.F. Ioffe Institute, St. Petersburg, Russia (CAS)The Siberian Branch of the Russian Academy of Sciences, Russia (MVG)Acotec Castilla, Boecillo, Spain (ISG)Caixa d’estalvis de Catalunya, Barcelona, Spain (ISG)Fatronik, Spain (ISG)Fundación Robotiker, Spain (ISG)GMV, Spain (ISG)Starlab Barcelona, Spain (MVG)TB-Solutions, Spain (ISG)Universidad Autonoma de Madrid, Spain (MVG)Universitat Pompeu Fabra, Barcelona, Spain (ISG)Chalmers University of Technology, Sweden (WICS)Ericsson, Sweden (ISG)Luleå University of Technology, Sweden (ISG)Malmö University, Sweden (MVG)Royal Institute of Technology, Stockholm, Sweden (ISG)Sonic Studio Interactive Institute, Piteå, Sweden (ISG)University of Lund, Sweden (CAS)Volvo Technology Corporation, Gothenburg, Sweden (CAS)


BIOMETRY.com AG, Switzerland (MVG)Ecole Polytechnique Fédérale de Lausanne, Switzerland (ISG, MVG)IdeArk, Switzerland (MVG)IDIAP Research Institute, Martigny, Switzerland (ISG, MVG)KeyLemon, Switzerland (MVG)Swiss Federal Institute of Technology Zurich (ETH), Switzerland (WICS)University of Geneva, Switzerland (ISG)EyePmedia, Switzerland (MVG)Asian Institute of Technology, Thailand (WICS)Microsoft Research Ltd., UK (ISG)National Infrastructure Security Co-ordination Centre (NISCC), UK (ISG)St. George’s University of London, UK (ISG)University of Manchester, UK (MVG)University of Oxford, UK (MVG)University of Southampton, UK (MVG)University of Surrey, UK (MVG)University of York, UK (CAS)Carnegie Mellon University, Pittsburgh, USA (ISG)Columbia University, New York, USA (ISG, MTO)Dartmouth College, Hanover, USA (ISG)Massachusetts Institute of Technology, USA (WICS)Nokia Research Center, Cambridge, USA (ISG)Purdue University, West Lafayette, USA (WICS)Rice University, Houston, USA (WICS)Stanford University, USA (WICS)University of California, Santa Barbara, USA (MVG)University of Georgia, USA (MTO)University of Maryland, USA (ISG, MVG, WICS)Virginia Tech, USA (WICS)Worcester Polytechnic Institute, USA (WICS)

Associate Member GroupsScientifi c Production Enterprise of Material Science, Armenia (EMPART)Graz University of Technology, Austria (INTERACT)Paracelsus Medical Private University, Austria (BME)Vienna University of Technology, Austria (INTERACT)Interuniversitair Microelektronica Centrum (IMEC), Belgium (EMPART)Katholieke Universiteit Leuven, Belgium (BME)Carleton University, Ottawa, Canada (EMPART)Communications Research Centre, Ottawa, Canada (EMPART)University of Waterloo, Canada (BME)Tsinghua University, China (BME, EMPART)Czech Technical University, Prague, the Czech Republic (BME)Academy of Sciences of the Czech Republic, Prague, the Czech Republic (EMPART)Aalborg University, Denmark (INTERACT)French National Center for Scientifi c Research - Laboratory for Analysis and Architecture of Systems (CNRS-LAAS), Toulouse, France (EMPART)XLIM, Limoges, France (EMPART)Fraunhofer FIT, Bonn, Germany (INTERACT)Fraunhofer ISC, Germany (EMPART)Laser-Laboratory Göttingen GmbH, Germany (EMPART)RWTH Aachen University, Germany (EMPART)Technical University of Ilmenau, Germany (EMPART)Technische Universität Darmstadt, Germany (EMPART)Institute of Communication and Computer Systems, Greece (EMPART)National Technical University of Athens, Greece (EMPART)Budapest University of Technology and Economics, Hungary (EMPART)

Debrecen University, Hungary (BME)University of Szeged, Hungary (EMPART)Council of Scientifi c and Industrial Research, Thiruvananthapuram, India (EMPART)Association for European Cardiovascular Pathology, Italy (BME)Consorzio per la Ricerca nell’ Automatica e nelle Telecomunicazioni (CRAT), Italy (BME)Istituto Nazionale per le Ricerche Cardiovascolari (INRC), Italy (BME)Klopman International, Italy (BME)Labor S.r.l., Italy (BME)STMicroelectronics S.r.l., Italy (BME)Telbios S.p.A., Italy (BME)Unità di Telemedicina Ospedale San Camillo, Italy (BME)City University of Tokyo, Japan (INTERACT)Tohoku University, Japan (BME)Kyonggi University, Seoul, Korea (EMPART)Yonsei University, Seoul, Korea (EMPART)University of Latvia, Riga, Latvia (EMPART)Vilnius University, Lithuania (EMPART)University of Twente, the Netherlands (BME)SINTEF, Norway (EMPART)Gdansk University of Technology, Poland (EMPART)Institute of Electronic Materials Technology, Poland (EMPART)Medical University of Gdansk, Poland (BME)Warsaw University of Technology, Poland (EMPART)Wroclaw University of Technology, Poland (EMPART)Aveiro University, Portugal (EMPART)Portugal Telecom Inovacao, Portugal (BME)Kurchatov institute, Moscow, Russia (EMPART)St. Petersburg Electrotechnical University, Russia (EMPART)Josef Stefan Institute, Ljubljana, Slovenia (EMPART)Blekinge University of Technology, Sweden (INTERACT)Chalmers University of Technology, Sweden (EMPART)Ericsson, Sweden (EMPART)Royal Institute of Technology, Stockholm, Sweden (INTERACT)Linköping University, Sweden (EMPART)Luleå University of Technology, Sweden (BME)Malmö University, Sweden (INTERACT)Royal Institute of Technology, Stockholm, Sweden (EMPART)Umeå University, Sweden (EMPART)Ecole Polytechnique Fédérale de Lausanne, Switzerland (EMPART)National Taiwan Normal University, Taipei, Taiwan (EMPART)National Taiwan University, Taiwan (EMPART)National United University, Taiwan (EMPART)Cranfi eld University, UK (BME)Imperial College London, UK (EMPART)University of Birmingham, UK (BME, EMPART)University of Cambridge, UK (INTERACT)University of Hull, UK (BME)University of Manchester, UK (EMPART)University of Nottingham, UK (EMPART)University of Salford, UK (EMPART)University of Southampton, UK (BME, EMPART)University of Surrey, UK (EMPART)Arizona State University, USA (EMPART)Rensselaer Polytechnic Institute, Troy, USA (EMPART)Rice University, Houston, USA (EMPART)Rutgers University, New Jersey, USA (EMPART)University of Arizona, USA (EMPART)University of California, Berkeley, USA (EMPART)University of Minnesota, USA (BME)University of Missouri - St. Louis, USA (EMPART)


INFOTECH OULU VUONNA 2010

Oulun yliopiston yksi neljästä strategisesta painoalasta on informaatioteknologia, jonka fokusalueiksi on määritelty konenäkö ja läsnä-älyn teknologiat, langaton viestintä, nopea elektroniikka ja fotoniikka, lääketieteen tekniikka, sekä tietojärjestelmät ja ohjelmistot. Infotech Oulu on Ou-lun yliopiston vuonna 1996 perustama informaatioteknii-kan tutkimuksen sateenvarjo-organisaatio. Tutkimuksen pääaloina ovat elektroniikka, tietoliikennetekniikka sekä tietojenkäsittelytiede ja -tekniikka.

Infotech Oulun tehtävänä on luoda ympäristö, jossa kan-sainvälisen tason tutkimusryhmät voivat kehittyä ja toi-mia parhaalla mahdollisella tavalla. Infotech Oulu edistää pitkäjänteistä tutkimusta, tutkijankoulutusta, kansainväli-syyttä sekä tutkimusryhmien ja eri sidosryhmien välistä yhteistyötä. Perus- ja soveltavan tutkimuksen vuorovaiku-tus nähdään tärkeäksi, mutta parhaat tulokset saadaan, kun liikkeellepanevana voimana on uteliaisuus.

Infotech Oulun tutkimusryhmät valitaan ulkopuolisen ar-vioinnin perusteella. Kansainvälisessä arvioinnissa kiin-nitetään huomiota ryhmien tieteelliseen tasoon. Vuoden 2010 alusta alkaneelle nelivuotiskaudelle valittiin viisi ryhmää täysjäseniksi ja kolme liitännäisjäseniksi. Kaksi ryhmää on Oulun yliopiston ja VTT:n yhteisryhmiä, joi-den varajohtaja on VTT:ltä.

Infotech Ouluun kuuluvat tutkimusryhmät toimivat pää-osin ulkopuolisella rahoituksella. Päärahoittajat ovat Tek-nologian ja innovaatioiden kehittämiskeskus Tekes, Suo-men Akatemia, opetusministeriö, EU ja teollisuus. Infotech Oulun täysjäsenten ulkopuolinen tutkimusrahoitus vuonna 2010 oli 11,8 miljoonaa euroa. Mukaan lukien liitännäis-jäsenet rahoitus oli 15,5 miljoonaa euroa. Laskentatavasta riippuen ulkopuolisen rahoituksen osuudeksi koko tutki-musrahoituksesta voidaan katsoa 70–80 prosenttia. Lop-puosa tuli Oulun yliopiston ja VTT:n omista budjeteista. Suurin ulkopuolinen rahoittaja oli Tekes yli kolmanneksen osuudellaan.

Vuoden 2010 lopussa täysjäseninä toimivissa tutkimus-ryhmissä työskenteli 287 henkeä. Liitännäisjäsenet mu-kaan lukien henkilömäärä oli 376. Tilastojen mukaan Infotech Oulun tutkimusryhmät ovat kansainvälistyneet entisestään. Koko henkilöstä ulkomailta oli 21,2 % ja tohtoriopiskelijoista peräti 27,8 %. Koko Infotech Oulun tutkimusryhmien ulkopuolinen tutkimusrahoitus ja hen-kilöstön määrä pieneni edellisvuodesta, koska tutkimus-ryhmien lukumäärä on yhtä pienempi kuin edelliskaudella ja osa valituista ryhmistä haki Infotechiin pienemmällä koostumuksella. Kun tämän ottaa huomioon ja tarkastelee muutosta tutkimusryhmäkohtaisesti, niin kokonaishenki-lömäärän ja myös ulkopuolisen rahoituksen suhteen on tapahtunut pientä kasvua.

Oulun yliopisto myöntää vuosittain Infotech Oululle pe-rusrahoitusta, joka vuonna 2010 oli suuruudeltaan 400 000 euroa. Suurin osa, noin 75 % rahoituksesta, käytettiin suo-

raan Infotech Oulun täysjäseninä toimivien tutkimusryh-mien tukemiseen. Loppuosaa rahoituksesta käytettiin mm. tutkijankoulutuksen ja kansainvälisen toiminnan edistä-miseen tukemalla jatko-opintokursseja, tutkijakokouksia ja -seminaareja sekä tutkijanvaihtoa. Yksittäisen ryhmän rahoitusosuuden määrää sen menestys tutkimusryhmien valinnan yhteydessä olleessa ulkopuolisessa arvioinnissa, sekä ryhmän viimeaikaiset näytöt tutkimuksen tuloksista. Tärkeimpinä kriteereinä ovat ryhmästä väitelleiden tohto-reiden määrä ja arvokkaimmat julkaisut lehtien merkittä-vyyttä kuvaavilla impact-faktoreilla mitattuina. Laadun mittarina käytetään myös julkaisujen viittausten määrää ja Suomen Akatemian vuosittaisen rahoituksen suuruutta.

Tutkimus ja tutkijankoulutus tukevat läheisesti toinen toi-siaan. Tohtorinkoulutus on oleellinen osa Infotech Oulun toimintaa. Infotech Oulun tutkijakoulu muutti nimensä tohtoriohjelmaksi uuden valtakunnallisen käytännön mu-kaisesti. Vuoden 2011 aikana Oulun yliopistolle tulee yksi tutkijakoulu ja entiset tutkijakoulut ovat sen tohtoriohjel-mia. Vuoden 2010 alusta alkaen professori Markku Juntti aloitti Infotech Oulun tohtoriohjelman johtajana. Valinnan suoritti tohtoriohjelman yleiskokous, joka koostuu tutki-musalamme kaikista Oulun yliopiston professoreista ja dosenteista.

Opetus- ja kulttuuriministeriön rahoittamat 25 tutkija -opiskelija paikkaa ja koordinaattorin paikka ovat merkit-tävä resurssi Infotech Oulun tohtoriohjelmalle ja koko yliopistolle alan pitkäjänteisen tutkimuksen kannalta. Toh-toriohjelma järjestää vierailevien kansainvälisten huip-pututkijoiden pitämiä luentoja ja intensiivikursseja. Sen toimintaa tehostaa Suomen Akatemialta saatava rahoitus opetustarjonnan järjestämiseksi ja muihin menoihin. Vuo-sille 2010–2013 tämä tuki on yhteensä 430 000 euroa, josta vuoden 2010 osuus oli 107 500 euroa. Vuoden 2010 lopussa koulussa oli lähes 200 jatko-opiskelijaa, jotka työskentelevät Infotech Oulun tutkimusryhmissä. Vuonna 2010 Infotech Oulun piiristä väitteli 17 jatko-opiskelijaa, joista kahdeksan sai tutkijakoulun rahoitusta.

Tieteelliset tulokset ovat parantuneet vuosi vuodelta, kun mittarina pidetään parhaiden julkaisujen impact-faktoreita ja viittausten määrää. Tekemämme sitaatioanalyysi osoit-taa, että eniten viitatut professorimme eivät jää jälkeen ICT-alan Suomen Akatemian huippuyksiköiksi valittujen tutkimusryhmien johtajista ja akatemiaprofessoreista, kun lasketaan viittauksia vuosina 2003–2008 tehtyihin julkai-suihin.

Kaksi tutkimusryhmiemme johtajaa aloitti työnsä tärkeil-lä paikoilla vuoden 2010 alussa. Professori Heli Jantunen valittiin Suomen Akatemian luonnontieteiden ja tekniikan tutkimuksen toimikunnan jäseneksi ja professori Olli Sil-vén Oulun yliopiston opetuksesta vastaavaksi vararehtorik-si. Heidän lisäkseen professori Juha Kostamovaara jatkaa Suomen Akatemian nimittämänä akatemiaprofessorina.

table of contents - infotech oulu

Documents