research opportunities - interactive visual representations, otto j. anshus, university of tromsø

Research Opportuni0es

Interac0ve Visual Representa0ons

INREMO Resource NetworkInterac0ve Representa0ons and Models

Anders Kluge, Intermedia, UiOOAo J. Anshus, IFI, UiT

inremo.no1

1Wednesday, 2.May, 2012

Research Opportuni0es

Interac0ve Visual Representa0ons

INREMO Resource NetworkInterac0ve Representa0ons and Models

Anders Kluge, Intermedia, UiOOAo J. Anshus, IFI, UiT

inremo.no2


INREMO is a Resource NetworkInterac(ve Representa(ons & Models

• Started in 2011• Goal

– Fostering and support of research & innova(on in design and use of interac0ve representa(ons and models with the purpose to aid understanding of complex phenomena and ideas

• Objec(ves– Spread results from the projects run by the partners– U(lize exis(ng contacts & networks for seminars and other relevant

ac(vi(es– Create new projects with new composi(ons of partners– Advice RCN (NFR) and EU


INREMO Partners• 15 partners, combining basic and applied research, and use of

the research: – inremo.no/partners/

• University & colleges (6)• Applied research (3)• Industry (4)• Museum (1)• NORSIGD – Norwegian Associa(on for Computer Graphics

• Steering group– Anders Kluge (PI/PM), InterMedia, University of Oslo– Wolfgang Leister, Norwegian Compu(ng Centre (Norsk Regnesentral)– Dag Andreassen, Norwegian Museum of Science, Technology and

Medicine (Norsk Teknisk Museum)– O]o Anshus, IFI, University of Tromsø


INREMO Partners• Aschehoug & Co. (independent

publishing house)

• Dept. of Media and Communication, UiO

• Dept. of Informatics, UiO

• Dept. of Computer Science, UiT• Dept. of Computer and Information

Science, NTNU

• Diiz (creative media production studio)

• Expology AS (consultancy services and solutions within communication, learning and experience)

• InterMedia, UiO (multidisciplinary research center)

• Kalkulo AS (technical software based on mathematics, geometry and visualization)

• Norwegian Association for Computer Graphics (NORSIGD)

• Norwegian Center for Integrated Care and Telemedicine (NST - research centre on telemedicine services)

• Norwegian Computing Center (contract research and development projects)

• Norwegian Museum of Science and Technology

• Oslo School of Architecture and Design

• Vestlandsforskning (assignment-based research institute)

5


INREMO Partner Projects• INVENTIO 2006-‐2011 (VERDIKT)• CONTACT 2009-‐2013 (VERDIKT)• YOUrban 2010-‐2013 (VERDIKT)• MIRACLE 2010-‐2014 (VERDIKT)• RHYME 2010-‐2015 (VERDIKT)• E-‐Me: 2010-‐2013 (VERDIKT)• SCY 2008-‐2012(IP EU)• KP-‐Lab 2007-‐2011 (IP EU)• Ship Bridge Design (MAROFF)• SFI in Service Innova(ons• SFI in Telemedicine – Tromsø Telemedicine Laboratory (TTL)• SHARE 2004-‐2008 (IKT-‐2010)• Display Wall with Compute Cluster 2004/2011 (RCN AVS, Tromsø Forskningss(gelse)• Verdione 2008/09-‐2012/13 (Stor IKT)• Several smaller projects

INREMO broad experience, can do both • design and implementa(on of real systems for

interac(ve digital representa(ons and models• apply them for conceptual understanding and

interpreta(on of phenomena and ideas


Relevance of Interac(ve Visual Representa(ons• Combining interac(ve models and visualiza(ons aid humans in both

finding facts about and understanding of an increasingly more complex and complicated world

• To understand complex rela(onships more is needed than text (editor), images (viewer, cameras) and video (media player, cameras)– Interac(ve Models

• Climate, Weather, Tsunamis, Heat pump, Nuclear Power Plant, Economy• A city and new buildings, The Earth

– Interac(ve Visualiza0ons• ”Storytelling”, where a user is taken through pre-‐defined ways of understanding the data • Dynamic visualiza(ons, where humans can steer on-‐demand the visualiza(on allowing for

an efficient and flexible explora(on of the data sets and the models

– Real Systems • Create and process models• On-‐demand fetching and processing of local and remote data sets (including streaming

data from models and images and video)• Interac(ve Visualiza(on and Human-‐Computer Interac(on


• Execu(on of models (simula(ons) is included in many parts of the EU work program for ICT -‐ “ICT Research Driver” for many fields and sciences

• Environmental challenges, robo(cs, produc(on, cultural reconstruc(ons, augmented, mixed, virtual reali(es, system simula(ons, teaching, biomedicine, pa(ent security, anatomy, global collabora(on, energy research, climate research, universal access, modeling of poli(cs, nano technology

Relevance of Interac(ve Visual Representa(ons

Gene0cs: Microarray Explora0on & Visualiza0on tool at Princeton University

Developed in coopera0on with an INREMO partner (IFI, UiT)


Performance

Number

iPhone

Desktop

TV (Android, iOS)

Laptop

Display Wall

Tablet

Sensors

Tabletop

• The size of accumulated data doubles every year– 2012: All stored data adds up to about 2-‐3ZB– Big Data sets

• Big Volume, Big Velocity, Big Variety – Small Data sets

• Big Number of small data sets from sensors, phones, tablets, lap and desktops

• Many devices with a rich set of characteris(cs

Observa(ons1 Ze]aB1000 ExaB1Million PetaB1Billion TeraB


Opinions by INREMO Partners

• Sogware should help humans to go from fact finding to explana(ons and insights: understanding

• Computers can be used much more than they presently are in storytelling and explora(on of data sets on cultural heritage, art and architecture

• InReMo: An interes(ng and relevant topic in the overlap between technology and learning for all, e-‐Inclusion


Opinions by INREMO Partners• To use computers in educa0on to help understanding

– Realis(c use of computers hard, but huge untapped poten(al– Need new content, not only re-‐packaging of old

• Need more collabora(on computer science-‐physics-‐mathema(cs

• We observe that Use implies that users Create and Modify – Pupils try different parameters and approaches to explore a model

– They learn how to create and do computa(ons, not only use ready apps

– Do-‐first (Project-‐first), worry about abstrac(ons later

11


12

Intermedia, UiOExplore and Understand the Climatewww.intermedia.uio.no/elevprodukt/Elevprodukt.swf


http://www.intermedia.uio.no/elevprodukt/Elevprodukt.swf







Size can be important

Also the case when going in the other direc(on from laptop to tablet to iPhone



Resolu(on can be important

12 pixels

~10x pixels

~100x pixels

~1000x pixels


Applica(on Domain Challenges• Users can’t easily develop and produce high-‐quality visualiza(ons

because of data complexity & size, and tools complexi(es– Need to research beAer interfaces and plaWorms for the development of

models, and their interac0ve visualiza0on

• Hard to tell a story and do efficient visualiza(ons– Need to research how to apply models and visualiza0ons to convey and

aid interac0ve experiences, learning, e-‐Inclusion • 3D visualiza(ons, storytelling, interac(vity, exaggerate behavior (higher

mountains, stretch & compress (me, change physics)

• User interfaces giving increased usability & accessibility, leading to possibly new usage domains, and synergy & redundancy increasing the bandwidth into human beings for informa(on transfer

• How to increase user par(cipa(on in the modeling vs. using what experts in the field have prepared


How to Design a CO2 Neutral HouseExploratory – Experimental – High Complexity

INTERMEDIA, UiO


Explora(on of the Weather SystemExploratory – Experimental – High Complexity

Recipe• Model of the weather system, and a model of the Earth and how it moves in space• Geo spa(al data set over Earth (locally stored for performance)• Computed Meteorological data set of latest weather forecast from met.no (locally stored for performance)• Sound & Gesture-‐based HCI support (cameras locate arms, microphones locate sound)• A display side (interac(vely fast display earth in high-‐res)• A compute side (integrate data sets, compute local high-‐res forecast within 2 minutes on-‐demand interac(vely

steered by user)• Add a user, and voila, for business and play, the weather can be forecasted and explored to increase

understanding of the weather system

hpds.cs.uit.no


Applica(on Domain Challenges• Iden(fy and experiment with new genres

– Games and lessons learned with regards to genre, architectures and algorithms applied

– If you can build a game, can you build an informa(on or scien(fic visualiza(on?

– Social media and representa(ons – space of possibili(es

18

hpds.cs.uit.no


Applica(on Domain Challenges• Remote Collabora0on & Presence

– Huge poten(al (learning, traveling, efficient collabora(on, exploring)– Possibly synchronized close enough in (me into a coherent virtual

stage suppor(ng “distributed performances”

19


Applica(on Domain Challenges

• Mediated reality– Manipulate the perception of reality, a visual filter

between the real world and what the user perceives– Mixed reality (augmented reality, augmented virtuality)• Physical and digital objects co-exist and interact in

real time– Virtual reality (including situated VR)• Replaces the real world with a simulated one

20


Augmented Reality Weather Forecasts

21

Point the tablet somewhere to view the landscape or map through the tablet’s camera An on-‐demand weather forecast of the relevant small region is computed interac(vely fast The forecast is visualized and visually integrated with the output from the camera Ques(ons

To do the System: How to (i) to compute a fast forecast? (ii) support HCI? (iii) support fast weather visualiza(ons? (iv) scale with number of users and size of region? To support Understanding: (i) Which weather system dimensions should the user control? (ii) Which HCI and visualiza(ons will be effec(ve for understanding?

hpds.cs.uit.no


Systems Domain Challenges

• Need interac(vely fast model processing and visualiza(ons in both low and high resolu(on, at high frame rates, with on-‐the fly integra(on of model output with data sets

• Infrastructures & their architecture, design & implementa0on– Support for diverse plaWorms (smart phones, tablets, laptops, desktops,

free-‐standing displays, (led displays) u(lizing embedded sensors (GPS, camera, accelerometers, compass, future sensors)

– Should scale with regards to number of platorms, number of clients, characteris(cs of clients => Solu(ons will primarily be Distributed and Parallel

– Robust, simple, efficient: We want it all to “just work”, every (me, all the (me

• Easier said than done

22


Systems Domain Challenges• How to build systems suppor(ng & enabling the applica(on domain

– Data sets (latency, bandwidth, high complexity and size, many data sets, local and remote storage)

– Compute resources (on-‐demand processing of data sets and models, response (mes)

– Visualiza0on resources (high frame rates, low and high resolu(on, placeshiging output)

– Mul0modal input & output beyond keyboard and mice• Placeshiging input (event handler)• Gestures (gaze, head, body) • Sound & speech• Tac(le feedback, olfac(on

Tromsø Touch Table (T3)


Gestures for Efficient & Flexible Visualiza(onhpds.cs.uit.no


Conclusions

• Hard to know what will happen in science, technology, and society, but we do know that:– Data sets grow exponen(ally and faster than we can analyze

them: they represent important informa(on– Achieving beAer and more efficient understanding of a complex

world is important for all human endeavor– Flexible interac(ve models and visualiza(ons are cri(cal tools to

achieve this– Norway needs innova(on -‐ To do so we need people who can

build systems and who can apply them


Conclusions• Interac(ve representa(ons and models using many big and small data sets are increasingly ...– means we learn by– forming our thoughts – influen0al on societal level– built by the users

• We recommend funding projects able to both ...– Apply models and interac(ve visualiza(ons– Build the necessary infrastructure systems and tools


research opportunities - interactive visual representations, otto j. anshus, university of tromsø

Technology

applied research

computer science

support of research

inremo partners aschehoug

interacve models

verdikt contact

verdikt rhyme

verdikt yourban