structuring user involvement dimitri schuurman et al summer school research day
DESCRIPTION
3rd ENoLL Living Lab Summer SchoolTRANSCRIPT
10/04/2023 1
Structuring User Involvement in ICT-Innovation: a Panel-based Living Lab-approach
Dimitri Schuurman
Bram Lievens
Lieven De Marez
Pieter Ballon
10/04/2023 2
Overview & Methodology
Literature research Analysis of 9 Living Lab-conceptualizations Construction of modified Living Lab definition Assess the implications of panel-based approach Illustration
Insight into the differences and similarities
between conceptualization and actual
practice in Living Labs
3
Evolution of Innovation ManagementOrtt & van der Duin (2008) Technology push: +/-‘60s Market pull: +/- ‘70s
No user needs vs. incremental flood
Interactionist approach: +/- ‘80s Combining both, still in-house
Open innovation: +/- ‘90s- ‘00s More open process Cooperation & interaction
Contextual innovation: now Approach depends on contextual factors More cyclical & non-linear approach ‘Innofusion’ & ‘social learning’ usage!
10/04/2023 4
Evolution of Living Labs
Concepts/methods related to Living Labs Vision on innovation management
Houses of the Future, demohomes Technology or science push
Ethnographic/observational methods Market or need pull
American Living Labs Interactionism
European Living Labs Open Innovation
Panel based Living Lab-approach Contextual innovation
Variety of practices under LL-umbrella: need for clearer conceptualization
10/04/2023 5
Conceptualizing from practice
Living Labs as Test and Experimentation Platform Commercial maturity lower than in market & societal pilots Focus less on technical testing than in field trials & testbeds Living Labs as open innovation platforms
Ballon et al., 2007
10/04/2023 6
Conceptualizing from practice (2)
Research phase Actions
contextualizationan exploration of the technological and social implications of the technology or service under
investigation; technological scan and state-of-the-art study
selectionidentifying potential users or user groups; this can be done on a socio-demographic level, based on selective or criterion sampling, allowance for theoretical variation of previously defined concepts
concretization an initial measurement of the selected users on current characteristics, behavior and perceptions regarding the research focus, in order to enable a post-measurement
implementationthe operationally running test phase of the Living Lab; research methods: direct analysis of usage by means of remote data collection techniques (e.g. logging), indirect analysis based on e.g. focus groups, interviews, self-reporting techniques…
feedbackan ex-post-measurement of the users (same techniques of initial measurement) and a set of technological recommendations from the analysis of data gathered during the implementation-
phase, which makes it possible to assess the added-value
Pierson & Lievens (2005), re-used by Shamsi (2008)
10/04/2023 7
Conceptualizing from practice (3)9 general ICT Living Lab-characteristics by Følstad (2008) – bottom-up approach analyzing 32 Living Labs-papers
1 = Research into the usage context; 2 = Discover unexpected ICT-uses and new service opportunities; 3 = Co-creation with the users; 4 = Evaluation of new ICT-solutions by users; 5 = Technical testing of the innovation in a realistic context; 6 = Familiar usage context for the users; 7 = Experience and experiment in a real-world context; 8 = Medium- or long-term user studies; 9 = Large scale user studies.
10/04/2023 8
9 general ICT Living Lab-characteristics by Følstad (2008) – bottom-up approach analyzing 32 Living Labs-papers
1 = Research into the usage context; 2 = Discover unexpected ICT-uses and new service opportunities; 3 = Co-creation with the users; 4 = Evaluation of new ICT-solutions by users; 5 = Technical testing of the innovation in a realistic context; 6 = Familiar usage context for the users; 7 = Experience and experment in a real-world context; 8 = Medium- or long-term user studies; 9 = Large scale user studies.
Only 4 ‘shared’ characteristics! Another indication of the conceptual ambiguity of the Living Lab-concept
Conceptualizing from practice (3)
10/04/2023 9
Analysis of LL-conceptualizations
ENoLL-related scholars 1 2 3 4 5 6 7 8 9
Frissen & van Lieshout, 2004 X X X X X
Pasman, Stappers et al., 2005 X X X X X X
Eriksson, Niitamo et al., 2006 X X X X X X X X
Ballon, Pierson et al., 2007 X X X X
Feurstein, Hesmer et al., 2008 X X X X
Ståhlbröst & Bergvall-Kåreborn, 2008 X X X X X X X X X
Almirall & Wareham, 2009 X X X X
Turkama, 2010 X X X X X X
Mahr & Schuurman, 2011 X X X X X X X X
Sum /9 4 5 9 3 2 9 9 9 4
10/04/2023 10
ENoLL-related scholars 1 2 3 4 5 6 7 8 9
Frissen & van Lieshout, 2004 X X X X X
Pasman, Stappers et al., 2005 X X X X X X
Eriksson, Niitamo et al., 2006 X X X X X X X X
Ballon, Pierson et al., 2007 X X X X
Feurstein, Hesmer et al., 2008 X X X X
Ståhlbröst & Bergvall-Kåreborn, 2008 X X X X X X X X X
Almirall & Wareham, 2009 X X X X
Turkama, 2010 X X X X X X
Mahr & Schuurman, 2011 X X X X X X X X
Sum /9 4 5 9 3 2 9 9 9 4
2 = Discover unexpected ICT-uses and new service opportunities; 4 = Evaluation of new ICT-solutions by users; 6 = Familiar usage context for the users; 8 = Medium- or long-term user studies;
3 = Co-creation with the users; 6 = Familiar usage context for the users; 7 = Experience and experiment in a real-world context; 8 = Medium- or long-term user studies;
Analysis of LL-conceptualizations
10/04/2023 11
Modified consensus definition
A Living Lab-approach consists of medium- or long-term research co-creating innovations with users in a familiar and real-world context, taking into account the ecosystem surrounding the innovation.
Missing aspect: where to get your users?
12
IBBT: Flemish (virtual) research institute, incubator and innovation intermediary for ICT, funded by Flemish government
Mission: IBBT aims to add economic and social value through excellent research and the creation of human capital in the domain of ICT
IBBT-iLab.o’s panel-based approach
13
iLab.o: IBBT’s Living Lab-division
A toolbox for any project type: ICON, Living Lab, CIP, FP7, …
Panel Management
We’ll find and motivate your test-users
Living Lab Methodology
We’ll show you how to set up a living lab project
Prototyping & testing
We’ll model a rough idea into a usable app for daily life and test it through
Simulate Your Business
Draw, discuss and simulate your value chain and business model on the fly
European Network of Living Labs
iLab.o hosts the Brussels Office for ENoLL
A toolbox for any project type: ICON, Living Lab, CIP, FP7, …
The iLab.o Living Lab-approach
Baseline measurement
•SotA Market: Environmental scan•SotA User: Current habits & practices•Selection Test Users from existing user panels•co-creation sessions
Live-phase
• Field Trials• User Research• Logging• Intermediary co-
creation sessions
Added Value assessment
• Exit and Debriefing
• Post-usage Validation
• Business Model Simulation
• Added Lab Tests (isolating Variables)
10/04/2023 15
Added value of panel based-approach 1) contextualization: through the longitudinal data the panel generates,
a permanent ‘contextualization’ is taking place for the surveyed topics
2) selection: the identification test-users is only a matter of selecting the right profiles out of the panel database. This avoids the time- and budget consuming surveying and recruiting of relevant user profiles.
3) concretization: a lot of data already present, so only a brief extra intake survey is required
4) implementation: panel members have ‘opted in’, panel management ensures practical organisation of research activities & device handling, panel manager as SPOC
5) feedback: all data added with existing panel data to further add to profile building
PANEL WITH THEMATIC FOCUS!
10/04/2023 16
Illustration: LeYLab Living Lab Sept 2010 11 industrial partners IBBT-iLab.o as
research partner Fibre internet
connection
LeYLab panel
32% course surfing35% course SNS
58% course working with computer/tablet
3% has already developed innovative apps
10% has innovative ideasregarding the Internet
20% is among the first to test innovative apps
115 fibre connections 98 households 43 tablets 36 mini PC >200 profiled panel
members
Phase 1 Phase 2 Phase 3
SotA-research: habits & practices network
problems
Initial concept CloudFriends-app
Co-creation session with experts & Lead
users
1st iteration CloudFriends-app
Test user co-creation session
2nd iteration CloudFriends-app
Post-usage validation
Project CloudFriends (home network diagnostics app)
19
Conclusions
Living Labs as promising innovation methodology, involving the end-user as key stakeholder through co-creation
Still a large variety in definitions and concrete set-ups of Living Labs
Added-value of a panel-based approach, in practice especially for entrepeneurs & start-ups
10/04/2023 21
Framework for customer characteristics
10/04/2023 22
Results of codings (N= 64)Characteristics Mean % high % low 1 = Research into the usage context 2,20 34,4% 65,6% 42 = Discover unexpected ICT-uses and new service opportunities 2,19 29,7% 70,3% 53 = Co-creation with the users 2,55 50% 50% 94 = Evaluation of new ICT-solutions by users 2,63 50% 50% 35 = Technical testing of the innovation in a realistic context 2,56 48,4% 51,6% 26 = Familiar usage context for the users 3,05 71,9% 28,1% 97 = Experience and experiment in a real-world context 2,44 54,1% 46,9% 98 = Medium- or long-term user studies 3,61 91,9% 8,1% 99 = Large scale user studies 2,36 50,8% 49,2% 4Sum /9
Co-creation with the users only in half of the sample
10/04/2023 23
Results of codings (N= 64)Characteristics Mean % high % low 1 = Research into the usage context 2,20 34,4% 65,6% 42 = Discover unexpected ICT-uses and new service opportunities 2,19 29,7% 70,3% 53 = Co-creation with the users 2,55 50% 50% 94 = Evaluation of new ICT-solutions by users 2,63 50% 50% 35 = Technical testing of the innovation in a realistic context 2,56 48,4% 51,6% 26 = Familiar usage context for the users 3,05 71,9% 28,1% 97 = Experience and experiment in a real-world context 2,44 54,1% 46,9% 98 = Medium- or long-term user studies 3,61 91,9% 8,1% 99 = Large scale user studies 2,36 50,8% 49,2% 4Sum /9
Co-creation with the users only in half of the sample Familiar usage context more often than real-world context
10/04/2023 24
Results of codings (N= 64)Characteristics Mean % high % low 1 = Research into the usage context 2,20 34,4% 65,6% 42 = Discover unexpected ICT-uses and new service opportunities 2,19 29,7% 70,3% 53 = Co-creation with the users 2,55 50% 50% 94 = Evaluation of new ICT-solutions by users 2,63 50% 50% 35 = Technical testing of the innovation in a realistic context 2,56 48,4% 51,6% 26 = Familiar usage context for the users 3,05 71,9% 28,1% 97 = Experience and experiment in a real-world context 2,44 54,1% 46,9% 98 = Medium- or long-term user studies 3,61 91,9% 8,1% 99 = Large scale user studies 2,36 50,8% 49,2% 4Sum /9
Co-creation with the users only in half of the sample Familiar usage context more often than real-world context Lack of research into the actual usage context Lack of discovery of unexpected usage or new opportunities
10/04/2023 25
Results of codings (N= 64)Characteristics Mean % high % low 1 = Research into the usage context 2,20 34,4% 65,6% 42 = Discover unexpected ICT-uses and new service opportunities 2,19 29,7% 70,3% 53 = Co-creation with the users 2,55 50% 50% 94 = Evaluation of new ICT-solutions by users 2,63 50% 50% 35 = Technical testing of the innovation in a realistic context 2,56 48,4% 51,6% 26 = Familiar usage context for the users 3,05 71,9% 28,1% 97 = Experience and experiment in a real-world context 2,44 54,1% 46,9% 98 = Medium- or long-term user studies 3,61 91,9% 8,1% 99 = Large scale user studies 2,36 50,8% 49,2% 4Sum /9
Co-creation with the users only in half of the sample Familiar usage context more often than real-world context Lack of research into the actual usage context Lack of discovery of unexpected usage or new opportunities Medium- or long term is a given, large scale is not