virtual reality and augmented reality in assembly applications · © competence centre for virtual...
TRANSCRIPT
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Prof. Dr.-Ing. Dipl.-Kfm. Christoph Runde Virtual Dimension Center (VDC) Fellbach Auberlenstr. 13 70736 Fellbach www.vdc-fellbach.de
Virtual Reality and Augmented Reality in Assembly Applications VDC Whitepaper
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Content
challenges & opportunities
in assembly
assembly line planning
product evaluation
assembly work place
ergonomics
assembly process evaluation
assembly process support
summary
Content
Fig.: Daimler
car assembly line at Daimler
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Assembly overview
in industrial production, assembly is the entirety of all processes for the
assembly of bodies with a geometrically determined shape (lengths, angles).
in addition to work preparation and parts production, assembly forms part of
the production system of an industrial company.
the essential partial operations of an assembly process are:
o connecting (screwing, nailing, welding, gluing, soldering, clipping)
o handling (grab, place, turn over, move, secure, control)
o check
o adjust (e.g. settings)
o auxiliary operations (eg cleaning, heating or cooling for press connections, deburring,
unpacking, sealing, oiling, ...)
challenges & opportunities
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Assembly Challenges
permanent market changes
need for transformation ability of
assembly systems
growing and falling lot numbers
competition leads to need for high
assembly efficiency
new products and shorter product
life cycles
mass customization, individualized
products with high variance
increasing work safety regulations
increasing age of work force
Assembly Opportunities
digital design as provider for 3D
planning and simulation data
hardware and software for 3D
simulation, 3D visualization and
virtual reality:
o new and cost-effective
solutions available
o increasing performance
o increasing acceptance
challenges & opportunities
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Assembly Planning Functional Requirements
collision checks
assembly tree for sequence planning
product variations for sequence planning
spatial assembly paths
expand and melt parts for clearance
checks
represent assembly tools
represent flexible parts
(cables, seals, hoses, …)
represent assembly aids within product
design
3D model intersections
forward and backward kinematics
space requirements for workers,
manufacturing resources, goods, stocks,
conveyors, factory building and
infrastructure
representation of arms and hands
visualization and documentation of
reaching distances
visualization and documentation of
area of visibility
link to MTM tables, to not-traceable
human operations, e.g. picking screws
ergonomic analysis
efficiency assessment
speech recognition to control mobile
AR solutions and to add new content
to them during assembly process
challenges & opportunities
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Assembly Line Planning
Fig: Fraunhofer IPA
body-in-white assembly
station planning using
virtual reality at Daimler
Use of Virtual Reality (VR)
combined layout of
o conveyors,
o robots and
oother delivery systems
robot programming, ensuring
o collision free synchronization of robots
o synchronization of robots with conveying
systems
oprocess efficiency
line production versus dock production
choice of conveying / transportation systems
space requirements of work places, scaffolds
interactive VR exploration of virtual assembly
line mock-up
Fig.: Fraunhofer IPA
Fig: Fraunhofer IPA
aircraft assembly
planning at Fairschild-
Dornier
electric stator assembly
line planning using VR at
Electrolux
line planning
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Assembly Line Planning
Fig.: Volkswagen
brown field planning of
robots (coloured) using
augmented reality
Use of Augmented Reality (AR):
graphical overlay of new, digital 3D content
(assembly systems) onto existing / brown field
factory structure
graphical overlay of new, digital 3D products
onto existing / brown field factory structure
visual check for
o collisions
o spaces for transportation systems such as
forklifts
avoiding LASER scan of factory structure
photo documentation
Bild: TU München
Fig.: Volkswagen
brown field planning of
new conveying system
(coloured) using
augmented reality
collision check of new
(digital) car body within
existing conveying line
line planning
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Product Evaluation
Fig.: OPTIS
virtual check of design
for assembly of an
aircraft interior by
exploration using a VR
powerwall
virtual check of design-for-assembly (DFA)
subjective evaluation of buildability due to
interactive exploration by experienced product
assembly expert
accessibility studies
o by vision
o by hands
o with tools
two-hand exploration is feasible
use of projection based VR,
however could lead to visual conflict
with own physical hands
user of head mounted display (HMD)
based VR and controllers or VR gloves
Fig.: ESI Group
Fig.: ESI Group
virtual check of two-
handed assembly
feasibility using a VR
powerwall
virtual check of
reachability with head-
mounted-display-based
(HMD-based) VR and two
3D controllers
product evaluation
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Work Place Ergonomics
Fig.: OPTIS
evaluation of overhead
work process ergonomics
by an anthropometric
human model, controlled
by a VR user with MoCap
processing of assembly tasks within a virtual
environment
figuring out ergonomic process properties
approaches: o interaction with virtual environment through a
virtual avatar. This avatar is controlled
- either by a motion capturing (MoCap)
system or by
- programmed animation
The avatar may deliver objective and
documented ergonomic analysis data.
o direct interaction with virtual objects, also
through a MoCap system, however the obtained
results are depending on the user (subjective)
Fig.: ESI Group
Fig.: Plavis
evaluation of reachability
and bending ergonomics
using MoCap technology
within VR
evaluating work place
ergonomics of manual
assembly stations using
digital human models
within virtual reality
workplace ergonomics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Work Place Ergonomics
Fig.: Volkswagen
check of inter space and
reachability of a manual
assembly station in a
brown field environment
using a digital human
model (colored) in
augmented reality
Use of Augmented Reality:
graphical overlay of different work
space zones, e.g.
o red area: contains all elements within a
grasping distance, thus with no need
to bend forward
o other area: out of direct grasping distance,
thus walk and/or bend forward
the AR user may walk around the work
stations and inspect all cask and tools,
if necessary correct their positions
photo documentation
workplace ergonomics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Assembly Process Evaluation
Fig.: ESI Group
Virtual evaluation of tool
use in interactive VR
Use of assembly tools:
relevant aspect of assembly analysis
examples:
opower screwdrivers
o lifting appliances
check for accessibility
check for ergonomics using tools
co-working
Fig.: ESI Group
Virtual evaluation of tool
use with 3D digital
human models in VR
process evaluation
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Assembly Process Evaluation
Fig.: Daimler AG)
Assembly path
visualization in VR
Assembly path identification:
identification of optimum assembly paths by
interactive assembly studies in VR:
recording part movement
identification of impossible paths
simple VR system: collision detection and
3D controllers (e.g. flystick, gesture
recognition)
technical approaches:
o direct interaction with virtual environment
o programming a virtual avatar
check of assembly process considering
flexible parts (push aside, stretch away
from interfering region)
Fig.: ESI Group
Fig.: ESI Group
Assembly path
generation with 3D
digital human model
Interactive exploration of
flexible parts during
assembly process
process evaluation
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Assembly Process Evaluation
Fig.: Virtalis
Haptic assembly
evaluation with force
feedback in VR
Assembly path identification:
in addition to visual and/or acoustic collision
response, haptic force feedback may be used
intuitive 6DOF directing of physical handle will
control virtual object position and orientation
haptic system blocks in case of collision
two force feedback systems may be joined to
achieve 6DOF (force and torque) feedback
assembly path generation
3D prints of additive manufacturing may be
attached to the handle to simulate realistic
grasping (grasp type and grasp point) by
human hand
Fig.: Fraunhofer IPK
Grasp type and grasp
point investigation using
a 3D print attached to a
haptic device
process evaluation
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Assembly Process Evaluation
Fig.: ESI Group
Virtual evaluation of
assembly sequencing
Assembly sequencing:
evaluating correct assembly sequencing
by variant comparison
in principle all aspect mentioned in this
presentation (line planning, product evaluation,
ergonomics) can be checked within this step
Virtual prototyping of assembly process:
for process optimization
for product optimization (e.g. assembly
support features)
standard times determination with MTM
(methods time measurement)
Fig.: Industriehansa / now Altran]
Fig.: Industriehansa / now Altran]
Guided assembly process
Identification of standard
working times for an
assembly operation by
the means of VR
process evaluation
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Process Support in Assembly
Fig.: Testia/Airbus
Assembly positions and
instruction are projected
as AR onto aircraft body
assembly assistance systems using
augmented reality
showing next process step during assembly
itself
indicating tools and parameters (e.g. torques)
applicable technologies:
oAR projection
oAR smart glasses
oAR tablet PCs and AR smartphones Fig.: BMW
Fig.: Volkswagen
Perspective through AR
smart glasses, showing
assembly instructions,
part positions, tools and
tool parameters
Disassembly instruction
as an AR overlay shown
on a tablet PC
process support
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Process Support in Assembly
assembly assistance systems: remote assistance with shared camera view
Fig.: Re’flekt
Assistance during assembly
process using remote
teleconferencing with a shared
camera view and augmented
reality support on a tablet PC
process support
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Process Support in Assembly
Fig.: Metaio
Comparison of planned
versus as-built status of
weld studs at car
assembly using AR
Quality assurance in assembly using
augmented reality
comparison of planned versus as-built status
visual check of deviations
check of installation space and positions
photo documentation
technologies tested in practice:
oAR tablet PCs and AR smartphones
ooffline AR systems (screen and camera
detached)
Fig.: Daimler
Fig.: Testia / Airbus
Check of installation
space and part positions
after assembly using AR
Quality check of
assembled parts using
AR comparison on a
tablet PC
process support
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Process Support in Assembly
Quality assurance in assembly
Fig.: Testia / Airbus
Quality check of assembled
parts: Augmented Reality
overlay shows assembled clip
as-planned and its status in
reality. The user visually
compares both and either
confirms of rejects the quality.
process support
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Summary
diverse field, many assembly applications for VR and AR
many specific VR / AR tools, thus no single solution for all
digital 3D design data is a necessity
Main benefits of VR / AR use
virtual prototyping of assembly process, optimizing and supporting
o assembly lines,
o products (DFA),
o workplace ergonomics and
o assembly process
already in a early, digital planning stage
human-integrated manufacturing engineering, considering
o human factors
o subjective assessments and tacit knowledge
summary
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Requirements on VR / AR products for assembly applications
CAD import
CAE import
collision detection
physics and sliding simulation
forward / backward kinematics
tolerance analysis
flexible components (cables, hoses, seals)
demonstrate (design-based) assembly aids
adding and melting of material for
clearance tests
priority graph for assembly order planning
present assembly sequence variations
identification of assembly paths
cuts through the 3D model
illustration of assembly tools
illustrate and document manual reach
distances
illustrate and document visible areas
illustration of arms / arm movements
anthropometric human model with
ergonomic analysis
consider space requirements for operating
equipment, products, people, materials
handling technology, building technology
links to MTM tables, for example to
detailed manual work which is hard of
impossible to track (e.g. separation of
screws)
evaluation not only of working hours,
but also of ergonomics, distances
AR engine for process support
speech recognition to easily supplement
documentation with spoken annotations
content authoring for training applications
summary
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
Literature
Deuse, J.; Busch, F.: Zeitwirtschaft in der
Montage. In: Bruno Lotter und Hans-Peter
Wiendahl (Hg.): Montage in der industriellen
Produktion. Ein Handbuch für die Praxis. 2. Aufl.
Berlin: Springer Vieweg
ESI: VR im Mittelpunkt der Produktionstechnik;
online unter: k https://www.virtual-reality-
magazin.de/vr-im-mittelpunkt-der-
produktionstechnik ; abgerufen am 13.5.2018
Heinig, M.: Nutzung von Virtuellen Technologien
für die Montageplanung von Unikaten.
Technischen Universität Hamburg-Harburg zur
Erlangung des akademischen Grades Doktor-
Ingenieur (Dr.-Ing.) genehmigte Dissertation,
Hamburg: 2015
Konold, P.; Reger, H. (2003): Praxis der
Montagetechnik. Produktdesign, Planung,
Systemgestaltung. 2., überarb. und erw. Aufl.
Wiesbaden: Vieweg
Lotter, B.; Wiendahl H.-P. (Hg.): Montage in der
industriellen Produktion. Ein Handbuch für die
Praxis. 2. Aufl. Berlin: Springer Vieweg
Patron, C.: Konzept für den Einsatz von
Augmented Reality in der Montageplanung.
Forschungsberichte IWB, Band 190, Herbert Utz
Verlag, München, 2004
Reinhart, G.; Zäh, M.F.; Patron, C.; Doil, F.; Alt,
T.; Meier, P.: Augmented Reality in der
Produktionsplanung. In: wt Werkstattstechnik
online 93 (2003), Nr. 9, S. 651-653
Roosen, G.: Virtuelle Technologien in der
Montageplanung ; online unter:
http://www.cadplace.de/SIM-VR/VR-
Produkte/Virtuelle-Technologien-in-der-
Montageplanung ; abgerufen am 13.5.2018
summary
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Content challenges & opportunities line planning product evaluation workplace ergonomics process evaluation process support summary
VDC members in VR / AR assembly applications
summary
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Thank you very much for your attention If you are interested in further details of this topic, please refer to
Virtual Dimension Center (VDC) Fellbach Auberlenstr. 13 70736 Fellbach www.vdc-fellbach.de [email protected]