1
1
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Integration of Emerging Inspection Deviceswith
Rapid Manufacturing SystemsAnath Fischer
Laboratory for CAD & Life Cycle EngineeringFaculty of Mechanical Engineering
Technion - Israel Institute of Technology, Haifa, Israel
CIRP-2007-SPCIES-Production System EvolutionDresden, Germany, August, 2007
Integration of Emerging Inspection Deviceswith
Rapid Manufacturing SystemsAnath Fischer
Laboratory for CAD & Life Cycle EngineeringFaculty of Mechanical Engineering
Technion - Israel Institute of Technology, Haifa, Israel
CIRP-2007-SPCIES-Production System EvolutionDresden, Germany, August, 2007
2
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Industry Globalization ChallengesIndustry Globalization Challenges
• Globalization has forced industries to operate in a highly competitive environment.
• The challenge is to increase the volume and variety of global transactions: products, services, knowledge.
• Technological developments are need to improve production systems.
• Globalization has forced industries to operate in a highly competitive environment.
• The challenge is to increase the volume and variety of global transactions: products, services, knowledge.
• Technological developments are need to improve production systems.
2
3
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Current Status of Digital InspectionCurrent Status of Digital Inspection
• At the end of the manufacturing process, a produced part should be verified to determine whether it fits the CAD model prototype under a given tolerances.
• The importance of automatic inspection for manufacturing is growing.
• Current production demands fully automatic, fast and accurate inspection.
• At the end of the manufacturing process, a produced part should be verified to determine whether it fits the CAD model prototype under a given tolerances.
• The importance of automatic inspection for manufacturing is growing.
• Current production demands fully automatic, fast and accurate inspection.
4
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Scan data problemsScan data problems
Physical Physical
ObjectObject
Scanned model: Scanned model:
large‐scale, noisy, unorganized, missing
information on properties of the sampled object
Scanning
3
5
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Traditional Inspection Methodsvs. Emerging scanning technologies
Traditional Inspection Methodsvs. Emerging scanning technologies
• Existing inspection computational methods do not take diverse scan information into account.
• Therefore, traditional inspection methods are limited and do not satisfy industry demands.
• Emerging scanning technologies are capable ofcapturing diverse data regarding geometrical and physical properties of the sampled object, such as surface normals, color, material.
• Existing inspection computational methods do not take diverse scan information into account.
• Therefore, traditional inspection methods are limited and do not satisfy industry demands.
• Emerging scanning technologies are capable ofcapturing diverse data regarding geometrical and physical properties of the sampled object, such as surface normals, color, material.
6
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Inspection technologies andcomputational methods
Inspection technologies andcomputational methods
• Inspection technologies and computational methods are embedded and strongly interconnected components of a single process.
• Utilizing the synergy between them should have a significant impact on global inspection by accelerating digital geometric processing and improving reconstructed object accuracy.
• Inspection technologies and computational methods are embedded and strongly interconnected components of a single process.
• Utilizing the synergy between them should have a significant impact on global inspection by accelerating digital geometric processing and improving reconstructed object accuracy.
4
7
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
The GoalThe Goal
Developing a new approach for
inherently handling diverse data
and
utilizing scanning technologies
for inspection.
Developing a new approach for
inherently handling diverse data
and
utilizing scanning technologies
for inspection.
8
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Utilizing the Diverse Data for Inspection: The Approach
Utilizing the Diverse Data for Inspection: The Approach
Scanning process
Cloud of points
Analysis of scanned points
Calculation of normals
Sharp feature detection
Segment identification
Diverse scan data
Surface normals
Sharp feature knowledge
Segment information
Color
Material
Inspection
Data verification and HSDM reconstruction
Scan data filtering
5
9
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
3D Non-Contact Measurement Technology
3D Non-Contact Measurement Technology
• Recent advances in computer vision and optics have led to development of emerging non-contact scanning devices.
• Non-contact scanners are capable of sampling a very dense cloud of points from a part’s surface in seconds.
• The following emerging non-contact technologies are described:
o Laser scannerso Lasers based on Conoscopic Holography technologyo 3D cameras
10
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
3D Non-Contact Technology:Laser Scanners
3D Non-Contact Technology:Laser Scanners
• Laser scanner uses stereoscopic technique(Cyberware, Nextech).
• The distance of a sampled point is computed by means of a directional light source and a video camera.
• The CCD camera’s 2D array captures the image of the surface profile along the laser sampling.
• The disadvantage is that the camera collects a small percentage of the reflected energy and therefore additional processing is needed.
6
11
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
NEXTEC- Digital CMMbased on laser scanning technology
NEXTEC- Digital CMMbased on laser scanning technology
ProductLaser 3D scanning for quality control, inspection and reverse engineering applications.
Specifications• Automatic Alignment Time: 50 sec• Total Machine Accuracy: 14 µm (2 sigma) • Data capture rate: 50 points per sec
Advantages• Fully compatible with traditional CMM equipment • Highly reliable measurements• High flexibility
ProductLaser 3D scanning for quality control, inspection and reverse engineering applications.
Specifications• Automatic Alignment Time: 50 sec• Total Machine Accuracy: 14 µm (2 sigma) • Data capture rate: 50 points per sec
Advantages• Fully compatible with traditional CMM equipment • Highly reliable measurements• High flexibility
12
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
NEXTEC- Digital CMMadapts globalization strategy
NEXTEC- Digital CMMadapts globalization strategy
Nextec- Laser scanner
CMMs- Coord3, Brown&Sharp, Mitutoyo
Blades Technology -Pratt & Whitney and Rolls Royce
7
13
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
NEXTEC- Digital CMMNEXTEC- Digital CMM
Product Laser 3D scanning for quality control and reverse engineering applications.Typical Applications
• Injection molding• Stamping• Inspection & RE • Dimensional verification
Product Laser 3D scanning for quality control and reverse engineering applications.Typical Applications
• Injection molding• Stamping• Inspection & RE • Dimensional verification
14
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
3D Non-Contact Technology:Conoscopic Holography Technology
3D Non-Contact Technology:Conoscopic Holography Technology
• Conoscopic holography is a polarized interference process based on crystal optics.
• The measurement process retrieves the distance of the light sampled point from a fixed reference plane.
• Advantages:•Accuracy measurement is in microns.•CH can handle large measurement angles.•CH measures metals with high accuracy.
• Disadvantage:•CH process is slow (minutes) with respect to 3D cameras.
8
15
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
OPTIMET- Laser ScannerOPTIMET- Laser Scanner
ProductHigh precision , 3-D measurement sensors, based on conoscopic holography technology
Specifications • Accuracy absolute : 2 - 35 µm• Angle of reading up to 85 degrees• Reading speed: 800 points per sec
Advantages• Very high accuracy• Measurement from extremely acute angles• Robust to variety of surface properties
ProductHigh precision , 3-D measurement sensors, based on conoscopic holography technology
Specifications • Accuracy absolute : 2 - 35 µm• Angle of reading up to 85 degrees• Reading speed: 800 points per sec
Advantages• Very high accuracy• Measurement from extremely acute angles• Robust to variety of surface properties
16
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
OPTIMET- Laser ScannerOPTIMET- Laser Scanner
ProductHigh precision , 3-D measurement sensors, based on conoscopic holography technology
Typical Applications• Quality control• Jewelry• Laser drillings • In-process inspection
ProductHigh precision , 3-D measurement sensors, based on conoscopic holography technology
Typical Applications• Quality control• Jewelry• Laser drillings • In-process inspection
9
17
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
3D Non-Contact Technology:3D Cameras
3D Non-Contact Technology:3D Cameras
• 3D photography is based on reconstructing 3D data from 2D images, taken from different points of view.
• Matching problem: for a given sampled point, its projected point should be identified in each image.
• To this end, a pattern is projected on the part where the same pattern points are identified in each image (Cognitens).
• The advantage is that a large number of measurements can be produced in one shot.
18
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
COGNITENS- 3D CameraCOGNITENS- 3D Camera
Product 3D camera for fast measurement of manufactured parts in plant floor environmentSpecifications
• Image acquisition: <0.001 sec• 3x1.3 mega pixel CCD cameras• Overall system accuracy: 100µm (3 sigma)
Typical Applications• automotive industry• aerospace industry
Advantages• Very high speed of measurement• Wide area measurement at one shot
Product 3D camera for fast measurement of manufactured parts in plant floor environmentSpecifications
• Image acquisition: <0.001 sec• 3x1.3 mega pixel CCD cameras• Overall system accuracy: 100µm (3 sigma)
Typical Applications• automotive industry• aerospace industry
Advantages• Very high speed of measurement• Wide area measurement at one shot
10
19
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Distribution of 3D scanning Technologies-Universities & Research Institutes
Distribution of 3D scanning Technologies-Universities & Research Institutes
Data acquisition technologies
72.73%
63.64%
27.27%
45.45%
9.09%
18.18%
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
CMM LaserScanners
2D Cameras 3D Cameras CT/MRIScanners
Other
Labs [%]
20
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Distribution of 3D Scanning Technologies- Hi-Tech Industry
Distribution of 3D Scanning Technologies- Hi-Tech Industry
11
21
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Distribution of Diverse Data Utilization-Hi-Tech Industry
Distribution of Diverse Data Utilization-Hi-Tech Industry
22
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Traditional Shape Retrieval MethodsTraditional Shape Retrieval Methods
• Most commercial methods that deal with shape retrieval do not utilize emerging technologies.
• They use only the 3D coordinates of the sampled object, while ignoring other diverse data provided by non-contact technology.
• As a result the digital geometric process becomes very difficult because it must extract implicit or missing information from the 3D coordinates.
• Moreover, it retrieves the object shape without preserving sharp features and smoothness, or detecting a functional surface.
12
23
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Shape Retrieval ProblemsShape Retrieval Problems
• Non-detection of sharp features
• Reconstructing inefficient mesh
• Ignoring mesh physical-based properties
24
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Using Diverse Scan DataUsing Diverse Scan Data
• Potential of emerging scan technologies is not utilized. • New methods needed to provide synergy between RT developers and
users.
Information on sharp features of the sampled objects
Surface normals associated with sampled points
Surface curvature associated with sampled points
Coordinatesof sampled points
Coordinates of sampled points
RT usersInspection developers
13
25
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
The GoalThe Goal
Developing new reconstruction methodsthat
utilizes diverse scan data
Developing new reconstruction methodsthat
utilizes diverse scan data
26
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Utilizing the Diverse Data for Inspection: The Approach
Utilizing the Diverse Data for Inspection: The Approach
Scanning process
Cloud of points
Analysis of scanned points
Calculation of normals
Sharp feature detection
Segment identification
Diverse scan data
Surface normals
Sharp feature knowledge
Segment information
Color
Material
Inspection
Data verification and HSDM reconstruction
Scan data filtering
14
27
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Utilizing the Diverse Data for Inspection: The Grid-based Approach (2)
Utilizing the Diverse Data for Inspection: The Grid-based Approach (2)
Diverse scan data
Classify scan pointsto voxels
Inspection process and shape retrieval
For each voxel set of points
Apply 3D GBF filterto calculate
representative surface point
New reduced points
28
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
2D Bilateral Filter2D Bilateral Filter
[ ] [ ]( , ) ( , )1n c n
nc f d n c g J J JJ
kδ
∈ Ω
⋅ ⋅= ∑
cc
n
15
29
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Stage 2: Applying3D Geometric Bilateral Filter (3D GBF)
for Data Fusion
Stage 2: Applying3D Geometric Bilateral Filter (3D GBF)
for Data Fusion
• 3D GBF calculates Representative Surface Point (RSP), which replaces the voxel set of points.
• The proposed 3D GBF is based on 2D bilateral filter technique.
• 3D GBF belongs to edge-preserving filters.
• 3D GBF utilizes diverse scan data and surface normal information to prevent smoothing across edges.
30
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
3D Geometric Bilateral Filter(3D GBF)
3D Geometric Bilateral Filter(3D GBF)
( , ) ( , ) ( , ) ( , )1i j i j
j ij j ij j ij j ij j ij jp V p Vj
s f d p c g Lp Ls p k f d p c g Lp Lsk
δ δ∈ ∈
⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤⋅ ⋅ = ⋅⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦= ∑ ∑
sj – RSP point of voxel Vj.
f [ ] – closeness weight function.
g [ ] – similarity weight function.
d(pij , cj) – similarity of positions (Euclidian distance) between point pijand centroid cj .
δ(Lpij ,Lsj) – similarity of property L between points pij and sj .
16
31
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
3D Geometric Multilateral Filter(3D GMF)
3D Geometric Multilateral Filter(3D GMF)
1 1 2 2
1 1 2 2
( , ) ( , ) ( , ) ( , )
( , ) ( , ) ( , ) ( , )
1i j
i j
n nj ij j ij j ij j ij j ij
p Vj
n nj ij j ij j ij j ij j
p V
s f d p c g Lp Ls g L p Ls g L p Ls p
k f d p c g Lp Ls g L p Ls g L p Ls
kδ δ δ
δ δ δ
∈
∈
⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎡ ⎤⋅ ⋅⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦
⎡ ⎤ ⎡ ⎤ ⎡ ⎤⎡ ⎤= ⋅ ⋅⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦
= ∑
∑
…
…
δ(Lnpij ,Lnsj) – similarity of property Ln between points pij and sj .
3D GBF can be extended to use several types of diverse data.
Several properties L1, L2, …, Ln can be involved in the filtering process
32
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
The proposed 3D GMF approach advantages
The proposed 3D GMF approach advantages
• Removes noise
• Preserving the underlying sampled surface including fine details and sharp features
• Reduces data, leading to further fast data processing or surface reconstruction
• Is robust, can handle complex topology
• Is fast
• Is simple to implement
17
Diverse data based methodDiverse data based method
ExamplesExamples
34
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
HSDM ReconstructionHSDM Reconstruction
18
35
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Neural-Network Grid-based Re-meshingfor micro-MRI Technology
Neural-Network Grid-based Re-meshingfor micro-MRI Technology
Rapid Prototyping models of bone micro-structure: (a) specimen from lumbar spine; (b) specimen from femoral bone
Micro-structures re-meshing using Neural Network: (a) Original mesh with 47%
failure -angles test;
(b) Re-meshed micro-structure results with only 22% failure in the same test
36
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Complexity analysisComplexity analysis
O(n)O(n)Mesh generation
O(n)O(dn)Connectivity graph construction
O(n)O(n)Scan data filtering
O(dn)O(dn)HSDM construction
SpaceTimeStage
19
37
TECHNION-Institute for Science and technology
Laboratory forCAD & LCE
Summary & ConclusionsSummary & Conclusions
• The non-contact scanning technology can significantly improve automatic inspection in real time for rapid manufacturing (RM).
• Therefore, this technology enhances globalization.
• Hi-tech industries and RM use non-contact laser scanners, but do not utilize diverse data.
• A complete system that consists of non-contact technology and a robust surface reconstruction method is needed.
• In this talk we described diverse data based reconstruction methods that can be well integrated, in real time, with the non-contact technology.
Thank youThank you