nor nabilah syazana binti abdul rahmaneprints.utem.edu.my/18619/1/automatic computer vision system...
TRANSCRIPT
i
AUTOMATIC COMPUTER VISION SYSTEM FOR INDUSTRIAL PRODUCT
QUALITY INSPECTION
NOR NABILAH SYAZANA BINTI ABDUL RAHMAN
This Report Is Submitted In Partial Fulfilment Of Requirements For The Bachelor
Degree of Electronic Engineering (Industrial Electronic)
Fakulti Kejuruteraan Elektronik dan Kejuruteraan Komputer
Universiti Teknikal Malaysia Melaka
June 2016
ii
UNIVERSTI TEKNIKAL MALAYSIA MELAKA
FAKULTI KEJURUTERAAN ELEKTRONIK DAN KEJURUTERAAN KOMPUTER
BORANG PENGESAHAN STATUS LAPORAN
PROJEK SARJANA MUDA II
Tajuk Projek :
Sesi Pengajian
: 1 5 / 1 6
NOR NABILAH SYAZANA BINTI ABDUL RAHMAN Saya …………………………………………………………………………………………………..
(HURUF BESAR) mengaku membenarkan Laporan Projek Sarjana Muda ini disimpan di Perpustakaan dengan syarat-syarat kegunaan seperti berikut:
1. Laporan adalah hakmilik Universiti Teknikal Malaysia Melaka.
2. Perpustakaan dibenarkan membuat salinan untuk tujuan pengajian sahaja.
3. Perpustakaan dibenarkan membuat salinan laporan ini sebagai bahan pertukaran antara institusi
pengajian tinggi.
4. Sila tandakan ( √ ) :
SULIT*
*(Mengandungi maklumat yang berdarjah keselamatan atau
kepentingan Malaysia seperti yang termaktub di dalam AKTA
RAHSIA RASMI 1972)
TERHAD**
**(Mengandungi maklumat terhad yang telah ditentukan oleh
organisasi/badan di mana penyelidikan dijalankan)
TIDAK TERHAD
Disahkan oleh:
__ ________________________ ___________________________________
(TANDATANGAN PENULIS) (COP DAN TANDATANGAN PENYELIA)
AUTOMATIC COMPUTER VISION SYSTEM FOR INDUSTRIAL
PRODUCT QUALITY INSPECTION
iii
“I hereby declare that the work in this project is my own except for summaries and
quotations which have been duly acknowledge.”
Signature : .......................................................
Author : NOR NABILAH SYAZANA BINTI ABDUL RAHMAN
Date : ....................................................
iv
“I acknowledge that I have read this report and in my opinion this report is sufficient
in term of scope and quality for the award of Bachelor of Electronic Engineering
(Industrial Electronics) with Honours.”
Signature : .......................................................
Supervisor’s Name : NORHASHIMAH BINTI MOHD SAAD
Date : .......................................................
vi
ACKNOWLEDGEMENT
First and for most, praise to Allah S.W.T, because of His bless that giving me
the strength to complete my full project report. The special thanks go to my helpful
supervisor, Dr. Norhashimah Binti Mohd Saad for the valuable guidance and advice.
She inspired me greatly to work on this project. Her willingness to motivate me
contributed tremendously to my project for showing me some example that related to
the topic of my project. I am also very thankful to my co-supervisor Dr. Abdul Rahim
Bin Abdullah for his guidance, advice, and motivations.
I also take this opportunity to express a deep sense of gratitude to Universiti
Teknikal Malaysia Melaka for giving me a chance to undergo this Final Year Project
as partial fulfilment for my degree. Thanks again to Universiti Teknikal Malaysia
Melaka for providing me with a good environment and facilities to complete this
project.
Finally, an honourable mention goes to my families and my fellow
undergraduate for their understandings and supports in completing this project. My
sincere appreciation also extends to all my colleagues and others who have provided
assistance on various occasions. Their views and tips are useful indeed.
vii
ABSTRACT
Product quality inspection is the life business enterprises, only the objective, and strict
inspection can promise the best quality of products left the factories. Therefore, the
product quality inspection is a key job of business enterprises. Product quality
inspection becomes a major issue in production and industrial. The problem is
associated with a manual inspection that is done by a human. In facts, human as an
inspector is slower and their efficiency is affected by their states of tiredness, illness
or other human being shortcomings. This project presents an automatic computer
vision system for industrial product quality inspection as a solution from the issue that
has been raised. Soft drink is a product that is tested for quality inspection in this
system. The offline system was designed to inspect the product based on color and
water level classification using Visual Studio 2010 software. There are several image
processing technique are used which are thresholding, morphological operation and
segmentation. The system used Otsu’ method for threshold the image and quadratic
distance classifier to classify the color of the soft drink based on color classification.
The water level will read the value of a white pixel that shows the height of the bottle.
It is used to measure the range of water level. This system is also do in real-time testing
that using conveyor, sensor and webcam. The experiment result shows that the system
is 100% accurate for both offline and online system.
viii
ABSTRAK
Pemeriksaan kualiti produk adalah hayat sesuatu perusahaan atau perniagaan dan
hanya objektif yang berkaitan, dan pemeriksaan yang ketat boleh menjanjikan sesuatu
produk berkualiti setelah keluar daripada sesebuah kilang. Oleh itu, pemeriksaan
kualiti produk adalah tugas utama sesebuah perusahaan atau perniagaan. Pemeriksaan
kualiti produk menjadi isu utama dalam pengeluaran sesebuah industri. Masalah ini
berkaitan dengan pemeriksaan manual yang dilakukan oleh manusia. Fakta telah
menunjukkan bahawa manusia sebagai pemeriksa adalah lebih perlahan dan
kecekapan mereka dipengaruhi oleh keletihan, sakit atau lain-lain kelemahan. Projek
ini membentangkan sistem penglihatan secara automatik oleh komputer untuk
pemeriksaan kualiti produk industri sebagai penyelesaian daripada isu yang telah
dibangkitkan. Minuman ringan adalah produk yang telah diuji untuk pemeriksaan
kualiti dalam sistem ini. Sistem luar talian telah direka untuk memeriksa produk
berdasarkan klasifikasi warna dan paras air yang menggunakan perisisan Visual Studio
2010. Terdapat beberapa teknik pemprosesan imej yang digunakan seperti
pengambangan, operasi morfologi dan segmentasi. Sistem ini menggunakan kaedah
Otsu ' untuk ambang imej dan pengelas jarak kuadratik untuk mengelaskan warna
minuman ringan berdasarkan klasifikasi warna. Paras air akan membaca nilai piksel
putih yang menunjukkan ketinggian botol. Ia digunakan untuk mengukur julat paras
air. Sistem ini juga dilakukan dalam talian yang menggunakan penghantar, pengesan
dan kamera web. Keputusan eksperimen menunjukkan bahawa sistem ini adalah 100%
tepat untuk kedua-dua luar talian dan sistem dalam talian
ix
TABLE OF CONTENTS
CHAPTER TITLE PAGES
PROJECT TITLE i
REPORT STATUS FORM ii
DECLARATION iii
SUPERVISOR DECLARATION iv
DEDICATION v
ACKNOWLEDGEMENT vi
ABSTRACT vii
ABSTRAK viii
TABLE OF CONTENTS ix
LIST OF TABLES xii
LIST OF FIGURES xiii
LIST OF ABBREVIATION xv
LIST OF APPENDIX xvi
I INTRODUCTION
1.1 Project Background 1
1.2 Problem Statement 2
1.3 Objectives 3
1.4 Scope of Work 3
1.5 Research Methodology 4
1.6 Thesis Organization 4
x
II LITERATURE REVIEW
2.1 Introduction 6
2.2 Theory and Basic Principles 7
2.3 RGB Color Model 9
2.4 Classification System 10
2.5 Level Classification 11
2.6 Hardware Setup 12
2.7 Commercial Value and Market Survey for 13
Product Inspection
2.8 Review of Previous Related Work 14
III PROJECT METHODOLOGY
3.1 Introduction 16
3.2 Visual Studio 2010 17
3.2.1 Integrated Development Environment 17
(IDE)
3.2.2 New Project 18
3.2.3 Solution Explorer 19
3.2.4 Toolbox 20
3.2.5 Editors and Designers 21
3.2.6 Properties Window 22
3.2.7 Build and Debug Tools 23
3.3 Project Development Workflow 24
3.4 Analysis Framework 27
3.5 Image Acquisition 28
3.6 Pre-Processing 28
3.6.1 RGB to HSV 28
3.7 Color Segmentation 30
3.7.1 Thresholding (Otsu’ Method) 31
3.7.2 Morphological Operation 31
3.7.3 Region of Interest (ROI) 32
3.8 Histogram Analysis 32
xi
3.9 Quadratic Distance Classifier 33
3.10 Water Level Classifier 33
IV RESULTS AND DISCUSSION
4.1 Introduction 34
4.2 Experimental Setup 35
4.3 Color Classification Analysis 35
4.4 Water Level Analysis 43
4.5 System Accuracy Analysis 45
V CONCLUSION AND RECOMMENDATION
5.1 Conclusion 47
5.2 Sustainability and Implementation 48
5.3 Recommendation for Future Development 48
VI REFERENCES 49
VII APPENDIXES 53
xii
LIST OF TABLES
NO TITLE PAGE
2.1 Hardware Component 13
2.2 Different Method Review 14
4.1 Total Sample Image 35
4.2 Red Histogram Value of Reference Image 38
4.3 Green Histogram Value of Reference Image 38
4.4 Blue Histogram Value of Reference Image 39
4.5 RGB Histogram Value of Reference Image 39
4.6 Red Histogram Value of Test Image 40
4.7 Green Histogram Value of Test Image 41
4.8 Blue Histogram Value of Test Image 41
4.9 RGB Histogram Value of Test Image 42
4.10 Threshold Value for PASS Color 43
xiii
LIST OF FIGURES
NO TITLE PAGE
2.1 Different Levels Image Processing 8
2.2 Schematic of the RGB Color Cube 10
2.3 RGB 24-Bit Color Cube 10
2.4 Bottle Level Filling System 12
3.1 IDE of Visual Basic 17
3.2 “New Project” From Start Page 18
3.3 “New Project” From the File Menu 18
3.4 New Project 19
3.5 Solution Explorer 20
3.6 Toolbox 20
3.7 Graphical User Interface 21
3.8 Graphical User Interface Designer 21
3.9 Code Editor 22
3.10 Properties Window 22
3.11 Build Tool 23
xiv
3.12 Error Message 23
3.13 Debug Tool 24
3.14 Project Workflow 26
3.15 Analysis Framework 27
3.16 RGB Color Cube 29
3.17 Top View of RGB Cube 29
3.18 HSV Color 29
4.1 Color Process of Reference Image 28
4.2 Color Process of Test Image 29
4.3 Red Histogram of Reference Image 30
4.4 Green Histogram of Reference Image 30
4.5 Blue Histogram of Reference Image 31
4.6 RGB Histogram of Reference Image 31
4.7 Red Histogram of Test Image 32
4.8 Green Histogram of Test Image 33
4.9 Blue Histogram of Test Image 33
4.10 RGB Histogram of Test Image 34
4.11 Water Level Pass 36
4.12 Water Level Overfill 36
4.13 Water Level Underfill 37
4.14 GUI before Running 38
4.15 GUI after Running 38
4.16 Accuracy System Bar Chart of Test Image 39
xv
LIST OF ABBREVIATION
IKS – Industri Kecil Sederhana
RGB – Red, Green, and Blue
HSV – Hue, Saturation and Value
VB – Visual Basic
VS – Visual Studio
3D – 3 Dimension
2D – 2 Dimension
LED – Light Emitting Diode
ROI – Region of Interest
GUI – Graphical User Interface
IDE – Integrated Development Environment
xvi
LIST OF APPENDIX
APPENDIX TITLE PAGE
A Analysis data of RGB value for Reference 53
image and Test image
1
CHAPTER 1
INTRODUCTION
1.1 Project Background
Product quality inspection is the life business enterprises, only the objective,
and strict inspection can promise the best quality of products left the factories.
Therefore, the product quality inspection is a key job of business enterprises. Visual
inspection is a sub-item from machine vision that contributed so much modern
manufacturing field. In the old days, this can be done with human inspection those
using human eyes. In facts, human as an inspector is slower and their efficiency is
affected by their states of tiredness, illness or other human being shortcomings [1]. In
the modern days, visual inspection systems reduce human interaction with the
inspected goods, classify generally faster than human beings and tend to be more
consistent in their product classification. Visual inspection also has broad application
in industrial automation and covers the full range of technical difficulty in computer
vision [2].
The technique of automated visual inspection has an extensively applied
foreground in the most modern automatic inspection of product quality [3]. This
technique is based on computer vision is kind of method does not touch with inspecting
2
objects. This method uses integrated technology of image processing, exactitude
measure, pattern recognition, artificial intelligence and so on. It is basic principle is to
analyze the target images obtained by the computer vision system in order to get
necessary measuring information, and judge whether the measured target is the norm
or not according to the known knowledge. The image processing, process, and judges
the collected images using the image processing and inspecting software and gets a
conclusion whether the inspected object is in accord with the beforehand established
request or not. Such inspection system extracts and analyzes a large number of features
and automatically reject products that do not satisfy the set quality standards. As a
result, automated visual inspection is undergoing substantial growth in manufacturing
because of its cost effectiveness, consistency, superior speed, and accuracy.
The global beverage industry is growing constantly and the people are
spending more and more money on it. The growing demand requires more efficient
production. At the same time also demands on quality are increasing. Therefore, the
manufacturers are forced to invest in faster machines and plants respectively. In the
beverage industry, this has already started with the production of the preform and of a
bottle and repeats itself, when the empty bottle is recycled and has become a new
bottle.
Industri Kecil Sederhana (IKS) is defined as manufacturing companies that
providing manufacturing related services with an annual turnover not exceeding RM25
million and employs a staff of full-time and does not exceed 150 people. Industri Kecil
Sederhana (IKS) is also a manufacturing company that uses a small capital to produce
their own product. This report will describe a software framework that used for this
project and developing an automatic visual inspection system for inspecting the quality
of beverage product.
1.2 Problem Statement
There exists no industrial vision system capable of handling all tasks in every
application field. Global consumption of soft drink is increasing year by year. It is very
3
important to take care of the quality of the product delivered. To maintain the quality
of such larger number of bottles is not an easy job. There are some IKS that still used
manual inspection in their quality inspection process. With manual inspection, it is
impossible maintaining the easy manufacturing flow. In the IKS, they are searching
new ways to package and deliver their products to minimize their impact on the
environment and reduce costs.
Based on the manual inspection and quality of product issues, this project is
introducing an automatic computer vision system for industrial product quality
inspection. It is because the vision applications have achieved a significant role in the
packaging industry. It is also a much more efficient way of checking products and
easier to be used. This project is proposed to overcome the problem of manual
inspection by a human that is not efficient anymore and to reduce cost by industry. The
quality control processes are based on color and level classification of soft drink bottle.
1.3 Objectives
There are several objectives that need to be achieved in this project. These
objectives are:-
i. To implement image processing technique for the detection and
classification of color and level of the beverage product.
ii. To develop the system based on classification method.
iii. To verify the performance of the system.
1.4 Scope of Work
This project mainly focuses on the software development that is designing an
automated system to classify the soft drink bottle based on color segmentation and
4
level classification of each bottle. The hardware design is not involved in this project.
The system will be designed using the Visual Studio software. The input image of soft
drink bottle will be captured using a webcam. Then, the image will be analyzed,
segmented and classified using the VS software. This system will be able to inspect of
soft drink bottle based on color segmentation and level classification. This project is
also focusing on five different flavors of soft drink, which are Strawberry, Grapes,
Orange, Tropical and Root beer.
1.5 Research Methodology
This project only discusses on software development which is involving the
algorithm development. The research methodology is an important part that shows the
work procedure in order to complete the project. There are some research
methodologies:-
Data collection: An input image of soft drink bottle was taken by using
a webcam as a sample image.
Pre-processing: Color was analyzed by using RGB component, HSV
component, and saturation component.
Color segmentation: Color is segmented using the process of
thresholding (Otsu’ method), morphological operation and region of
interest (ROI).
Color classification: Quadratic Distance Classifier was used to
calculate the threshold value between two sample images.
Level classification: Each bottle has been set by a certain range of water
level, which is passed, overfill and underfill.
Performance verification: It shows the accuracy of the system.
Field testing: Real-time testing.
5
1.6 Thesis organization
The report of this project is divided into five main chapters. There are Chapter
1: Introduction, Chapter 2: Literature Review, Chapter 3: Methodology, Chapter 4:
Results and Discussion, Chapter 5: Conclusion and Recommendation.
Chapter 1 was explained about project background or project overview of the
automated visual system inspection system. The problem statement of the project has
been described and the reason for developing this project. The objectives are stated is
related to the problem statement of this project. The scope of work is described the
project requirement and research methodology is described what method used in this
project. Thesis organization shows the report structure and organization.
Chapter 2 is an important part that needs to be understood by previous research
was done that related to this project. This chapter will gather information about method
and approaches that suitable to be used in this project from any resources. The previous
research will be reviewed, discussed and summarized.
Chapter 3 will be discussed a method that used in this project. This chapter has
explained the procedures that have been made in order to complete this project. The
method used are RGB and HSV color model, Otsu’ method and Quadratic Distance
Classifier. All of this method will be applied in this project.
Chapter 4 is about finding the results and discussion of this project. This
chapter was explained about the results of a simulation using VS software. The
algorithm has been developed in the previous chapter and it will use to figure out the
result in this chapter. The results will be discussed the acceptable or unacceptable of
the beverage product, based on color segmentation and level classification.
Chapter 5 was described conclusion and recommendation in this project. An
overall conclusion of this project has been discussed in this chapter. A specific
recommendation has been recommended due to the issue that has been raised by the
previous researchers.
6
CHAPTER 2
LITERATURE REVIEW
2.1 Introduction
In order to gain enough information to recommend the appropriate method that
can be used to complete the research, comparison between certain techniques is
required to detect and classify of color and level of the beverage product. This chapter
explains and discussed related to the theory and basic principles, review of previous
related work, and summary of this project. The research covered image acquisition,
RGB color model, and level classification method.
The review of previous related works focuses on image processing technique
for automatic product quality inspection that designed by respective researchers. The
commercial value and market survey this project will also be discussed in this chapter
so that the project will be beneficial to the market.
7
2.2 Theory and Basic Principles
The computerized method of image analysis is recently widely used in industry
and is attracting a lot of attention and interest. It is caused by an increasing number of
production requirements and development of computer technology [4]. Computer
vision is a disciplined study how to reconstruct, interpret and understand a 3D scene
from its 2D images in terms of the properties of the structures present in the scene.
Besides, the ultimate goal of computer vision is to model, replicate and, more
importantly, exceed the human vision using computer software and hardware at
different levels. Computer vision systems are commonly used in many fields such as
Science, engineering, and industry. Their applications allow automating many
scientific and technological process.
Image processing is a technique to enhance raw images received from cameras
or sensors placed on satellites, space probes and aircraft or pictures taken in normal
day to day life for various applications. Image processing systems are becoming
popular due to easy availability of powerful personal computers, large size memory
device, and graphic software. There are two methods available for image processing
which is analog image processing and digital image processed. But, this project was
focused on digital image processing. The term digital image processing generally
refers to the processing of a two-dimensional picture by a digital computer [5]. In
digital image processing, a digital computer is used to process the image. The image
will be converted to digital form and then process it. It is defined as subjecting
numerical representations of objects to a series of operations in order to obtain the
desired result.
Image processing or analysis involves a series of steps, which can be broadly
divided into three levels: low-level processing, intermediate level processing, and
high-level processing as indicated in figure 2.1.
8
Figure 2.1: Different Levels Image Processing
Figure 2.1 shows Low-level processing includes image acquisition and pre-
processing. Before any video or image processing can commercially an image must be
captured by the camera and converted into a manageable entity. This is a process
known as image acquisition. Image pre-processing refers to the initial processing of
the raw image data for correction of geometric distortions, removal of noise, graylevel
correction and correction for blurring. Pre-processing aims to improve image quality
by suppressing undesired distortions or by the enhancement of the important features
of interest. Intermediate level processing involves image segmentation, and image
representation and description.
Image segmentation is typically defined as an exhaustive partitioning of an
input image into regions, each of which is considered to be homogenous with respect
to any image property of interest. For example intensity, color or texture. This process
is considered an essential component of any image analysis system and the problem
associated with it have received considerable attention [6], [7]. Image representation
and description are critical for successful detection and recognition of objects in a
scene. After an image has been segmented into object and background regions, one
intends to represent and describe them in characteristic feature for computer
processing during pattern recognition or in quantitative codes for efficient storage
during image compression. Image description deals with the extraction of quantitative
information from the previously segmented image regions.
High-level processing involves recognition and interpretation. It is using
statistical classifiers or multi-layer neural networks of the region of interest. These