eco-solar factory: 40%plus eco-efficiency gains in the...

Project no.:

679692

Project acronym:

Eco-Solar

Project full title:

Eco-Solar Factory: 40%plus eco-efficiency gains in the

photovoltaic value chain with minimised resource and energy

consumption by closed loop systems

Research and Innovation Actions (RIA)

FOF-13-2015

Start date of project: 2015-10-01 Duration: 3 years

D 3.4

Report on automatic inspection tool for 50% less scraped solar cells (WP 3)

Due delivery date: 2018-04-30

Actual delivery date: 2018-04-30

Organization name of lead contractor for this deliverable: AIMEN

Project co-funded by the European Commission within the Framework Programme Horizon 2020 (2014-2020)

Dissemination Level

PU Public

CO Confidential, only for members of the consortium (including the Commission Services) x

Ref. Ares(2018)2622096 - 22/05/2018

EU-RES Classified Information: RESTREINT UE (Commission Decision 2005/444/EC)

EU-CON Classified Information: CONFIDENTIEL UE (Commission Decision 2005/444/EC)

EU-SEC Classified Information: SECRET UE (Commission Decision 2005/444/EC)

Deliverable number: D 3.4

Deliverable name: Report on automatic inspection tool for 50% less scraped solar cells

Work package: WP3 Remanufacturing, resource efficiency and reuse in solar cell processing

Lead contractor: AIMEN

Author(s)

Name Organisation E-mail

Francisco Rodriguez Lorenzo AIMEN [email protected]

Abstract

One of the goals of ECOSOLAR project is to develop a full automatic system for inspection and

(PV) photovoltaic cell repair. In a previous European funded project (REPTILE, FP7-GA286955)

AIMEN, ISC and INGESEA has proposed a pilot line concept for inspection and cell repair (called

Cell-Doctor). In this system, defects detection in PV cells is based on machine learning software

and allows selection of functional PV cell areas by cutting or isolating non-defective areas thanks

to laser process. In Eco-Solar project, ISC, AIMEN and INGESEA has implemented this concept

at TRL6 aiming to avoid at least 50% of the scrapped cells. This report will describe main

functionalities associated to Cell doctor and will be focused on core vision technologies used for

defect detection and classification.

Public introduction1

1 All deliverables which are not public will contain an introduction that will be made public through the project WEBsite

TABLE OF CONTENTS

Page

1 INTRODUCTION ........................................................................................................ 4

2 GENERAL STRUCTURE OF CELL DOCTOR ........................................................ 5

3 AUTOMATIC ELECTROLUMINESCENCE INSPECTION SYSTEM ................... 6 3.1 General description ............................................................................................ 6

3.2 Mechanical structure and control and detection components ........................... 6 3.2.1 Mechanical structure and PV cell contacting system ............................ 6 3.2.2 Electrical and control system ................................................................. 7

3.2.3 Electroluminescence sensor system....................................................... 8 3.3 Defect detection software .................................................................................. 9

3.3.1 CCD sensor acquisition software .......................................................... 9 3.3.2 Software vision for defect detection and system management............ 10

3.3.3 Defects detection tests and processing strategy ................................... 11

4 VISION SOFTWARE AND MACHINE LEARINING STRATEGY FOR DEFECT

DETECTION ............................................................................................................. 13 4.1 ECOSOLAR labeler ........................................................................................ 13 4.2 ECOSOLAR learner ........................................................................................ 14

4.2.1 Features extraction ............................................................................... 14

4.2.2 Database creation and management .................................................... 15 4.2.3 Training ............................................................................................... 16 4.2.4 Test ...................................................................................................... 16

5 CONCLUSION .......................................................................................................... 18

6 APPENDICES ............................................................................................................ 19 6.1 APPENDIX nº1 ECOSOLAR labeler manual ................................................ 20

6.2 APPENDIX nº2 ECOSOLAR learner manual ................................................ 34 6.3 APPENDIX nº4 Mounting system for CCD/InGaAs sensor .......................... 47

1 INTRODUCTION

One of the ECOSOLAR project goal is to develop a full automatic system for inspection and cell

repair. In a previous EU funded project (REPTILE, FP7-GA286955) AIMEN, ISC and

INGESEA has proposed a concept for inspection and cell repair system (called Cell-Doctor)

based on different software and hardware validations, able to automatically select and cut or

isolate non-defective areas in defective cells and wafers. In Eco-Solar project, ISC, AIMEN and

INGESEA has implemented this initial concept and developed the Cell-Doctor system, to fully

integrate it in the manufacturing and recycle of Eco-Solar cells, aiming to avoid at least 50% of

the scrapped cells (Figure 1).

Figure 1: Cell Doctor System Flow Chart

In parallel to development done by INGESEA related to integration of cells handling systems

and complementary inspection systems (IV characterization station) and laser processing station,

AIMEN and ISC has centered and intensified their work on automated electroluminescence

inspection system based on machine learning vision technologies. This report will describe main

functionalities associated to Cell doctor and will be focused on core vision technologies used for

defect detection and classification.

2 GENERAL STRUCTURE OF CELL DOCTOR

Cell doctor 3D lay out and actual development is shown in Figure 2 and Figure 3.

Figure 2: Cell Doctor 3D lay out

Figure 3: Actual Cell Doctor prototype in INGESEA facilities

3 AUTOMATIC ELECTROLUMINESCENCE INSPECTION

SYSTEM

3.1 General description

Electroluminescence system is constituted by an

- Enclosure for avoiding spurious light effect during image acquisition, Contacting system

for electrical cell excitation, with a pneumatic positioning system,

- Mechanical mounting system for adjusting image focalization and field of view,

- CCD sensor with enhanced sensitivity in near infrared band (≈1100nm),

- Electrical power supply and embedded PC and for controlling electrical excitation and

image acquisition and image processing,

- A software system with different modules for image acquisition, defect detection,

classification, laser processing parameter calculation (area, shape, power,), and

communication with Cell Doctor PLC and laser processing station,

Figure 4: General functioning of the Electroluminescence inspection system

3.2 Mechanical structure and control and detection components

3.2.1 Mechanical structure and PV cell contacting system

Electric contacts for cells excitation is composed by a table where a pneumatic system moves up

and down pins system and affix PV cells with suction system in order to avoid any cell

breakingFigure 5: Table for cells fastening with pin contact detailed view (Figure 5). Mechanical mounting

(

Figure 6) has been designed to allow easy height and angular adjustment of CCD sensor. X and

Y positioning are adjusted during assembly to be centered on cell sample. A head adaptor allows

using InGaAs sensor available at AIMEN to perform further acquisitions with this technology.

On the objective is to compare image quality between CCD and InGaAs technology for defect

detection. Aditional details on mounting structure are provided in appendix 4

Windows Service

Cutting andIsolation area

definition

Parameter forCutting and

isolation

FTP transfer

Figure 5: Table for cells fastening with pin contact detailed view

a) General 3D view b) Mounting systen installed

in enclosure

c) Detailed view of sensor

mounting head

Figure 6: Mounting system developed for ECOSOLAR EL inspection system

3.2.2 Electrical and control system

For ensuring electrical excitation a controllable programable power supply from Delta

Electronika (SM 70-AR-24) has been selected. Its auto-ranging output option allows supplying

35V with 24A or up to 70V with 24A (Figure 7). Electroluminescence phenomena happens at

excitation threshold voltage of 0,6V. Electrical parameter can be varied up to 8A for adjusting

electroluminescence emission intensity. Power supply is controlled in remote mode through

ethernet communication bus. Industrial PC selected for controlling power supply and images

acquisition is the RCO-6000i7 (Figure 7) fitted with an Intel Core Quad i7-6700TE processor,

4GB RAM memory, and 256 GB SSD (Solid State Disc).

a) SM 70-AR-24 from Deleta Eelectronika b) RCO-6000i7 industrial PC

Figure 7: Electrical power supply and embedded PC selected for ECOSOLAR electroluminescence inspection

system

3.2.3 Electroluminescence sensor system

Technology selected for inspection system development is a CCD sensor, GE 1024 1024 DD

NIR from Greateyes (Figure 8) with enhanced quantum efficiency in near infrared (1100nm)

with a pixel size equal to 13 µm × 13 µm and 1024x1024 resolution. Total sensor size is 13.3

mm × 13.3 mm and it is possible to cool sensor temperature from 20°C to -60°C with air cooling

or water cooling system.

a) CCD sensor DD NIR from greateyes

selcted for ECOSOLAR b) Quantim efficency of DD NIR sensor is around

35% @ 1000nm

Figure 8: CCD sensor general characteristics

The GE 1024 1024 DD NIR from Greateyes is a deep depletion (DD) sensor with NIMO (Non-

inverted mode) architecture, with higher sensitivity in the NIR at 1000nm. Even if NIMO

technology has higher sensitivity, this architecture has higher dark current (0,017

electrons/pixel/sec) compared to AIMO sensors (0,0003 electrons/pixel/sec). The noise produced

by dark current can be reduced by cooling DD sensors at low temperatures. For ECOSOLAR, it

has been obtained good results for -10ºC with air cooling process without losing quantum

efficiency (QE).To ensure maximum photon capture during luminescence phenomena, an

objective optimized for 900 nm to 1350nm range with special coating for NIR region has been

selected. The “inspec.x M” large aperture lens from QI Optics (with large aperture of 1,4, focal

length of 50mm, and F-mount system) has been mounted on camera with a Thorlabs adaptor.

Transmission in the NIR region is higher than 95%, reaching 98% at 1000nm (Figure 9).

a) Inspec.x M NIR 1.4/50 lens b) Tranmission spectra for inspec.x M 1.4/50 lens

Figure 9: Inspec.x M NIR 1.4/50 characteristics

3.3 Defect detection software

Software for general control is composed of different modules (Figure 10):

- The software for images acquisition, developed with C++ programing language, that

synchronizes power supply and camera and allows control of sensor parameters.

- The vision software, developed with Python script, which integrates classifier module and

functions related to processing strategies, automatic parameters generation and communication

with laser station for information file transfer. This last one is managed by a window service

program,

- The “wrapper” software in charge of communication with cell doctor PLC and synchronizing

EL inspection. It is planned to develop this short software modules in the next month final

integration and its implementation would be done in C# programing language.

Figure 10: Inspection system software modules

3.3.1 CCD sensor acquisition software

This software has been developed in C++ integrating Greateyes SDK librairies. It is a console

application that accepts input and sends output to the console. “LibTiff”2 libraries have been

implemented for image management and saving to disc. Application can be launched in

command prompt (Figure 11) using the following command:

GreatEyesAcquisition.exe Exposure time Voltage Temperature

Once application has been launched, cooling process on camera starts (Figure 11).

2 http://simplesystems.org/libtiff/

http://simplesystems.org/libtiff/

Figure 11: “GreatEyesAcquisition” application launched from command prompt in windows OS

Once stated temperature is reached image excitation current is sent to cell at the fixed voltage

and intensity defined by user and images is acquired and stored in the embedded PC disc.

Figure 12: Image acquisition process from “GreatEyesAcquisition”.

3.3.2 Software vision for defect detection and system management

Software for defect detection has been elaborated starting from software modules developed for

learner. Classification modules has been integrated in an application together with function for

definition of areas to be cut and/or isolated (cutting/isolation strategy) and with automatic

parameter files generation. Automatic launching of this application is controlled with a window

service.

Windows service is a computer program that operates in the background and is similar in concept

to a Unix daemon. Windows services can be configured to start when the operating system is

started and run in the background as long as operating system as Windows is running

In ECOSOLAR case, window service manages a thread that initializes loading classification

model in memory. Each time an image is stored in an input directory, thread applies classifier to

the captured image, then returns and stores classification result (i.e., image with detected defects)

and original image in an output directory. This window service has been developed in python

using win32serviceutil. Directories are automatically created in the same directory containing

installation script files, one for input image (named “predict”) and the other for output results

(named “classified”). Window service installation can be done through a command prompt with

the following instruction:

python service.py --startup auto --username USERNAME --password PASSWORD.

3.3.3 Defects detection tests and processing strategy

After labeling, training has been done on more than fifty images, using specific option for

features extraction and training (see section 4.2.1, 4.2.2 and 4.2.3). Classifier presents good

results to crack, shunts and finger interruption defects. In ECOSOLAR project, it has been

decided to recycle half part of the cell once defect is detected. Processing strategy (cutting and

isolation) applied in ECOSOLAR will consider this recycling strategy. Figure 13 shows an

example of defects detection and area de delimitation for processing strategy.

a) Original image b) Busbar and boundary cell localization

c) Defect detection d) Area to be cutted

Figure 13: Example of defect detection and processing strategy application to PV cell

Information related to area (shape, localization in PV cell) to be processed is then stored in a text

file with the specific format. In this file, it is possible to define isolation process (i), executed by

laser as ellipses and cutting (c) process processed as lines. The file structure proposed is the

following:

#i | x (mm) y (mm) w (mm) h (mm) a (deg) | p (W) f (Hz) v (mm/s)

#c | x1 (mm) y1 (mm) x2 (mm) y2 (mm) | p (W) f (Hz) v (mm/s) l (n)

For isolation #i where (x, y) is the ellipse center, (w, h) its dimensions and (a) the rotation angle.

For cutting #c initial point is (x1, y1) and final point is (x2, y2) for processing line.

4 VISION SOFTWARE AND MACHINE LEARINING STRATEGY

FOR DEFECT DETECTION

4.1 ECOSOLAR labeler

This software module allows labelling images acquired with electroluminescence system. This

software tool will ease the process of manually labeling defects in solar cell images. It is aimed

to be used by characterization experts, as for example ISC. Core components has been developed

in Python language, with PyQt and OpenCV libraries. This software module is used for

classifying defects and save information in specific mask images (corresponding to different

defects) that will be used by learner software (Figure 14).

a) EL PV cell image with labelled defects b) Different mask images corersponding to labelled

defects

Figure 14: Example of labelling process applied to an electroluminescence PV cell image

Software is prepared to work with both 8-bit and 16-bit (the original images captured by the EL-

System at ISC. It allows visualization of Histogram adjustment. It is easy to deploy in new

computers through standalone installable versions available for Linux and Windows. It is also

prepared to easily label cell boundary and other problematic zones (busbars, logo), to incorporate

that knowledge to the training database (Figure 15).

Figure 15: Results of labelling process for other categories applied to an EL PV cell image

A catalog of supplementary data of interest for solar cell image characterization has been added.

A protocol to embed these parameters as metadata in the header of TIFF images (Pillow Library)

has been established. EcoSolar labeler allows the user to read/edit/write these embedded

metadata (Figure 16).

a) Meta data list b) Cell metadata dialog box

Figure 16: Metadata parameters list and its dialog box software implementation

4.2 ECOSOLAR learner

Ecosolar Learner is a software tool that allow extraction of features, databases creation of

samples, training and testing of defect classifiers. It is built on the base software framework and

on the EcoSolar Labeler. It is geared to learn from a large-scale database, and to easily

incorporate new features and algorithms. Graphic User Interface has been designed to be user-

friendly and has four main modules: features extraction, database creation and management,

training, and test.

4.2.1 Features extraction

Features extraction is done from labeled images (obtained with EcoSolar Labeler). Software uses

algorithm based on Gabor and Log-Gabor for features extraction (Figure 17, Figure 18). Option

for multi-resolution (images with resolution from 1024x1024 to 512x512) has been implemented.

Features extraction has been parallelized with threaded computing process to reduce computing

time. Option for flipping images allows multiplying by four number of samples used for training.

Scope Metadata

Global Image Modality

Cell Type

Cell Area [cm2]

Number of Busbars

IV Short Circuit Current ISC [mA]

Open Circuit Voltage VOC [mV]

Fill Factor FF [% ]

Efficiency η [% ]

Shunt Resistance RSH [Ω]

Series Resistance RS [mΩ]

Maximum Power Point Current IMPP [mA]

Maximum Power Point Voltage VMPP [mV]

EL Exposure Time [ms]

Sample Voltage [mV]

Sample Current [mA]

Sample Temperature [K]

PL Exposure Time [ms]

Laser Current [A]

a) Dialog box for images selection b) Dialog box for feature extraction method selection

Figure 17: Example of process for electroluminescence images features extraction

Figure 18: Example of features extraction process in progress

4.2.2 Database creation and management

In order to obtain acceptable classification results it is necessary to perform training on a large

number of images, to obtain numerous samples per class. This leads to possible store issues. Due

to memory limitations all the data will not be processed at once. The solution applied in this case

is the use of specific files format, named Hierarchical Data Format3 (HDF). HDF is a set of file

formats designed to store and organize large amounts of data. In our case, HDF5 files database

is adapted to our requirements. Moreover, database files can be dynamically updated and

extended when once new labeled images are available.

The processing steps for database creation and store is the following:

- Solar cell detection

- Identification of pixels of different classes (through labels)

- Features extraction (Gabor, Log-Gabor by the moment)

- features packaging and database creation in HDF5 format

3 https://www.hdfgroup.org/

https://www.hdfgroup.org/

4.2.3 Training

Training is applied from HDF5 files database with a multiclasss classification and an SV

Classifier (with/without preprocessing). Incremental learning option for large datasets has been

implemented and opened framework allows progressive inclusion of new learning algorithms (as

for example AdaBoost) (Figure 19). The threaded execution allows user stopping the process at

any point. Once trained, the classifier can be serialized and stored in a single file for future use.

Figure 19: Dialog boxes for learning process configuration; from left to right, learning, incremental learning and training

scheme configuration.

4.2.4 Test

The trained classifiers must be run on new images to test its performances. The user can use a

classifier just after training it or loading a previously trained one from a file. Defect maps are

shown after classification. Tests can be performed over labeled images, and we obtain

performance figures related to ground truth (Figure 20, Figure 21). Options as time analysis and

automatic distinction between mono and multi-crystalline cells, detection of cell boundary and

number of busbars has been integrated. As in previous software modules computing has been

implemented in threads for accelerating execution.

Figure 20: Performance dialog box for defects detections

Figure 21: Dialog box with performance results

5 CONCLUSION

To the date main hardware structure of electroluminescence inspection system has been

developed and has been successfully tested in laboratory. Software for images acquisition has

been implemented. Vision software with classifier has been developed and a first version

integrated with window service system is running on embedded PC. Classifier is currently under

new training process to improve defect detection and adapt processing strategy. It is expected to

develop communication software module in the next months during integration stage of EL

system in “Cell doctor” line. Part of pilot line developed by INGESEA will be received in the

next months at AIMEN for final integration and testing.

6 APPENDICES

6.1 APPENDIX nº1 ECOSOLAR labeler manual

ECOSOLAR LABELER (v0.3.2)

User Manual

The EcoSolar Labeler is a software tool developed at AIMEN to ease the process of manually

labeling defects in solar cell images (electroluminescence, photoluminescence). It has been

conceived to provide solar cell characterization experts at ISC with a user-friendly tool to delimit,

at pixel level, the defects of a given solar cell image and classify them with minimal effort, in a

user-friendly way also allowing the user to embed and edit measurement parameters as metadata

of the original image file.

The final goal behind this software is to build a large labeled dataset of defective solar cell

images. Such dataset will be the core input of the training platform we will develop to obtain the

automated inspection tool for EcoSolar Task 3.4.

The application has been programmed in Python, using PyQt for Graphic User Interfaces,

OpenCV and Pillow for image processing, and Matplotlib for graphic representations.

General Operation

The user must select a collection of images, and then interactively “paint” the defective areas as

new layers of each image. These layers are finally stored as binary images (masks), of the same

size of the original picture, having white pixels in the area of the specific defect and black pixels

elsewhere.

Illustration 1: Screenshot of EcoSolar Labeler

According to ISC requirements, the tool (since version 0.2.0) is geared to process both 8-bit and

16-bit grayscale TIFF images, allowing the user to introduce, edit and save a collection of

metadata (mainly electrical measurements) relevant for characterization purposes. These

metadata are stored as application specific TIFF tag headers in the image file itself.

The software is prepared to label 8 different kinds of defects (this list can be flexibly modified

and adapted throughout the lifetime of the software tool) as well as other three categories related

to the cell segmentation (cell boundary, busbars and manufacturer logo). Each category is

associated with a color used to display the semi-transparent masks during labeling (color code is

inherited from that used in the REPTILE project), a numeric value and an identifiable

abbreviation used for the file naming protocol.

Defect Color Code Numeric Value Abbreviation

Finger Interruption Blue 1 FINGERINT

Shunt (not on busbar) Red 2 SHUNTNOTBUS

Shunt (on busbar) Dark Red 3 SHUNTBUS

Crack Yellow 4 CRACK

Back Metalization Defect Cyan 5 BACKMETAL

Breakage Magenta 6 BREAKAGE

Front Metalization Defect Olive 7 FRONTMETAL

Other Artifact Green 8 ARTIFACT

Illustration 2: Example of the General Operation of EcoSolar

Labeler

Table 1: Solar Cell defects, colors and abbreviations.

Category Color Code Numeric Value Abbreviation

Cell Boundary Sky Blue 100 CELLBOUNDARY

BusBars Green Yellow 101 BUSBAR

Logo Orange 102 LOGO

Table 2: Extra label categories, colors and abbreviations.

Saving and Naming Protocol

Let us suppose that we want to label the defects of an image, stored at directory/image001.tiff,

that has a shunt (not on bus bar), a crack, two finger interruptions and one additional unknown

artifact. Once labeled, all those defects will be stored by the application in the subdirectory

directory/image001_DEFECTS, (concatenation of the original file name, without extension, and

the string _DEFECTS).

Each individual defect mask will be, then, stored as follows:

directory/image001_DEFECTS/image001_000_SHUNTNOTBUS.bmp

directory/image001_DEFECTS/image001_001_CRACK.bmp

directory/image001_DEFECTS/image001_002_FINGERINT.bmp

directory/image001_DEFECTS/image001_003_ARTIFACT.bmp

Illustration 3: Example of extra categories labeling.

Thus, the defect file naming protocol concatenates: the original image file name (without

extension), a series number, and the abbreviation of the corresponding defect.

Metadata Collection

From version 0.2.0, EcoSolar Labeler includes the possibility of introducing, editing, showing

and saving a set of parameters (mainly electrical) related to the solar cell and the measurements

performed. These parameters are stored as metadata of the original TIFF image (see appendix

for more information on the tags used).

The list of available parameters, as defined by ISC experts is the following one:

• Global:

Modality [Electroluminescence, Photoluminescence]

Cell Type [Monocrystalline, Multicrystalline]

Cell Area [cm²]

Number of Busbars

• Current-Voltage (IV):

Short Circuit Current ISC [mA]

Open Circuit Voltage VOC [mV]

Fill Factor FF [%]

Efficiency η [%]

Shunt Resistance RSH [Ω]

Series Resistance RS [mΩ]

Maximum Power Point Current IMPP [mA]

Maximum Power Point Voltage VMPP [mV]

Illustration 4: Defects saving and naming protocol

• Electroluminescence4 (EL):

Exposure Time [ms]

Sample Voltage [mV]

Sample Current [mA]

Sample Temperature [K]

• Photoluminescence5 (PL)

Exposure Time [ms]

Laser Current [A]

Regarding Electroluminescence images, from version 0.3.2, the labeler also includes the option

of automatically import EL data from the filename of the image. This automatic import is

only activated when EL metadatas are not found in the headers of the file and the filename meets

several rules. EL parameter interpretation is compatible with the current image file naming

protocol defined by ISC, as well with the legacy protocol used to capture images during the

Reptile Project.

Current:

IMAGEID_MODALITY_SAMPLEVOLTAGE_SAMPLECURRENT_SAMPLETEMPERATURE_EXPOSURE

TIME.tif MODALITY → "EL"

SAMPLEVOLTAGE → #mV (Integer)

SAMPLECURRENT → #mA (Integer)

SAMPLETEMPERATURE → #K (Integer)

EXPOSURETIME → #ms (Integer)

Legacy:

IMAGEID_MODALITY______EXPOSURETIME___SAMPLECURRENT___SAMPLEVOLTAGE.tif

MODALITY → "EL"

EXPOSURETIME → #.#s (Float)

SAMPLEVOLTAGE → #.#V (Float)

SAMPLECURRENT → #.#A (Float )

Welcome window and main window

Once the program is started, a welcome window with the EcoSolar logo and the project

consortium is shown for a few seconds.

4 Only for Electroluminescence images.

5 Only for Photoluminescence images.

Then, the main window of the application appears.

Main ToolBar

All the options of the application can be alternatively accessed via the menu bar, toolbars, and

also by contextual menus (left mouse button click in the image viewer area). In this document

we will focus on toolbar interaction, since it is the most visual, handy and easy to learn interaction

method to use the software. However, most of these functionalities can also be found on the

different menu bar and contextual menu options.

Illustration 5: Welcome Window

Illustration 6: Main Window

Icon Action Meaning [Shortcut]

Exit [Ctrl+Q]

Open Image File(s) [Ctrl+O]

Close Current Image [Ctrl+C]

Load Previous Image [Ctrl+P]

Current Image Indicator (current/total)

Load Next Image

View MetaData [Ctrl+M]

Adjust Visualization Histogram [Ctrl+H]

Zoom In [Ctrl+I]

Zoom Out [Ctrl+U]

Reset Zoom (1:1 Scale) [Ctrl + R]

Adjust Zoom to span available space [Ctrl+A]

New Defect

Edit Defect

Manage Defect Layers Visibility

Table 3: Main ToolBar Icons

At first, the user must select the image or set of images to label. After clicking the “Open Image

File(s)” option, a dialog will appear, allowing the user to select the files (*.tiff or *.tif images)

to open. Once selected, the images can be sequentially loaded (Load Next/Previous Image),

individually closed (Close Current Image) or open additional images to include to the load list.

When a specific file is loaded, the user will be able to explore it by zooming, panning (holding

down the left mouse button Hand Mode) and also inspecting the pixel values (8 or 16-bit values

depending on the original image) through the status bars.

Illustration 7: Main ToolBar

Cell

Metadata

Once a cell image is loaded, the user can visualize and edit its application-oriented parameters

by pressing the “View Metadata” button. Then, the “Cell Metadata” window will appear. Any

change in these parameters will be saved in the original TIFF file itself, using specific tags (see

Appendix).

Histogram

When an image is loaded from file (either 8 or 16-bit depth), the application automatically

computes its minimum and maximum values and then adjusts the image representation on screen

to maximize the dynamic range and optimize the contrast.

However, in many cases the user can be interested in changing this automatic dynamic range

adjustment and specify a different one (to emphasize a given defect, or to adapt the visualization

due to a pixel outlier outside the desired dynamic range). For that purpose, the user can press the

“Adjust Visualization Histogram” button and change the visualization thresholds interactively

over a graphic representation of the histogram of the image (changes are immediately visualized

in the main window).

It is important to note that, regardless of the image representation on screen, the pixel value

shown in the status bar will always be the value in the original image.

Illustration 8: Status Bars

Illustration 9: Cell Metadata Windows (for EL and PL Images)

New Defect

Once an image is loaded, the user can start the labeling process by pressing the “New Defect”

button. Then, a pop-up menu will be displayed to select the specific type of defect that is going

to be labeled (determining the color code to display de defect, and the abbreviation for the naming

protocol).

Illustration 10: Adjust Visualization Histogram Window

Then, the “Draw Defect ToolBar” will appear below the Main ToolBar.

Draw Defect ToolBar

The user will be able to draw over the image, via a semi-transparent layer with the color

associated to the defect. There are available brushes to paint and erase with different shapes

(circular or square) and sizes (diameter/side slider), via freehand drawing (Paint and Erase

Modes) or by straight lines (Line Mode). To make further flexible the labeling process, the user

can undo and redo the last operation, as well as clean the whole layer.

Once the drawing is finished, the user must validate the defect, so the mask layer is saved to file.

Alternatively, at any point, the defect can be discarded, and no information will be saved to disk.


Defect Identifier (SeriesNumber – TypeOfDefect)

Validate and Save Current Defect Layer [Ctrl+V]

Discard Current Defect Layer [Ctrl+D]

Undo Last Drawing Operation [Ctrl+Z]

Redo Last Drawing Operation [Ctrl+Shift+Z]

Clean Defect Layer

Defect Mode

(icon shows

current mode)

Paint Mode (add defective pixels) [Ctrl+P]

Erase Mode (remove defective pixels) [Ctrl+E]

Line Mode (draw defective straight lines) [Ctrl+L]

Polygon Mode (draw a polygon6) [Ctrl+G]

Hand Mode (pan movement) [Ctrl+H]

6 Only to draw cell boundary.

Illustration 12: Draw Defect ToolBar

Brush Shape

(icon shows

current shape)

Circular Shape

Square Shape

or + components Components to set the diameter (circle) or side (square) of

the current brush

Table 4: Defect ToolBar Icons

Edit Defect

A defect layer that has been previously labeled, can be edited and modified by pressing the

“Edit Defect” button and selecting the desired layer in the pop-up menu. Then, the “Draw

Defect Toolbar” will be shown again, allowing the user to continue/correct the drawing with the

same tools as when labeling from scratch. In this case, “discard” the drawing means that no

change will be saved from the previous labeling.

Defect Visibility

When several defects have been drawn, the different layers can be disturbing to label another

one. To avoid these problems, the user can select the visibility of the layers via the “Defect

Visibility” button.

Cell Boundary

To ease the annotation process when selecting the option to draw the Cell Boundary, the

Illustration 13: Edit Defect Pop-Up Menu

Illustration 14: Defect Layer Visibility Pop-Up Menu

“Polygon” drawing mode is automatically activated7. This mode allows the user to define a

polygonal contour (with a predefined contour thickness) by simply clicking its respective

vertices.

Polygon labeling ends when the user clicks in the neighborhood of the first vertex annotated, so

the line is considered closed. In case a mistake is made during the polygon drawing process, the

user must force the closure of the line (i.e. click the first vertex again) and then undo the last

operation (erasing the whole polygon) to start again the annotation.

7 In fact, this mode is the only one available for Cell Boundary labeling, and it is not available for the rest

of defect/categories.

Illustration 15: Example of Cell Boundary polygonal drawing.

APPENDIX: TIFF METADATA TABLE (ECOSOLAR TAGS)

Following the TIFF standard recommendations, we have used the “reusable” range of tags

(65000-65535) to store the metadata for solar cell images.

Scope Meaning Tag Content

Global Image Modality 65000 “EL”, “PL”

Cell Type 65001 “MONO”, “MULTI”

Cell Area [cm2] 65002 Number (as String)

Number of Busbars 65003 Number (as String)

IV Short Circuit Current ISC [mA] 65010 Number (as String)

Open Circuit Voltage VOC [mV] 65011 Number (as String)

Fill Factor FF [%] 65012 Number (as String)

Efficiency η [%] 65013 Number (as String)

Shunt Resistance RSH [Ω] 65014 Number (as String)

Series Resistance RS [mΩ] 65015 Number (as String)

Maximum Power Point Current IMPP

[mA]

65016 Number (as String)

Maximum Power Point Voltage VMPP

[mV]

65017 Number (as String)

EL Exposure Time [ms] 65050 Number (as String)

Sample Voltage [mV] 65051 Number (as String)

Sample Current [mA] 65052 Number (as String)

Sample Temperature [K] 65053 Number (as String)

PL Exposure Time [ms] 65070 Number (as String)

Laser Current [A] 65071 Number (as String)

6.2 APPENDIX nº2 ECOSOLAR learner manual

ECOSOLAR LEARNER (v0.1.3)

User Manual

The EcoSolar Learner is a software tool developed at AIMEN to interactively train, run and

evaluate solar cell defect detectors/classifiers based on images (electroluminescence,

photoluminescence). It has been built on the basis of the EcoSolar Labeler software and includes

all its functionalities. In fact, to make use of all the features of the EcoSolar Learner, a large

dataset of defective solar cell images labeled via the EcoSolar Learner is needed.

It has been conceived to dynamically configure, train, and test different classifiers in an easy and

objective way, so we can directly compare results and draw conclusions towards the optimal

design and configuration of the classifier required by EcoSolar Task 3.4.

The application has been programmed in Python, using PyQt for Graphic User Interfaces and

threading, OpenCV and Pillow for image processing, Matplotlib for graphic representations,

SciKit Learn for machine learning algorithms and PyTables for HDF5 support,

General Operation

The main window of the EcoSolar Learner is almost identical to that of the EcoSolar Labeler.

Only a new “Processing” menu and its respective toolbar button have been added to the interface.

This menu and button give access to all the functionalities related to classifier training, execution

and test.

All the previous options and operation from the Labeler remain intact.

Illustration 16: Processing Menu and ToolBar Button


Exit [Ctrl+Q]

Open Image File(s) [Ctrl+O]

Close Current Image [Ctrl+C]

Load Previous Image [Ctrl+P]

Current Image Indicator (current/total)

Load Next Image

View MetaData [Ctrl+M]

Adjust Visualization Histogram [Ctrl+H]

Zoom In [Ctrl+I]

Zoom Out [Ctrl+U]

Reset Zoom (1:1 Scale) [Ctrl + R]

Adjust Zoom to span available space [Ctrl+A]

New Defect

Edit Defect

Manage Defect Layers Visibility

Processing

Table 5: Main ToolBar Icons

Processing Menu

The core of the EcoSolar Labeler is the “Processing” menu options:

• Create Training Database: Extract features from a collection of images, pack and store

them for optimized processing.

• Train Classifier: Configure and Train a classifier from a previously generated training

database.

• Load Classifier: Load a previously trained classifier from disk.

• Save Classifier: Save a trained classifier to disk.

• Info Current Classifier: Shows the information of the current classifier.

• Run Classifier (Current Image): Run the classifier over a given image.

• Locate Cell and Busbars: Run an image processing routine to precisely locate the cell

limits, and the busbars positions, automatically determining the cell type (mono or

multicrystalline).

• Ground Truth (Current Image): See the ground truth information of the current image.

• Match Classification (Current Image): Compare classification performance on the

current image with ground truth information.

• Process Image Collection: Process a set of images (including classification, matching

and time analysis).

Create Training Database

A training database consists of the collection of features extracted from the labeled pixels of a

set of training images. Since the amount of information may be huge and unmanageable as a

whole by a standard computer, we decided to implement the database following the HDF5

format, easily allowing partial indexed reading of the matrices (via PyTables package) almost

transparently for the python programmer.

As in the EcoSolar Labeler, at first, the user must select the set of labeled images via the “Open

Image File(s)” option. Once all the desired images (or a larger collection) have been selected and

can be visualized, the “Create Training Database” option will be enabled.

Once clicked the option, a first dialog to select the training images appears:

Illustration 17: Options of the Processing Menu

In this dialog, the user can define the desired subset of training images to use from the bunch of

files previously selected. At the same time, the user can select a Downsampling Factor to

process the image in a resolution lower than the original (to find the optimal size to accelerate

processing and keep performance) and also to include Flipped Versions of the selected images

to virtually multiply the number of training samples and improve the generalization capabilities

of the system.

Illustration 19: Downsampled versions of the training images

Illustration 18: Select Training Images Dialog

Once the images and their configuration have been selected and confirmed, the user must select

and configure the kind of features to extract for them. By the moment, our software supports the

extraction of Gabor and Log Gabor features, and the user can manually configure the number

of scales, number of orientations, scale factor and minimum wavelength for both cases.

Illustration 20: Flipped versions of the training

images

Illustration 21: Feature Type Selection Dialog

After that, the user will select a directory to store the resulting HDF5 files. If the selected

directory already contains HDF5 files colliding with the ones that are going to be generated, the

user will be warned. In case the existing files are fully compatible with the new data, the software

will allow the user to “append” the previous files with the new information (thus, allowing to

dynamically expand the database as more labeled images are available). During the processing

time, an informative progress bar dialog will be shown. Once a right directory is selected, the

extraction process will start.

Illustration 23: Feature Extraction Progress

Illustration 22: Log

Gabor Features Configuration Dialog

Once the extraction progress has finished, the user will find a collection of *.hd5 files in the

selected directory named DEFECT_***.hd5. Each file stores the samples from all the selected

images related to a single defect, and ”***” refers to the numeric value associated to the specific

defect (see EcoSolar Labeler Manual) written with 3 digits.

The features information is stored in the HDF5 file as a single earray (enlargeable array) called

“features” that is child of the root node. The shape of this array is (number_of_samples, number

of features_per_sample) and in its “User Attributes” is stored the extraction information (label,

type of features, image flip options, downsample factor, image files...). These HDF5 files can be

externally explored with compatible software like ViTables.

Train Classifier

The first requisite to train a classifier, is having available a HDF5 database of features as

described in the previous paragraphs. Thus, when the user selects the “Train Classifier” option,

the first action is to indicate a directory containing compatible HDF5 files for training. Once the

directory has been selected, a learning scheme must be selected:

• Standard Learning: The classifier is trained in a single iteration, seeing a single bunch

of examples for training.

• Incremental Learning: The classifier is trained in an iterative way and, in each iteration,

a different bunch of samples (randomly selected from the main pool) is used. This gives

the learner the potential to see a huge number of examples that would be unmanageable

(due to memory restrictions) with a standard learning strategy.

Illustration 24: Learning Scheme Selection Dialog

In any case (standard or incremental), the user must select in the next dialog the number of

samples from each class to use for training. For standard learning, these parameters refer to the

total number of examples, while for incremental learning they refer to the number of samples

gathered per iteration. In both cases, if (as usual) the selected number of samples is smaller than

the total number of samples available, the selection is made randomly. Additionally, in this same

step, the user must select the number of iterations for the incremental learning scheme.

The next step is to select the training scheme used by the learner. By the moment, we have

available:

• Principal Component Analysis (PCA) + Scaler +Support Vector Classifier (SVC)8

• Scaler + Support Vector Classifier (SVC)

8 Parameters for Standard Learning - PCA: 0.99999 of the total variance; SVC: Penalty parameter C =

1.0, Kernel='rbf', class_weight='balanced'. Parameters for Incremental Learning - PCA: all the components,

SVC: Stochastic Gradient Descent (SGD) equivalent to linear SVM, all classes with weight one,

Illustration 25: Database Configuration for Standard (left) and Incremental

(right) learning schemes

Once the algorithm selection is confirmed, the training process starts, and a progress dialog is

shown. Once the process ends, the main window comes to its original state and the classifier

remains loaded into memory (enabling the options requiring an already trained classifier).

Save Classifier

When we have loaded in memory an already trained classifier we have the option of saving it as

a persistent file in the hard disk unit. Clicking the option “Save Classifier”, the user will be able

to select the path and name of the classifier file.

Despite the extension of the generated file will be *.clas, the file is stored following the ZIP

format. Thus, the contents of the classifier file can be explored with a standard file compressor

software. The most interesting part of the ability to explore the contents of the file is that, among

the different files (classifier.pkl*...) there is an 'info.txt' file with plain text information about the

classifier (training configuration) stored for traceability.

Load Classifier

The inverse step of “Save Classifier”. The user can select a classifier file (*.clas) with a

previously trained classifier and load it into memory.

Info Current Classifier

Displays a window showing the information of the classifier currently loaded (the training

configuration information stored in' info.txt' ).

Illustration 26: Training Scheme Selection Dialog

Run Classifier (Current Image)

Clicking this option, the user will run the classifier loaded into memory on the current image.

Once the classification is finished a pixel map with the obtained result is shown in a new window

(busbars are considered as “out of cell pixels”)

Illustration 28: Example of a classification result

Illustration 27: Current Classifier Information Window

Locate Cell and Busbars

This option launches a routine to precisely determine the cell boundary, the cell type (mono or

multicrystalline) and the position of the busbars. Cell boundary and cell type estimations are

purely based on visual information. Busbars location, for a more robust decision, in addition to

visual information uses also geometric restrictions based on the number of busbars information

stored in the metadata.

Ground Truth (Current Image)

This option allows the user to show the ground truth pixel map of the current image.

Match Classification (Current Image)

Using the ground information, this option runs the classifier over the current image and matches

the obtained results with the real labels. As well as the result and ground truth maps, two error

maps are also shown: one identifying the error in defect vs no defect areas, and another taking

into account also the defect category.

Illustration 29: Example of cell and busbar location

Illustration 30: Example of ground truth pixel map

Additionally, a window with a bunch of different performance figures extracted from the test is

displayed (as selectable and copiable text).

Process Image Collection

To perform larger tests the tool also includes this option to perform classification and matching

over several images in batch mode. At first, the set of images of interest (including its flipped

versions if desired) must be selected. Then the user can indicate different operating modes:

• Automatic/Manual: Execution over each image is performed sequentially with

automatic or manual transition between consecutive images.

• Match with ground truth: This check box activates the “matching” and compares the

results of the classification with the respective ground truth information. In case of

activation, after processing each image the four pixel maps (Illustration 13) are shown,

and the pixel performance result dialog, computed across all the selected images, is shown

at the end of the batch processing. If not activated, only the classification result is shown.

• Time Analysis: If the user wants a time analysis of the computational burden required

by the classifiers, this option allows such a measurement. Includes the possibility of

repeating the measurements a given number of times, and then averaging the results for

more reliable results. If activated, at the end of the execution, a time analysis result

window will be shown (with selectable and copiable text).

Illustration 31: Pixel Performance Results Dialog

Illustration 33: Time Analysis Results Window

Illustration 32: Image Collection Processing Options Dialog

6.3 APPENDIX nº4 Mounting system for CCD/InGaAs sensor

eco-solar factory: 40%plus eco-efficiency gains in the...

Documents