Implementing very high-speed hierarchical MLP-based classificationsystems in real-time industrial environments

Riccardo Parenti (* ), Carla Penno (* )Daniela Baratta (** ), Francesco Masulli (*** )

(* ) Ansaldo Ricerche S.r.l., Corso F.M. Perrone, 25, 16161- Genova (IT) - E-mail : parenti@ari.ansaldo.it(** ) DIBE Università di Genova, via all ’Opera Pia, 11A, 16145 – Genova (IT) – E-mail : dany@dibe.unige.it

(*** ) DISI Università di Genova, via Dodecaneso, 33, 16146 – Genova (IT) – E-mail : masulli @ge.infm.it

Abstract

Using a proper combination of special HW and SW,it is possible to exploit the capabiliti es of the Multi-Layer-Perceptron-based “ tree architecture” even invery high-speed industrial classification problems.In particular, the paper shows as, adopting aspecially developed 128 MCPS ASIC Neural co-processor (able to synthesize a whole 64 input – 128hidden – 64 output MLP on-chip), it has beenpossible to build an industrial board, suitable forboth PCI and VME bus, capable of more than 50000highly detailed pattern classification per second. Theboard aims to become a widely diffused tool for theindustrial implementation of data treatment systems.It is now entering in the commercialization phase.The paper gives many details about the chip + board+ SW kit developed and shows an example ofapplication.

IntroductionIt is widely known that Neural Networks (NN)constitute a cost-efficient solution for classificationproblems [1][2][3][4]. It is also assessed that theycan handle a large class of classification problemsyielding the reliabili ty needed in the industrialenvironment [7], even if the availability ofindustrial-graded development environments istoday not so diffused.

A typical industrial-graded development kit should:

(i) be able to achieve the performance typicall yrequired in industrial applications;

(ii ) be easy to be operated, both during the set-upand the installation phase;

(iii ) be equipped with tools useful to validate thesystem performances both in speed andclassification quali ty;

(iv) be implemented in such a way to be easilyintegrated in the industrial data treatmentsystems, e.g. by means of a standard industrialbus interface.

Of course, in order to be able to assure a goodcommercial exploitation, a really friendly humaninterface of the development kit has to be developed,in order to allow the user to apply the NN-based

classification system, also without a specific NNskill. In particular, the SW development toolkitshould hide as much as possible to the industrialoperator the complexity related to the set-up andtraining of the NN system.

Many of the problems of coupling advanced datatreatment techniques to real industrial applicationsare quite complex; they have been here solvedthanks to a deep experience matured by some of theauthors in the implementation of AdvancedInformation Technologies in the industrial field[5...14].

To meet the real-time constraints of a very high-speed application, a dedicated HW implementationhas been designed and manufactured. It is based onan ASIC neural co-processor [5][6] (calledMLP_chip) whose architecture has been optimizedto execute in real-time a MLP-based hierarchicalnetwork.

An application board, containing two MLP_chips,some weight RAMs and a glue logic chip, has beendesigned and manufactured according to the IP busANSI standard specifications (hence compatiblewith many commercial carrier boards). Both a VMEbus based system, and a PCI bus version, compliantwith the emerging standard for low cost industrialsystems, have been developed and tested.

A Windows based user interface has beenimplemented for the development kit, able to let theoperator be in condition to obtain a full y workingsystem simply feeding the system with a gooddatabase, collected on the plant.

The system has been designed in order to implementin HW the TMLP (Tree of Multi-Layer Perceptron)architecture (see later), a hierarchical NN topologyable to yield the same performances of any MLPapplication, requiring at the same time very muchless synapses, obtaining a very much faster trainingphase [6].

For instance in the sample application presented, theTMLP required about 5 times less connections thanits equivalent MLP, performed about 5 times faster,and required about 25 times less training time.

The structure of this paper is as follows. Section 2introduces the TMLP architecture and its capabili ty.

Section 3 deals with the basic neural HW developedand gives some information related to the WindowsNT driver developed for the commercial carrierboard used. Section 4 describes the developmenttoolkit, that is the SW developed in order to allowthe non-specialist to manage in details the NNsystem. Section 5 shows a VME-based sampleapplication developed for the steel industry. Section6 reports conclusions and final observations.

The TMLP ArchitectureMulti Layer Perceptron (MLP) based architectureshave been proved, among others, reliable and theyare widely adopted in the real-world applications.The TMLP architecture introduces a particularhierarchical architecture.

In order to explain more clearly the TMLParchitecture let us consider the application examplereported. Since such an application concerns theclassification of samples belonging to hierarchicallyorganized classes, it seemed natural to use ahierarchical network, reflecting the same hierarchyof the data to be classified. Anyway the TMLParchitecture can be suitable also for problems wherethe hierarchical structure is not so clearly defined.

In principle, the hierarchical classification scheme,likewise any associated TMLP network, could spanover many levels, yielding very complex topology.

In practice, it is extremely difficult to find real-worldproblems requiring more than two levels in order tobe well solved. For that reason the system developedrefers to a generic second order tree, composed by amaximum of a single root MLP driving a maximumof 63 leaf MLP. Each of the MLP involved (root,leaves) can have as much as 64 input, 128 hiddenunits, 64 output nodes.

The overall problem complexity solvable by such arich structure is very high and, very probably, able towell cover all the main industrial applications. It canbe considered that such a structure can classify asmuch as 8192 different objects described using a 64features input data set.

d1 d10 d83 d90 d30 d66 d92 d16 d35 d62 d72 d75 d77 d79 d82 d60 d70 d26 d27

L eaf 4 L eaf 5 L eaf 6L eaf 3L eaf 2L eaf 1

Root

Sc ratches Seams T ransverse Stains D ir tines s M arks

I nput Sample

Figure 1. Structure of the TMLP network

In order to clarify the TMLP network use, the reader

can look (Figure 1) at the hierarchical structure usedfor the sample problem reported in Section 5. Eachnode shown in the picture is a single-hidden-layer,multiple-output MLP. The MLP at the top level (rootMLP) has to classify the input data with respect tosuperclasses (i.e. defect families, e. g. seams, marks,stains, etc.). It selects only one leaf MLP at thesecond level in charge of classifying the inputsample, within the chosen family, into detailedclasses (e.g. d1, d30, d92 etc.). Both the first and thesecond level MLPs refer to the same input data set.

Concerning training, each MLP is trainedindependently from the others (e.g. using the BackPropagation algorithm [1]) on the correspondingdata sub-set.

Generally speaking, the main advantages of usingTMLPs with respect to MLPs implementation inreal-world problems are at least the following [15]:(i) a significant reduction of the training

complexity respect to MLPs;(ii ) a significant increment of the classification

accuracy, thanks to the exploitation of the“ inherent data structure” of the problem;

(iii ) a very much easier determination of the singleMLP topology (hidden neurons required,overall number of weights, etc.).

The Basic Neural HW DevelopedIn this section they will be briefly shown the maincomponents of the HW system developed: thecustom VLSI MLP_chip, the Industry pack busbased mezzanine board, both in its block structureand in its layout. Moreover a technical summary ofthe mezzanine board is reported.

The MLP_chip

The MLP_chip is devoted to be used as a basicmodule in hardware networks implementing MLPbased neural networks. The topology of the MLParchitecture implemented by the MLP_chip isprogrammable; the weight memory (each weight isan 8-bit data) is implemented off chip. To implementa TMLP network then each MLP must be run-timeconfigurable into a wide variety of differenttopologies [6]. Possible network topologydefinitions are down-loaded from a host computerinto internal RAMs. Weights are stored in externalweight memories directly addressed by theMLP_chip. The MLP_chip has a 20-bit weightaddressing space.

Some examples of hierarchical MLP-basedarchitectures are shown in Figure 2: TMLP, CMLP(Coupled MLP) and TCMLP (Tree of CMLP).

The MLP_chip implements a two-layers MLP usingtwo different matrix-vector multipliers (FIRSTLAYER and SECOND LAYER). The FIRSTLAYER and SECOND LAYER blocks implement

the matrix-vector multiplication, inherent to theMLP execution, and the neuron activation functionand they have been optimized to speed-up theexecution of the whole MLP. The computationperformed by the two layers has been decoupledmaking them work independently from each otherand following two different computational schemes[5],[6]. The two layers work in parallel. Being themodule designed for classification problems, it givesas output a list of the more likely classes indecreasing order (e.g. classification): the first classcorresponds to the neuron with the highest outputvalue, the second one to the second neuron highestoutput value, and so on.

The processing rate obtained is 128 Mega-Connection Per Second (MCPS) @32 Mhz clockfrequency (taking into account a single MLP_chip).The chip complexity is about 8000 standard cells. Itcontains 7040 bits of RAM, organized as two dualport RAMs and ten single port RAMs, and fourmultipliers. The chip has been designed andmanufactured in CMOS 0.7 µm technology, doublemetal, single poly and its area is 6.7 × 6.3 mm2 . Themain chip features are summarized in Table 2; amagnified photo of the sili con chip of the MLP_chipis shown in Figure 3.

Table 1 – IP BUS interfaces

INDUSTRIAL BUS Max IP single size modulesfor each carrier board

VME 3U 2VME/VXI 6U 4ISA 4PCI and Compact PCI 2/4

Technology ATMEL ES2 CMOS 0.7 µmchannel length.

Complexity 8000 standard cells and16macroblocks

PADs count 148

Total Area 6.7 × 6.3 mm2

Power consumption 1 W @ 32 MHz

On-Chip Ram 7040 bits

Package PGA181

Processing rate 128 MCPS @ 32 MHz.

Table 2 – MLP_chip characteristics

ΣN

� � � � � � � � � � � � �

Figure 2: Some examples of hierarchical MLP-based architectures.

The industry pack bus

The Industry Pack Bus is ANSI/VITA 4/1995standard, defined on standard high density 50 pinconnectors. It can be better defined a “meta-bus” , asit is possible to easily find on the market carrierboard able to interface IP bus modules to the majorindustrial rack level buses (e.g. Table 1).

The bus main features are:

• 16 bit data bus / 6 bit address bus / Byte or wordaddressing

• 2 Interrupts for each module• Main clock frequency: 8 or 32 Mhz• Peak continuous data rate:

> 32 Mbyte/second (single size module at 32Mhz) > 64 Mbyte/second (double size moduleat 32 Mhz)

The block structure of the mezzanine module

At the board level the real-time classification system

can be considered an application where specializedprocessors (the MLP_chips) co-operate withstandard microprocessors to accomplish applicationspecific number crunching.

Figure 3: MLP_chip Photograph

Physicall y MLP_chips are placed on a custommezzanine board, which is a private resourcesuitable to be interfaced by means of a standard IPbus to many commercial boards. With reference tothe classification real-time requirement, the datatransfer between the MLP_chip and the ImageTreatment System (ITS) plays a crucial role.

The choice of using a private board, with a standardbus (Industry Pack Bus) has been adopted in order tomake it uncoupled from the DCS handshake andtime independent from the Real Time Kernel.Moreover, a Shared Memory is used in order tocommunicate commands, to feed input data and toget output classifications between the NN Systemand the Application Level.

Buffer

WeightMemory1

WeightMemory2

Control

Data

Address

NN Chip 1 NN Chip 2

ControlLogicChip

IP

BUS

Figure 4: Mezzanine Board Block Schematic

U8 U7

J3

C63 C61 RP5 C62 C60 C48 RP3 C37

C53 C42 C57 C55 C46 C44

RP6 RP2 RP4 RP7 RP3 C22_4 U18 C6 U6 U11 U30

U38 C11 C16_4 C20 C18 C13 C22_3 U37 R1 DL1 R2 DL2 C16_3 U29

U20 U14 C14 U13 C16_2

C22_2 U36 U28 C10 C2 U10 C16_1 C27 C22_1 U35 U27 U2 C21 JP1 C23 + R25 R24 +

ONNI-ADC 32 MHz NEURAL NETWORK BOARD

ANSALDO

RICERCHE

Figure 5: Mezzanine Board Serigraphy

The Mezzanine Board Main Characteristics

The main Industry Pack Neural Network Mezzanine(Figures 4 and 5) characteristics are:

• Industry Pack plug-in SMD Mezzanine boardcompliant with IP 32 MHz specifications

• Neural Network subsystem(s) implemented oncustom CMOS 0.7 µm High-Speed VLSI Chips(MLP_chip)

• 64 input-128 hidden-64 output Multi LayerPerceptron Neural Network maximum size

• 64 different Neural Network structures can bestored at the same time on the on-board highspeed memory (1 Mbyte-12 ns) and selected atrun time for each chip

• Very fast run time selection (< 32 ns) of the NNstructure to run (one out of the 64 stored)

• Output classifier based on winner-take-allstrategy implemented on chip

• Peak continuous data rate between host andmezzanine:

− 16 Mbyte/second at 32 Mhz clock− 4 Mbyte/Second at 8 Mhz clock

• Each mezzanine board can support from one totwo MLP_chips and implement, directly in HW,both MLP (one chip) and TMLP (two chips)architectures

• Very high NN processing speed:− with internal pipeline architecture not

used (single shot classification):10 input-128 Hidden-64 output <128 µs64 input-128 Hidden-64 output <144 µs

− with internal pipeline architecture used(continuous sustained rate):

64 input-128 Hidden-64 output <40 µs.

• Double mezzanine board size: 91.5 x 109 mm.• Power supply: 5V• Power consumption: 3 W (idle) - 5 W (running).

The mezzanine is interfaced to the PCI bus via aGreensprings PCI-40 carrier board and using acustom SW driver developed in the Windows NT4.0 version. It is possible to have drivers for PCIcarrier boards both for LINUX and Windows 95/98.

The driver simply has to feed the IP bus connectorsof the commercial carrier boards with the datacoming from the SW application, that implementsmemory write/read cycles in the mezzanine addressspace. In particular the driver is a “dumb SW” in thesense that it does not implement any special functionother than address mapping.

The Development KitIn general the industrial operator is not so muchinterested in knowing technical details about NNs.His main aim is to quickly implement the industrialapplication. This simple statement has a veryimportant consequence for the NN systemdevelopers: the development kit needs to be drivenby a very friendly user interface. It has to be simplebut able to provide help and/or to assume rightdefault decisions anytime the user is in troubleand/or not interested in knowing more in details theNN technical aspects.

Given that the development flow of the NN basedreal-time classification system can be assumed verysimilar to the one reported in Figure 6; a consistentuser aiding system has been implemented. Using aWindows-like GUI, the development system helpsthe operator in properly implementing the variousphases needed to obtain the full workingclassification system.

REAL-TIMECLASSIFICATION

DATABASESAMPLES TRAINING

DATABASESGENERATION

NEURAL NETWORKDVP & TRAIN & TEST

(offline)DOWNLOADFINAL NN TO

NN SYSTEM

REAL-WORLD

DATA

DATA FLOWREAL-TIME

Figure 6: NN based real-time classification systemdevelopment flow

Figure 7: The active desktop

Figure 8: The database elaboration

In order to describe the development system some ofthe main panels are reported and briefly commented.

In Figure 7, the active desktop implemented for thedevelopment environment is shown. A symbolicdevelopment flow is traced and the operator isforced to implement the various phases in the correctorder, starting from the training database validationand treatment phase up to the NN parametersdownload to the specialized HW, ending with theaided on-line or off-line testing.

In particular the main steps are:

1. Data Elaboration phase (see Figure 8)

• starts from the databases directly collectedon the site/plant;

• organizes the data in a proper hierarchicalway in order to make the classificationuseful (i.e.: to develop the classes by whichthe incoming data should be divided in)

• automatically prepares the files required bythe NN trainer for all the application NNs.

2. Training Phase (see Figure 9)

• starts from the training files worked out bythe previous phase;

Figure 9: The training tool

Figure 10: The on-line test

• allows the changing of both the algorithmand the default training parameter set, ifrequired by the operator (otherwise,automatic default choices are selected bythe system);

• monitors the training cycle andautomatically stops it when some stopcriteria are met, in order to assure asatisfactory training for the on-goingindustrial application;

• tests the developed structure using thespecialized NN HW, in order both to besure of the results and to functionally testthe download link.

3. On-line classification (see Figure10)

• downloads the parameters worked out inthe previous phase to the HW subsystemand switches to the classification ‘on-line’mode.

At this point the development system is no morerequired and can be quitted. The classificationtask is then performed in real-time at full speedby the NN HW.

A Sample ApplicationA system implementing in real time the surfacedefect classification of steel strips in flat rolli ng millhas been developed as sample application.

Surface defect recognition of steel strips plays abasic role in quali ty management of steel makers.Surface defects are one of the most important quali tymetrics for customers, as they affect heavily thequali ty (and price) of the products.

Figure 11: A sketch of surface defect classes andfamili es in flat rolled mill s.

Better information on defects may provide valuabledirect feed-back for process control to reduce costsof quali ty (internal costs of scrap-rework) and toincrease manufacturing productivity and yield.Surface defects are numerous, and may be organizedby famili es such as marks, scratches, stains, etc. TheFigure 11 shows a sketch of defect classes andfamili es in flat rolled mil ls. Defects belonging todifferent families may have similar visualcharacteristics, and, as such, they are very difficultto classify. Current approaches rely on humaninspectors specifically trained for the job. The insetof Figure 12 shows the surface of a flat rolled millcontaining a scratch.

Figure 12: A schematic of the quality controlsystem.

The features of samples of steel-ribbon impuritiesand flaws (i.e. the defects) are obtained with on-lineand on-plant measurements. Some features representthe geometrical properties of the imperfections onthe strip, while others provide information aboutill umination, width and thickness of the strip etc.The measurements were collected through an on-lineCCD camera coupled with an on-line imageacquisition/pre-processing system, which performedfiltering and feature extraction tasks on the imagescollected on the surface of the flat rolled strip.

� � � � � � � � � � � �

SW DEVELOPMENT SYSTEM CONSOLE DEFECT COLLECTION AND TREATMENTEMULATOR SYSTEM CONSOLE

VME BUS

NETWORKINTERFACE

ETHERNET

� � � � � � � � � � � � � � � � � � � �� � � � � ! " # � " $ �

PC-BASED BUS ANALYZER167 162

NNSYSTEM

386

Figure 13: The performance measurement set-up ofthe steel defect classification system

The databases, used for tailoring the application,were obtained with on-line and on-plantmeasurements; they include samples of steel-ribbonimpurities and flaws. Each sample consists of 16features, some representing the geometricalproperties of the imperfections on the strip, othersproviding information about illumination, width andthickness of the strip etc., and a code number, whichidentifies the type of defect. Most of the inputfeatures come from the on-line imageacquisition/pre-processing system.The image processing system has been tested in asimulated environment (shown Figure), using realworld data collected on the real plant, in order to

make a reliable assessment of the performancesachievable.

It can be mentioned that, on the real plant, the NNsystem has been implemented using a MotorolaMVME162-533 board as carrier board, and it hasbeen integrated in an already existingVME/VMEexec/pSOS+ based defect detectionsystem. The on-line system is shown in Figure 14.

IP bus IP bus

IMAGE TREATMENT

MVME162 board

DEFECT CLASSIFICATION

Weight Memory

Mezzanine boardNeural Chips

ApplicationCPU

ApplicationCPU

SYSTEM SYSTEM

NN_CPU

VME bus

Figure 14: Host CPU: Commercial CPU MotorolaMVME162_533 (68040@33 MHz)

Physicall y MLP_chips are placed on the custommezzanine board. With reference to theclassification real-time requirement, the data transferbetween the MLP_chip and the Image TreatmentSystem (ITS) plays a crucial role; in particular:

(i) special care has been taken in developing anoptimized custom control logic (Glue Chip) onthe mezzanine board;

(ii ) the weight data throughput to the MLP_chipshas been maintained at 64 Mbyte/s, usingspecial design techniques;

(iii ) a MVME162-533 (32MHz 68040 CPU, 16Mbyte RAM) has been adopted as host for theneural mezzanine, because it guarantees boththe data transfer rate and the compliance withthe whole application system (ITS) required.

Any communication between the Neural DefectClassification System (DCS) and the ITS modules isdone throughout the NN_CPU that executescommands from the ITS System properly interfacingwith the mezzanine board. Commands, dataexchanges and neural network parameters arecarried out through shared memories, accessiblefrom the address space of the VME bus.

Starting from a collection of less than 800 samplesof defect collected on a real plant, the systemdemonstrated to be able to classify the defects betterthan a skilled operator, with a mean classificationtime of less than 380ms considering all thecomputational overhead due to the simple VME-buscommunication mechanism adopted for thedemonstration unit (one by one classification). Thesystem architecture resulted in a hierarchical neuralstructure, shown in Figure 15.

ROOT NN-CHIP(classification ofsuper-classes)

LEAF NN-CHIP(classification of defects

of the kind "LUNGHI")

LEAF NN-CHIP

(classification of defectsof the kind

"METALLURGICI")

LEAF NN-CHIP

(classification of defectsof the kind

"IRREGOLARI")

LEAF NN-CHIP(classification of defects

of the kind "NON

DIFETTI")

LUNGHI METALLURGICI IRREGOLARI NON DIFETTI

GRAFFIO

STRISCIATURA

INCLUSIONE

SFOGLIA

SPORCIZIA

MAL DECAPATO

IMPRONTA

VARIAZIONE

FONDOMACCHIA

.....

.....23 input

20 hidden

4 output

.....

.....23 input25 hidden

9 output.....

.....23 input

25 hidden

9 output.....

.....23 input

25 hidden

9 output.....

.....23 input

25 hidden

9 output

Figure 15: The NN classification systemarchitecture.

The database had 794 samples, collected on the plantby the detection system and classified by the plantoperator using 4 main famili es of objects and 9particular defect types. From the statistical point ofview, the database collected is "poor": the variousdefect types are very unbalanced. For technicalreasons 773 samples out 794 have been used for thetest, while the remaining ones have been rejected,because affected by some recording error (missingfields, meaningless “0”s, etc.).The results obtained have been extremely good,outperforming the mean requirement given by theplant owner (80% of correct classification on adefect of the order of 500 defect per second): thesystem yields a good 83.6% correct classificationratio at a very good 2611 defect per secondclassification speed.

Given the poor statistical quali ty of the databasecollected from the point of view of the requirementof a good NN training, those results are to beconsidered even more interesting. As a consequenceof this, it is reasonable to expect that the system willoutperform very much more the specification of theplant managers if a good statistical quali ty databaseis used. Anyway, the system is very much faster thanspecified even if in the test application it is not usedthe available pipeline subsystem implemented on thechip to fully develop the maximum classificationrate achievable (1 classification every 20 µs).

ConclusionsThe paper presents a HW system implementinghierarchical MLP_based NNs (e.g.: TMLP) for realindustrial applications and the SW toolkitimplemented for it. A Neural ASIC co-processor hasbeen developed for the real-time execution of MLP-based NNs: the processing rate of the ASIC is 128MCPS. An ANSI standard IP bus based neural co-processor board has been developed and tested. Botha VME and a PCI version of the system have beenmanufactured and carefull y tested. The systemfocuses first of all on the industrial imaging market.It is provided with the completely automatic trainingwizard, able to:

(i) record directly from the plant imaging system adatabase suitable for the Neural Networklearning phase;

(ii ) perform the learning phase starting from thatsimple Excel-like database;

(iii ) validate the learning obtained;

(iv) download to the on-line system the NeuralNetwork architecture realized;

(v) start the on-line classification task.

References[1] Widrow, B., Lehr, M.A., “30 Years of

Adaptive Neural Networks: Perceptron,Madaline, and Backpropagation” , Proceedingsof the IEEE, Vol. 78, No. 9, pp. 1415 - 1442,Sept. 1990.

[2] Tsoi, C., Pearson, R. A., “Comparison of threeclassification techniques, CART, C4.5 andMulti-Layer Perceptrons” , in R. P. Lippmann,J. E. Moody, D. S. Touretzky (Eds.), Ad. inNeural Information Processing System,Morgan Kaufmann, Vol. 3, pp. 963-969, 1990.

[3] Atlas, L., et al., “A performance comparison oftrained multil ayer perceptrons and trainedclassification trees” , Proc. of the IEEE, Vol.78, No 10, pp. 1614 - 1619, Oct. 1990.

[4] Guo, H., Gelfand, S. B., “Classification Treeswith Neural Network Feature Extraction” ,IEEE Transaction on Neural Networks, Vol 3,pp. 923-933, November 1992.

[5] Parenti, R., Penno, C., Caviglia, D.D., Bo,G.M., Baratta, D., Valle, M., Canepa, G., “AHardware Implementation of HierarchicalNeural Networks for Real-Time Quali tyControl Systems in Industrial Applications” ,ICANN’97 – Int. Conf. on ANN, Lausanne(CH), 1997.

[6] Baratta, D., Valle, M., Caviglia, D.D.,“Hierarchical Neural Networks for Quali tyControl in Steel-Industry Plants” . Journal ofMicroelectronic Systems Integration,1997.

[7] Colla, A., Parenti, R., “Neural networksapplications in process control” , Proc. of Int.Conf. SNN’97 - Neural Network Best Practicein Europe, Amsterdam (NL), 1997.

[8] Caviglia, D., Valle, M., Baratta, D., Baiardo,V., Marchesi, M., ”A Neural ASICArchitecture for Real-Time Classification”Proc. of EUROMICRO95 (1995) 632 - 638.

[9] Parenti, R., Bagnasco, A., "Industrialapplications of Neural Network Tecnologies",Proc. of the Int. Society for HybridMicroelectronics Congress, Milano (IT), 1992.

[10] Parenti, R., Bogi, S., Massarani, A., "IndustrialApplication of Real-Time Neural Networks inMultistage Desalination Plant", Proc. of IDAWorld Congress on Desalination and WaterSciences, Abu Dhabi, 1995.

[11] Parenti, R., Masulli , F., "Drive Train ForElectric/Hybrid Vehicles Improved By Soft-Computing", Proc. of the International ICSCSymposia on Intelligent Industrial Automation- IIA’96, Reading (UK), 1996.

[12] Parenti, R., Penno, C., "A Neural NetworkBased Approach For The Prediction of Rolli ngForces in Steel Hot Rolli ng Mil ls", Proc. of theInt. Conf. on Engineering Applications ofNeural Networks EANN96, London (UK),1996.

[13] Parenti, R., Penno, C., Oriati, M., "Set-upSystem for Hot Roll ing based on NeuralNetworks", Proc. of IIA97 - InternationalICSC Symposia on Intell igent IndustrialAutomation 1997, Nîmes (FR), 1997.

[14] Parenti, R., "The Soft-Computing In The RealWorld: Some Data And Some Considerations",invited paper, pages 298-304 in (A.Bonarini etal. - Ed.s) "New trends in fuzzy logic", Proc. offirst Italian Workshop on the Fuzzy LogicWILF’95 - World Scientific, 1996.

[15] Baratta D., Diotalevi F., Valle M. and CavigliaD., “Gradient Descent Learning Algorithm forHierarchical Neural Networks: a Case Study inIndustrial Quality” , IWANN99, Brno (CZ),1999.