optimization of processes & products using …focapo-cpc.org/pdf/macgregor.pdfoptimization of...

(c) 2004-2013, ProSensus, Inc.

Optimization of Processes & Products using

Historical Data

John MacGregor

Distinguished University Professor Emeritus, McMaster University

Chairman, ProSensus Inc.

FOCAPO/CPC 2017, Jan. 8-12, 2017

Mark-John Bruwer

CTO, ProSensus Inc. (now with Aspentech)


ProSensus – changes have occurred

• ProSensus Software Assets recently Purchased by Aspentech

– ProMV: BIG data analytics focussed on process industries

• Analysis and diagnosis of historical process data (batch & continuous)

• Robust multivariate LV models for inferentials/soft sensors

• Building and simulating multivariate monitoring plots

• Optimization of processes using models built from historical data

– ProMVOnline : Monitoring/soft sensors for continuous processes

– ProBatchOnline: Monitoring/soft sensors for batch processes

– ProBatchControl: MPC of final quality for batch processes

• ProSensus today:

– Engineering consulting on the analysis and optimization of processes

– ProVision: Industrial vision systems for monitoring and control

– ProFormulate: Rapid development of new products


Outline of this presentation

• No new optimization methods

• Focus is on:

– What can be done using historical data collected from processes and with R&D data

– Optimization using empirical models

– Industrial examples

1. Optimization of processes using models built from historical data

• Latent variable models

• Uniqueness and causality issues

• Optimization of batch and continuous processes from historical data

2. Optimization of products

• Development of new products

– Brings modeling, multivariate DOE and optimization into R&D


Concept of latent variables

Measurements are available on K physical variables: matrix=X

But, the process is actually driven by small set of “A” (A « K) latent

variables t1, t2, … tK (T).

– Raw material variations

– Equipment variations

– Environmental (temp, humidity, etc.) variations

K columns

= X


Projection of data on a lower dimensional latent variable space (T)

Latent variable spaceMeasured variables

TX

t1

t2

•SPE and T2 distances provide information on differences among observations

•Analysis, Monitoring, Control and Optimization done in latent variable space

Window into process


Latent variable regression models (PLS)

Two data matrices: X and Y

Choice of X and Y variables depend mainly upon the objectives of the study.

Symmetric in X & Y

PLS provides models for both X and Y and relationship between X and Y (Maximizes Cov(X,Y)).

X = TPT + E Y = TCT + F

TX Y


Optimization

• Using theoretical models

– Optimization based around fundamental models is widespread throughout the process industries

• Off-line optimization (eg in design)

• Real-time optimization

• Using empirical models

– What if no fundamental model exists or only partial theoretical information is available?

– Can we make use of the abundant historical data to optimize processes?

• For the active use of empirical models (eg. optimization and control), we need causal models.

– Causality for any active changes in the MV’s the model will predict the effect on the y’s


Causality in Empirical Models

• In the passive use of empirical models no causality is required• Model use only requires that future data have the same correlation structure

– Eg. Calibration; soft sensors; process monitoring

– Historical data is great for these

• But for active use of models as in optimization and control, one needs causal models

– For empirical models to be causal in certain x-variables – we need to have independent variation (DOE’s) in those x’s.

• Such data is often not readily available in industry.

– But industry has massive amounts of undesigned historical data.

• Causal models for the effects of individual x’s on the y’s cannot generally be extracted from such historical data

• All classical regression models (eg MLR, ANN, …) provide no causal information from such data

– Due to extreme correlation among the variables, these regression approaches provide an infinite number of indistinguishable solutions.


Causality in Latent Variable models

• LV models do provide unique and causal models

– Why? Because they simultaneously model both the X and Y spaces

• But they provide causality only in the low dimensional LV space

– i.e. if we move in LV space (t1=w1*x, t2=w2*x, …) we can predict the causal effects of these moves on Y (Ypred= TCT)

– Can use this fact together with the model of the X-space (X = TPT) to perform optimization and control in the LV spaces

– Now illustrate this idea of causality and optimization with some industrial examples of batch processes.


Optimization using batch production data (herbicide)

– Initial chemistry and charges (Z)

– Trajectories on 10 process variables (X)

– 12 final quality measures (Y)

– >200,000 measurements

Z X Y

Initial Conditions Variable Trajectories

End Properties

variables

bat

ches

LV model captures all relevant variation in Z, X and Y in 2 LV’s

Want to optimize y’s using all of Z and X.


Batch-Wise Unfolding of the data array

– The batch trajectory observations are extracted horizontally in a time-wise fashion

– Append both Quality (Y) and Initial condition data (Z)

– Each batch becomes a single row of data in the model

– PLS models built on these unfolded data matrices

Variable 1 Variable 2 Variable 3 Variable 4 Variable KBatch

Trajectory Data

Variables, K

N Batches

Time, T

Unfolded trajectory dataN Batches

Variables x Time, TxK

YZ


Trajectory alignment


Analysis of Z block

Score plotAnalysis reveals obvious clustering

among on- and off-spec batches.

The model also allows an understanding

of why off-spec batches occur.

The critical to quality variables were

process related, not chemistry related.

tz1

tz2

VIP’s for Z model


Analysis of unfolded X trajectories

• Loading vector w*1

for X model(Note the time varying w*s – captures the nonlinear time-varying behavior of the batch)

Score plot for X

• Clustering of good/bad batches

•Good batches have high t1

•Loadings w1* show that for high t1:a. Low tank level and used time for

entire batch

b. High Press, Jtemp, Dtemp in first

half only


Model Explore Tool in ProMV

• location on score plot

• save data from current pointer position

• view different data blocks

• X & Y values at the location on the score plot

• view different data blocks

• view the variables scaled


Often have multiple blocks of process data

• Latent variable software (ProMV) easily handles such multiple data blocks– Which blocks affect Y most (BIP) ?

– Which variables in each block are most important to Y?

– Get monitoring (MSPC) models for each block and the overall evolving process.

– Optimization in any of the blocks to maximize yield and quality

Z2 X1 Y

Initial Conditions

Batch Variable Trajectories

End product

quality

Z1

bat

ches

X3X2 X4 X5 X6

Intermediate processing steps


Ex. 2: Optimization of a Batch Polymerization

Joint study with Air Products and Chemicals.


Data after alignment


Batch polymerization (Air Products & Chemicals)

• 13 variables in YDesire a new product with the following final quality attributes (Y’s):

• Solution– Build batch PLS latent variable model on existing data (Z, X, Y)

– Perform an optimization in LV space to find optimal LV’s

– Use LV model of X-space to find the corresponding recipes and process trajectories

Maintain in normal ranges: Y1 Y2 Y3 Y4 Y5 Y6 Y8

Constraints: Y7 = Y7des

Y9 = Y9des

Y10< Y10const

Y11< Y11const

Y12 < Y12const

Y13 < Y13const

… and with the minimal possible batch time (*)


Optimization was initially done as a 2 step

procedure (Sal Garcia)

• Step 1: Solve for with constraints on T2 and on y’s

• Step 2: Solve for xnew that yields subject to certain constraints on SPE and x’s.

T T

* * * *new newnew 2 new new new new new new

new

minW W PW PW x τ G x τ x x Λ x x ηx

x

ˆnew

ˆnew

2

,T

des xnew 1 des xnew 21

ˆmin τˆ ˆ( ) ( ) ρ

ˆ

.

ˆ

Axnew a

axnew as

s t

xnew

y Q τ G y Q ττ

BQ τ b

Aspen-ProMV does this in one step by projecting all the constraints into the LV

space and optimizing in the constrained LV space.


All solutions satisfy the requirements on ydes

Case 1 to 5: weight on time-usage is gradually increased

Different solutions: change the penalty for total time


ProFormulate: Modeling & Optimization in

R&D

Rapid development of new products


Motivation

• Innovation is important: 93% of executives surveyed say their companies depend on innovation for long-term success1

• Current innovation strategies aren’t working: only 18% of executives believe their company derives a competitive advantage from its innovation strategy1

• The situation is getting worse: executives are less satisfied with innovation performance today than they were in 20091

• Product development at many big companies is characterized by:

– Almost trial and error approach

– Long development times

– Intuitive experience of individual researcher is often only way past information is carried forward (little information from other developers)

– No model-based optimization approaches used.

1Accenture Report: Why Low-Risk Innovation Is Costly

http://www.accenture.com/us-en/Pages/insight-low-risk-innovation-costly.aspx

http://www.accenture.com/us-en/Pages/insight-low-risk-innovation-costly.aspx


Ex. Development of a new biologic media for mAb

• This example had 65 raw materials variables from 13 categories.

• Trial history (in 2-D LV space) shows typical clusters of experiments.


What do we mean by Product Development?

• We mean:– Developing new chemical/food products for new

applications

– Improvement / reformulation of existing products

– Make existing products using alternative materials at lower cost

– Consolidation of products using fewer raw materials


Degrees of Freedom in Product Development

RawA

RawB

RawC

……

Chemical

Molecular

Appearance

Texture

Taste

RawA

RawC

40%

30%

30%RawP

RawZ

e.g.

Flowrates

temperature

mixing

Desired properties

To capture the synergisms among these, they must all be considered

simultaneously in the experimentation, modeling and optimization

Otherwise many experiments & long development times

Formulation

ratios

Selection of

raw materials

Process

conditions

Final product

properties


Product Development: An Integrated Program

1. Set up and maintain good databases• Never lose past development data

• Take sufficient measurements

2. Use multivariate models to continually extract the information from all past and new data

• Provides a repository for all the company’s development knowledge

• Continually add to and update models

3. Design Experiments to augment existing knowledge• Don’t start over with each new project - augment knowledge as needed

• DOE’s in LV space

4. Reformulate products via optimization in LV space• To achieve products with desired quality attributes, at lowest cost


Some ProSensus Product Development Projects

• Srixon golf ball cores – (Mitsubishi Chemicals/Sumitomo))

• Functional polymers for medical devices – (Mitsubishi Chemicals)

• High performance polymeric coatings

• Muffin batters – (Pepsico Foods)

• Cake compounds


Mitsubishi Chemicals example

Rubbers

Polypropylene

Oil

Functional

Polymers

Select materials from each class, their

blend ratios and the process conditions

Critical!

Pro

cess c

onditio

ns*

Develop functional polymers for medical devices, golf balls, etc

(Compounding problem)


Expt

No.

MaterialFinal

properties

Ru

bb

er

pro

pe

rty R

ub

be

rs

Rubberproperty

T

oilX

proY

Oil

pro

pe

rty

PP

pro

pe

rty

T

rubberX

T

PPX

Rubber

Oil

PP

DBrubber_X

Oils

Oilproperty

DBoil_X

DBPP _X

PPproperty

PP

Raw material property data

Formulations

Partial data from database of

raw material properties

rubberR

Raw material

property data

(subset)

oilR PPR

XDB

Data structure

Process

conditions

Z


00

01

10,1

0)ˆ(

..

)()(

,

,

,

1

,

1

2

,2

1

2

1

3

1

,21

jnew

jnew

j

jnew

NN

j

jnew

A

a a

anew

new

Kk

k

newmixnewmixnew

DBnewnewmix

NN

j

j

NN

j

jjnewPLSnewmixdes

T

PLSnewmixdesr

r

r

rr

consts

T

xxSPE

rx

ts

wcrwBxyWBxyMinnew

X

Mixture

constraint

PLS model

constraint

Ideal

mixing rule

Binary variable

constraint

00

01

10,1

0)ˆ(

..

)()(

,

,

,

1

,

1

2

,2

1

2

1

3

1

,21

jnew

jnew

j

jnew

NN

j

jnew

A

a a

anew

new

Kk

k

newmixnewmixnew

DBnewnewmix

NN

j

j

NN

j

jjnewPLSnewmixdes

T

PLSnewmixdesr

r

r

rr

consts

T

xxSPE

rx

ts

wcrwBxyWBxyMinnew

X

Mixture

constraint

PLS model

constraint

Ideal

mixing rule

Binary variable

constraint

Estimation error

Total

material cost

The number

of materials

Optimized variables:

The mixture ratios of all

possible raw materials available

in the database XDB, and

process conditions (Z)

Optimization using the models


DOE’s for Product Development

• Often industrial data bases are very rich but only in limited regions/clusters

– Limited choices of materials

– Limited formulation ratios

– Limited processing conditions

• Optimal DOE’s can be used to provide a small number of runs that can:

– Upgrade these databases for better development

– Optimize for a particular product


Concept of DOE in latent variable spaces

• Points show existing products

• Note regions of latent variable space where are no data

• Optimal DOE’s to find those scores (t1, t2, ….) that would fill in these holes

Latent variable

space

1

235

67

89 10

1112

13

14

15

18

19

20

23

24

2526

27

29

31

3233

34

35

3738

39

42

43

44

t1

t2


DOE in latent variable spaces (to improve DB)

• Experiments ( ) in score space

• From the DOE in the scores (t1, t2, …) use LV model to provide corresponding DOE in the raw materials, formulations and processing conditions: [Z, X, R]

• DOE in low dimensional score space provides a corresponding DOE in the high dimensional original variable space

1

235

67

89 10

1112

13

14

15

18

19

20

23

24

2526

27

29

31

3233

34

35

3738

39

42

43

44

t1

t2

Latent variable

space


Summary

• Latent Variable models have many advantages with “BIG” historical data:

– Easily handle many variables and missing data

– Models are unique and interpretable

– Causality in the Latent Variable space

• Useful for optimization of processes from historical data

– Allows for more powerful use of BIG process data

– Combination with fundamental models

• Useful for rapid development of new products

– Brings Multivariate modeling, DOE and optimization into R&D

optimization of processes & products using …focapo-cpc.org/pdf/macgregor.pdfoptimization of...

Documents