Knowledge and Learning in Complex Business Systems

Zuobing Xu University of California , Santa Cruz (Silicon Valley Center)

Ram Akella, Kristin Fridgeirsdottir, Eric Wang, Arvind VidyarthiINFORMS Pittsburgh, November 6, 2006

2

Overview: Perspective on Learning and Knowledge

- Semiconductor process learning (KLA, AMD, INTEL)

- Automotive product development learning (Delphi)

- IT service center knowledge mining and learning (IBM, Cisco)

3

Perspective on Learning and Knowledge - 1

Semiconductor process yield management and learning.The models were new, but still in the OR framework

4

Perspective on Learning and Knowledge - 2

- In the automotive environment, OR modeling ran into a roadblock

- Forced to learn ontology to speed up learning in new product development

- Subsequently, discovered that learning and knowledge management are closely related to text mining and information retrieval, Applicable in airlines, health care , service/call centers

5

Perspective on Learning and Knowledge - 3

IT Service Center

- Millions of documents (services call logs, Technical documents)

- We use text mining and information retrieval to manage and extract knowledge efficiently.

6

Is There a Unifying Framework for Knowledge and Learning?

Do not know, but..Need to combine

• Operations Management• Knowledge Management• Text Mining

Specific research examples follow

Process Learning in the Semiconductor Industry

8

In-line Monitoring for Semiconductor Product-Process Learning and Manufacturing

Isolation Metal 2Poly 1 Poly2 Metal 1 WaferProbe

Þ 2110Þ 2110KLA KLA

PhotoPhoto

DepoDepoEtchEtch

Wafer processing Inspection Classification

Off-lineReview

In-lineADC

or

10 days

30 daysYield

9

Business Need

• Methodology: For rapid product and process learning– How to use inspection machines to improve yield

per machine

10

7 20 9

Detection Delay

EventOccurring

EventDetected

SourceIsolated

FixValidated

SourceIsolation

RootCauseAnalysis

Corrective Actions

17 2Hours

50

In-Control

Defect level

REDEFINED GOAL: MINIMIZING TIME TO DETECT AND FIX YIELD EXCURSION ( AN EXCURSION @ ETCHER IN FAB A)

Goal Optimize procedures and inspection-review machine usage to reduce delay to detect and fix yield excursion•Using defects as surrogates ( linking defects to yield is a

technology problem in electrical/computer engineering)

11

Resources &strategies

Isolation Poly1 Poly2 Metal1 Metal2

Market

Review &classification

Review &classification

Sourceidentification

Sourceidentification

Data/information flowData/information flow

Root-causeanalysis

Root-causeanalysis

Correctiveactions

Correctiveactions

InspectionInspectionValidationValidation

Goal Detect killer defect excursion faster through efficient integrated inspection-review cyclesTrade-off: Time versus benefit and cost

Conversion Of Defect Data To Yield Information And Action

12

Analytics

Developed generalized Neyman-Pearson Lemma for multi-dimensional control charts

Model queuing effects of inspection-review-isolation-fix-validation cycle

Business and Knowledge Management in the Automotive Industry

14

Shift in Learning Paradigm: From Numerical to Text

Knowledge and learning are now associated with text

Traditional OR models cannot be directly used!

What do we do?

15

Triggered Learning Process: “Dynamic’ Ontology Usage

Step #2 – Update

Step #3 –Communicate

Step #4 – React

Step #1 – Feedback

GM Dealer Customer

TLP is a structured approach that 1. Feeds back lessons learned created by downstream organizational

personnel2. To a staff that condenses these lessons learned into an ontology and,3. Communicates these items to NPD4. NPD personnel reacts to this information as it arrives by incorporating

it into the new product or process under development

First timequality

Safety and ergonomics

Formal OEM complaints

Warranty Long-term durability

NPD Production OEM Field <3 yrs Field >3 yrs

Delphi GM Dealer Customer

Lessons Learned Ontology

16

Step 2: Update Ontology

Staff used • To summarize the Lesson

Learned in the ontology • “Attach” the Lesson Learned

documents provided

Business and Knowledge Management in IT Services

18

Service Center Work Flow

Search or browse for solution(self service)

Call service center

Solve the problem by their own knowledge

Search or browse internal database

for solution

Customer have trouble with product

Deskside Expert help

19

Improve IT Service Performance by Text Mining and Information Retrieval

- Text Mining : Organize Technical Documentation Automatically - Information Retrieval : Retrieve and Find the Correct Solution

Can we trust computer algorithm?

- We need human guidance ( Relevance Feedback).

How can we reduce human effort ?

- Active learning ( Actively selecting document for human evaluation )

20

Query Retrieval Engine

Retrieval Engine’s Ranking d1=3.4d2=2.0

…d10=1.0

Judgments d1(Non-relevant)d2(Non-relevant)

…

d10 (Relevant)

FeedbackQuery Update

DocumentCollection

Active Learning in Information Retrieval

Service Engineer or Customer

21

Active Learning to Choose Feedback Documents

Goal : choose a set of documents containing most information for user to evaluate.

Relevance Select highly ranked retrieval documents given by search engine

Diversity Increase distance between selected documents .

Density Choose document in high density region.

22

Experimental Results

BaseLine Top K Gap K Cluster Active improv. improv.No feedback New Method (BaseLine) (Cluster)

MAP 0.1919 0.205 0.203 0.2145 0.2403 21.05% 12.03%Pr@10 0.434 0.444 0.44 0.5 0.526 21.20% 5.20% MAP 0.3151 0.3339 0.3306 0.3751 0.3781 20.00% 0.80%Pr@10 0.5 0.506 0.506 0.572 0.61 22% 6.64%

Existing Feedbak Methods

Hard 2003

Hard 2005

Data Eval.

Mean Average Precision (MAP) - Measure of overall ranking accuracy.

Precision at 10 documents(Pr@10) - Measure of the precision for the first 10 documents.

23

Organize Documents

- Technical documents and case logs always have some meta data indicating its product category.

- But problem description, root cause , solution are not well categorized.

- Design a new active semi-supervised text clustering algorithm to group documents.

24

Text Mining: Semi-supervised clustering with pair-wise constraints (Organize Trouble Tickets)

Can not be linked!!!!

25

Future Research for Knowledge Management of Service Center

- Using OR to model the benefit and cost for human interaction. - Feed the lessons learned from service center to New product develop process. - Combine text analytics with call center scheduling.

26

Conclusions: Unified Model for Knowledge and

Learning

Data (Defect signal, Documents)

Information Knowledge Management

Reduceoperations cost +Increase overall efficiency

Statistical DetectionBuilding Ontology

Text ClusteringInformation Retrieval

… …

Queuing Model

Resource Allocation

… …

Goal