Help-Desk Systems

Stephen Lee-Urban

November 17, 2006

2

References

• Experience Management: Foundations, Development Methodology, and Internet-Based Applications, Ralph Bergmann.– Chapter 9: Developing and Maintaining

Experience Management Applications– Chapter 11: Experience Management for Self-

Service and Help-Desk Support

3

Outline

• Motivation / Goals / Background

• The HOMER architecture

• Developing & Managing EM* Applications

• Evaluation of HOMER

*EM = Experience Management

4

Motivation

• Complexity of technology increasing– Operation, Maintenance, Repair– Probability of failure grows exponentially with

complexity– Expertise in all features unreasonable

• Problem solving experience distributed– Experience overlap is variable

Experience Management System (EMS)

5

Goals of EMS

• Create knowledge repository– Store problem solving experience– Simplify trouble-shooting complex technical

domain– Domain is likely to change over time

• Use knowledge repository– By varying levels of expertise– In time-critical situations

6

Experience Management vs CBR

Experience Management

CBR

(Organization)

(IDSS)2. Reuse3. Revise

4. Retain

Case Library

1. RetrieveBackground Knowledge

Experience base

Reuse-related

knowledge

Problem acquisition

Experience evaluation and retrieval

Experience adaptation

Experience presentation

Complex problem solving

Developm

ent and M

anagement

Methodologies

BO

OK

Slide: Dr. Munoz-Avila

7

Help Desk Support Levels

• Key-user

• Specific technician

• Vendor

• Help-Desk sys.

• Bottlenecks at each level. – Problems recur.– Frustration for all parties, and wasteful of resources!

• “Managerial” Aids to Help-Desk: – Inventory systems, trouble-ticket tools, call-tracking software– Don’t support diagnosis nor serve as knowledge repository

http://www.simpsonstrivia.com.ar/simpsons-photos/wallpapers/homer-simpson-wallpaper-brain-1024.jpg

CBR

8

Outline

• Motivation / Goals / Background

• The HOMER architecture

• Developing & Managing EM* Applications

• Evaluation of HOMER

9

The HOMER System

• German: “HOtline Mit ERfahrung”• Developed as part of INRECA-II project• For CAD/CAM help-desk at

DaimlerChrysler in Sindelfingen• Generic experience management

architecture & set of related tools– Stores experience in CB for access, reuse

and extension– Drastically decreases problem solving time

http://www.aeria.phil.uni-erlangen.de/photo_html/portraet/griechisch/dichter/homer/homer3.JPGhttp://gattaca.com.ar/weblog/wp-content/homer.gif

10

Structure and Representation• Determines experience management approach• HOMER uses an object oriented approach to model experience. • Advantages of OO approach:

– Structure of system to diagnose is representable in detail– Symptoms (attr.) easily related to owning object– Semantics of prob. description can be captured and used for selecting

appropriate prior exp.– High retrieval accuracy can be achieved– Particularly suited for diagnosis help-desks w/ complex equipment

Can show many hard to diagnose faults– Can be used to help guide help-desk operator while describing and

entering cases– Better domain model = easier maintenance and use

• Disadvantages of OO approach: – Harder to create model– Additional effort in beginning of knowledge acquisition

Representation should be discussed with system admin. Shallow?

11

Case Structure

• Modeled according to approach used by help-desk operators to solve problems

• Feasible for most complex help-desks

• Topic: problem area (e.g. hardware)• Subject: failing object (e.g. printer)• Behavior: way (miss-) behaves (e.g crashes)

• Minimum set of attribute-value pairs describing symptoms necessary to fault diagnosis.• In OO, all attributes are related to certain object.

• Fault: problem cause• Remedy: problem fix• Represented as (hyper-) text

Case: Complete path from prob. to solution

12

Experience Base Partitioning

• Domain size & complexity + demanded accuracy & consistency = partitioning

• Two kinds of cases:– Approved: experience reviewed by top-level experts.

“Best Practice” for problem solving– Open: recently captured, pending validation

(correctness, completeness, & utility)

• Case base separated into:– Case Buffer – all open cases– Main Case Base – all approved cases

All available to help-desk operators

13

HOMER Architecture

• Client-Server– Shared domain model (DM) + case base (CB)– Eases maintenance of DM + CB– Server accessible via intranet / internet– “Fat” client reduces network traffic

• Object-Oriented experience model– Not “shallow” users are experienced– For non-trivial problems

14

• Lowest access rights• Daily user• Retrieval & acquisition

• Middle rights• Case maint. & case approval• Redundancy & consistency checks• Buffer to base• May modify value ranges of attributes

• Highest rights• Creates & maint. domain and case model• +/- attr & concepts• Admin users and rights

15

The Server

• CBR-Works server stores model & CB,

• CBR-Works modeling tools provided by server for domain modeling, case & model maintenance, & initial case acquisition

• Domain model snapshot…

16

Hierarchical domain

concepts

Help Desk Case

Attributes

• Problem, holds failure• Situation, holds symptoms - Hierarchical, sub-concepts - Structure helps maint. & retrieval• Loesung, holds solution• Administrativa, holds organizational & statistical information

17

The Client

• Interface to server for case retrieval, case acquisition, & case browsing

• Written in Java• “Hotline” component for help-desk operator

– Only shows relevant information– Assists via two modes: user/system driven

• “Case Browser” component for experience author– For access to CB and case buffer– Revision, extension, approval of “open” cases– Removal of outdated cases

18

Hotline Component (HC)

• Used mainly during hotline operations

• Supports help-desk operator during problem-solving process

• Four main modes of execution– Problem description (initialization/refinement)– Situation description (user/system driven)– Solution retrieval (manual/automatic)– Retain w/ feedback

questions

answers to questions

solutions (cases)

problems

19

HC: Problem Description

• Gives initial info. on user’s problem

• Answer following questions consecutively:– Problem’s topic? (e.g. output, software, …)– Topic’s subject? (e.g. output: plotter or printer)– Behavior? (e.g. “no output at all”)

problems

question (free text)

values

value range units

20

HC: Situation Description

• Problem determines relevant case structure (i.e. situation template)– Contains possible symptoms for prob.– Each symptom associated w/ question

• Symptom attribute values complex/simple• Two modes of operation

– User-driven: (no guidance, direct entry)– System-driven:

• Shows sorted view of most relevant questions • Based on attributes with highest info gain

21

HC: Solution Retrieval

• Experience retrieved based on problem and situation descriptions

• Manual (batch) / automatic (per question)

• Each row a solution, sorted by relevance (similarity)

• Solutions viewed (read-only)

22

HC: Retain/Feedback

• Retain case when it – Has new experience– Requires case model extension (e.g. new

value to type)

• Case entry interface allows– Final modifications (e.g. finish case descr.)– Annotating why op. thinks case should be

retained

23

Case Browser Component

• Used by experience author to manage CB

• Works on both approved and open cases

• Can perform following operations:– Case creation– Case copy– Case deletion– Case approval

Problem, situation, solution, administrativa

question (free text)

values

value range units

24

Outline

• Motivation / Goals / Background

• The HOMER architecture

• Developing & Managing EM* Applications

• Evaluation of HOMER

25

Developing & Maintaining EM Applications

• IT companies require efficient, effective application development– Guidelines/Methods of implementing apps– Also seek to preserve past project experience

• INRECA methodology– Targeted at industrial EM applications– Developed in the INCRECA-II European

ESPRIT project (1996-9)

26

EM Model

Knowledge Kernel

Problem Solving Cycle

• Knowledge Acquisition & Knowledge Maintenance• Technical, organizational & managerial aspects

27

Three Process Types

• Technical– Describe development of system & required documentation– E.g. requirements analysis, system design, implementation and

testing

• Organizational– Address parts of business process in which software will be

embedded– E.g. training end users, archiving request records, updating &

maintaining the help-desk system

• Managerial– Provides environment and services for developing software that

meets product requirements and project goals– E.g. project planning, monitoring, quality assurance

28

Methodologies

• Makes development an engineering activity, rather than an art

• Use of methodology provides benefits:– Productivity, Quality, Communication, Management

decision making• INRECA methodology based on software

engineering principles, covering:– Project management (cost assessment, schedules,

project plans, etc.)– Product/Deliverable specification– Product development and maintenance– Analysis/Organization of target env. for CBR system

29

INRECA

• Basic philosophy: experience based construction of experience management applications

• Self-application of principles of experience reuse

• Experience modeled as structured text documents, hyper-text linked.– Origin: Software process modeling

30

Software Process Modeling

• Well defined terminology to describe the engineering of a product– Process, “what”: basic step to carry out, transforming input into

output. Defined by a goal, a set of alternative methods, input/output/modified products, & required resources (agent/tool)

– Method (simple/complex) “how”: Detailed specification of a way to achieve a process’ goal

– Product: goal of the process

INRECA notation

31

INRECA Process Models

• Make explicit all processes, products, methods, resources and interactions

• Diagrams insufficient Each element must be described in detail– Called a process model– Solid basis for project planning (e.g. effort

calculable based on processes involved)

• General vs. Specific descriptions– Common Generic/Cookbook vs. Project level

32

The INRECA Experience Base• Collection of processes, products, & methods common across EM apps.

• Processes, products, & methods tailored to class of applications (e.g. help-desk). • Each class has recipe• Recipe: has process models describing how app. of that class be developed & maintained

• Describes experience of an instance of a completed project• Project-specific info. (e.g. processes carried out, effort of processes, etc.)

33

The Common Generic Level (CGL)

• Overview of top-level processes and products in Bergmann, chapter 9– General enough to occur in many projects– Problem statement “vision document”

goal checklist feasibility study detailed analysis of organization project plan software development evaluation

– Meanwhile, experience base can be consulted for all of the above processes

• Consult book for thorough exposition

34

Go / No-go(analysis &elicitation)

Organizational(planning)

Implementation

Maintenance

35

CGL: The Three Processes

• Managing an AI / EM project differs from other IT projects– Concepts & technologies altogether new– Emphasize early awareness training– Ensure user-participation in iterative prototyping

process

• Technical processes very typical of software development projects

• Organizational includes identifying perceived problems and opportunities at the human & organizational levels

36

Documenting in INRECA

• Processes, products, & methods documented and stored in experience base

• “Sheets” structured page w/ all relevant information in a predefined format– Standardize documentation– Created as web pages for easy access & use– CGL has more than 150 linked sheets– 8 type of description sheets:

[ generic | specific ] [process | product | simple method | complex method]

– Contains references to respective input, output, and modified products of the process

37

Process Description Sheets

• Recipe/Project Name• Process Name• Process Goal• Input/Output/Modified Products• Set of Applicable Methods (names)• Agents (e.g. personnel) • Required Tools• Administrative Information (e.g. sheet author,

version, last modified)

38

Product Description Sheets

• Module/Project Name

• Product Name (e.g. “requirements doc”)

• Product Description

• Administrative Information

39

Simple Method Description Sheets

• Module/Project Name

• Method Name

• Method Description

• Administrative Information

40

Complex Method Description Sheets

• Module/Project Name

• Method Name

• Method Description

• Details– Links to a product flow description which

contains the relevant sub-processes (byproducts)

• Administrative Information

41

The INRECA Reuse Procedure

• Recipes at cookbook level are most useful for building a new application

• Even projects not fitting a recipe can use processes described in CGL

42

Don’t Forget to Remember!

• After completing project, include it in the experience base continuous improvement of the EM software development process

43

Tool Support for INRECA

• Experience modeling methodology tool implemented in Visio®– Natural choice: shapes, hierarchical, HTML, VBA, database

access

• Knowledge modeling tools– Integrated in the CBR-Works tool (tec:inno GmbH 1999)– CBR-Works Concept & Type Hierarchy Editor for vocabulary

modeling – Similarity modeling tools integrated in the Concept & Type

editors

• Also has an editor for adaptation and completion rules

44

Outline

• Motivation / Goals / Background

• The HOMER architecture

• Developing & Managing EM* Applications

• Evaluation of HOMER

45

Evaluation of HOMER

• Evaluation performed by INCRECA-II project partners to identify benefits of EMS

• Incoming calls monitored. Help-desk operator 1st solves conventionally, then with HOMER– No solution new case created– Two month test period: 102 calls; 45 unsuitable for

HOMER

46

Evaluation (II)

• Of the 57 (/102) suitable problems– 18 solved (i.e. 32%)

• Ave. resolution time w/out HOMER: 141 min.• Ave. resolution time w/ HOMER: 9 min.• Correct result a limited bias HOMER sys. Mode

• HOMER + CB transferred to a new site– Operators have no experience with the process chain

so the cases are of great value– Initial knowledge acquisition gave operators insight

into domain.

47

Evaluation (III)

• Methodology recipe created and used– Impact: productivity, quality, communication, &

management decision making– Customer and developer both benefited

• Creation of project definition from scratch– Took three months– New development team reused recipe to define three

new projects, each in < one week (12x speedup!)• Were sure all relevant aspects taken into account• Basic recipe available developers focus on domain

peculiarities quality/detail of descriptions greatly enhanced

48

Evaluation (IV)

• Development & testing of 1st prototype– About 6 months– Subsequent efforts: 2 weeks (13x speedup)– Less qualified personnel w/out sacrifice of quality

• Useful as training tool for maintenance & use• Message: development of EM system a science,

not art– Each process describable– Validity of approach defensible– Realistic expectations of effort by whom and when– Measurable project process

49

Thank You!

(Questions?) (Comments?)