impact 2359 optimizationandbusinessprocesses revmay04 · – in bpm it points to business process...

Session Title goes here

Speaker name

#1111Optimization and Business Processes

Juan Carlos Morales, XM

Greger Ottosson, ILOG, an IBM Company

2359A

Agenda, Part I

• What is Optimization?

• Two Global 500 Stories

• Industries and Applications

• Business Processes, Optimization and Analytics

One expression, many interpretations

• Different reference points

– For a software developer, it refers to code optimization

– For a webmaster, it’s about search engine optimization

– In BPM it points to business process improvement

– At IBM ILOG, it means decision-support based on mathematical optimization

• They’re all different but they all aim at increasing efficiency.

• Using Google, 18 of the top 20 references are about mathematical optimization, a.k.a. Operations Research

What is Optimization?

What is Optimization?

• Optimization helps businesses:

– create the best possible plans and schedules,

– explore alternatives and understand trade-offs, and

– respond to changes in business operations.

• It answers questions like:

– (Manufacturing/Supply Chain) “How much of each product should I

produce, at what plant, and when? How should I price it?

– (Finance) “What trades should I perform to re-balance my portfolio

to maximize return and minimize transaction and tax costs?”

– (Retail) “What offers should I extend to what customers, when and

through what channel? What inventory should I keep? How should I

allocate my shelf space, and how should I place the products?”

What-If Analysis

Collaboration

Planning and Scheduling Processes

Agenda





Automotive Sales & Operations Planning

How many cars of

each type?

Delivered to what

market?

At what time?Produced where?

Business needs

• Business needs

– Efficiently allocate production capacity to market demand

– Make difficult supply-demand trade-offs

• Current issues

– Lacks agility (planning 3 months in advance)

– Difficult to accommodate campaigns & promotions

– Continuous re-adjustments of plan

• Solution

– Solution supporting collaboration

– Optimization for efficient supply-demand balancing

– Increased agility (planning 2 months in advance)

Feb

Mar

Apr

May

Actual

Orders

Improving

Forecasts

Solution

• Solution: Planning application

– Optimization for efficient balancing of Supply and Demand

– Collaboration to support multi-user planning process

• Technology: IBM ILOG Optimization, with

– Repository for multi-user collaborative planning

– Optimization Server & Engines for scalability

– Tools for optimization modeling

• Additional software

– IBM Websphere Application Server

– IBM DB2

– IBM Cognos for reporting

Retail Distribution Planning

How much of each

product?

Procured from

whom?

At what day?Delivered where?

Business Needs

• Business Process

– Distribution Planning for 130 supermarkets

• Business Goals

– Reduce distribution and inventory costs

– Eliminate risk of running out of stock

– What-if analysis for “stock up before price increase”

• Planning process

– Daily planning process

– One planner per distribution center

Solution

• Solution: Planning application, supporting

– Trade-offs between costs and inventory levels

– Scenario management & What-if analysis

• Technology: IBM ILOG Optimization, with

– Repository for scenario management

– Optimization Server & Engines for scalability

– Tools for optimization modeling

• Additional software

– IBM Websphere Application Server





Agenda

Other Optimization Customers

IBM ILOG Optimization for

• Sales & Operations Planning

• Collaborate Planning with Scenario

Sharing

• Optimization-based Supply-

Demand Balancing

IBM ILOG Optimization &

Visualization for

• Aircraft assembly scheduling and

monitoring

• Sophisticated visualization of project

dependencies

• Optimal sequencing of activities

Whiskey Manufacturer

IBM ILOG Optimization &

Visualization for

• Optimize loading of pallets and kegs

on trucks and rail cars

• Reduce inventory by direct loading


• Strategic production planning,

maturation and inventory

liquidation

• What-if analysis and reporting

Major Car Manufacturer


• Schedules 600 soccer games annually

in this German league of 36 clubs

• Satisfies league requirements and

team preferences


• Power generation scheduling

• Reduction of carbon emissions

• Leverage wind power

Industry Applications

Well-Documented Optimization ROI Cases

2 Chilean Forestry firms

Timber Harvesting$20M/yr + 30% fewer trucks

UPS Air Network Design$40M/yr + 10% fewer planes

South African Defense Force/Equip Planning $1.1B/yr

Motorola Procurement Mgmt $100M-150M/yr

Samsung Electronics Semiconductor Mfg50% reduction in cycle times

SNCF (French RR) Scheduling & Pricing$16M/yr rev + 2% lower op ex

Continental Airlines Crew Re-scheduling $40M/yr

AT&T Network Recovery35% reduction spare capacity

Grant Mayo van Otterloo

Portfolio Optimization $4M/yr

Source: Edelman Finalists, Science of Better, http://www.scienceofbetter.org

Agenda





Business Processes & Business

Process Management

• A business process is

– "a collection of related, structured activities

that produce a service or product that meet

the needs of a client.“

• Traditional process addressed with BPM

– Loan approval/Credit extension

– Underwriting/Servicing/Claims processing

– Order fulfillment/Available to promise

– Payment processing/Invoicing

– …

BPM processes vs P&S processes

• BPM processes

– Fine-grained transactions and decisions• Single order/application/claim

– Multiple process instances• One per transaction

– Real-time and automatic orchestration• Might involve human tasks/decisions

• Planning and Scheduling processes

– Global view of “supply and demand”• Decides how to allocate resources to tasks

– Single process instance• Exceptions include per-geo/per-channel/per-product decisions

– Usually tactical/strategical planning on snapshots• Exceptions include product configuration and real-time optimization

Connecting BPM and P&S Processes

Planning and Scheduling Process

Import snapshot

Export plan/schedule/rules

BPM Processes

Monthly/Weekly/Daily

ERPERP

CRMCRM

HRMHRM

AMSAMS

Example: Marketing Campaign Optimization

• Industry– Retail Bank

• Goal– 1-1 targeting of product offerings to customers

– Personalization (email vs phone, etc)

– Consistency and timing (no repeat offers, etc)

– Maximize resource efficiency (call centers, etc)

• Two intertwined processes– Tactical optimization-based planning process to determine the

• Which offer to be madem to what particular client, at what time, and through what channel

– Transactional processes to extend & track the offers, through

• Web, Emails, Calls, Mailings

5/13/2009

Scoring

(Optimization)

Decision Management

Query/Report

Time Series

Regression

Data Warehouse

Models,

Scores,

Forecasts

Decisions, Reasons

OperationalApplication

Optimization,

Simulation,Policy Management

PredictiveAnalytics

Segmentation

Policy Rules,

Models

Decision Warehouse

Business RulesRepository

PlannersPlanners

Plan or

Schedule

AnalystsAnalysts

Optimization Rule Management

Rule Execution

Prediction`

Decision

Execution

Analysis

Decision Management – Marketing Example

Prediction`

Decision

Execution

Analysis

Likehood to:

• Accept offer

• Use product

• Generate profit

• Generate loss

Decision points:

• Online

• Event-driven

• Mailings

• Inserts

Decisions (plan):

• Who’s eligible for what offer

• When to make the offer

• Subject to: cap on total risk exposure

Outcome:

• Accepted offers

• Use of products

• Profit/loss

Optimization & Analytics

• Davenport slide…

Co

mp

eti

tive A

dvan

tag

e

Degree of Intelligence

Std Reports

Ad hocReports

QueryDrilldown

Alerts

StatisticalAnalysis

Forecasting

Optimization

Predictive Modeling

What happened?

How many, often, where?

Where exactly is the problem?

What actions are needed?

What will happen next?

What if these trends continue?

Why is this happening?

What’s the best that can happen?

Source: Davenport, “Competing on Analytics”

Summary

• Optimization helps businesses:

– create the best possible plans and schedules,

– explore alternatives and understand trade-offs, and

– respond to changes in business operations.

• Planning and Scheduling Processes vs BPM

– Complementary and sometimes intertwined

• Optimization vs Analytics

– Analytics to analyze and anticipate

– Optimization for efficient decisions and trade-offs

Experiences with the Application of

Optimization Technology for the

Colombian Electricity Market

Juan Carlos Morales

XM S.A E.S.P.

Agenda, Part II

1. Description of the Colombian Market

2. Economic dispatch model of the Colombian electricity system and other solutions developed by the Colombian ISO using IBM ILOG – CPLEX – OPL

3. Conclusions and Results

Market Structure

XM is in charge of operating the National Interconnected Power

System through the National Dispatch Center -CND- and of

administering the Wholesale Electricity Market -MEM-.

DISTRIBUTION (wires)

Suppliers pass their costs through to their clients

COMMERCIALIZATION

� Regional Monopolies

� Free access to the grid

� Regulated charges

� Competition

� OTC contracts, negotiated prices

� Bid Competition in the spot market� Imports from other countries (No TIE)

GENERATION

� Regulated

� Unregulated

� Public Lighting

� Exports to other countries (No TIE)

CLIENTS

NationalDispatchCenter

Financial Operations

SYSTEMOPERATION

MARKET ADMINISTRATION

� Natural Monopoly

� Competition since 1999 in expansion(for the market)

� Free access to the grid and regulated charges

TRANSMISSION

� Buy and sell energy

� Competition� Commercialization margin approved by CREG

for the regulated market

Ecuador Market

Short Term International Energy Transactions (TIE)

XM – WHOSALE ENERGY MARKET NUMBERS

“XM, Compañía de Expertos en Mercados” is in charge of the operation of the Colombian National Interconnected System, and also administers the Wholesale Energy Market –MEM– that in 2008 recorded transactions worth over USD$5,524.00 millions, of which28% corresponded to transactions in the Colombian Electricity Market.

Market Transaction – Whosale Energy Market

WHOLESALE ENERGY MARKET

Long Term Energy Market

a bilateral contracts market in which generators and electricity suppliers sell and purchase energy at freely-negotiated prices and quantities

Short Term Energy Market

a daily spot market that determines the spot energy price in every hour as

well as the dispatch of resources

Energy is traded on three competitive markets

Electricity Pool

• On the Colombian Wholesale Electricity Market, the Pool is where all generation plants compete on the basis of bidding prices to supply the electricity demand of the National Interconnected System.

• Traders do not participate in an active way in the Pool since they cannot submit bids.

• When prices are not the result of competition, generation plants are paid based on their variable costs which have regulated maximum values.

Firm Energy Market

a firm energy market to ensure that there will be sufficient energy

resources, particularly in exceptionally dry periods

Market Clearing Function – System Operator

PLANNING AND

CONTROL

Supply Energy and

services

Demand Energy and

services

Inelastic

Demand

The Generators

Submit bids that

compete hourly

Reconcile

supply and

demand

The market clearing function determines the quantities that market

participants produce or consume, and the prices that are used to settle

the quantities of electricity services produced or consumed maintaining

secure and reliable operation of the physical system

33

Planning and Control Activities

Planning and Control Operation

Planning

Control

Years Months/weeks Days/Hours

Time Horizon

Sig

nific

an

ce

of

Pla

nn

ing o

r C

on

tro

l

Long-Term PlanningStochastic Problem• Hydrology

• Aggregated demand

Medium-Term PlanningStochastic Problem• Hydrology

• Partially aggregated demand

• Set operations sets

Short-Term Planning and ControlDeterministic Problem• Bid prices

• disaggregated demand

• Set operations sets

Po

st O

pera

tion

• Prices bid – Startup prices

• Declared availability

• Transmission constraints (represented by the area import/export limits and network constraints)

• Up and down reserve for AGC

• Unit characteristics: Minimum up/down times, ramp constraints, others

• Demand forecast

• Reliability Information





• Unit characteristics: Minimum up/down times, ramp constraints, others

• Demand forecast


08:00

• Interconnections price curves (exportation)

• Import price curve

• Interconnections price curves (exportation)

• Import price curve

Coordinated Dispatch13:00

14:45

A 24-hour optimization process aimed at the minimization of operating costs subjected to generation and network constraints.

The System Operator computes the optimal schedule for each thermal unit and hydro plants for each hour of the next day and for the international interconnection with Ecuador (Implicit auction)

15:05

Dispatch with Venezuela

Short-Term Planning and Control Applications

Economic Dispatch (Day – Ahead Hourly Based)

Basic characteristics of the DRP program:

• Mathematical formulation using Mixed Integer Programming.

• The DRP has been developed by components.

• Extremely fast execution times.

• Module of mathematical modeling.

• Matrix generation in standard formats (LP).

• Advanced User Interface Graphic.

• High performance.

• DRP allows the modeling of different scenarios (dispatch, test, training, etc)

• DRP models the technical characteristics of the units and the transmission network with

voltage levels equal to or greater than 110 kV. The dispatch problem is modeled with a

network of 212 generation units, 600 transmission lines and 438 buses.

Short-Term Planning and Control ApplicationsEconomic Dispatch Process

General Characteristics

XM has developed an Application called DRP based on Mixed Integer

Linear Programming (MILP) approach to determine the quantities that

market participants produce or consume maintaining a secure and

reliable operation of the physical system.

Short-Term Planning and Control Applications Economic Dispatch Model

Problem Description

The dispatch problem is solved by optimization techniques using Mixed Integer Programming. Minimizing the operational cost of the system subject to the technical characteristics of the units, the frequency reserve and the network constraints.

Matrix LPMatrix LP

SolveSolve

SolutionSolution

ReportsReports

Mathematical InterfaceMathematical Interface Data BaseData Base

CPLEX Callable Libraries

Planned in a second phase

DRP models a system of 212 generation units, 600 transmission lines and 438 buses and gets the solution in an average of 83 seconds

CPLEX

Hydrothermal dispatch system with linear network constraints for short, medium and long term operation studies (Work in progress

with OPL)

Stochastic optimization methodology� Thermal plant modeling� A Complete hydro plants modeling� Transmission network� Market constraints� Risk

Long-Term Planning and Control Applications Hydrothermal Dispatch Model

Spot Market Calculation

Short-Term Energy Market Application

ActualGeneration

IdealGeneration

Ideal

PAYS (R-)

Actual

Actual

Ideal

RECEIVES (R+)

dif dif

Constraint cost

Correctly

determining spot

electricity prices is

critical to

successful

competitive market

operation

Day (d) Day (d+1)





• Unit characteristics

• Demand forecast







• Demand forecast


• Prices bid – startup prices

• Real generation availability


• Real Demand

• Prices bid – startup prices

• Real generation availability


• Real Demand

The spot energy market is a single-zone hourly market with supplier offers submitted one day ahead of dispatch. The process computes the Spot Price Market for each hour of the day d.

Bilateral Contracts Market in which generators and electricity suppliers sell and purchaseenergy at freely-negotiated prices and quantities. This market is fundamentally financial: the purpose of the contracts is to reduce the exposure of both the supplier and the end-user to price volatility in the spot market, while the physical delivery of the energy committed in these contracts occurs via the spot market.

OPERATIONAL PROCESS

COMERCIAL PROCESS

Wholesale Electricity

Market

Measures

Operational Information

1 5 9 13 17 21Long Term Contracts

COMMERCIALPROCESS

BY AGENTS

AGENTSGenerators

Traders

Over 150,000 gathered every day

Over 310 contracts are dispatched every day

Over 45,000,000.00 operations every month

Ex –

po

st

pro

cess

Dispatch Contracts

Long-term Market Energy Market ApplicationContract Dispatch (Work in progress with OPL)

Firm Energy for the Reliability Charge (ENFICC)

ENFICC refers to the maximum electric energy that a generation

plant is able to deliver on a continual basis during a year, in

extreme conditions of hydro inflows.

Basic characteristics:

The ENFICC of hydraulic plants is calculated using a

computational model (available in the Web Page

www.creg.gov.co), which maximizes the minimum energy that a

hydraulic generation plant can produce monthly during dry periods,

using CPLEX as the optimization engine.

Energy Firm CalculationWholesale Energy Market Application

The firm energy market pays generators a Reliability

Charge determined by a competitive auction for each

MWh of firm energy committed. Generators in turn are

obligated to supply energy at a fixed price whenever

spot prices exceed a pre-defined “Scarcity Price”.

CPLEX Callable Libraries

CPLEX

SummaryXM is in charge of operating Colombia’s National Interconnected Power System through the National Dispatch Center (CND) and administering the country’s wholesale electricity market (MEM).

XM as the Independent System Operator in Colombia is in charge of maintaining secure and reliable operation of the physical system, providing correct economic signals to enable competitive market activities. This presentation describes XM’spractical experiences with the application of optimization technology for the Colombian Electricity Market.

43

Conclusions and Results

Results• Since March 1st 2003, DRP is being used as the official software

to calculate the economic dispatch in the Colombian Power

System. Although the economic dispatch is a big and a complex

problem which must be solved in a very short period of time, DRP

has allowed us to reach high levels of reliability and opportunity,

nearly to 100%, in the delivering of the final dispatch program.

• Sum up, DRP is a Unit Commitment problem with Network Linear

Constraints (NLC - Optimal Power DC Flow), which resolves the

economic dispatch problem according to all the rules of the

Colombian Market.

• XM is working to develop in house some of the most critical

applications for the business (power systems and energy

markets); as the DRP, Ideal Dispatch, Firm Energy, Contracts,

Hydrothermal dispatch.

• Among others, DRP obtains the following results: The dispatch

program for each thermal unit and hydroelectric plant, marginal

nodal prices, the marginal system price, power flows between

electric areas and transmission lines, the frequency reserve

program and international transaction programs (TIE), etc.

Conclusions

• According to the dynamics of the Colombian Power Markets, it has

become necessary to control our own application software. DRP and

the other applications allow us to implement new and very complex

constraints, making it possible to incorporate new rules into any

power and energy market solutions.

• The Applications have been developed according to the

specifications and requirements of proper users in order to reduce the

processing time and the human mistakes in the manipulation of data.

• Despite the size and complexity of the Dispatch problem, the

application’s speed and accuracy have enabled XM to reach high

levels of reliability and opportunity – nearly 100 percent in delivering

the final dispatch program. The solution obtained by this formulation

also provides marginal price information associated with system

constraints such as load demand requirement

Calle 12 Sur N°18 – 168, PBX: 57(4) 317 22 44 FAX: 57(4) 317 08 33, Medellín –

Colombia

www.xm.com.co

XM Compañía de Expertos en

Mercados

Characteristics of the Colombian Power System

http://portal.xm.com.co/Pages/DescripciondelSistemaElectricoColombiano.aspx

The total installed capacity is 13,457 MW composed of 64%

Hydroelectric power and 36% Thermal.

In 2008, the peak load was 9,079MW. The

energy demand accounted for 53,870 GWh,

with an increase of 1.64% compared with 2007.

End User Tariffs (USDc/kWh)

Residential 9.1

Industrial 8.4

Hydraulic Plants 8,525.00

Thermal - Coal 967.00

Thermal - Gas 2,757.00

Thermal - Fuel Oil 619.00

Self-producers and co-generators 25.00

Small Plants 564.00

Total 13,457.00

Maximum Volume 16402

500 kV 2399

220 - 230 kV 11674

110 - 115 kV 11,290.00

Annual (GWh) 53,870.00Annual Growth (2008 vrs 2007) 1.640%

Peak Load (MW) 9,079.00

Transmission Networks (km)

Electricity Demand

Generation Capacity (MW)

Reservoirs (GWh)

Characteristics of the Colombian Power System and International Interconnections

336 MW336 MW

500 MW500 MWCol - Ecuador

Implicit AuctionsDecoupled Systems

Col - VenBilateral Contracts

National

networks

•Colombia – Venezuela

Bilateral contracts

•Colombia – Ecuador

Implicit Auctions (2003)

•Colombia – Panama Future

Col - Panamá

300 MW300 MW

Venezuela

Colombia

Ecuador

Perú

We love your Feedback!

• Don’t forget to submit your Impact session and speaker feedback! Your feedback is very important to us, we use it to improve our conference for you next year.

• Go to www.impact09guide.com on a smartphonedevice or a loaner device

• From the Impact 2009 Online Conference Guide;– Select Agenda

– Navigate to the session you want to give feedback on– Select the session or speaker feedback links

– Submit your feedback

© IBM Corporation 2009. All Rights Reserved.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

The following are trademarks of the International Business Machines Corporation in the United States and/or other countries: ibm.com/legal/copytrade.shtmlAIX, CICS, CICSPlex, DataPower, DB2, DB2 Universal Database, i5/OS, IBM, the IBM logo, IMS/ESA, Power Systems, Lotus, OMEGAMON, OS/390, Parallel Sysplex, pureXML, Rational, Redbooks, Sametime, SMART SOA, System z , Tivoli, WebSphere, and z/OS.

A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at ibm.com/legal/copytrade.shtml.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government CommerceJava and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark OfficeIntel and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.UNIX is a registered trademark of The Open Group in the United States and other countries.Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.

impact 2359 optimizationandbusinessprocesses revmay04 · – in bpm it points to business process...

Documents