© 2015 IBM Corporation

Auto-Scaling and Dynamic Routing for WebSphere Liberty Collectives Steve Fontes Senior Software Engineer WebSphere Application Server Intelligent Management

Please Note: • IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

• Information regarding potential future products is intended to outline our general

product direction and it should not be relied on in making a purchasing decision.

• The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.

• The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

1

Dynamic Routing in a Liberty Collective

BACKGROUND Dynamic Routing

Static routing (1 of 2)

• Web server with plug-in routes web requests

• Requests are routed based on routing information in the plug-in configuration file

• This file is generated using Liberty utilities • Unavailable servers and applications are determined by failed requests

Server1 AppA

Server2 AppB

Web server

WebSphere Plug-in

Routing Information

(plugin-cfg.xml)

Static routing (2 of 2)

• Routing information file must be regenerated as the routing endpoints change

• Servers added, removed, or updated • Applications added, removed, or updated

Cluster1

Member1 AppA

Member2 AppA

Web server

WebSphere Plug-in

Cluster2

Member3 AppB

Member4 AppB

Routing Information

(plugin-cfg.xml)

What is Dynamic Routing

• Feature of Liberty collectives

• Maintains routing information about the web applications in a collective

• Dynamically delivers routing information to the WebSphere plug-in

Liberty collective

Dynamic Routing topology

• Dynamic Routing provides a service that keeps the plug-in routing information up-to-date with the routing topology

Cluster1

Member1 AppA

Member2 AppA

Web server

Intelligent Management

enabled WebSphere

Plug-in Cluster2

Member3 AppB

Member4 AppB

Collective Controller

Dynamic Routing Service

Benefits of Dynamic Routing

• Application requests are routed correctly as servers and applications in a collective are

• Added / Removed / Updated

• Started / Stopped

• No need to update WebSphere plug-in configuration file as the collective changes

• Plug-in is informed when servers and applications are started and stopped

• Particularly beneficial for auto scaled clusters (more later)

• Dynamic routing enabled controllers are discovered automatically

Dynamic routing is based on Liberty Collectives

• Liberty Collective

• Loosely coupled collection of Liberty servers

• Collective members

• Liberty servers that are part of a collective

• Collective controller

• Liberty server that maintains collective information

• The collective controller is also a collective member

• Liberty cluster

• A name for a logical grouping of collective members

Collective controllers • Maintain a repository of information about the state of the

collective

• Collective members publish information to the collective controller repository

• Collective controllers can be part of a replica set

• For high availability of collective controller function

• Repository information is replicated between collective controllers in the replica set

Liberty Collective

Collective Controller Replica 1

Collective Controller Replica 2

Collective Controller Replica 3

Replica Set

Collective clusters • A cluster is a name for a logical

group of collective members • A collective member can be part of at

most one cluster • Members are not required to be in a

cluster • All members of a cluster must be in

the same collective • It is possible to perform operations

on a cluster as a whole • Start, stop, ...

Cluster1

Server1

Server2

Server3

Example Liberty Collective

Collective Controller Replica 1

Collective Controller Replica 2

Collective Controller Replica 3

Cluster1

Server1

Server2

Server3

Cluster2

Server4

Server5

Server6

Server7

Server8

Server9

DETAILS Dynamic Routing

Dynamic Routing

• Updates routing information in the WebSphere plug-in as servers and applications are

• Added, removed, and updated

• Started, and stopped

• Enhances request routing to Liberty collective members

• Uses the Intelligent Management enabled WebSphere plug-in

• Plug-in receives routing information from Dynamic Routing enabled collective controllers

• The plug-in configuration file has minimal information

– How to connect to collective controllers

Dynamic Routing components

• Collective members

• Publish member specific routing information to a collective controller

• Collective controller

• Maintains routing information published by members

• Provides the dynamic routing service for the plug-in to connect to

– Delivers routing information to the plug-in

• Intelligent Management enabled WebSphere Plug-in

• Gets routing information from the Dynamic Routing service

• Chooses destination for requests

Dynamic Routing components Cluster1

Member1 AppA

Collective controller replica set

Member2 AppA

Collective Controller Replica

Web server

Cluster2

Member3 AppB

Member4 AppB

●Application Info (URI's, …) ●Application State (Started, Stopped) ●Server Info (host, port, cloneID, …) ●Server State (Started, Stopped)

Web Requests Collective Communication Routing Information

Dynamic Routing Service

Intelligent Management

enabled WebSphere

Plug-in

17

How are collective members used?

• Collective members create MBeans that contain necessary information • The MBean information is published to the collective controller

• Information published by the MBeans: • ApplicationMBean (one per application)

– The state of the application (STARTED, STOPPED, …) • ApplicationRoutingInfoMBean (one per application)

– The application contents (Module names, URI’s, Virtual Hosts, …) • EndpointRoutingInfoMBean (one per server)

– Communication information (host, ports, timeouts,…) • SessionManagerMBean (one per server)

– Session information (Clone id, cookie name, …) • Collective members might be part of a cluster

• The cluster feature is published to the collective controller – Provides the cluster name

18 <Feature name>

How are collective controllers used?

• Collective controllers contain services that maintain information used for routing

• Services in the collective controller: • Repository service

– Contains the MBean information published by members • Routing Information Manager service

– Maintains routing information about the collective • Dynamic Routing service

– Delivers routing information to the Intelligent Management enabled WebSphere plug-in

CONFIGURATION Dynamic Routing

Administrator enables dynamic routing

1. Configure a collective

2. Configure a collective controller replica set (optional)

3. Configure clusters (optional)

4. Add the dynamicRouting-1.0 feature to server.xml of collective controller(s)

5. Use the dynamicRouting command to set up Dynamic Routing

6. Configure communication with the plug-in

7. Put generated Dynamic Routing artifacts where they can be read by the WebSphere plug-in

Configuring a Collective • Steps 1 – 3

• Search for “Configuring a Liberty collective” http://www.ibm.com/support/knowledgecenter/SSD28V_8.5.5/com.ibm.websphere.wlp.nd.doc/ae/tagt_wlp_configure_collective.html

• Search for “Configuring Liberty collective replica sets” http://www.ibm.com/support/knowledgecenter/SSD28V_8.5.5/com.ibm.websphere.wlp.nd.doc/ae/tagt_wlp_configure_replicas.html

• Search for “Configuring a Liberty server cluster” http://www.ibm.com/support/knowledgecenter/SSD28V_8.5.5/com.ibm.websphere.wlp.nd.doc/ae/twlp_config_cluster.html

The dynamicRouting feature

Add the feature to server.xml of collective controller(s)

<featureManager> <feature>dynamicRouting-1.0</feature>

</featureManager>

The dynamicRouting command ● Creates the plugin-cfg.xml file needed by the WebSphere

plug-in ● genPluginCfg

– Generates WebSphere plug-in configuration file with <IntelligentManagement> stanza that enables Intelligent Management in the WebSphere plug-in

● Creates the security artifacts needed for communication with the WebSphere plug-in ● genKeystore

– Generates a keystore containing a personal certificate and signer certificates required to enable secure communication between the Dynamic Routing service and clients

● Performs both of the above operations ● setup

dynamicRouting genPluginCfg Option Meaning

--host The host name of the Dynamic Routing enabled controller.

--port The HTTPS port number of the controller.

--user An Administrator user for the controller.

--password The password for the Administrator user for the controller. If no value is defined you will be prompted.

--pluginInstallRoot Fully qualified path of the WebSphere plug-in root directory on the web server host.

--webServerNames Comma separated names of the web servers for which WebSphere plug-in configuration files must be generated.

Example: ./dynamicRouting genPluginCfg --port=9444 --host=controller1.acme.com --user=admin --password=password --pluginInstallRoot=/opt/IBM/WebSphere/Plugins --webServerNames=webserver1

Creates file plugin-cfg.xml in directory where the command is run

Example plugin-cfg.xml file ● The generated plugin-cfg.xml file will include the

<IntelligentManagement> stanza ● There will be one <Connector> stanza for each collective controller in

the collective with dynamicRouting enabled. Example stanza: <IntelligentMangement> <Property name="webserverName" value="webServer1"/> <ConnectorCluster enabled="true" maxRetries="-1" name="default" retryInterval="60"> <Property name="uri" value="/ibm/api/dynamicRouting"/> <Connector host="controller1.acme.com" port="9444" protocol="https"> <Property name="keyring“ value="/opt/IBM/WebSphere/Plugins/config/webserver1/ plugin-key.kdb"/> </Connector> </ConnectorCluster> </IntelligentManagement>

dynamicRouting genKeystore Option Meaning

--host The host name of the Dynamic Routing enabled controller.

--password The password for the Administrator user for the controller. If no value is defined you will be prompted.

--port The HTTPS port number of the controller.

--user An Administrator user for the controller.

--keystorePassword The password for the generated key store. If no value is defined you will be prompted.

--certificateSubject Optional. The DN for the generated SSL certificate. Default DN is CN=<value of --user argument>,OU=client,O=ibm,C=us

--keystoreType Optional. The type of the generated keystore. Default type is jks. Valid values are jks and pkcs12.

Example: ./dynamicRouting genKeystore --port=9444 --host=contoller1.acme.com --user=admin --password=password --keystorePassword

Creates file plugin-key.jks in directory where the command is run

dynamicRouting setup • Uses the same arguments as genPluginCfg and

genKeystore • Performs both operations Example: ./dynamicRouting setup --port=9444 --host=controller1.acme.com --user=admin --password=password --keystorePassword=keypass --pluginInstallRoot=/opt/IBM/WebSphere/Plugins --webServerNames=webserver1

Creates file plugin-cfg.xml Creates file plugin-key.jks

dynamicRouting help • Prints out usage information for the dynamicRouting

command

Setting up secure communication with the WebSphere plug-in

• Copy the keystore file to the host where the web server with WebSphere plug-in is located

• Convert the keystore to a format that can be used by the plug-in • Use the gskcmd that comes with the web server

Example: On the liberty controller host, from the wlp/bin directory scp plugin-key.jks <user>@<webserverHost>:/tmp/

On the web server host /opt/IBM/HTTPServer/bin/gskcmd -keydb -convert -pw <keystorePassword> -db /tmp/plugin-key.jks -old_format jks -target /tmp/plugin-key.kdb -new_format cms -stash -expire 365 rm /tmp/plugin-key.jks

• Put the generated plugin-cfg.xml file to the plug-in config directory • Put the converted keystore files in the plug-in config directory Example: On the liberty controller host, from the wlp/bin directory scp plugin-cfg.xml <user>@<webserverHost>:/tmp/

On the web server host mv /tmp/plugin-key.kdb /opt/IBM/WebSphere/Plugins/config/webserver1 mv /tmp/plugin-key.rdb /opt/IBM/WebSphere/Plugins/config/webserver1 mv /tmp/plugin-key.sth /opt/IBM/WebSphere/Plugins/config/webserver1 mv /tmp/plugin-cfg.xml /opt/IBM/WebSphere/Plugins/config/webserver1

Make Dynamic Routing artifacts available to the WebSphere plug-in

Discovery of Dynamic Routing enabled controllers

1. A plugin-cfg.xml has been generated for a Dynamic Routing enabled controller

2. A new controller is added to the replica set 3. The new controller has the dynamicRouting-1.0 feature

added 4. The connection information to the new controller is

delivered to the plug-in • No need to update the plugin-cfg.xml

5. The plug-in will failover to a new controller if error encountered with the currently connected controller

Auto scaling in a Liberty collective

BACKGROUND Auto Scaling

What is Auto Scaling

• Auto scaling dynamically adjusts the number of running Liberty servers in a cluster based on workload

• Auto scaling decision making is defined through scaling policies and the current state of the cluster members

– Factors for this decision making include

– The minimum or maximum number of server instances

– The threshold values for monitored server resources

• Auto scaling decision making is performed at the cluster level

– All severs participating in auto scaling must belong to a cluster

– Policy is applied at the cluster level and can be different between clusters

Auto Scaling Topology

• Auto Scaling provides automated control over all participating clusters and their members

Cluster1

Member1 AppA

Member2 AppA

Web server

Intelligent Management

enabled WebSphere

Plug-in Cluster2

Member3 AppB

Member4 AppB

Collective Controller

Dynamic Routing Service

Liberty Collective

Auto Scaling Service

Benefits of Auto Scaling

• Provides elasticity for application clusters

• Automated monitoring and control over clusters

– Host and server resources are monitored

– As resource demand fluctuates, servers are started and stopped

• Centralized policy management

– Policy is specified at the controller

– Default policy may be specified and then customized by cluster

• Beta Capability

• As resource demand increases, Liberty servers are created

Details

Auto Scaling Components

• Scaling Controller

• Maintain view of all managed clusters’ topology • Analyze incoming resource metrics • Make scaling decisions

– Start or Stop servers as needed

– Beta – add Liberty servers as needed

• Highly Available – uses collective replica sets function

• Scaling Members

• Monitor Host and Server resources

• Elect one server per host as “host leader”

– Optimizes member traffic to controller

38

<scalingDefinitions> <defaultScalingPolicy enabled="false" /> </scalingDefinitions>

Scaling Policy

• Scaling policies are used by the scaling controller to determine when to start or stop members of a cluster

• There are two ways to scale a cluster

• Scaling to meet a minimum or maximum number of servers per cluster

• Scaling to meet resource demands of a cluster

• Scaling policies must be placed in the server.xml of the scaling controller

39

• Scaling Policies allow you to set a minimum and maximum number of scaling members for a cluster.

• The controller uses this information to ensure the number of running servers falls within this bound.

• Ex: If min=2 and only one scaling member is started, the controller will start another member

• Ex: If max=3 and four members are started, the controller will stop one member

<scalingDefinitions>

<defaultScalingPolicy enabled="true" min="2" max=“3"/> </scalingDefinitions>

Scaling Policy – Min and Max

40

Scaling Controller

Scaling Member 1

Scaling Member 2

Scaling Member 3

Cluster

Scaling Controller

Scaling Member 1

Scaling Member 2

Scaling Member 3

Cluster

Policy Min=2 enabled=false

Enable Scaling Policy

Policy Min=2 enable=true

Scaling Policy – Min and Max (Continued)

41

Scaling Controller

Scaling Controller

Cluster

Policy Min=2 enabled=true

Modify Scaling Policy

Policy Min=4 enable=true

Scaling Policy – Min and Max (Beta)

Host (registered

with controller)

Scaling Member

1

Scaling Member

2

Scaling Member

3

Cluster

Host

Scaling Member

1

Scaling Member

2

Scaling Member

3

Scaling Member

4

42

Scaling Policy allows you to set minimum and maximum thresholds for server resources such as CPU, memory, and heap.

The scaling controller will use resource data sent by the scaling members in combination with policy to determine whether a scaling action should occur

• Ex: If CPU Max Threshold = 90 and CPU usage = 95, the scaling controller will start a member

•Beta – The controller will add a member

• Ex: If all metrics Min Threshold is breached, then the scaling controller will stop a member (When no metric is specified in stated policy, default policy is used).

<scalingDefinitions> <defaultScalingPolicy enabled="true" min="1" max="4” > <metric name="cpu" min="30" max="90" /> </defaultScalingPolicy> </scalingDefinitions>

Scaling Policy – Threshold

43

Scaling Controller

Scaling Member 1

cpu=95

Scaling Member 2

Cluster

Web Server

Scaling Controller

Web Server

Cluster

Scale Out

Policy Policy

cpu=90

Scaling Member 1

cpu=45

Scaling Member 2

cpu=45

cpu=90

Scaling Policy – Threshold (Continued)

Host

44

Cluster

Scaling Controller

Web Server

Cluster

Scale Out

Policy

Scaling Member 1

cpu<90

Scaling Member 2

cpu<90

cpu=90

Scaling Policy – Threshold (Beta)

Scaling Controller

Web Server

Policy

Scaling Member 1

cpu=95

Scaling Member 2

cpu=95

cpu=90

Host (registered

with controller)

Scaling Member 3

cpu<90

45

Auto Scaling allows for scaling policies to be nested to fine tune your scaling preferences for specific clusters. Nested scaling policies can override default policy values for specific clusters. A cluster’s nested policy will inherit all default policy settings it does not override.

• Ex: Increasing the minimum and maximum number of members for a specific cluster

<scalingDefinitions> <defaultScalingPolicy enabled="true" min="1" max="2” /> <scalingPolicy enabled="true" min="2" max="4" > <bind clusters="cluster1”/> </scalingPolicy> </scalingDefinitions>

Nested Scaling Policy

46

Default Scaling Policy

Min = 1 Max = 2 CPU: max=60 min=10 Heap: max=50 min=15 Memory: max=70 min=25

Cluster 1 Scaling Policy

Min = 2 Max = 4 CPU: max=60 min=10 Heap: max=50 min=15 Memory: max=70 min=25

Inherited

A nested scaling policy specifies only the attributes it intends to overwrite, all other attributes are inherited from the default scaling policy.

Nested Scaling Policy (Continued)

Overridden

CONFIGURATION Auto Scaling

Configuring Auto Scaling 1. Create a controller and configure a collective

2. Configure a collective controller replica set (optional)

3. Specify the scalingController-1.0 feature in the collective controller(s) and define policy.

4. Start controller

5. Create a member(s) and join the collective

6. Add cluster identity to member

7. Specify the scalingMember-1.0 feature in the server.xml of the member

8. Start member (repeat steps 5-8 as needed for all members)

9. After all members are configured, enable policy

Specify Scaling Controller Feature

<featureManager> … <feature> scalingController-1.0</feature> </featureManager> <scalingDefinitions> <defaultScalingPolicy enabled=“false” min=“1” max=“3”/> </scalingDefinitions>

• Augment featureManager list in controller’s server.xml • Define defaultScalingPolicy (if desired)

• While defining collective, it is best to disable policy

•Start Controller

Controller server.xml

Create a member and join collective

• Use step two from procedure described on the following document.

http://www.ibm.com/support/knowledgecenter/SSD28V_8.5.5/com.ibm.websphere.wlp.nd.doc/ae/tagt_wlp_configure_collective.html

• Define cluster http://www.ibm.com/support/knowledgecenter/SSD28V_8.5.5/c

om.ibm.websphere.wlp.nd.doc/ae/twlp_config_cluster.html

Specify Scaling Member Feature

<featureManager> … <feature> scalingMember-1.0</feature> </featureManager>

• Augment featureManager list in member’s server.xml

•Start the member

Member server.xml

Stop Members and Enable Policy

<scalingDefinitions> <defaultScalingPolicy enabled=“true” min=“1” max=“3”/> </scalingDefinitions>

• Enable policy in the controller’s server.xml

• Stop members after completing collective registration process http://www-

01.ibm.com/support/knowledgecenter/SSD28V_8.5.5/com.ibm.websphere.wlp.core.doc/ae/twlp_admin_startstopserver_cmd.html

Controller server.xml

Configuring Auto Scaling (Beta)

1. Create packages for deploying to new hosts • Liberty runtime

• JRE

• Liberty server

2. Provide packages on collective controller

3. Register available hosts with collective controller

4. Enable a scaling policy

QUESTIONS? (Then Summary)

Auto Scaling and Dynamic Routing Summary

• Auto Scaling in Liberty provides elasticity to a clustered server environment.

• Scaling based on minimum and maximum number of servers

• Scaling based on resource consumption

• Support for centralized control over Auto Scaling policy is provided • Default scaling policy applies to all clusters

• Nested scaling policies apply to explicitly stated clusters

• HA capability is offered for the management function

• Dynamic routing uses the auto-scaled resources as they become available

• Dynamic routing stops using resources that have been scaled-in

Notices and Disclaimers Copyright © 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.

Notices and Disclaimers (con’t)

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

• IBM, the IBM logo, ibm.com, Bluemix, Blueworks Live, CICS, Clearcase, DOORS®, Enterprise Document Management System™, Global Business Services ®, Global Technology Services ®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, SoDA, SPSS, StoredIQ, Tivoli®, Trusteer®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

Thank You Your Feedback is

Important!

Access the InterConnect 2015 Conference CONNECT Attendee Portal to complete your session surveys from your smartphone,

laptop or conference kiosk.