1. IBM SmartCloud Orchestrator A Technical OverviewDate: February 24, 2014

2. Table of Contents 1Executive summary ...............................................................................................3 2Why do you need an orchestrator in your cloud? ...............................................5 3IBM SmartCloud Orchestrator Architecture.........................................................8 3.1IBM SmartCloud Foundation...........................................................................8 3.2IBM SmartCloud Orchestrator.........................................................................8 3.2.1IBM SmartCloud Provisioning at a glance ........................................8 3.2.2IBM SmartCloud Monitoring at a glance ...........................................9 3.2.3IBM SmartCloud Cost Management at a glance ..............................9 3.2.4IBM SmartCloud Orchestrator per se at a glance ........................... 10 3.3IBM SmartCloud Orchestrator High Level Architecture ................................. 10 3.3.1IBM SmartCloud Provisioning at a glance ...................................... 11 3.3.2IBM SmartCloud Orchestrator at a glance...................................... 12 4IBM SmartCloud Orchestrator Content .............................................................. 13 5Custom Operations ............................................................................................. 16 5.1.1Event operations ........................................................................... 16 5.1.2Instance operations ....................................................................... 17 5.1.3Service operations ......................................................................... 17 6IBM SmartCloud Orchestrator Service Catalog ................................................. 18 6.1SmartCloud Automation Modeling User Experience ..................................... 22 6.2Operations Context ...................................................................................... 23 6.3Custom data persistency .............................................................................. 24 6.3.1Use of Operation Context through request (example) .................... 25 7IBM Cloud Marketplace ....................................................................................... 27 7.1What is cloud automation content ................................................................. 28 7.2How to build Cloud Automation Content? ..................................................... 28 7.3IBM Cloud Automation Marketplace Value Proposition ................................. 28 8More information ................................................................................................. 30 9Trademark Terms ................................................................................................ 31 AAppendix: IBM SmartCloud Foundation ............................................................ 32 A.1IBM SmartCloud Control Desk ..................................................................... 32 A.2IBM SmartCloud Application Performance Management .............................. 32 3. List of figures Figure 1: IBM SmartCloud Orchestrator at a glance .............................................4 Figure 2: Example of real workflow ........................................................................5 Figure 3: Data Center Domains ..............................................................................6 Figure 4: IBM SmartCloud Orchestrator High Level Architecture ...................... 11 Figure 5: Event operations for typical pattern deployment ................................ 16 Figure 6: IBM SmartCloud Orchestrator Service Catalog UI............................... 18 Figure 7: Operations Registry .............................................................................. 19 Figure 8: Create a new operation ......................................................................... 20 Figure 9: The three types of operations ............................................................... 20 Figure 10: SmartCloud Automation Modeling User Experience ......................... 23 Figure 11: Operations Context ............................................................................. 24 Figure 12: Use of Operation Context through request (example) ...................... 26 Figure 13: IBM Cloud Marketplace ....................................................................... 27 4. 1Executive summary In todays ever changing business environment, IT is becoming a fundamental business process that allows enterprises to become more effective at delivering solutions to their customers and users. The cloud paradigm can dramatically increase the speed of delivery of new business services while reducing operational cost and improving quality of services. As enterprises look to integrate and optimize their cloud infrastructure, they need advanced cloud capabilities for automating the management of virtual environments and building a dynamic service delivery model. The ability to support user-driven services requests and orchestrated resource deployment is pivotal to an optimized infrastructure. IBM SmartCloud Orchestrator is a comprehensive service delivery platform for cloud environments that can help drive down costs and increase speed to deliver business services. As part of the IBM SmartCloud Foundation products suite, IBM SmartCloud Orchestrator provides advanced cloud management capabilities for enterprises which need to integrate their business policies and systems together in a customized cloud service via workflow orchestration. Simply put, IBM SmartCloud Orchestrator provides end to end service deployment across infrastructure and platform layers with integrated IT workflow capabilities for runbook automation and IT governance, resource monitoring and cost management. It offers a consistent, flexible and automated way of integrating the Cloud with your data center policies, processes and infrastructure across your various IT domains. You can define and implement business rules and IT policies with an intuitive graphical tool. It helps you connecting the aspects of your different data center domains into a consistent orchestration of automated tasks and human tasks achieving business goals and ensuring compliance. This assembly provides an extensible approach for integrating with your existing environments such as network and storage management tools as well as integration with your specific service management processes such as those defined in IT Infrastructure Library (ITIL). IBM SmartCloud Orchestrator is based on the IBM Common Cloud Stack (CCS) which also includes OpenStack (note IBM is a platinum sponsor of OpenStack Foundation). CCS aims to provide a common realization of the core cloud technologies for effective and efficient management of cloud systems. One very critical piece in the overall IBM orchestration strategy is the IBM Cloud Marketplace. Accessible from the Internet, it acts as the center for either publishing or consuming contents (a kind of AppStore). In a nutshell, the Cloud Marketplace is the IBM cornerstone for establishing an ecosystem between creators of contents and users of contents for SmartCloud Orchestrator. It hosts extensions developed by IBM, partners and even customers to help accelerate the integrations of SmartCloud Orchestrator and related partners / customers products. At the core, IBM is focusing on integration with additional products from its hardware and software divisions, in the areas of networking, storage, service management, etc. All these IBM contents are downloadable, free of charge, from the IBM Cloud Marketplace. Therefore, you 5. will be able to directly download your needed contents to your cloud platform (for example in your test systems) and try out. In summary, IBM SmartCloud Orchestrator is an open and scalable platform that allows integration of Cloud with your data center policies, processes, and infrastructure in a consistent, flexible, and automated way, across various IT domains.An open and scalable cloud platformA marketplace for content sharing and re-useAn easy to use orchestrator for cloud service automationA ready to use automation library 2013 IBM CorporationFigure 1: IBM SmartCloud Orchestrator at a glanceIt comes with ready to use orchestration capabilities; further additional contents, toolkits, and other building blocks can be easily found, used, adapted, and shared by the network system that is centered on the IBM Cloud Marketplace. Next sections in this document are focusing on: Why do you need an orchestrator in your cloud? IBM SmartCloud Orchestrator High Level Architecture IBM SmartCloud Orchestrator Content Custom operations IBM Cloud Marketplace 6. 2Why do you need an orchestrator in your cloud? There are various reasons you need an orchestrator; here some examples: Customers and users are looking for end to end automation of cloud service delivery to achieve greater returns; Provisioning play a key role, but is just one of many steps that must be automated; Each customer or user has unique requirements to integrate with existing data center processes and tools. In the following real environment, the orchestration is limited to the provisioning of VMs only; as you can see it, only the technical part is their focus.VM ProvisioningReal customer example 2013 IBM CorporationFigure 2: Example of real workflowMore often than not, the need goes beyond than just provisioning image templates to create VMs in the cloud, even if you are leveraging powerful patterns to instantiate the service. In real environments, there are more processes required which are surrounding the actual VMs provisioning to enable an end-to-end orchestration. A typical example: before triggering the real provisioning, you may want approval workflows which deal with accepting or rejecting certain services requests from users for creation of virtual systems; at the end of the provisioning, you may also want to add some VM entries in your CMDB in order to track the resources available in your data center. Another example: in case of a private cloud, there is often a need to call out multiple departments in the organization to collect the right information to prepare 7. items such as VLANs/storage/.. in order to start the provisioning afterwards. So, all these kind of additional extensions to the actual VM provisioning can be coded with workflows and actual processing. This is not only interesting for the deployment phase but also for other lifecycle operations such as termination of the services, de-registration of the services, etc. You may need to return resources, clean up the CMDB accordingly. Again, all these activities can be done in addition of the VM provisioning. In other words, there are extra things that have to be done to enable a holistic cloud solution. To address these, IBM is leveraging the notion of domains automation as illustrated in the following figure:Orchestration Service CatalogToolingDomains Data Center Automation DomainService Assurance DomainCloud DomainSecurity DomainInfrastructure & Application Resilience DomainNetwork DomainStorage DomainAsset, License Chargeback ITIL & Work Management DomainWorkload Scheduling & Automation Domain 2013 IBM CorporationFigure 3: Data Center DomainsThe most obvious one is the Data Center Automation Domain. It is dealing with configuring hypervisors, copying images to the right location, starting an image to get an instance of it, etc. As illustrated in the above figure, there are other obvious domains such as networking, storage that are needed by the virtual machines. Note for the storage domain, you may also need for a Storage as a Service in which the file system has its own lifecycle which is independent of virtual machines. All these domains can integrate with existing SmartCloud Orchestrator workflows or be tailored as needed to drive related activities in their respective domain. SmartCloud Orchestrator unfolds the power of cloud computing in your data center by providing a consistent, flexible and automated way to integrate with the policies, processes and infrastructure across IT domains. 8. In summary, here are the key characteristics of SmartCloud Orchestrator: Integrated orchestration of automated IT operations and human tasks; Automation across all components of the cloud stack and across IT domains; Production-level cloud in compliance with IT and business rules, achieving business goals; Intuitive graphical tooling to design and manage orchestration; Protect investment in orchestration-level artifacts while exploiting domain-specific technology updates. Now its time to look over the high level architecture of SmartCloud Orchestrator. 9. 3IBM SmartCloud Orchestrator Architecture Before jumping on the architecture of SmartCloud Orchestrator, lets briefly introduce the IBM SmartCloud Foundation initiative.3.1IBM SmartCloud Foundation IBM SmartCloud Foundation is a set of capabilities which helps you more easily build and rapidly scale cloud environments with improved time-to-market, integration, and management. IBM SmartCloud Monitoring, IBM SmartCloud Provisioning, IBM SmartCloud Control Desk, IBM SmartCloud Orchestrator, IBM SmartCloud Cost Management, and IBM Application Performance Manager are the offerings that are part of the IBM SmartCloud Foundation family of products. IBM SmartCloud Foundation provides integrated cloud capabilities which give you varying entry points to cloud computing. IBM continues to enhance how the SmartCloud Foundation capabilities are integrated based on the Open Services for LifeCycle Collaboration (OSLC) approach, to deliver an open and integrated cloud ecosystem.3.2IBM SmartCloud Orchestrator IBM SmartCloud Orchestrator encompasses the following capabilities: IBM SmartCloud Provisioning; IBM SmartCloud Monitoring; IBM SmartCloud Cost Management; and IBM SmartCloud Orchestration per se. Depending on requirements, additional capabilities are available such as IBM SmartCloud Control Desk (including its Service Provider option) and IBM SmartCloud Application Performance Management (a short description is available in the appendix), and of course other Tivoli products.3.2.1 IBM SmartCloud Provisioning at a glance IBM SmartCloud Provisioning capabilities include: A simple, easy-to-use deployable solution to help you move from no cloud to a base cloud that may be up and running in less than a few days; Multi-hypervisors support: Microsoft Hyper-V Server, Red Hat Kernel-based Virtual Machine, VMware ESX, and Xen OpenSource, IBM Power Virtual Machine, (zVM in the future), so you have choices and possibly save on licensing costs; Image lifecycle management: control image sprawl and reduce business risk with rich analytics, image versioning, and federated image library to standardize images; expand use cases to support security and governance models by 10. searching for images with particular content for patching and security auditability, analyzing images for similarity to consolidate image types and storage; Image Construction and Composition Tool that provides a graphical tool to build an image recipe, using your company's approved operating systems and software as ingredients, and then create an image that delivers the user's requirements while providing an audit trail; Highly automated self-service deployment of virtual machines in a reliable nonstop cloud environment that allows you to add capacity by powering on a new compute node, patch the infrastructure by simply dropping the patch in a directory, and work around failures with a complete, distributed, redundant infrastructure; OpenStack Folsom; Deliver on highly scalable fault tolerant architecture but also deploying multitiered applications and services for scalable and elastic cloud.3.2.2 IBM SmartCloud Monitoring at a glance IBM SmartCloud Monitoring provides performance and availability capabilities as part of the SmartCloud Foundation. As businesses consolidate and virtualize application workloads along their journey toward cloud, the cost savings that they had envisioned often prove elusive. True efficiency comes from the ability to right-size both the environment and the virtual workloads - in response to actual performance data, rather than theoretical estimates - in order to create an optimized cloud infrastructure that runs densely enough to provide true consolidation while maintaining application service levels and room for expansion. IBM SmartCloud Monitoring monitors the health and performance of a cloud infrastructure, including environments containing both physical and virtualized components. This software provides the tools needed to assess current health and capacity and model expansion, as needed. IBM SmartCloud Monitoring provides: Visibility into the cloud infrastructure including environments containing both physical and virtualized components; Monitoring of heterogeneous environments for visibility and control into all areas of the infrastructurephysical, virtual and cloud; What-if capacity analysis designed to model changes, reduce risk, improve availability and lower energy and server costs; and Policy-driven analytics for intelligent workload placement.3.2.3 IBM SmartCloud Cost Management at a glance As end users from IT and non-IT organizations request cloud services, they need insight into the cost tied to specific SLAs of a dynamic service delivery. Service Providers and their financial organization need to understand the service delivery costs within and outside IT to maximize profitability of existing and new cloud offerings. 11. IBM SmartCloud Cost Management provides a comprehensive financial cost management environment where you can collect, assess, and bill based on the usage and cost of the cloud service delivered and help drive down the cost to deliver business services. SmartCloud Cost Management provides value particularly as part of SmartCloud Orchestrator; however it also integrates with lower entry solutions like SmartCloud Provisioning to provide cloud cost management metering and showback reporting. IBM continues to support financial management customers outside the cloud service management space and will continue to deliver new capabilities and benefits for such deployments.3.2.4 IBM SmartCloud Orchestrator per se at a glance IBM SmartCloud Orchestrator is built on a common cloud stack and common pattern modeling. The IBM Business Process Manager - Standard Edition (also known as Lombardi) is used for workflow modeling and runtime capabilities. It is integrated in such a way that event triggered automation provides the basis for cloud-centric workflow enablement. The major components of SmartCloud Orchestrator are: The Runbook Automation Modeling UI; and The Runbook Automation Engine. Out-of-the-box, there are various runbooks and associated workflows provided as content packs with SmartCloud Orchestrator as well as for Monitoring, and Cost Management. SmartCloud Orchestrator therefore is packaging the appropriate products and providing the necessary runbooks or runbook activities to leverage these domains specific functionality. Last but not least: contents, toolkits, and other building blocks can be easily found, used, adapted, and shared by the network system that is centered around the IBM Cloud Marketplace (see related chapter in this document). Now its time to look at the high level architecture of IBM SmartCloud Orchestrator.3.3IBM SmartCloud Orchestrator High Level Architecture In this section, and as illustrated on the figure below, we have deliberately expanded the high level architecture around two of the four core components of IBM SmartCloud Orchestrator only, i.e.: IBM SmartCloud Provisioning (boxes in green); and IBM SmartCloud Orchestrator (boxes in blue) per se. 12. Cloud UserSmartCloud Orchestrator (SCO)SmartCloud Provisioning (SCP) SmartCloud MonitoringAutomation Automation Modeling UI(SCM)Self ServiceOrchestration UI UIIBM Business Process IBM Business Process Manager (BPM)Modeling and AdminICCT Image CreationAPI APISmartCloud Cost ManagementAPI APIImage LibrarySingle Image & Composite Patterns Management(SCCM) Automation Engine Automation Engine BPM Process ServerHypervisor ManagementIBM Extensions for IBM Extensions for Workload Aware Workload Aware optimizations optimizations (DRS & HA) (DRS & HA)OpenStack GatewayOpenStackHybrid Extension Hybrid ExtensionSmartCloud EnterpriseContent packages Content packagesService DeskLicense MgmtAccountingCost MgmtNetworkStorageAmazon EC2 Amazon EC2KVM KVM VMware vCenter VMControl PowerXEN XEN ESX ESX Hyper-V Hyper-V 2013 IBM CorporationFigure 4: IBM SmartCloud Orchestrator High Level ArchitectureThe various components of the architecture are articulated as follows: From SmartCloud Orchestrator self service UI, you can view patterns, edit virtual system patterns, collect pre-deployment information, create virtual systems, register operations. You can collect pre-deployment information from the user during deployment; SmartCloud Provisioning can dispense applications and topologies into a pool, or cloud of virtualized hardware, and can manage resources. However, in case of SmartCloud Orchestrator, provisioning is just a part of cloud orchestration; The workflow design component, that is, the Automation Modeling UI easily extends the design to a full orchestration use case in the Business Process Manager. To deploy applications, SmartCloud Orchestrator provisions the pattern and creates virtual machines for the database, application, and webservice. It also takes care of orchestration with integration across various domains. For example, it sets up backup monitoring, requests quota change, opens ticket configuration, and sets up storage network. The resources of SmartCloud Orchestrator run on hypervisors mentioned earlier in this document; and Infrastructure as a service (IaaS) federation API normalizes APIs across different vendor clouds, such as IBM SmartCloud Enterprise, Amazon EC2, OpenStack and similar such products. Lets start with IBM SmartCloud Provisioning.3.3.1 IBM SmartCloud Provisioning at a glance The main components are: 13. Self Service UI: the front end for users / consumers to request services; Pattern Modeling: the modeling capabilities to manage virtual systems and virtual applications patterns; API: REST APIs you can leverage for integration with your own portal; Image Construction and Composition Tool: help create images, SW bundles; Image Library: manage the entire lifecycle of images and VMs; Single image and composite patterns management: flows hardcoded to deploy, copy, images; OpenStack Folsom; IBM OpenStack gateway, a normalized API in order to get interactions with: hypervisors (ESX, Hyper-V, KVM, Xen, and zVM in the future);hypervisor managers (VMware vCenter, IBM VMControl); andExtensions added to the UI to drive specialized deployment with respect to high availability and disaster recovery. Lets look over the IBM SmartCloud Orchestrator components per se.3.3.2 IBM SmartCloud Orchestrator at a glance IBM SmartCloud Orchestrator includes: Automation Modeling UI: IBM BPM Process Designer; Automation Engine: IBM BPM Process Server; Additional orchestration features to the self service UI (more panels and contents to deal with the automation pieces), note SmartCloud Orchestrator and SmartCloud Provisioning have exactly the same UI; API: REST APIs you can also leverage for integration with your own data center domains; Hybrid cloud extensions: for IBM SmartCloud Enterprise and Amazon EC2; and Content packages: out-of-the-box orchestration contents providing additional capabilities for various data center domains (service desk, network, storage, firewall, load balancer, etc.). All of these various packages can be used by the workflows defined in BPM Process Server, and they will be called out by the activities defined in these content packages. In next section, we are introducing the various Content types supported by IBM SmartCloud Orchestrator. 14. 4IBM SmartCloud Orchestrator Content IBM SmartCloud Orchestrator Content is an automation that enables SmartCloud Orchestrator to make use of the functionality delivered by external software and infrastructure devices. In other words, when we talk about content for orchestration, we mainly talk about workflows, snippets of workflows so called, i.e. a set of reusable entities of a workflow which are the activities, the individual nodes that make up a complete workflow and also things like user interfaces and widgets to name a few. The workflow designer in BPM is not only capable of designing and creating workflows which are executing things, but also able to create additional UI panels for collecting specific information which is needed to drive any tailored orchestration. There are different types of SmartCloud Orchestrator content that you can create and deploy onto the cloud: Content TypeDescriptionVirtual imagesVirtual images provide the operating system and product binary files that are required to create a virtual system instance. For example, the IBM OS Image for Red Hat Linux Systems virtual image is preinstalled on your appliance. These images can be extended to customize the virtual images and the operating system.Software BundlesA software bundle contains and describes the software available for use within a virtual image. It includes information about how to install the software, prerequisites of the software, and parameters available for customizing the software. This combines your own operating system definition along with custom software bundles to create virtual bundles that can be provisioned on the cloud.Toolkits and ApplicationsToolkits provide resources that you can use as you build your process applications. Process applications can share library items from one or more toolkits, and toolkits can share library items from other toolkits. If you have access to a toolkit, you can create a dependency on the toolkit and use the library items within it for your process development efforts. If the operations can be reused, go for Toolkits. A process application is a container for process models and their supporting implementations; it is stored in the repository. After the artifacts have been authored or otherwise created, they are assembled into a process application. If you target any specific process implementation, then go for application. 15. Content AdaptersThe integration service calls the Content adapter and it implements the logic required to invoke a target device on the data center. The implementation is a Java class that accepts BusinessObjects/Json. The content adapter invokes the target device-sdk. It is the code that bridges the Business Process Manager process and domain adapter. The integration service invokes the content adapter to perform the following tasks: Receives Business Process Manager artifacts Maps the received artifacts to Java objects Invokes Domain AdaptersDomain AdaptersIt is a Java class that implements the logic required to invoke a target device on the data center. The inputs and outputs are Java objects and it is completely independent of the Business Process Manager.PatternsCreate virtual application patterns to model virtual applications that you can deploy to the cloud. A virtual application is defined by a virtual application pattern. It is a complete set of platform resources that fulfill a business need, including web applications, databases, user registries, messaging services, and transaction processes. Each virtual application pattern is associated with a pattern type, which is a collection of plug-ins that provide these resources and services for a particular business purpose in the form of components, links and policies. The pattern types, product extensions of the cloud system, and the types of virtual application that you build depend on the pattern types that you have enabled.Workflow AutomationIt is used for pre/post operations during pattern deployment. In SmartCloud Orchestrator, they are triggered by various operations. The different types of triggers are: Event Operation - event operations are related to processes that are executed during pattern deployment. For example, this can be an automated approval process. Instance Management Operation - Instance operations are management extensions for instances. For example, change password. Service Operation - it is a process that is not related to any pattern. For example, add a user. Triggers can be either blocking (synchronous, that is, this workflow must complete) or non-blocking (asynchronous, that is, activities continue in 16. parallel). In next section, we are focusing on the first three operations in bold under the Workflow Automation row here above, called Custom Operations. 17. 5Custom Operations As mentioned earlier, the orchestration is realized by the BPM workflows. SmartCloud Orchestrator allows definition of custom operations for various granularities: Event operations; Instance operations; and Service operations.5.1.1 Event operations If we look at a typical pattern deployment process:SCP base processingResolve ModelDefine PlacementCopy imagesConfigure hypervisorPre-provision EventSCO Custom BPM processesStart VMPre-VM-start EventExecute Config ScriptsNotify userPost-provison Post-provision-failed EventTrigger one or multiple custom processes on certain events 2013 IBM CorporationFigure 5: Event operations for typical pattern deploymentIt consists of various steps which are basically hard coded in the pattern management engine. In SmartCloud Orchestrator, IBM introduces EXIT at various points where you can register additional dedicated workflows. A typical example: in pre-provision event you would register an approval process; while in post-provision event, you would register an automation which adds the established VM in a CMDB database for example. You could register one or multiple of these processes and they will then be executed depending on pre-, -post whenever the exits are defined. 18. We are not only providing these event operations for the deployment process but for any lifecycle of operations.5.1.2 Instance operations Instance operations apply to instances. Once you have deployed either a single image or pattern, you will get a service instance (i.e. one or more virtual systems). While SmartCloud Provisioning already provides a few instance operations such as start / stop of VMs, there may be specific instance operations needed: as example, the deployment of a WebShop service with a three-tier patterns (ex: HTTP server, Application Server, Database Server) for test purpose in which there is a demand to reset the data or import very specific data set. This can be implemented through instance operations. You would write a workflow that exactly picks the data and deploy it into the database server; a user or developer could via a simple click drive the reset of data and import these data. This could also be applicable for backing up the system, taking snapshots, etc. So, various types of use cases are possible here.5.1.3 Service operations Service operations are the kind of workflows which are primarily independent of any service instances or any patterns. There are different types of service operations; here are two examples (i.e. use cases) triggered by the Self Service UI: Administrative operations: as example, the on-boarding a new tenant or user for which some specific things have to be done per context; you may want to include your own workflows according to your needs and register them as service operations; Offering: you may want to offer cloud services which are independent from VMs, an example is storage as a service: manage independent lifecycle of file system where you can create a file system / delete a file system / mount a file system to a VM. Another use case, normalizing data center resources: its about providing a simple UI to a low end user so to speak in which the number of data to type in is minimal. In this use case, the user will select a small, medium or large system instead of presenting him a number of CPUs, various memory sizes and disk spaces. Of course behind the scene, this information is hard coded in related workflows. 19. 6IBM SmartCloud Orchestrator Service Catalog Service operations described here above will show up in the SmartCloud Orchestrator Service Catalog: 2013 IBM CorporationFigure 6: IBM SmartCloud Orchestrator Service Catalog UIFor a low end user, you may want to lighten the content of the service catalog. This is also addressed by SmartCloud Orchestrator. At this point, there is one piece missing when we talk about all the above custom operations: how do they relate in SmartCloud Orchestrator? This is where the Operation Registry entry comes into play in the SmartCloud Orchestrator Administrator Self Service UI under Service Catalog tab shown hereafter (note: this entry doesnt exist in SmartCloud Provisioning): 20. 2013 IBM CorporationFigure 7: Operations RegistryOperations Registry is used to register these newly created operations. The Operation Registry is the central registry in SmartCloud Orchestrator where you must register and configure all Business Process Manager-based custom extension operations. An extension to SmartCloud Orchestrator consists of user UI panels and a Business Process Manager Business Process Definition: The UI panels eventually collect more data that are then implemented by a Business Process Manager Human Service (its optional); and The Business Process Manager Business Process Definition defines the activities of the extension operation. The extension consists of a Business Process Manager Human Service and Business process. The business process must be developed from the standard Process Designer and then registered as extension points in the SmartCloud Orchestrator Operation Registry. Lets describe the three basic operations now. The create new operation panel shown below invites the cloud administrator to provide a name for the operation, a description, select an association type (i.e. one of the three type of operations: event, instance, service) and some additional entries to fill out depending on selected association type: 21. 2013 IBM CorporationFigure 8: Create a new operationFor the sake of concision, the following figure illustrates the three types available in SmartCloud Orchestrator in one screen:Event Operation Instance OperationService OperationService OperationSelect a custom BPM process for this operationSelect corresponding user interface (BPM coach) 2013 IBM CorporationFigure 9: The three types of operations 22. Event Operation It is a custom operation which is triggered by a specified event during predefined management operations. For example, the predefined operation Deploy pattern calls custom event operations that are registered in the Operation Registry for any of the deploy pattern events. Approval operation (subset of event-triggered operation) is a special case of an event-triggered operation. These operations are registered as an event-triggered operation for all patterns, and are triggered on the pre event before any default processing takes place. If required, the approval operation can include a manual Accept or Reject to continue with the operation. SmartCloud Orchestrator provides an approval panel that lists all pending approvals, and a means to accept or reject. If you look at Event Operation panel and select Event Operation under Association Type, you will be prompted for selecting one of the defined user exits. There are various Event Operations for each of the lifecycle operations that is already implemented by SmartCloud Provisioning (again part of IBM SmartCloud Orchestrator), such as pre- and post-events. Instance Operation It is a custom action that is run by the user on a single image deployment, virtual system instance, or virtual application instance. These kinds of operations implement additional lifecycle management action that extends the set of predefined actions. For example, as the list of predefined operations does not include a system backup, this can be implemented as instance-level operation. The instance level operations are accessible through the list of actions on the instance details view panel. If you look at Instance Management Operation under Association Type, you will see the Apply to which is also valid for Event Operation of course. Here you could define whether you want to apply this Operation to a very specific pattern or image, or to all. For example: an approval workflow is typically apply to all patterns, and all images; you will do approval all the time but resetting some test data, in our above WebShop example, will of course apply to this specific pattern; it will not make sense to have this Instance Operation for any other service or pattern. So, you can limit Instance Operations to one or multiple images or patterns. Service Operation Service operation is a custom operation, which can be run in the context of the data center. These operations are administrative operations and are used to automate the configuration. Service operations can also be used to enhance the catalog of available services with additional functionality. Service operations are independent of any existing instance or process or events. You can register your process and your UI. As example, you way want the following BPM workflow to be executed on-boarding a new user or customer: a process (i.e. the steps) would be to associate certain resources pool to that user or customer, you 23. will provide him dedicated storage resources, dedicated VLANs, etc. he can work with. In order to do so, you may also need to collect additional data, for example: you would ask the amount of storage he will get; that is the reason why you can register a user interface here which corresponds to this actual process. With that UI which is called BPM coach, you would collect the necessary properties. We will see later on how this process looks like and the interface creation. In designing SmartCloud Orchestrator, a goal was also to drive all these operations and all functionalities through an external portal because many customers have already their own portal, and they want to integrate with that one. Thats the reason IBM decided to split the approach in two parts, in other words: The actual execution part; and The user interface part which collects the parameters. By doing that way, you could always call the actual process from an external portal. This external portal has the duties to collect, get or set data the necessary parameters for the process. Next section introduces the process modeling capabilities in a context of IBM SmartCloud Orchestrator.6.1SmartCloud Automation Modeling User Experience The following figure is a screenshot from the IBM Business Process Designer (note: a few callouts are added for information purpose):Rich tooling functions to edit, version, debug, optimize workflowsAccess to rich libraries (toolkits) of reusable automation assets that enable to speed automation creationGraphical editor for composing and connecting workflowsPalette of library assets enable easy workflow composition through drag and dropRich set of actions types, flow control, data handling primitives that simplify creation of complex automationsEasy workflow action editing for managing: data mapping, error recovery options, implementation details , etc. 2013 IBM Corporation 24. Figure 10: SmartCloud Automation Modeling User ExperienceIn the central pane, you can see a set of various nodes in the process and activities with decision points. You can select existing controls from the vertical palette on the right side. The good news is that you can start small (so to speak) by wiring things without having a real implementation behind it. You can check if this is the process you want, you can change it. When you are done with the modeling, you can start with the implementation underneath each of these activities; and BPM allows you to leverage various technologies. The simplest one is that you use Javascript to implement these activities. If there are any activities to just get the right VM ip address, this can be easily implemented via Javascript. There are more complex ones where you want to call out external programs or products or even hardware, firewall, load balancer, what ever. For this kind of situations, BPM allows you to create these via REST calls, webservices, SSH calls,... which are typically used to callout to external systems. So, each of these activities could have different a implementation and there are basically wired together and kept together by leveraging and using a kind of data container (so called private data) which is floating throughout all the process and which could be accessed by each of these individual nodes. To ease now the creation of a very customized workflow, there are some toolkits coming to play. Some are already provided with SmartCloud Orchestrator but there can be specific toolkits provided which are implementing very specific implementations (for example: adding some backup capabilities). What is a toolkit? Basically, a toolkit is really the implementation of such an activity box. With a toolkit, you have a sort of implementation. In the above figure, there is a list of various backup related integration services such as cancel, create, delete backup policies. If you have such a toolkit available you can just kick this implementation and drag them to such an activity; and by that, it is bound to that implementation. You, as creator of your process, you dont care about the actual implemenation. The only thing you care about is the data mapping. For each of these implementations of individual node activities, there will be some inputs and outputs; and BPM allows you an easy way to do the mapping of these parameters which are available for example in your workflow context and map them to the input of your individual implementation. It is very easy to work with this kind of editor especially if you have a bunch of toolkits available; you can drag them to build your own processes as needed. Lets share a few words on Operations Context.6.2Operations Context The following figure shows an operation context and a service instance: 25. 2013 IBM CorporationFigure 11: Operations ContextWith each call of such an operation (this is especially the event operation and the instance operation), they are executed in a certain context; and the context is the actually deployed system or the actual request you are processing and therefore each of these operations on workflows are getting called with so called operations context objects. This is the data that you can directly leverage in your workflow by simply using this data mapping capability. So, for example, beside the actual operational context with some metadata, there is also the service instance object included which lists all the VMs of your deployed service. You can easily pick the data from these VMs (e.g. hostname, IP address) and they can then be used in your process; for example: by adding those as input for CI (Configuration Item) creation activities and passing this IP address to this CI creation code. Next section addresses custom data persistency: a feature that is often asked by our customers.6.3Custom data persistency If you know Tivoli Provisioning Manager, you know it has a Data Center Model called DCM which keeps data representing your data center such as hypervisors, routers, servers, software, etc. Within SmartCloud Provisioning and therefore SmartCloud Orchestrator, IBM uses a different approach. In SmartCloud Orchestrator, it only keeps the absolutely 26. necessary data, and instead uses points or references to external systems in order to get to certain data. This has the advantage of using an actual view of your systems and not relying on having always the actual correct data in your own model. If we look at the persistency of data, each of these extensions may also need to persist additional data; typically, it will also be a kind of references to external objects. Here we are providing a set of capabilities depending on the use cases (especially depending on the lifecycle) whether you need to persist data just for certain requests or service instances or even cross service instances. Again, the good news is that are various toolkits to help enabling these kinds of data persistently. Last but not least, there are also some descriptions about how we deal with these additional data in typical flow. How data is passed: Extensions build for SmartCloud Orchestrator usually need custom data; Persistency options for custom data: Save with operation context (request lifecycle),Save with instance object,Use persistency toolkit,Use external database,Use CMDB (IBM SmartCloud Control Desk toolkit available);Mixed models are regularly used, i.e. instance keeps reference to externally stored data and all additional data which is not available externally. Now that we have defined Operation Context and Data Persistency, lets have a look at the use of operation context through request example.6.3.1 Use of Operation Context through request (example) The following figure shows how data is passed. 27. SCO1 Once per registered event operationSCO REST API3Pre-process eventOperation contextCore CCS processingPost-process event45SCO REST APICustom Custom Data DataOperation context2 BPM Human ServiceOperation context A Operation context BBPM ProcessA Bpre-provision event operationpost-provision event operation 2013 IBM CorporationFigure 12: Use of Operation Context through request (example)If we look at the various components like what we talk about we do have the BPM Human Service which represents the UI and collects the data from an end user. The data is then persisted in the operations context. Later on, its passed through the individual BPM processes which are doing the actual execution and again; and as earlier said, the separation of UIs actual execution you would also be able to drive all these operations from an external portal by just providing the necessary operations context. Of course data is needed by these execution processes, but this can be done by REST calls and no UIs will be involved in that case. Next chapter introduces the IBM Cloud Marketplace. 28. 7IBM Cloud Marketplace The IBM Cloud Automation Marketplace is a hosting solution for cloud services which are complicated to find, a one-stop shop that allows marketers to come together, find, discover cloud content and extend IBM's cloud solution in a collaborative manner with a goal to improve the ecosystem and assist sales, services and business partners with deploying and managing cloud solutions . Typical marketers would be IBMers, customers, business partners, and ISV's. The Marketplace federates relevant content from ISML and other existing repositories into a single content catalog and provides various business models for selling content and support. The IBM Cloud Automation Marketplace delivers capabilities for: End users to collaborate and discover solution capabilities to address business needs; End users to request & have content automatically delivered & configured for use; and IBM and Business partners to publish, update, meter and monetize content using a Web-based delivery model. Establish cloud platform agility and value by rapidly delivering content (organic & partner ecosystem) using the AppStore model. 2013 IBM CorporationFigure 13: IBM Cloud Marketplace 29. Its one central point that helps enable: Customers to rapidly discover and implement cloud solutions (use a simple oneclick download option to download content into the IBM Business Process Management software's designer view to build data center orchestration); Business partners and ISVs to create value by adding content to IBM cloud solutions; IBM services and development to rapidly deliver and iterate on capabilities in response to changing customer requirements and integration needs; and Collaboration within the ecosystem to accelerate customer value creation.7.1What is cloud automation content Any content that can be uploaded on the Cloud Stack and having the following characteristics: Leverage existing extension points on the cloud stack; and Easily upgradable to new releases and fix packs.7.2How to build Cloud Automation Content? IBM provides dedicated documentation and examples on how to build cloud automation content through the SmartCloud Orchestrator Content Development Guide.7.3IBM Cloud Automation Marketplace Value Proposition There are various advantages to leverage cloud content on the IBM Cloud Automation Marketplace. Some of the major ones are: Time to value for cloud implementation: Cloud Automation Content will be available as a 1 click download; Sharing Content; and Finding content for your problem: Marketplace would serve as a central repository for all the content that you want for your cloud implementation. These induce the following business and technical benefits for your enterprise: Business benefits Enable customers to rapidly discover and implement cloud solutions; Enable business partners and ISVs to create value by adding content to IBM cloud solutions; Enable IBM services and development to rapidly deliver & iterate on capabilities in response to changing customer requirements and integration needs; and Enable collaboration within the ecosystem to accelerate customer value creation. Technical benefits 30. Simplify the lifecycle of content creation, delivery and deployment; Provide a consolidated view to content spanning the build, manage and create phases of the cloud deployment; and Enable social collaboration platform to enable tighter interlock between customers, partners and IBM. In summary, a user can upload content to the IBM Cloud Marketplace, search for content and use a simple one-click download option to download content into the IBM Business Process Management software's designer view to build data center orchestration. 31. 8More information The technical documentation of the product can be found on the IBM website at http://pic.dhe.ibm.com/infocenter/tivihelp/v48r1/topic/com.ibm.sco.doc_2.2/welcome. html. 32. 9Trademark Terms International Business Machines Corporation 2013 All Rights Reserved. IBM CorporationITIL is a registered trade mark of the Cabinet Office Linux is the Registered Trademark of Linus Torvalds in the U.S. and other countries. OpenStack is trademark of OpenStack, LLC Red Hat Enterprise Linux is a Registered Trade Mark of Red Hat SUSE Linux Enterprise Server is a Registered Trade Mark of Novell. VMware, ESX, ESXi, vCenter are Registered Trade Mark of VMware, Inc. Windows, Windows Server are Registered Trade Mark of Microsoft Corporation. 33. AAppendix: IBM SmartCloud Foundation This appendix briefly describes two additional capabilities that are part of the IBM SmartCloud Foundation: IBM SmartCloud Control Desk; and IBM SmartCloud Application Performance Management.A.1IBM SmartCloud Control Desk IBM SmartCloud Control Desk unified asset and service management software provides a common control center for managing business processes for both digital and physical assets. This IT Infrastructure Library (ITIL)-compliant software is accessible through mobile devices and integrates with social media and development tools. Now you can choose the delivery model you need such as on-premise, software as a service (SaaS) or VM image and seamlessly change it when your business needs evolve. IBM SmartCloud Control Desk includes: Service request management gives you an efficient service desk for handling service requests and managing incidents; Change, configuration and release management provides advanced impact analysis and automated change procedures designed to reduce risk and support integrity of services; IT asset lifecycle management provides inventory management and software license compliance capabilities: helps to manage assets throughout their lifecycle, optimizing usage of digital and physical assets and minimizing compliance risks; Service catalog helps users solve their own problems: provides an intuitive selfhelp portal and a complete catalog of services; and Support for service providers supply service support and service delivery capabilities for multiple customers in a single deployed instance. This can help increase profitability and improve customer or user satisfaction.A.2IBM SmartCloud Application Performance Management The growing cloud computing model has created challenges for traditional performance and availability management practices. In traditional environments, one rely on disciplines like root cause analysis. However, today's cloud administrator is focused on resilience and elasticity. In this new cloud model, application availability is an imperative, and performance shortcomings will directly affect a business' bottom line revenue. IBM SmartCloud Application Performance Management extends the performance and availability capabilities of IBM SmartCloud Foundation to include transaction tracing, monitoring of end-user performance, and prevention of transaction bottlenecks which cause performance degradations. With options of agent or agentless tracking combined together in a single topology view, IBM SmartCloud 34. Application Performance Management may be used by IT Operations or development teams, with the ability to specify the level of detailed analysis necessary. IBM SmartCloud Application Performance Management is a single, comprehensive solution that intelligently manages performance, availability, and capacity for complex application infrastructures in cloud and hybrid environments. SmartCloud Application Performance Management offers the right visibility, control, and automation for critical applications. With its modular design, SmartCloud Application Performance Management lets you get started quickly and easily to add capabilities as they are needed. Additionally, SmartCloud Application Performance Management delivers: Easy-to-understand, role-based, web dashboards with smart drill downs, making it easier to sense, isolate, and diagnose availability, performance, and capacity issues; Analytics to improve capacity utilization and optimize performance; A common reporting tool, IBM Cognos based, to help make reporting simple and easy to customize Breadth of domain coverage in combination with a single trusted source of information for more accurate and faster problem diagnostics; and An entry edition that intelligently manages essential infrastructure and applications in cloud and hybrid environments. SmartCloud Application Performance Management can help enable IT operations and application support teams to optimize the performance of enterprise level IBM Business Process Manager Standard Edition and Advanced Edition solutions. Businesses need to react quickly to market demands and need to support execution of their business strategies with dynamic business processes that can be implemented quickly and easily. Business process management (BPM) enables you to discover, model, execute, rapidly change, govern, and gain end-to-end visibility of your business processes.