sprint 48 review

ManageIQ

Sprint 48 Review - Sprint End October 24, 2016

October 26, 2016

Overview● Sprint Statistics (Oleg Barenboim)● Classic UI (Harpreet Kataria)● Service UI (Chris Kacerguis)● Providers (Greg Blomquist)● Automate (Greg McCullough)● Platform (Gregg Tanzillo)● API (Alberto Bellotti)● Quality Engineering (Pete Savage)● Discussion

Sprint Statistics (O. Barenboim)

404 Pull Requests Merged for ManageIQ/manageiq

Sprint Statistics - Euwe Backports(O. Barenboim)

266 Pull Requests Backported to Euwe

PR Breakdown by Feature Category (O. Barenboim)

* Note that some PRs have more than one category.

Providers

Sprint Comparison (O. Barenboim)

Dar

ga-2

Dar

ga B

eta

Dar

ga -3

Dar

ga G

A

Dar

ga R

C

Dar

ga -

4

Euw

e B

eta

All Repo Stats (O. Barenboim)

Data source:

https://github.com/ManageIQthese are listed by most recent activity. go into each one that has been updated in the past 20 days and add up the PRs that have been merged

Order descending by #PRs merged

Repository PRsmanageiq 404integration_tests 99manageiq.org 53manageiq-ui-self_service 49manageiq-pods 14guides 12manageiq_docs 12manageiq-providers-amazon 11manageiq-providers-lenovo 9manageiq-appliance-build 7azure-armest 6miq_bot 6wrapanapi 6manageiq-appliance 4manageiq-api-client 1Total 693

https://github.com/ManageIQ

https://github.com/ManageIQ

http://manageiq.org

http://manageiq.org

Classic UI(Harpreet Kataria)

● PRs merged (171)○ Bugs (77)○ Enhancements (35)○ Tech Debt / Refactoring (20/27)○ Enhancements Backported to EUWE (10)

● Tech Debt / Refactoring○ Removed some old redundant and commented code○ RJS: cleanup of flash message div rendering○ Restored all previous TreeNodeBuilder#build functionality○ File chooser form control was moved into a common single partial○ Font icon cleanup(ongoing)○ Network provider converted to use RESTful routes like other providers


● Enhancements○ Host Aggregates UI (No screenshots available)○ Snapshots actions support for Volumes○ Snapshots actions support for Openstack VMs○ Support to create/update/delete OpenStack Cloud Tenants○ Support to create/update/delete Cloud Networks.○ Added Container Templates subtab under Containers○ Support for Cinder backup actions○ Added Ironic Controls support for OpenStack Infrastructure Hosts○ Support to show Cloud Volume Backup - Link to list of Cloud Volume

Backups was added on Cloud Volume summary screen, from there you can drill into its details view screen.(No screenshots available)

○ Support to allow self signed certificates on Automate Import screen (covered by GregM’s team)


● Enhancements○ Middleware Datasource validation enhancement○ Added Topology views for Cloud & Infra providers○ Support to generate authorization keys for remote regions○ Added Dashboard view for Infrastructure Providers○ Basic support for displaying chargeback report data in Service UI○ Option to hide deprecated template images by default during provisioning○ Schedule Automate Tasks (Demo: GregM’s team)○ Support for proxying the VNC or SPICE connection(Demo: Martin

Povolny)○ Hide VMs & Instances in explorer trees (Demo: Zita Nemeckova)


Snapshots actions support for Volumes Snapshots actions support for Openstack VMs


Support to create/update/delete OpenStack cloud Tenants


Support to create/update/delete Cloud Networks


New sub-tab Container Templates under Containers


Added Ironic Controls support for OpenStack Infrastructure Hosts


Support for Cinder backup actions


Middleware Datasource validation enhancement


Added Topology views for Cloud & Infra providers


Support to generate authorization keys for remote regions


Added Dashboard view for Infrastructure Providers


Option to hide deprecated template images by default during provisioning


Basic support for displaying chargeback report data in Service UI


● Schedule Automate Tasks (Demo: GregM’s team)

● Support for proxying the VNC or SPICE connection (Demo: Martin Povolny)● Hide VMs & Instances in explorer trees (Demo: Zita Nemeckova)

Service UI(C. Kacerguis)

● Back of the House Changes○ Completed the renaming of the Repo (ManageIQ-UI-Service)○ Migrated to yarn for dependency management○ Increased test coverage to 58%

● Added confirmation when removing items from the shopping cart

● Added basic support for displaying chargeback data● Added the Service Power operations for Start, Stop and

Suspend● Lots of bug fixes and technical debt cleaned up

Providers(G. Blomquist)

Containers(G. Blomquist)

● Container Templates● Chargeback for container images● Support Custom Attributes


Container Templates


Chargeback for Container Images


Support Custom Attributes

Google(G. Blomquist)

● Add load Balancer health checks● Hide deprecated images during provisioning

Hawkular(G. Blomquist)

● Validate when there is no daily data for live metrics● Conversion of Provider form to Angular● Fixed 'Summary' Link in the accordion menu● Better Datasource validation● Hide operation toolbar buttons

OpenStack(G. Blomquist)

● Lots of UI updates (Harpreet will cover this)● Enable node start/stop

RHEV(G. Blomquist)

Refresh performance improvements when creating a new VM

Lenovo(G. Blomquist)

Getting started … first commits rolling in

vSphere(G. Blomquist)

Provisioning validation when selecting hosts/clusters

vCloud(G. Blomquist)

Checkboxes in Provisioning

Pluggable Providers(G. Blomquist)

● supports :timeline● supports :terminate● supports :shutdown_guest● supports :reboot_guest

Automate(Greg McCullough)

Schedule Automate Tasks● Specify options similar to Automate -> Simulate

○ System/Process Instance○ Message○ Request○ Optional

■ Target Object■ Additional Key/Value Pairs

● Creates Automation Request object● Run Once or on a recurring schedule


Schedule Automate Tasks


New notification methods enable users to create custom notifications from Automate:● $evm.create_notification

● $evm.create_notification! - raises errors.

Currently supports User level notifications.

Notification and NotificationType Service Models added.


New $evm.create_notification Method

Basic notification:

Specify only a message:

$evm.create_notification(:message => “my custom provisioning job has started”)

Generates a “user” “info” notification.



Custom notification:

Specify level and subject:

$evm.create_notification(:level => “error”, :subject => vm, :message => “my custom provisioning job has failed”)

Generates a “user” “error” notification.



Predefined notification:

Specify type with subject:

$evm.create_notification(:type => :vm_provisioned, :subject => vm)

Generates a “user” “info” notification.



Next Steps:

● Global/Tenant level support.

● The addition of $evm.create_notification to life cycle

state machine methods. (start/finish/error)


Retirement Enhancements:

An unsuccessful retirement can leave a VM in an archived/orphaned state with an “unknown” power state.

Although the majority of these VM’s have been removed from the provider, our current retirement process prevents these VMs from fully retiring.


Retirement Enhancements:● Allow VM’s with unknown power state to retire.

○ Changed check_pre_retirement method to allow a

VM with an “unknown” power state to proceed to

retirement.

● Allow Archived/Orphaned VMs to retire.

○ Changed remove_from_provider method to allow an

archived/orphaned VM to complete retirement.


● Import Automate model from Git Repository○ Support for Self Signed Certificates○ Support for Rake command○ Route the import request to the appliance with the

‘Git Repository Owner’ role


● Self Signed Certificate support○ Uncheck the “Verify Peer Certificate”


Rake command● Required Parameters

○ GIT_URL● Optional Parameters

○ USERID○ PASSWORD○ REF = Branch name (default is master)○ VERIFY_SSL = 0 | 1 (default 1)○ REF_TYPE = BRANCH|TAG (default is BRANCH)


Catalog Items

● Default provisioning state-machine entry-point updated○ Catalog Item - CatalogItemInitialization○ Catalog Bundle - CatalogBundleInitialization


Generic Objects● Pass “param_#_type” value to automate for method calls:

○ Example:■ param_1 = “1.0”■ param_1_type = “Float”

● Expose service model methods:○ property_attributes○ property_associations○ property_methods○ Find_objects

■ Search by active_record and generic object attributes■ Search by associations (using association IDs)


Added cinder backup/restore actions to Automate

MiqAeServiceManageIQ_Providers_Openstack_CloudManager_CloudVolume

● Method Definition: ○ backup_create(backup_name = "", incremental = false, options = {})

● Method Definition:○ backup_restore(backup_id, options = {})

● Option○ :sync = true/false (Default: false)

Provisioning● No longer appending “_” to the VM Name when generating unique names

for service provisioning.○ Underscore is an invalid character on some Providers○ Example: VM Name = test_vmware

■ Before: test_vmware_001■ After: test_vmware001

○ If required, append VM Name/Instance Name with “_” in catalog item.



That’s it, the party is over.

Platform(Gregg Tanzillo)

Enhancements / Bug Fixes - 95 PRs Merged!● Chargeback

● Tenancy

● Centralized Administration

● Additional updates

● API

Platform: Chargeback(Gregg Tanzillo)

● Major rework of calculations

○ No longer based on the count of metrics

○ Instead, based on interval duration (hour/week/month)

○ Fixed rate costs now correct for weekly and monthly

intervals

○ Metrics are either avg or max of interval instead of the

sum

● Reinstated weekly and monthly rates

Platform: Chargeback(Gregg Tanzillo)

● JSON generation of chargeback report for service

○ Supports showing service cost in Service UI

○ Includes Rest API

GET /api/services/:id?attributes=chargeback_report

● VM GUID a reportable field

● Many PRs for refactoring and performance (thank you

Simon)

Platform: Tenancy(Gregg Tanzillo)

● Ad hoc sharing of resources across tenants

○ Backend complete

○ VMs, templates and Services

○ Work on API and user interface underway

Platform: Centralized Administration(Gregg Tanzillo)

● VM reconfigure

● Provision VM from a template

● User and group ownership working in global region

○ Without triggers :)

Platform: Centralized Administration(Gregg Tanzillo)

● Service provisioning - close

○ Addressing some last minute bugs

○ Classic UI only

● Regional method relay mixin for generic central

administration of models

○ New DSL defined by the mixin

○ Enables dynamic of methods routing to a remote

region

Platform: Additional updates(Gregg Tanzillo)

● Provider authentication retry

○ Default schedule changed from 1 day to 1 hour

○ Retry on recoverable errors

○ Incremental delay between retries - 2 min, 4, 8 and 16

● System memory/swap information saved with status

○ Will now be able to detect when swap is invaded

Platform: Additional updates(Gregg Tanzillo)

● Configure a different disk for log files

○ Through appliance console

○ Longer retention of log history

● Increased log volume from 2gb to 11gb

API(Alberto Bellotti)

● Added ability to Create and Edit /api/requests

Interface to MiqRequest:

create_request(values, requester, auto_approve) update_request(request, values, requester)

POST /api/requests{

“options” : {

“request_type” : “service_reconfigure”,

“src_id” : 4,

“config_param” : “some_value_1”

},

“requester” : {

“user_name” : “admin”

},

“auto_approve” : true

}

POST /api/requests/:id{

“options” : {

“config_param” : “some_value_2”

},

“requester” : {

“user_name” : “admin”

}

}


● Introduced new /api/automate_domains primary collection○ Query and Actions on MiqAeDomain’s

○ Moved refresh_from_source action from /api/automate to /api/automate_domains

- Simplifies implementation

- refresh_from_source can now be triggered from Api Client gem

miq.automate_domains.find_by(:name => “ManageIQ”).refresh_from_source

● Updated refresh_from_source to work asynchronously

● /api/automate is strictly for Automate Browser queries

miq.get(“automate/manageiq”, :depth => -1, :search_options => “state_machines”)


● Added support for Provider Custom Attributes○ Same signatures as the custom attributes support for Vms

Queries:

GET /api/providers/:id?expand=custom_attributes

GET /api/providers/:id/custom_attributes

Adding, Editing and Deleting custom attributes via actions on the subcollection:

POST /api/providers/:id/custom_attributes

{

“action” : “add” | “edit” | “delete”,

“resources” : [

{ “name” : “ca_name1”, “value” : “ca_value1” },

{ “name” : “ca_name2”, “value” : “ca_value2” },

...

]

}


● Added Service Power Operations to /api/services

All service power actions are asynchronous and include the task_id/href in the results.

○ Service resource operations on /api/services/:id

POST /api/services/:id{

“action” : “start” | “stop” | “suspend”}

○ Bulk Service operations on /api/services

POST /api/services{

“action” : “start” | “stop” | “suspend”,“resources” : [

{ “href” : “http://localhost:3000/api/services/101” },{ “href” : “http://localhost:3000/api/services/102” },...

]}

http://localhost:3000/api/services/101

http://localhost:3000/api/services/102


● Added ability to Create and Edit Service Dialogs○ Create Service Dialog:

POST /api/service_dialogs{

“description” : “...”,“label” : “...”,“dialog_tabs” : [ ]

}

○ Edit Service Dialog (individual and bulk edits)

POST /api/service_dialogs/:id{

“action” : “edit”,“resource” : {

“label” : “...”,}

}

Quality Engineering(Pete Savage)

Manual

● Continuing with upstream testing○ Full regression testing○ Sprint New Feature testing

● 38 new deliverables in Sprint 47○ 53% looking good○ 3% blocked, buggy or not fully implemented○ 44% unknown, testing in progress


Automation● Still seeing some UI quirks where page is not ready when we interact with it

○ Under investigation

● Navmazing (new navigation framework) conversion○ 677 -> 398 removals of old force_navigate○ More in PRs yet to be merged

● Widgetastic (new page model framework) taking shape○ Have several PRs now using this much more explicit design


Integration Tests

99 PRs Merged

Discussion

Sprint 49 Review - November 16

sprint 48 review

Software