vio updates - ibm · pdf file• ability to roll out updates to vio servers without...
TRANSCRIPT
Technical University/Symposia materials may not be reproduced
in whole or in part without the prior written permission of IBM. 9.0
© Copyright IBM Corporation 2014
VIO UpdatesFeatures You May Have Missed
James Nash
Power Systems Strategic Initiatives
IBM
© Copyright IBM Corporation 2014 2
© Copyright IBM Corporation 2014 3
© Copyright IBM Corporation 2014
James Nash
[email protected] Systems Strategic Initiatives
(formerly known as ATS)
Special Thanks to
Benoit Creau
Steve Knudson
Steve Nasypany
Mark Olson
Steve Pittman
4
Session Objectives What’s relatively new
© Copyright IBM Corporation 2014
PowerVM Editions and Versions
VIO Server 2.2.3
― Simplified SEA failover configuration
― Shared Storage Pool enhancements
― Updates to ‘part’
― NPIV Statistics
― Useful but oft forgotten
Live Partition Mobility
Power Enterprise Pools
Power Integrated Facility for Linux
VIO Alternatives
―SR-IOV
―RoCE
―PowerKVM5
PowerVM Editions
© Copyright IBM Corporation 2014 6
Virtual I/O Server
VIO Server
POWER Hypervisor (PHYP)
Client
LPAR
Client
LPAR
Client
LPAR
© Copyright IBM Corporation 2014 7
VIO Versions
• 2.2.1.5-FP25 SP03 December 2012
• 2.2.1.6-FP25 SP04 April 2013
• 2.2.1.7-FP25 SP05 August 2013
• 2.2.1.8-FP25 SP08 February 2014
• 2.2.1.9-FP25 SP09 August 2014
• 2.2.2.0 December 2012
• 2.2.2.1-FP26 March 2013
• 2.2.2.2 FP26 SP01 February 2013
• 2.2.2.3 FP26 SP02 August 2013
• 2.2.2.4-FP26 SP04 February 2014
• 2.2.2.5-FP26 SP05 August 2014
• 2.2.2.6 January 2015
• 2.2.3.1-FP27 SP01 November 2013
• 2.2.3.2-FP27 SP02 April 2014
• 2.2.3.3-FP27 SP03 August 2014
• 2.2.3.4-FP27 SP04 November 2014
© Copyright IBM Corporation 2014 8
Latest Levels as of August 2014
Keep it current
• 2.2.1.9-FP25 SP09 August 2014
• 2.2.2.6 January 2015
• 2.2.3.4-FP27 SP04 November 2014
© Copyright IBM Corporation 2014 9
HMC 8.8.2 SP1 – New view
© Copyright IBM Corporation 2014
HMC 8 requires:
Rack Mount CR5 or newer
Desk-side C08 or newer
At least 2 GB of memory
10
NIB and SEA Failover
© Copyright IBM Corporation 2014 11
SEA Failover with Load Balance
© Copyright IBM Corporation 2014 12
Simplify SEA Failover
• No Need for a Control Channel
• Requirements
– VIO server 2.2.3 or higher
– HMC 7.7.8 or higher
– System Firmware 780
• Note – Function is not available on “B” model servers
– IBM Power 770 (9117-MMB)
– IBM Power 780 (9179-MHB)
– VLAN ID 4095 not in use
© Copyright IBM Corporation 2014 13
© Copyright IBM Corporation 2014
VIO Server 1
ent0
Ethernet
Switch 1
Client
LPARClient
LPAR
VIO Server 2
ent3
13
ent0
11
ent0ent1
11ent3
13
ent2
12ent1
11
ent2
12
ent0
12
Simplify SEA Failover – VIO 2.2.3
Ethernet
Switch 2
14
© Copyright IBM Corporation 2014
padmin@vio1$ entstat –all ent1 | grep –iE “Priority|Port VLAN ID”
Priority: 1 Active: False
Port VLAN ID: 11
padmin@vio1$ entstat –all ent2 | grep –iE “Priority|Port VLAN ID”
Priority: 1 Active: False
Port VLAN ID: 12
padmin@vio1$ entstat –all ent3 | grep –iE “Priority|Port VLAN ID”
Priority: 1 Active: False
Port VLAN ID: 13
Simplify SEA Failover
• Verify VLAN IDs and Trunk Priorities
padmin@vio2$ entstat –all ent1 | grep –iE “Priority|Port VLAN ID”
Priority: 2 Active: False
Port VLAN ID: 11
padmin@vio2$ entstat –all ent2 | grep –iE “Priority|Port VLAN ID”
Priority: 2 Active: False
Port VLAN ID: 12
padmin@vio2$ entstat –all ent3 | grep –iE “Priority|Port VLAN ID”
Priority: 2 Active: False
Port VLAN ID: 13
15
© Copyright IBM Corporation 2014
padmin@vio1$ mkvdev –sea ent0 –vadapter ent1 ent2 ent3
–default ent1 –defaultid 11 –attr ha_mode=sharing
ent4 Available
Simplify SEA Failover
• Create your Shared Ethernet Adapter
padmin@vio2$ mkvdev –sea ent0 –vadapter ent1 ent2 ent3
–default ent1 –defaultid 11 –attr ha_mode=sharing
ent4 Available
ctl_chan=ent#
ctl_chan=ent#
16
© Copyright IBM Corporation 2014
padmin@vio1$ lsdev -dev ent4 –attr
attribute value description user_settable
accounting disabled Enable per-client accounting of network statistics True
adapter_reset yes Reset real adapter on HA takeover True
ctl_chan Control Channel adapter for SEA failover True
gvrp no Enable GARP VLAN Registration Protocol (GVRP) True
ha_mode sharing High Availability Mode True
hash_algo 0 Hash algorithm used to select a SEA thread True
jumbo_frames no Enable Gigabit Ethernet Jumbo Frames True
large_receive yes Enable receive TCP segment aggregation True
largesend 1 Enable Hardware Transmit TCP Resegmentation True
lldpsvc no Enable IEEE 802.1qbg services True
netaddr 0 Address to ping True
nthreads 7 Number of SEA threads in Thread mode True
pvid 15 PVID to use for the SEA device True
pvid_adapter ent1 Default virtual adapter for non-VLAN-tagged packets True
qos_mode disabled N/A True
queue_size 8192 Queue size for a SEA thread True
real_adapter ent0 Physical adapter associated with the SEA True
send_RARP yes Transmit Reverse ARP after HA takeover True
thread 1 Thread mode enabled (1) or disabled (0) True
virt_adapters ent1,ent2,ent3 List of virtual adapters associated with the SEA True
Simplify SEA Failover
• Verify VLAN IDs and Trunk Priorities
17
© Copyright IBM Corporation 2014
padmin@vio1$ entstat –all ent4 | grep –i “Control Channel PVID”
Control Channel PVID: 4095
padmin@vio1$ entstat –all ent4 | grep –i state
State: PRIMARY_SH
Simplify SEA Failover
• Where is the Control Channel
padmin@vio2$ entstat –all ent4 | grep –i “Control Channel PVID”
Control Channel PVID: 4095
padmin@vio2$ entstat –all ent4 | grep –i state
State: BACKUP_SH
18
© Copyright IBM Corporation 2014
VIO Server 1
ent0
ent4
SEA
Ethernet
Switch 1
Client
LPARClient
LPAR
VIO Server 2
ent3
13
ent0
11
ent0ent1
11
ent4
SEA
ent3
13
ent2
12ent1
11
ent2
12
ent0
12
VLAN 4095
Simplify SEA Failover – VIO 2.2.3
Ethernet
Switch 2
19
Remove Existing Control Channel
© Copyright IBM Corporation 2014 20
© Copyright IBM Corporation 2014
padmin@vio1$ oem_setup_env
root@vio1# rmdev –l ent6
ent6 Defined
or
padmin@vio1$ rmdev –dev ent6 -ucfg
ent6 Defined
Simplify SEA Failover
• Requires both SEA adapters in a defined state (outage)
padmin@vio2$ oem_setup_env
root@vio2# rmdev –l ent6
ent6 Defined
or
padmin@vio2$ rmdev –dev ent6 -ucfg
ent6 Defined
21
© Copyright IBM Corporation 2014
root@vio1# lsattr –El ent6 –a ctl_chan
ctl_chan ent4 Control Channel adapter for SEA failover True
root@vio1# chdev –l ent6 –a ctl_chan=“”
ent6 changed
padmin@vio1$ lsdev –dev ent6 –attr ctl_chan
ctl_chan ent4 Control Channel adapter for SEA failover True
padmin@vio1$ chdev –dev ent6 –attr ctl_chan=“”
ent6 changed
Simplify SEA Failover
• Blank out the control channel
root@vio2# lsattr –El ent6 –a ctl_chan
ctl_chan ent4 Control Channel adapter for SEA failover True
root@vio2# chdev –l ent6 –a ctl_chan=“”
ent6 changed
padmin@vio2$ lsdev –dev ent6 –attr ctl_chan
ctl_chan ent4 Control Channel adapter for SEA failover True
padmin@vio2$ chdev –dev ent6 –attr ctl_chan=“”
ent6 changed
22
© Copyright IBM Corporation 2014
root@vio1# mkdev –l ent6
ent6 Available
or
padmin@vio1$ cfgdev –dev ent6
ent6 Available
Simplify SEA Failover
• Requires both SEA adapters in a defined state (outage)
root@vio2# mkdev –l ent6
ent6 Available
or
padmin@vio2$ cfgdev –dev ent6
ent6 Available
23
© Copyright IBM Corporation 2014
padmin@vio1$ entstat –all ent4 | grep –i “Control Channel PVID”
Control Channel PVID: 4095
padmin@vio1$ entstat –all ent4 | grep –i state
State: PRIMARY_SH
Simplify SEA Failover
• Where is the Control Channel
padmin@vio2$ entstat –all ent4 | grep –i “Control Channel PVID”
Control Channel PVID: 4095
padmin@vio2$ entstat –all ent4 | grep –i state
State: BACKUP_SH
24
© Copyright IBM Corporation 2014
VIO Server 1
ent0
ent6
SEA
Ethernet
Switch 1
Client
LPARClient
LPAR
VIO Server 2
ent3
13
ent0
11
ent0ent1
11
ent6
SEA
ent3
13
ent2
12ent1
11
ent2
12
ent0
12
VLAN 4095
Simplify SEA Failover – VIO 2.2.3
Ethernet
Switch 2
25
Simplification Summary
© Copyright IBM Corporation 2014
Simplified SEA Failover setup
If you do not specify a control channel (ctl_chan) when
running mkvdev –sea … The system will “discover” SEA
Failover “partner” adapter in the other VIO
Discovery protocol uses VLAN id 4095. If this is one of your
actual tagged VLANs, you must continue to use control
channel on both sides
Multiple SEA pairs in the machine can share VLAN 4095 for
discovery
This is still SEA failover, and we still set priority 1 or 2 on the
trunked virtual adapter in the SEA
VIO server 2.2.3, HMC 7.7.8, Firmware 780. Not supported on
MMB or MHB at this time.
Perhaps we stay consistent with our current Power7 practices,
and use this for new Power8 machines.
26
Shared Storage Pools
Shared Storage Pool
PHYP
VIOS VIOS
PHYP
VIOS VIOS
PHYP
VIOS VIOS
PHYP
VIOS VIOS
© Copyright IBM Corporation 2014 27
VIO 2.2.3
© Copyright IBM Corporation 2014
Shared Storage Pool Enhancements
– Pool Resiliency – mirror the storage pool (two failover groups)
– Pool shrink by removing physical volume
– Dynamic disk growth in the pool
– Scaling – more clients, larger physical volumes
– New lu, pv, failgrp commands
– Cluster wide operations performed concurrently
– VIO 2.2.2 added
• ability to roll out updates to VIO servers without stopping the cluster
• In case of failed repository disk, cluster remains active, no changes,
until disk is replaced
• VLAN tagging supported in the cluster
developerWorks articles to get started
28
Shared Storage Pool Limits
Feature Min Max
• Number of VIOS Nodes in Cluster 1 16
• Number of Physical Disks in Pool 1 1024
• Number of Virtual Disks (LUs) Mappings in Pool 1 8192
• Number of Client LPARs per VIOS node 1 200
• Capacity of Physical Disks in Pool 10GB 16TB
• Storage Capacity of Storage Pool 10GB 512TB
• Capacity of a Virtual Disk (LU) in Pool 1GB 4TB
• Number of Repository Disks 1 1
• Capacity of Repository Disk 512MB 1TB
Recommended VIO server requirements – per SSP node:
• Minimum CPU: 1 CPU of guaranteed entitlement
• Minimum memory: 4GB
• Storage requirements per SSP cluster (minimum): – One fiber-channel attached disk for cluster repository, 1 GB
– At least one fiber channel attached disk for data, 10GB
© Copyright IBM Corporation 2014 29
New command: lu
$ lu -list
POOL_NAME: test_pool
TIER_NAME: SYSTEM
LU_NAME SIZE(MB) UNUSED(MB) UDID
aix2-rootvg 10240 0 ad4f5d...
aix3-rootvg 20480 18460 242eac...
aix4-rootvg 102400 0 56e22b...
aix5-rootvg 10240 0 1a4758...
aix6-rootvg 40960 0 bfa955...
aix7-rootvg 51200 0 599ad4...
© Copyright IBM Corporation 2014 30
New command: pv
$ lspv
NAME PVID VG STATUS
hdisk0 00f629dbc192abd7 rootvg active
hdisk1 00f629dbbe759aa5 caavg_private active
hdisk2 00f629dbbe7c9b29 None
hdisk3 00f629dbbd739cbe None
$ pv -list
POOL_NAME: test_pool
TIER_NAME: SYSTEM
FG_NAME: Default
PV_NAME SIZE(MB) STATE UDID
hdisk2 1048576 ONLINE 3C231IBM FlashSystem
© Copyright IBM Corporation 2014 31
VIO Sizing Recommendations
© Copyright IBM Corporation 2014
http://www.redbooks.ibm.com/redbooks/pdfs/sg248062.pdf32
VIO Sizing – System Planning Tool
© Copyright IBM Corporation 2014
http://www-947.ibm.com/systems/support/tools/systemplanningtool/33
System Planning Tool
© Copyright IBM Corporation 2014 34
VIO Sizing – Workload Estimator
• Updated with information on POWER8 and PowerKVM
© Copyright IBM Corporation 2014
http://www-947.ibm.com/systems/support/tools/estimator/35
Workload Estimator
© Copyright IBM Corporation 2014 36
Other options
• IBM Sales Representative
• IBM Client Representative
• IBM Architect
• IBM Client Technical Specialist
• IBM Business Partner
1-866-883-8901
http://www-304.ibm.com/partnerworld/wps/bplocator/landing
• IBM Techline
© Copyright IBM Corporation 2014 37
After you have the box…
© Copyright IBM Corporation 2014 38
Performance Advisors
• Run advisors on test or production systems.
• Advisors will evaluate the environment for performance
optimization opportunities
– Gives guidance on how to make the necessary changes.
• Three advisors available….
– PowerVM Virtualization
– Java (JPA)
– VIOS & Virtual Ethernet (built-in VIO with v2.2.2.1 via ‘part’)
• “Built in Smarts” to detect some of the most common
problems that are encountered
• Available on Developer Works
– FREE OF CHARGE
• https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power+Systems/page/Performance+Tools
© Copyright IBM Corporation 2014 39
Virtualization Performance Advisor
• Diagnoses performance issues within PowerVM partitions.
• Runs within an AIX partition (dedicated or shared)
• Analyzes performance characteristics
• Provides a health check report.
• Report includes system configuration (available resources, hardware & software levels, etc), partition (type, mode, available resources, s/w levels), partition settings and recommendations to improve performance (e.g. placement, SMT, processor folding, processor, and memory pool utilization).
© Copyright IBM Corporation 2014 40
Java Performance Advisor
© Copyright IBM Corporation 2014 41
VIO 2.2.2.1 – part command
© Copyright IBM Corporation 2014 42
Fibre Channel Statistics
© Copyright IBM Corporation 2014 43
Color-coded Icons
© Copyright IBM Corporation 2014 44
Severe Performance Problem
© Copyright IBM Corporation 2014 45
Investigate
© Copyright IBM Corporation 2014 46
part - Components Monitored
The following are the types of advisory reports that
are generated by the VIOS Performance Advisor
tool: System configuration advisory report
–CPU (central processing unit) advisory report
–Memory advisory report
–Disk advisory report
–Disk adapter advisory report
– I/O activities (disk and network) advisory report
© Copyright IBM Corporation 2014
https://www.ibm.com/developerworks/wikis/display/WikiPtype/VIOS+Advisor
An option for VIO servers below v2.2.2.1
47
Reminder – viosbr (added at v2.1.2)
© Copyright IBM Corporation 2014
$ viosbr -backup file backup -frequency daily numfiles 5
Creates a backup file once a day, prefixed with
“backup” in /home/padmin/cfgbackups
– backup.01.tar.gz
– backup.02.tar.gz
– backup.03.tar.gz
– backup.04.tar.gz
– backup.05.tar.gz
$ viosbr -restore file
/home/padmin/cfgbackups/backup.03.tar.gz
48
http://www.ibm.com/partnerw
orld/gsd/searchprofile.do?na
me=VIOS_Recognized_List
For VIOS Recognized applications:http://www.ibm.com/partnerworld/gsd
Click on "Search solutions" link on the left
side of the page
Click on "Validations" towards the top
middle right of the page
Type "VIOS Recognized" in the search field
and click “Search”
This will generate a current list of officially
tested applications that are supported on
the VIO Server.
© Copyright IBM Corporation 2014
Third party software - updated
49
Live Partition Mobility
© Copyright IBM Corporation 2014
Evacuate option
migrlapr –o m –m source_server –t target_server –all
Performance suggestions
– large_send, large_receive on physical adapters in VIO
(these SHOULD already be on)
– Suggesting tcp_sendspace, tcp_recvspace 524288 on
VIO interface
– Jumbo frames if your environment supports it
50
Power Enterprise Pools
© Copyright IBM Corporation 2014
Power Enterprise Pools allow mobile activations of CPU and
memory to be shared among a group of pool of Power Systems
All previous activations are static and do not move between
servers
New systems can be added to the pool
Pool information can be viewed, including pool resource
assignments, compliance, history logs
Up to 75% of system memory can be mobile
In a 770 pool, up to 60 mobile activations allowed (feature EP22)
In a 780-795 pool, up to 120 mobile activations for 780, up to 192
for 795 (feature EP23)
For 770, 780, up to 30 mobile memory activations of 100GB
(EMA4)
For 795, up to 120 mobile memory activations of 100GB (EMA4)
51
Power Enterprise Pools
© Copyright IBM Corporation 2014
HMC 7.7.8, with 2GB of memory required
770-780 D models (9117-MMD, 9179-MHD) and 795 (9119-FHB)
only
Firmware 780 only (FW780, AH780)
770, 780 must have at least 4 static core activations, 25% of
memory static activations
795 must have 24 static core activations, or 25% of installed
cores, whichever is higher
In all servers, 25% of memory must have static activations
GA Dec 6 2013 for 795
GA April 25 2014 for 770, 780
Two type of pool – 770 pool, and a high end pool (780, 795)
“Master” HMC, a redundant HMC acts as “backup master”
52
Power Enterprise Pools
© Copyright IBM Corporation 2014
Configuration file obtained from IBM CoD to create a pool, add server to pool,
add mobile activations to pool
53
Power Integrated Facility for Linux (IFL)
© Copyright IBM Corporation 2014
One Power IFL feature is 4 processor, and 32GB
activations, with 4 PowerVM EE entitlement and SWMA,
feature ELJ0 (zero not O)
Lower price activation than for AIX or IBM i, for Linux
only
Power customer agrees to segregate AIX and IBM i
partitions into a separate virtual shared processor pool,
away from Power IFL cores.
Any IBM Software Group (SWG) products run on Linux
on Power are also discounted - priced at 70 Processor
Value Units (PVU)
770, 780 B, C, D models, and 795
Firmware 780, HMC 7.7.8
54
Power Integrated Facility for Linux (IFL)
© Copyright IBM Corporation 2014
How do we keep AIX and IBM i from running on the less
expensive Linux cores? AIX, i, and VIO go in a sub pool, where
we can set a maximum CPU limit. Lower cost Linux cores are
used in the default pool
55
o
System
Topology
Node
DRILL DOWN
Partition
DRILL DOWN
V1.1.2 support for POWER8
PowerVP Performance Monitor
© Copyright IBM Corporation 2014 56
Servers supported
― POWER7
― POWER7+
― POWER8
Firmware version 770, 780, 810 or higher.
System and partition-level agents
Minimum OS levels:
― AIX 6.1, AIX 7.1
― RedHat Enterprise Linux 6.4
― SUSE Linux Enterprise Server 11 SP3
― IBM i 7.1 with TR6
PowerVP Performance Monitor
© Copyright IBM Corporation 2014 57
PowerVP Performance Monitor
© Copyright IBM Corporation 2014 58
© Copyright IBM Corporation 2014
PowerVP Performance Monitor
59
Do I need a VIO Server?
© Copyright IBM Corporation 2014
• RoCE or IB
• SR-IOV
• PowerKVM
• PowerNV
60
RDMA over Converged Ethernet
RoCE
• PCIe3 2-port 40 GbE NIC / RoCE QSFP+ Adapter
– FC EC3B; CCIN 57BD
© Copyright IBM Corporation 2014
http://www-01.ibm.com/support/knowledgecenter/POWER8/p8hcd/fcec3b.htm?cp=POWER8%2F4-3-10-1-1-1661
SR-IOV
© Copyright IBM Corporation 2014
NIC SR-IOV for POWER7+ 770/780 with
Specific 10GE Ethernet adapters #EN0K and #EN0H in Gen2 slots
Or with newest IMFC #EN10 and #EN11
62
SR-IOV
Single Root I/O Virtualization
Runs “closer to the silicon” potentially offering some
performance efficiencies
Doesn’t require VIOS as a pre-req and thus can do simple
virtualization under PowerVM without VIOS …. BUT …. VIOS
continues to offer many additional advanced functions
Architecturally can virtualize a resource like an Ethernet
adapter and allocate/provide a user-defined minimum level of
bandwidth to a partition … Quality of Service (QoS)
Ethernet NIC announced.
Could use VIOS & SR-IOV together
LPAR #2
LPAR #3
LPAR #1
© Copyright IBM Corporation 2014 63
Additional Linux Options
© Copyright IBM Corporation 2014
Power S822L8247-22L
Power S824L
Power S812L8247-12L
64
The Redbook
© Copyright IBM Corporation 2014
http://www.redbooks.ibm.com/abstracts/sg248198.html?Open
65
© Copyright IBM Corporation 2014
VIO Command Reference (v2.2.3.3)
http://public.dhe.ibm.com/systems/power/docs/hw/p8/p8hcg.pdf
66
© Copyright IBM Corporation 2014
Appendix
PowerVM Shared Ethernet Adapter simplification:
Get rid of Control Channel Adapterhttp://chmod666.org/index.php/powervm-shared-ethernet-apdater-simplification-get-rid-of-control-channel-adapter/
PowerVM developerWorks - wikihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/PowerVM
Virtual IO and virtualization developerWorks - wikihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Virtual%20I
O%20and%20virtualization
Virtualization Best Practices developerWorks – wikihttps://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Virtualizatio
n%20best%20practices
HMC 8.8.1 SP1https://www.ibm.com/developerworks/community/blogs/brian/entry/hmc_version_8_release_8_8_1_highlights?lang
=en
VIO and IVM Command Referencehttp://public.dhe.ibm.com/systems/power/docs/hw/p8/p8hcg.pdf
VIO 2.2.3.3 Release Noteshttp://www-01.ibm.com/support/knowledgecenter/POWER8/p8eeo/p8eeo.pdf
67
James Nash
Power Systems
Strategic Initiatives
Formerly known as
ATS
1177 S. Beltline RoadCoppell, TX 75019
© Copyright IBM Corporation 2014