pvm06 configuration of virtual io on power6

7/30/2019 pVM06 Configuration of Virtual IO on POWER6

1/38

2008 IBM Corporation

Session Title: Configuration ofVirtual I/O on POWER6

Speaker Name: Janel Barfield

Session ID: pVM06

2008 IBM Power Systems Technical Conference

October 20 - 23, 2008 - Brugge, Belgium

IBM UNIX Software Service Enablement

email: [email protected]


2/38

IBM Training

2008 IBM

Corporation

Agenda

Virtual I/O configuration concepts

Describe the new Integrated Virtual Ethernet (IVE) adapter

Architecture

Multi-Core Scaling

Describe and configure virtual Ethernet

Configure external network access using IVE

Describe and configure external network access using Shared

Ethernet adapter (SEA) With Link Aggregation

With SEA Failover

Describe and configure virtual SCSI

Configure new file-backed virtual devices

Answer questions as time permits

email me with any questions

[email protected]


3/38

IBM Training

2008 IBM

Corporation

Virtual I/O Configuration Concepts

On POWER5 and POWER6, virtual I/O devices include virtual

Ethernet, Integrated Virtual Ethernet Adapter, Shared Ethernet

Adapter, and virtual SCSI devices

Virtual SCSI and Shared Ethernet adapters require the

Advanced POWER Virtualization feature and a configured

Virtual I/O Server

Integrated Virtual Ethernet is a new POWER6 feature

The Virtual I/O Server is a dedicated special appliance

partition


4/38

IBM Training

2008 IBM

Corporation

Integrated Virtual Ethernet (IVE)

X2

External Layer 2 or Layer 3 switch

Port Group 110 Gb or

1 Gb physical

port

1 Gb

physical port

LPARs

Logical Switch Logical Switch

Three IVE models:

Dual-port Gigabit: 1 port group, 2 physical 1 Gb ports

Dual-port 10 Gigabit: 2 port groups, 1 physical 10 Gb port per port group

Quad-port Gigabit: 2 port groups, 2 physical 1 Gb ports per port group


5/38

IBM Training

2008 IBM

Corporation

lhea

ent

OS

IVE Ports Logical ports are associated with a specific physical port

Port group:

Set of 16 logical ports on 1 or 2 physical ports:

Logical ports can be split evenly between the 2 physical ports in a port

group, or unevenly 1 or 2 port groups per Host Ethernet Adapter (HEA), depending on model

1 or 2 physical ports per port group, depending on model, each with own Layer

2 switch

Port Group

External Switch

Logical

Port

PhysicalPort

Logical

SwitchHEA

IVE

Logical devices as

they appear in AIX

LPAR


6/38

IBM Training

2008 IBM

Corporation

Multi-Core Scaling (MCS) The MCS value sets the maximum number of logical ports for a port group (port group setting)

Think of 16 logical ports as 16 paths to the processor MCS groups ports so that assignedlogical ports have multiple queue-pairs in AIX

Best IVE adapter performance when MCS value equal to # of processors in the LPAR

Using MCS limits the number of assignable logical ports for LPARs Default value is 4, each assigned logical port gets 4 paths to the processor, so only 4 logical

ports per port group available to assign to LPARs Must power off/on managed system to change MCS

Port Group

Switch

PhysicalSwitch

Port Group

Switch

PhysicalSwitch

MCS = 1 MCS = 4

4 portseach has 4 pathsto the processor

16 portseach has 1 pathto the processor


7/38

IBM Training

2008 IBM

Corporation

Managed System IVE Configuration (1 of 2)1. Select the managed system, and choose:

Hardware Information -> Adapters -> Host Ethernet

Physical port

settings

(actual)

LPARs which are configured for logical

ports using this physical port

2. Choose which IVE adapter

3. Choose physical port

4. Click Configure

button to configure

physical port


8/38

IBM Training

2008 IBM

Corporation

A Virtual I/O Server partition must be listed as the promiscuous LPAR if it is

using a port on the IVE as part of a Shared Ethernet adapter device

Managed System IVE Configuration (2 of 2)4. button is used to configure physical port settings:

MCS

Set to None or to a

partition that will take

entire physical port

Enabled

means HMC

will attempt to

negotiate flowcontrol for

physical port


9/38

IBM Training

2008 IBM

Corporation

LPAR IVE Configuration

Configurelogical ports

from the LPAR

profile or with

DLPAR


10/38

IBM Training

2008 IBM

Corporation

IVE Demonstration

Configure an IVE logical port on the VIOS LPAR

1. Check the configuration of the physical adapter

2. Add a logical port for the VIOS LPAR and make it promiscuous

LPAR1 LPAR2

POWER Hypervisor

POWER6 System

VIOS

HEA

P0 P1

lhea0

ent0

Ethernet

Switch

IBM T i i


11/38

IBM Training

2008 IBM

Corporation

Virtual Ethernet: POWER5 Hypervisor Ethernet Switch

The POWER Hypervisor provides the Ethernet switch functionality

for virtual Ethernet adapters on POWER5 and POWER6 systems:

Implements a software Ethernet switch

Supports up to 4094 networks (VLAN IDs)

Copies packets between partitions (memory to memory) Consistent with VLAN IEEE 802.1Q standard

The ports on this switch are configured via the HMC and are part of the

partitions configuration

VLAN 1 VLAN 2

VLAN 3

LPAR 1 LPAR 2 LPAR 3 LPAR 4 LPAR 5

POWER Hypervisor (switch)

IBM Training


12/38

IBM Training

2008 IBM

Corporation

Virtual Ethernet Adapters

Virtual Ethernet adapters are configured using the HMC or

IVM, in the partitions profile or dynamically with DLPAR

Select the virtual slot

number and PVID Configures the port for

this slot on the

Hypervisor switch

Select additional VLANs

Allows the adapter to

communicate on multiple

VLANS

Requires additional

configuration in the LPAR

Virtual Ethernet interfaces are configured in AIX just like

physical adapter interfaces


13/38

IBM Training


14/38

IBM Training

2008 IBM

Corporation

Shared Ethernet Adapter (SEA)

Shared Ethernet adapter bridges external networks to

internal VLANs:

Forwards frames at OSI Layer 2 and is transparent to IP

layer

Virtual I/O Server Partition

Layer 2 Bridge (Shared Ethernet adapter)

Device Driver Device Driver Device Driver

PhysicalAdapter

VirtualAdapter

VirtualAdapter

External

LANs

1-16 virtual adapters

which connect to

VLANs (and clientpartitions)

IBM Training


15/38

IBM Training

2008 IBM

Corporation

SEA Example

Shared Ethernet adapter is used to extend VLAN 2 and

3 using the physical Ethernet in the Virtual I/O Server

Shared

Ethernet

VLAN 3VLAN 3

VLAN 2

VLAN 2

Virtual I/O

Server

Managed systemEthernet Switch

Standalone serversLPAR 1, 2, 3

IBM Training


16/38

IBM Training

2008 IBM

Corporation

Configure Virtual Ethernet Adapter for SEA

Virtual adapters that will be

associated with an SEA on the

VIO Server must have the

Access external network

checkbox selected

Specify a PVID and optionally

any VLAN IDs that the SEA will

extend to an external network

- PVIDs are stripped from

frames in the SEA, so use aunique VLAN for PVID when

supporting access to VLANs

on an external network

Virtual Ethernet adapters and Hypervisor switch ports

are configured here on the HMC/IVM

IBM Training


17/38

IBM Training

2008 IBM

Corporation

Configure SEA Device in VIO Server Command to create Shared Ethernet Adapter:

$ mkvdev sea ent0 vadapter ent1 \default ent1 defaultid 1

Creates device (ent2 in this case):$ lsdev virtualname status description

ent1 Available Virtual I/O Ethernet Adapter (l-lan)

ent2 Available Shared Ethernet Adapter

PVID 1ent0 ent1

ent2 (SEA)

VIOS

switch

physical

Hypervisor

virtual

IBM Training


18/38

g

2008 IBM

Corporation

SEA Demonstration Configure an SEA adapter on the VIOS

1. Create a virtual Ethernet adapter on the VIOS LPAR that can access external

networks

2. Create the SEA on the VIOS LPAR using the lhea port and the virtual adapter

Ethernet

Switch

LPAR1 LPAR2

POWER Hypervisor

ent0 ent0

en0 en0

VLAN 1

192.168.1.1 192.168.1.2

PVID=1ieee_virtual_eth=0

VIOS

HEA

P0 P1

lhea0

ent0

ent1


POWER6 System

SEA


IBM Training


19/38

g

2008 IBM

Corporation

SEA with Link Aggregation Create link aggregation Ethernet adapter:

$ mkvdev lnagg ent0 ent1

ent3 shows up as:$ lsdev adapter | grep ent3

ent3 Available EtherChannel / IEEE 802.3ad LinkAggregation

Create Shared Ethernet adapter:

$ mkvdev sea ent3 vadapter ent2 \

default ent2 defaultid 1

ent2 PVID 1

(Physical)ent4 (SEA)

ent0

ent1

ent3

(Aggregate)

Virtual I/O Server

switchHypervisor

(Virtual)

IBM Training


20/38

2008 IBM

Corporation

Virtual SCSI Overview

PhysicalStorage

S S S

PHY

Virtual I/O Server

C

Client

C

Client

C

Client

VTDVTD

Hypervisor

PHY

PHY

PHY Physical

Adapter

SVSCSI Server

Virtual Adapter

CVSCSI Client

Virtual Adapter

VTD Virtual Target Device

C

VTD

S

VTD

The red connections show two clientsaccessing the same physical storage

(A) via two different server adapters

(B) and virtual target devices (D)

VTD

PHY

The blue connection shows multiple

target devices (D) attached to a

single server adapter (B)

IBM Training


21/38

2008 IBM

Corporation

Virtual SCSI Adapters Configuration

Define virtual SCSI server in VIO Server partition and client

adapter in AIX or Linux partition

Check availability of virtual SCSI server adapters on VIO Server:$ lsdev -virtualname status descriptionvasi0 Available Virtual Asynchronous Services

Interface (VASI)vhost0 Available Virtual SCSI Server Adaptervsa0 Available LPAR Virtual Serial Adapter

IBM Training


22/38

2008 IBM

Corporation

Create vSCSI Adapters Demonstration

Create virtual SCSI adapters on the VIOS and AIX

LPARs

LPAR1

POWER Hypervisor

POWER6 System

VIOS

vscsi0vhost0

FC cardSAN

fcs0 fcs1

hdisk6

hdisk7

sas0

hdisk5

hdisk0

Internal storage Optical device

cd0

IBM Training


23/38

2008 IBM

Corporation

File-Backed Virtual Devices File-back (FB) virtual device types:

File-backed disk devices

Files created in storage pools can be used as hdisk on client

File-backed optical media devices

Create a Virtual Media Repository which can be stocked withDVD-ROM/RAM media

Clients can use images stored in repository as cd0 devices with media

FB virtual device characteristics:

Read-only FB devices can be shared by multiple clients

Bootable FB devices appear in SMS

Reside in FB storage pools

Mount Directory = /var/vio/storagepools/

LV_NAME =

Granularity as small as 1MB or as large as parent Logical Volume

FB virtual devices are new as of Virtual I/O Server V1.5

IBM Training


24/38

2008 IBM

Corporation

Creating File-Backed Virtual Disks

Files on the virtual I/O Server can be used as backing storage:

1. Create a volume group (mkvg) or storage pool (mksp -f)

2. Create a FB disk storage pool (mksp -fb) inside volume

group/storage pool

3. Create a device in the pool (mkbdsp) and map to a vadapter

4. The client associated with that vadapter sees new FB device as

an hdisk

Volume Group/Storage Pool - contains hdisk(s)

FB Disk Storage Pool (contains FB virtual disks)

Target dev Target dev Target dev

IBM Training


25/38

2008 IBM

Corporation

Create FB Virtual Disks Example (1 of 2)

Create new volume group/logical volume storage pool:$ mkvg -vg stpool1 hdisk1 OR mksp -f stpool1 hdisk1

Create new FB storage pool in the logical volume storage pool:$ mksp -fb fbpool -sp stpool1 -size 10gfbpool

File system created successfully.

10444276 kilobytes total disk space.

New File System size is 20971520

Create new file device with a certain size, create the VTD, and mapto vhost adapter:$ mkbdsp -sp fbpool 30m -bdfb_disk1 -vadapter vhost3

Creating file "fb_disk1" in storage pool "fbpool".

Assigning file "fb_disk1" as a backing device.

vtscsi3 Available

fb_disk1

New storage pool(stpool1)

New FB storage pool(fbpool) that is 10 GB

inside ofstpool1

Create new 30 MB file called fb_disk1

Resulting VTD is named vtscsi3

and is mapped to vhost3


26/38

IBM Training


27/38

2008 IBM

Corporation

Create FB Virtual Optical Device (1 of 2)

Optional: create volume group/logical volume storage pool to holdthe Virtual Media Repository:

$ mkvg -vgmedrep hdisk4 OR mksp -fmedrep hdisk1

Could put the media repository in the VIOS rootvg instead

What to do with all that freespace in the VIOS rootvg? Since the repository holds

static images, it is a better candidate to use some of that free space than logicalvolumes that will provide virtual disks for running client LPARs

Create 10 GB Virtual Media Repository in the LV pool:$ mkrep -spmedrep-size 10G

You can increase the size of the repository later if necessary with chfs

Create media (aixopt1) in repository from a file:

Media could be blank, loaded from cd# device, or a file

Make it read-only to allow multiple clients to access it simultaneously

$ mkvopt -name aixopt1 -file dvd.product.iso-ro

IBM Training


28/38

2008 IBM

Corporation

Create FB Virtual Optical Device (2 of 2)

View repository and its contents:

$ lsrep

Size(mb) Free(mb) Parent Pool Parent Size Parent Free

10198 6532medrep 69888 59648

Name File Size Optical Accessaixopt1 3666 None ro

Create FB virtual optical device and map to vhost adapter:

$ mkvdev -fbo -vadapter vhost4

vtopt0 Available

You can create multiple virtual optical devices for each client if desired

Name them with the dev flag

Load the image into the media device:$ loadopt -vtdvtopt0-disk aixopt1

Use the unloadopt command to unload

You can perform multi-cd installs by loading subsequent volumes by reloading the virtualoptical device with loadopt -f


29/38

IBM Training


30/38

2008 IBM

Corporation

FB Device Command Examples (2 of 2) List FBPOOL details:

$ lssp -bd -sp fbpool

Name Size(mb) VTD SVSA

fb_disk1 30 vtscsi3 vhost3

fb_disk2 4096 vtscsi4 vhost3

Show all mounts including FB devices:$ mount

mounted mounted over vfs date options

--------------- --------------- ------ ------------ ---------------

/dev/hd4 / jfs2 Apr 18 13:01 rw,log=/dev/hd8

/dev/hd2 /usr jfs2 Apr 18 13:01 rw,log=/dev/hd8

/dev/hd9var /var jfs2 Apr 18 13:01 rw,log=/dev/hd8

/dev/hd3 /tmp jfs2 Apr 18 13:01 rw,log=/dev/hd8

/dev/hd1 /home jfs2 Apr 18 13:01 rw,log=/dev/hd8

/proc /proc procfs Apr 18 13:01 rw

/dev/hd10opt /opt jfs2 Apr 18 13:01 rw,log=/dev/hd8/dev/fbpool /var/vio/storagepools/fbpool jfs2 Apr 28 12:04 rw,log=INLINE

/dev/VMLibrary_LV /var/vio/VMLibrary jfs2 Apr 28 14:36 rw,log=INLINE

(First column of mount output was removed to fit output on slide. All filesystems are locally mounted.)

IBM Training


31/38

2008 IBM

Corporation

File-Backed Virtual Devices Demonstration

Configure a file-backed virtual disk and file-backed

virtual optical device

VIOSLPAR1

vhost1 vscsi1

rootvg

hdisk0

Virtual

Media

Repository

AIX53_iso

AIX61_iso

cl_mksysb

hdisk1

stpool1(LV storage pool)

fbpool1(FB storage

pool)

fb_disk1

fb_disk2

vtscsi2

vtopt1

POWER Hypervisor

hdisk2

cd1

IBM Training


32/38

2008 IBM

Corporation

Use Logical Volumes as Backing Devices

3) On the VIO Server, define storage resources

To create a volume group:$ mkvg [ -f ][ -vg VolumeGroup ] PhysicalVolume ...

To create a logical volume:$ mklv [ -lv NewLogicalVolume | -prefix Prefix ]

VolumeGroup Size [PhysicalVolume ... ]

To create a storage pool:$ mksp [-f] StoragePool PhysicalVolume ...

To create a backing device from available space in a

storage pool:$ mkbdsp [-sp StoragePool] Size [-bd BackingDevice]

-vadapter ServerVirtualSCSIAdapter

IBM Training


33/38

2008 IBM

Corporation

Define Virtual SCSI Target Devices

On the VIO Server, define virtual target devices

Boot the client or run cfgmgr to use new virtual devices

$ mkvdev -vdev TargetDevice -vadapter VirtualServerAdapter[ -dev DeviceName ]

For example:

$ mkvdev vdev hdisk3 vadapter vhost0vtscsi0 Available$ mkvdev vdev lv10 vadapter vhost0vtscsi1 Available

$ mkvdev vdev cd0 vadapter vhost0vtopt0 Available

Check the target devices with lsdev:$ lsdev -virtualname status descriptionvtscsi0 Available Virtual Target Device - Diskvtscsi1 Available Virtual Target Device - Logical Volumevtopt0 Available Virtual Target Device Optical Media

physical volume

logical volume

optical device


34/38

IBM Training


35/38

2008 IBM

Corporation

Viewing VIO Configuration from the HMC

HMC command line example:

hmc:~> lshwres -m hurston -r virtualio --rsubtype scsilpar_name=VIOS,lpar_id=1,slot_num=16,state=1,is_required=0,adapter_type=server,remote_lpar_id=,remote_lpar_name=LPAR3,remote_slot_num=3,"backing_devices=""0x8100000000000000//""""/var/vio/VMLibrary/aixopt1"""""""

. . .

IBM Training


36/38

2008 IBM

Corporation

Virtual Target Device Demonstration

Configure virtual target devices for the vSCSI adapterwith differentphysical backing devices

clientVG

LPAR1

POWER Hypervisor

POWER6 System

VIOS

vscsi0vhost0

FC card

SAN

fcs0 fcs1

hdisk6

hdisk7

sas0

hdisk5

hdisk0

Internal storage Optical device

cd0

vtscsi1

cl_lv

vtopt0

vtscsi0

cd0

hdisk0

hdisk1

IBM Training


37/38

2008 IBM

Corporation

Conclusion

Virtual I/O devices on POWER5 and POWER6 systemsinclude virtual Ethernet, Integrated Virtual Ethernet (IVE),Shared Ethernet (SEA), and virtual SCSI devices

Virtual Ethernet and IVE, are part of the base feature and donot require PowerVM (or Advanced Power Virtualization forPOWER5)

Devices are defined in the HMC

Configuration in AIX is same as for physical devices

Virtual SCSI and SEA devices extend the use of physicaldevices to multiple clients using the Virtual I/O Server andrequire the PowerVM feature

Create virtual devices withmkvdev on VIO Server command line to bridge virtualEthernet adapters (mkvdev sea) and to associate virtual SCSI adapters withphysical storage

File-backed virtual disks and optical devices provide even more

flexibility in storage allocation


38/38

pvm06 configuration of virtual io on power6

Documents