washington washington university in st louis [email protected] the new msr plugin download...

WashingtonWASHINGTON UNIVERSITY IN ST LOUIS

[email protected] http://www.arl.wustl.edu/~jdd

The New MSRPlugin Download

Environment

2WashingtonWASHINGTON UNIVERSITY IN ST LOUIS

John DeHart - 04/20/23

First: The Old Way• Compile plugin code on NetBSD• Copy plugin directory onto File System

destined for SPC• Boot SPC• Log in to SPC

– requires user space and serial line or telnet access

• Use modload(1) on SPC to load plugin– links plugin against kernel in /netbsd on SPC

• Use modstat(1) on SPC to see list of loaded plugins



The New Way• Compile plugin code on NetBSD• Verify plugin has no undefined symbols

– There is no link against running kernel anymore– wu_arl/msr/usr/src/sys/msr/msr_lkm.h provides macros to needed

kernel functions and structures• We can add more as we need them.• Plugins do need to be recompiled if msr_lkm.* changes• But if msr_lkm.* is stable, even if rest of kernel changes, plugin does

not need to be recompiled.

• Use pluginDownload utility from CP (Linux) to download plugin to SPC– Only runs on Linux right now…

• Use instantiatePlugin utility from CP to config plugin, filters, etc

• Use pluginStat utility from CP to view list of loaded plugins



Exposed Functions• Kernel functions

– assert()– MALLOC()– FREE()

• Clock Functions– cpu_clock_1usec()– cpu_clock_10usec()– cpu_clock_1msec()

• LKM Functions– lkm_nofunc()– lkmdispatch()– lkmexists()



Exposed Functions (continued)• MSR Plugin Control Unit

– pcu_deregister_class()– pcu_free_all_instances()– pcu_register_class()– pcu_list_all_classes()



pluginDownload• Uses gnu binary utilities cross-linker

– Runs on Linux, builds a NetBSD binary– Used to:

• prelink to get size information• final link to relocate

• Uses new AAL5 control data channels to download plugin directly into kernel memory.– uses sendcmd API to:

• allocate kernel memory for plugin• set up AAL5 data channel• call plugin entry function to complete load operation

• Side Note: sendcmd API– should be expanded to include all of sendcmd, but right now we

have about 15 “commands”.

• Source files: wu_arl/utilities/SPC/plugDownload



pluginDownload: steps it takes– pre-link object file so we can get sizes

• use a bogus kernel address to force a.out file to have valid sizes

– open pre-linked plugin file to get size

– allocate two chunks of kernel memory• 1: plugin code

• 2: plugin symbols

• returns addresses of two chunks and slot in lkm table

– re-link plugin to relocate

– open two dchans into MSR kernel• plugin code

• plugin symbols

– download plugin code and symbols

– finish plugin loading by calling its entry function• sendcmd -c rp_pcu –s load causes kernel to call entry function

– release dchans



USAGE: pluginDownloadpluginDownload [-h] [-d] [-V <vpivci>] [-D <fileDchan#> -D

<symtabDchan#>] [-s] -p <port#> -e <entry> <input file>

-h : get this usage message

-d : turn on debug output

-V <vpivci> : specify the VPI/VCI to use for download.

: overrides default of 164 + port#

-D <chan#> : specify the MSR Control Data Channel ids to use.

: First use of -D is for file channel

: second use of -D is for symbol table channel

: if they aren’t specified, they’ll be allocated

-p <port#> : specify the port to be downloaded to

-e <entry> : specify the symbol name that should be called

: when module is loaded

-s : include symbols in the plugin download



USAGE: instantiatePlugininstantiatePlugin -p port [-h] [-ge] [-f filterId] pluginId flags

srcAddr srcMask srcPort dstAddr dstMask dstPort protocol

-p port : MSR Port to query

-g : bind to general match classifier

-e : bind to exact match classifier

-f filterId : use this filter table entry (gen match classifier only)

-h : Get this usage message



Example ../../../utilities/bin/$MYOS/pluginDownload -p 4 -e wvplugin -s combined.o

# Gen plugin src src src dst dst dst

# Port Cfy Fid Id flags addr mask port addr mask port prot instantiatePlugin -d -p 4 -g -f 0 200 0 0 0 0 192.168.220.2 255.255.255.255 5555

17



Extra Supporting Notes



Data Transfer

VaddrStart of Virtual address block where data frames should go (32b)

LengthLength of block in bytes (32b)

Next AddrWhere next frame should go. Initially set to <Vaddr> (32b)

Seq #Next expected frame sequence number (first frame should be 1) (16b)Index(dchan#)

0

1

. . .

DCHAN_MAX

MSR Kernel Dchannel Table



Data Transfer (continued) MSR Control Data Channel RATM AAL5 Frame Format

(as it appears in the MSR Kernel)

dchan#(8bits) seq# (16bits)

framelength (16bits) pad (16bits)

AAL5 Trailer (8 Bytes)

AAL5 Frame Data (<= 1992 Bytes)

Shim Reserved space (8 Bytes)

1992

B 2000

B

2016

B

APIC Rx descpoints here flags (8bits)

flags:1: Data2: Ack4: Nack

washington washington university in st louis [email protected] the new msr plugin download...

Documents