Windows System Programming using
PythonMark Hammond
OReilly Open Source Python ConferenceAugust 1999, Monterey, CA
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 2
About this PresentationMost content taken directly from upcoming book for
O’Reilly
Python Programming on Win32By Mark Hammond and Andy Robinson
http://www.ora.com/catalog/pythonwin32/
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 3
Who this talk is for?Existing Python programmers
– Even those without any Windows experience should follow this without problem.
Although there is not enough time to explain the relevant Windows APIs
Existing Windows Programmers– Even without Python experience, you should immediately see the
similarities between your existing language.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 4
Python and WindowsEach Python version comes with an installer package for
Windows.Standard Python port contains all cross-platform Python
features, but very few Windows specific features.Python for Windows extensions contains many useful
Windows extensions for Python.http://www.python.org/windows
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 5
Python Windows ExtensionsIncludes:
– Pythonwin: MFC based GUI environment and IDE/Debugger
– win32com: Interfaces Python and COM
– win32 Extensions: Interfaces to native Win32 API.Official releases can be found at http://www.python.org/windowsExtensions home is at http://starship.python.net/crew/mhammond
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 6
System Level Programming?For this talk, we define system level programming as
working with low-level features of Windows Files, Pipes, Processes, Threads, Services, Event Log and so forth.
Python and similar languages really not suitable for device-driver type development, and other more system-like Systems Programming!
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 7
Why not just use Python?Python has excellent native support for files, processes,
threads etc.These features are typically limited to those defined by ANSI C.
– Many advanced Windows features are not exposed using these interfaces.– Standard implementation of some of the standard library functions leaves a
little to be desired in a Windows environment.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 8
Portable Process Control (1 of 3)
Standard Python functions all work– Just often not quite how we would like!
os.system()import os
os.system(“notepad C:\\autoexec.bat”)
Problems– Creates a new console window when run from a GUI.– Waits for process to terminate.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 9
Portable Process Control (2 of 3)
os.execv family– Doesn’t search system path, and doesn’t parse command lines
os.execv("c:\\Winnt\\notepad.exe", \
("c:\\autoexec.bat",) )
– Does clobber your existing process - the call to os.execv() never returns!
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 10
Portable Process Control (3 of 3)
os.popen()>>> file = os.popen("echo Hello")>>> file.read()'Hello\012'– Works fine from Windows NT console programs, but fails miserably from
a GUI!– win32pipe module in the Win32 extensions provides a working
replacement.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 11
Better Process Control (1 of )
win32api module provides some high-level, Windows specific functions.
win32api.WinExec()– Very similar to os.system(), but overcomes limitations.– >>> import win32api– >>> win32api.WinExec("notepad")– Optional parameter allows you to specify the Window’s initial state (eg,
minimized)
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 12
Better Process Control (2 of 2)
win32api.ShellExecute()– Typically opens “documents” - eg, execute “foo.doc”, and (typically) Word
will open.– Finer control over the new process.– Can also execute arbitrary executables - not limited to documents.– For example, to print a specific document:win32api.ShellExecute(0, "print", \ "MyDocument.doc", None, "", 1)
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 13
Ultimate Process Control (1 of 2)
win32process module exposes the low level Win32 API.Full support for CreateProcess, CreateProcessAsUser,
CreateThread etc.Full support for Windows Handles
– Files can be passed as stdin/out/err– Process and thread handles are waitable using the win32event module.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 14
Ultimate Process Control (2 of 2)
Able to set thread and process priority and affinity levels– Very handy for bugs that only appear in multi-processor
machines.Able to do all this for both existing and new processes.Process usage samples included in distribution.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 15
Introduction to our SampleFull sample Windows NT Service
– Provides event log and performance monitor information. Clients connect using Named Pipes
– Less than 75 lines of code for all this functionality.– Still too big to present in one hit
Selected excerpts included in slides Full code on CD, and at
http://starship.python.net/crew/mhammond/conferences/ora99
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 16
Portable Files and PipesPython has excellent built-in file support
– Inherits platform stdio support - wont bother discussing them here.Native Windows files only useful for highly-advanced features, such
as:– Overlapped IO– IO Completion Ports– Named Pipes– NT Security requirements
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 17
Native Files (1 of 4)
win32file.CreateFile() used for most file operations– Create and open regular files, memory mapped files, etc.– Takes seven parameters - c.f. open()’s two!– Returns a PyHANDLE object
Can be passed to any Python API wrapper expecting a handle.Auto-closed when last reference removed.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 18
Native Files (2 of 4)
Overlapped IO for asynchronous operations– File opened for overlapped IO requires a Windows event object.– All IO operations return immediately.– Event object signalled when IO complete
Great for high-performance servers– Simple to support multiple concurrent file operations per thread.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 19
Native Files (3 of 4) NT Completion ports for even better asynchronous control
– Windows manages associating the completed IO operation with a connection
– More complex to use.– Often requires a state-machine implementation.– Offers excellent performance - Microsoft’s recommended architecture
for scalable, high-performance servers.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 20
Native Files (4 of 4) Full support for NT Security
– Default security can be used by passing None– Many real-world applications require explicit security configuration
Full support for Windows Impersonation– Processes can automatically impersonate the remote pipe client.– Impersonate any user given their password.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 21
File Sample (1 of 2)
Overlapped IO is used self.overlapped = \ pywintypes.OVERLAPPED()# create the event to be used.self.overlapped.hEvent = \ win32event.CreateEvent(None,0,0,None)
Special security for pipes is needed for services sa = win32security.SECURITY_ATTRIBUTES()# Allow full access!sa.SetSecurityDescriptorDacl ( 1, None, 0 )
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 22
File Sample (2 of 2)
Create pipe and connect to client pipeHandle = \ win32pipe.CreateNamedPipe(pipeName, openMode, pipeMode, win32pipe.PIPE_UNLIMITED_INSTANCES, 0, 0, 6000, # 6 second timeout. sa)...hr = win32pipe.ConnectNamedPipe(pipeHandle,\ self.overlapped)
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 23
Windows NT ServicesSimilar concept to a Unix daemonA few special requirements
– Must respond to asynchronous commands from NT to (e.g.) Shutdown– Must be capable of reporting status to NT
NT has built-in UI for configuring and controlling services.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 24
Python controlling ServicesFull exposure of the Windows NT Service Control Manager
API– Start, Stop, Pause Services, Install or Remove services, etc
win32serviceutil module makes it simple >>> win32serviceutil.StopService("Messenger")(32, 3, 0, 0, 0, 6, 20000)>>> win32serviceutil.StartService(...)>>>
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 25
Implementing ServicesSimple to implement Services in Python
– Smallest service is around 16 lines of code!Includes debug support, self-installation, and fully controllable from Windows NTFew more lines needed to make it something useful :-)
Simply sub-class Service base-class, and implement control features– Minimum required is StopService– Trivial to implement most controls
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 26
Windows NT Event LogCentral place for programs to log information.
– Ideal for Services - can not present effective GUIsBenefits to programmer
– Built in transaction and thread safety, maximum size ability, etc.Benefits to Administrator
– Central log of messages, and third party tools to help analysis.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 27
Python reading the Event LogComplex native API from win32evtlogSimpler interface from win32evtlogutil
– Define Feeder functiondef DumpRecord(record): print ”Got event ID”, record.EventID
– And feed it!win32evtlogutil.FeedEventLogRecords( \ DumpRecord)Got Event ID -2147483645Got Event ID -2147483645
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 28
Python writing the Event LogMore complex than reading - event sources
must be registered.win32evtlog and win32evtlogutil used here tooPython Service framework also supports
simple event log writing
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 29
Writing the Event Log SampleOur sample uses the Service Framework functions which
makes it trivialEvents are logged by ID, rather than explicit text.
– Our sample uses a built-in ID to log a “service starting” message import servicemanagerservicemanager.LogMsg( servicemanager.EVENTLOG_INFORMATION_TYPE, servicemanager.PYS_SERVICE_STARTED, (self._svc_name_, ''))
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 30
NT Performance MonitorBuilt-in NT tool for monitoring an applications performance.Application must be written to provide this data.API designed for smallest impact on program supplying data
– Moves the burden to the collecting application.– Not trivial to work with
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 31
Python reading from PerfMonUseful for overcoming limitations in built-in tool
– For example, sample the data hourly and log to a longer-term database.
– Useful for reading standard information about another process.Process ID, Memory usage, etc.
win32pdhutil module for reading data– Good sample code in the source file
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 32
Python supplying PerfMonComplex installation and setup procedure.
– C .h file and custom .ini file necessary at application installation– Supported by the Python Service framework
Quite trivial to use once running– Simply increment a counter - Performance Monitor handles
conversions to required units.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 33
PerfMon SampleInstallation more than we can cover here
– See InitPerfMon() methodWorking with counters trivial
– Increment our connections counter each connection self.counterConnections.Increment()
Perfmon manages converting to connections-per-second automatically.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 34
Our Sample in Detail (1 of 8)
PipeService2.py– The service implementation.
PipeService2_install.hPipeService2_install.ini– Required for performance monitor installation– Most services don’t need this!
PipeServiceClient.py– Sample client to connect to our service.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 35
Our Sample in Detail (2 of 8)
Service functionality in PipeService class– Base class handles most of the grunt– We simply supply service name and other optional attributes
class PipeService(\ win32serviceutil.ServiceFramework): _svc_name_ = "PythonPipeService" _svc_display_name_ = "A sample Python ... "
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 36
Our Sample in Detail (3 of 8)
We respond to an NT ServiceStop request by telling NT we are stopping, and setting a Windows Event
def SvcStop(self): # Before we do anything, tell the # SCM we are starting the stop process. self.ReportServiceStatus( \ win32service.SERVICE_STOP_PENDING) # And set my event. win32event.SetEvent(self.hWaitStop)
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 37
Our Sample in Detail (4 of 8)
Overlapped IO means we can wait for either a connection, or our Stop request
win32pipe.ConnectNamedPipe(pipeHandle,\ self.overlapped)
... # Wait for either a connection, or # a service stop request. waitHandles = self.hWaitStop, \ self.overlapped.hEvent
rc = win32event.WaitForMultipleObjects(\
waitHandles, 0, timeout)
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 38
Our Sample in Detail (5 of 8)
Install our service simply by executing the service script– If not for Performance Monitor, the command-line would be simple:
C:\Scripts> PipeService2.py install– But PerfMon needs an extra arg
C:\Scripts> PipeService2.py \--perfmonini=PipeService2_install.ini installInstalling service PythonPipeService to ...Service installed
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 39
Our Sample in Detail (6 of 8)
Once installed, we can start the service– Start from Control Panel, or using the script itself
C:\Scripts> python.exe PipeService2.py start
And start a client test sessionC:\Scripts> python.exe PipeServiceClient.py \ Hi thereThe service sent back:You sent me:Hi there
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 40
Our Sample in Detail (7 of 8)
We will have Event Log records...
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 41
Our Sample in Detail (8 of 8) And Performance
Monitor Data.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 42
SummaryPython library flexible, rich and portable enough for many
tasksLow-level Windows programming achieved by using
extensions that expose the raw Windows APIThis talk should have shown:
– An overview of how this programming is done in Python.– How simple it is to do complex Windows tasks.
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 43
More InformationPython and Python Documentation
– http://www.python.org– http://www.python.org/doc
Python for Windows Extensions– http://starship.python.net/crew/mhammond– http://www.python.org/windows– Reference Manuals and Samples included with the distributions
O’Reilly Python ConferenceAug 24, 1999
Windows System Programming using Python
Slide 44
Thanks for comingMark Hammond
– [email protected]– http://starship.python.net/crew/mhammond