Download - Mixed Language Programming
-
7/31/2019 Mixed Language Programming
1/43
Mixed Language
Programming
-
7/31/2019 Mixed Language Programming
2/43
Types of Languages Mixings
Embedding Python Using Python code
from code in a different language.
Extending Python Using code in adifferent language from Python.
We are going to focus only on extending
Python in this presentation.
-
7/31/2019 Mixed Language Programming
3/43
Why Mixing Python with Another
Language? Extending\Embedding Python:
To spare the development cycle of a code that wasalready written once (code reuse).
Extending Python: To improve Python code performance.
True multithreading support.
Type-safe language -> Less bugs (?).
Other reasons: To save testing time by testing low-level code using
Python high-level tests.
Other languages are more mature: Less bugs in the language.
Mission-specific tools that ease development.
-
7/31/2019 Mixed Language Programming
4/43
Python-C Integration
Using ctypes
-
7/31/2019 Mixed Language Programming
5/43
ctypes
Functionality:
Gives access to the OS specific DLLs via
Python.Allows calling C language functions in user-
defined DLLs\shared libraries from Python.
Pros:
No need to recompile the DLL (wrap librariesin pure Python).
Availability: External module up to V2.4. FromV2.5 part of the Python standard library.
-
7/31/2019 Mixed Language Programming
6/43
Loading a DLL cdecl calling convention (most of the
times):
dll = ctypes.cdll.LoadLibrary()
stdcallcalling convention (on Windows):dll =
ctypes.windll.LoadLibrary()
On Windows, some DLLs are beingloaded automatically:
Kernel32.dll is at ctypes.windll.kernel32
Msvcrt.dll is at ctypes.cdll.msvcrt
-
7/31/2019 Mixed Language Programming
7/43
Fundamental Data Types
Python typeC typectypes type
1 character
string
charc_char
int/longintc_int
int/longunsignedlongc_ulong
floatfloatc_float
floatdoublec_double
string or Nonechar * (NULL
terminated)
c_char_p
int\long or Nonevoid *c_void_p
-
7/31/2019 Mixed Language Programming
8/43
Fundamental Data Types (II)
An int data type:
i = ctypes.c_int(3) # or i=ctypes.c_int(), to get
an init val of 0. i.value = -99
A string data type:
s = "Hello, World"c_s = ctypes.c_char_p(s) # or c_s =
ctypes.c_char_p(), to an init val of NULL.
c_s.value = "Hi, there"
-
7/31/2019 Mixed Language Programming
9/43
Strings and Buffers
Assigning a new value to instances of thepointer types c_char_p and c_void_pchanges the memory locationthey pointto, not the contentsof the memory block.
If you need mutable memory blocks, usecreate_string_buffer():
buff = ctypes.create_string_buffer("Hello", 10)
# create a 10 byte bufferbuff.value = "Hi"
Use the raw attribute to get the entirebuffer contents.
ctypes.sizeof() == sizeof() in C
-
7/31/2019 Mixed Language Programming
10/43
Accessing Global Variables
Suppose that in your DLL, Test.dll, youhave the following exported global variabledeclaration:
__declspec(dllexport) double some_var;
To access it from ctypes, you must call thein_dll() method of the variables properctype, with the variables containing DLL
and name: test_dll = ctypes.CDLL(Test.dll)
test_dll_var = ctypes.c_double.in_dll(test_dll,"some_var")
test_dll_var.value = 3.14
-
7/31/2019 Mixed Language Programming
11/43
Calling Functions You can call every exported function (i.e.,
__declspec(dllexport)-ed function on Windows)in your DLL.
All Python types except integers, strings, andunicode strings have to be wrapped in theircorresponding ctypes type: libc = ctypes.cdll.msvcrt
printf = libc.printf
printf("An int %d, a double %f\n", 1234,c_double(3.14))
None is used as the NULL pointer: libc.time(None) # System time in seconds since
the UNIX epoch
-
7/31/2019 Mixed Language Programming
12/43
Windows Specific Issues
You must specify (for example)GetModuleHandleA or GetModuleHandleWexplicitly, and then call it with normal or unicodestrings respectively.
ctypes tries to protect you from calling functionswith the wrong number of arguments or thewrong calling convention (raises ValueError). It does this by examining the stack after the function
returns, so although an error is raised the functionhasbeen called.
ctypes uses SEH to prevent crashes fromgeneral protection faults due to invalid argumentvalues (raises WindowsError): >>> windll.kernel32.GetModuleHandleA(32)
-
7/31/2019 Mixed Language Programming
13/43
Specifying Functions Prototypes
Specifying the required argument types can be
done by the argtypes attribute - a sequence of
ctypes types:
strchr = libc.strchr strchr.argtypes = [ctypes.c_char_p, ctypes.c_char]
By default, functions are assumed to return the
C int type.
Other return types can be specified by setting
the restype attribute:
strchr.restype = ctypes.c_char_p
strchr("abcdef", "d") # Would now return def, and not
-
7/31/2019 Mixed Language Programming
14/43
Passing Parameters By Reference
The byref() function is used to pass
parameters by reference:
f = ctypes.c_float() # f = 0.0
s = ctypes.create_string_buffer(10)
# create a 10 byte empty buffer.
libc.sscanf("Input", "%f %s", ctypes.byref(f), s) pointer() does a lot more work, so it is
faster to use byref() if you don't need the
pointer object in Python itself.
-
7/31/2019 Mixed Language Programming
15/43
Structures and Unions Structures and unions must derive from the
Structure and Union base classes.
They must contain a _fields_ attribute: a list of2-
tuples, containing a field nameand a field type. The field type must be a ctypes type.
>>> class POINT(ctypes.Structure):
... _fields_ = [("x", ctypes.c_int), ("y",ctypes.c_int)]
>>> point = POINT(10, 20)
>>> print point.x, point.y
-
7/31/2019 Mixed Language Programming
16/43
Bit Fields in Structures and Unions
Bit fields are only possible for integer fields.
The bit width is specified as the third item in
the _fields_ tuples.
In C:struct {int word1 :16;
int word2 : 16;} DoubleWord;
In Python:>>> class DoubleWord(ctypes.Structure):
... _fields_ = [(word1", ctypes.c_int, 16),
... (word2", ctypes.c_int, 16)]
-
7/31/2019 Mixed Language Programming
17/43
Arrays and Sizes
The way to create array types is by multiplying a
data type with the array length.
>>> TenIntegers = ctypes.c_int * 10>>> ii = TenIntegers(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
# You can also write: ii = TenIntegers(), and
get an array of zeroes.
>>> for i in ii:
print i
-
7/31/2019 Mixed Language Programming
18/43
Pointers Pointer instances are created by calling the
pointer() function on a ctypes type: i = ctypes.c_int(42)
pi = ctypes.pointer(i)
pi have a contents attribute which returns theobject to which the pointer points (the i object,above): i2 = ctypes.c_int(99)
pi.contents = i2 It is also possible to use indexes to change the
pointed value, but you must know what you'rechanging, just as in C: pi[0] = 22 # i2.value is now 22.
-
7/31/2019 Mixed Language Programming
19/43
Pointer Types and NULL Pointers
A pointer type is being created via the
POINTER() factory function:
PI = ctypes.POINTER(ctypes.c_int)
pi = PI(ctypes.c_int(42))
Calling the pointer type without an
argument creates a NULL pointer:
null_ptr = PI()
As already mentioned, to set a POINTER
type field to NULL, you can assign None.
-
7/31/2019 Mixed Language Programming
20/43
Type Checking and Casting Usually, only instances of the specified type are
accepted.
cast() takes two parameters, a ctypes object thatcan be converted to a pointer of some kind, and
a ctypes pointer type. It returns an instance of the second argument,
which references the same memory block as thefirst argument:
a = (ctypes.c_byte * 4)() # Creates a zeroed 4-byte array.
pi = ctypes.cast(a, PI) # Converts thearray to an int ptr.
Array instances are auto-casted to the
compatible pointer types (ii to PI, for example).
-
7/31/2019 Mixed Language Programming
21/43
Forward Declarations We can write a linked list on C like this:
struct node; // forward declaration.
struct {
void *data;struct node *next;
} node;
In ctypes, we can define the cell class and set the
_fields_ attribute after the class statement:>>> class node(ctypes.Structure):
... pass
>>> node._fields_ = [(data", ctypes.c_void_p),
("next", ctypes.POINTER(node))]
-
7/31/2019 Mixed Language Programming
22/43
Callback Functions
You can pass Python functions as C callbacks.
The CFUNCTYPE factory function creates types
for callback functions using the cdecl callingconvention.
On Windows, the WINFUNCTYPE factoryfunction creates types for callback functions
using the stdcall calling convention. Both of these factory functions are called with
the result type as first argument, and thecallback functions expected argument types as
the remaining arguments.
-
7/31/2019 Mixed Language Programming
23/43
Callback Functions An example>>> from ctypes import c_int, POINTER>>> IntArray5 = c_int * 5
>>> ia = IntArray5(5, 1, 7, 33, 99)
>>> qsort = libc.qsort
>>> qsort.restype = None # A void return value.>>> CMPFUNC = CFUNCTYPE(c_int,
POINTER(c_int), POINTER(c_int))
>>> def py_cmp_func(a, b):
... return a.contents b.contents
>>> qsort(ia, len(ia), ctypes.sizeof(ctypes.c_int),CMPFUNC(py_cmp_func))
# ia is now sorted
-
7/31/2019 Mixed Language Programming
24/43
Callback Functions Warnings
Make sure you keep references toCFUNCTYPE objects as long as they areused from C code.
Otherwise, they may be garbage collected,crashing your program when a callback ismade.
The result type of callback functions is
limited to the predefined fundamentaltypes.
ctypes 1.0.1 (included in Python 2.5) behaves
strangely if you use other types.
-
7/31/2019 Mixed Language Programming
25/43
Function Pointers In Python In-order to call a function pointer from Python,
you need to wrap it with the appropriate callbacktype.
If a function pointer is defined as: typedef int (*MyFuncPtr)(void);
Then the equivalent ctypes type is:MyFuncPtr = ctypes.CFUNCTYPE(ctypes.c_int)
Use WINFUNCTYPE, instead of CFUNCTYPEfor the stdcall pointer: typedef int (__stdcall *MyStdFuncPtr)(void);
You can also use this type as a struct field, function
return type, etc.
-
7/31/2019 Mixed Language Programming
26/43
Auto-Generated Wrapper Code
You can try the ctypeslib code-generator
that parses the header file of the DLL and
generate the wrapper code from it(currently unstable).
-
7/31/2019 Mixed Language Programming
27/43
Alternatives For ctypes
SWIG
Boost.Python
Many moreUsing the Python\C API directly
Pyrex
Weave\PyInline
and many more:
SIP
Cython
PyCXX
Etc.
-
7/31/2019 Mixed Language Programming
28/43
Python-.NET
Integration UsingPython For .NET
-
7/31/2019 Mixed Language Programming
29/43
Why Extending Python with .NET
Code? Managed code -> no resource leaks.
An extensive runtime library (BCL) Windows Forms
ASP .NET ADO .NET
GUI developing is simpler on the Visual Studio .NETDesigner.
Cross-Platform: .NET Framework
Mono
DotGNU
Availability:
.NET 3.0 is installed on Windows Vista and Windows Server2008.
-
7/31/2019 Mixed Language Programming
30/43
-
7/31/2019 Mixed Language Programming
31/43
Assembly Loading
Before you can access a namespace in anassembly, you have to load it.
Loading an assembly: import clr
clr.AddReference("System.Windows.Forms") # No.dll prefix required
Assemblies are loaded from: The application directory.
The PYTHONPATH (= sys.path).
The GAC.
-> To ensure that you can load an assembly, putthe directory containing the assembly insys.path.
-
7/31/2019 Mixed Language Programming
32/43
Accessing CLR Namespaces
Once their assembly was loaded ,CLR
namespaces are treated as Python packages:
from System.Windows.Forms import Form
You must import the clr module BEFOREaccessing the CLR namespaces, even if their
containing assembly is already\automatically
loaded.
import clr
from System import String, Int32
from System.Collections import *
-
7/31/2019 Mixed Language Programming
33/43
Using Classes Python for .NET allows you to use any non-
private classes, structs, interfaces, enums ordelegates from Python.
To create an instance of a managed class, youcall one of its public constructors:
from System.Collections import Hashtable table = Hashtable()
table["key 1"] = "value 1
You can get and set fields and properties of CLRobjects just as if they were regular attributes.
You can subclass managed classes in Python,though members of the Python subclass are notvisible to .NET code: import System.Windows.Forms as WinForms
class HelloApp(WinForms.Form):
# Subclass Implementation
-
7/31/2019 Mixed Language Programming
34/43
Using Methods
All public and protected methods of CLRobjects are accessible to Python:
from System import Environmentenv_vars =
Environment.GetEnvironmentVariables()
for env_var in env_vars:
print env_var.Key, env_var.Value
Static methods may be called eitherthrough the class or through an instance of
the class (As above, for Environment).
-
7/31/2019 Mixed Language Programming
35/43
Type Conversions
Elemental Python types (string, int, long, etc.)
convert automatically to compatible managed
equivalents (String, Int32, etc.) and vice-versa. All strings returned from the CLR are returned as
Unicode.
Types that do not have a Pythonic equivalent
are exposed as instances of managed classesor structs (System.Decimal, for example).
-
7/31/2019 Mixed Language Programming
36/43
Using Generics When running under.NET runtime 2.0+, you can
use generic types.
A generic type must be bound to create aconcrete type before it can be instantiated:
from System.Collections.Generic import Dictionarymy_dict = Dictionary[String, Int32]()
my_dict[a] = 1
You can also pass a subset of Python types thatdirectly correspond to .NET types:my_dict = Dictionary[str, int]()
This also works when explicitly selecting genericmethods or specific versions of overloadedmethods and constructors.
-
7/31/2019 Mixed Language Programming
37/43
Overloaded and Generic Methods
There are cases where it is desirable to select a
particular method overload explicitly.
Methods of CLR objects have an
"__overloads__" attribute that can be used forthis purpose:
from System import Console
Console.WriteLine.__overloads__[bool](true) Generic methods may be bound at runtime
using this syntax directly on the method:
someobject.SomeGenericMethod[str]("10")
-
7/31/2019 Mixed Language Programming
38/43
Other Features If a managed object implements one or more indexers,
you can call the indexer using standard Python indexingsyntax.
You can raise and catch managed exceptions just thesame as you would pure-Python exceptions.
Managed arrays behave like standard Python sequence.
Multidimensional arrays support indexing one would usein C#: arr = System.Array.CreateInstance(String, 3, 2, 4) # Creating a 3
dimensions 3*2*4 array of strings. arr[1, 1, 1] = abc
Managed objects that implement the IEnumerableinterface (=Collections) can be iterated over using thestandard iteration Python idioms.
-
7/31/2019 Mixed Language Programming
39/43
Getting Help
The docstring of a CLR method (__doc__)
can be used to view the signature of the
method, including overloads:print Environment.GetFolderPath.__doc__
You can also use the Python help method
to inspect a managed class:help(Environment)
-
7/31/2019 Mixed Language Programming
40/43
(Explicit) Delegates And Events A delegate type can be instantiated and passed a
callable Python object to get a delegate instance:
>>> def my_handler(source, args):
print 'my_handler called!'
>>> from System import AssemblyLoadEventHandler,AppDomain
>>> deleg = AssemblyLoadEventHandler(my_handler)
>>> AppDomain.CurrentDomain.AssemblyLoad += deleg
# event+=deleg -> use deleg as an event handler.>>> from System.Drawing import Point #
The delegate would be called. You can also call itdirectly: deleg(None, None).
You can add callable objects to a delegate instanceusing: deleg+=method.
-
7/31/2019 Mixed Language Programming
41/43
(Implicit) Delegates And Events
You do not have to pass an explicitly instantiateddelegate instance to an event:
>>> class SimpleApp(WinForms.Form):
def __init__(self): self.button = WinForms.Button()
self.button.Click += self.button_Click# register the event handler (unregister it with -=)
self.Controls.Add(self.button) def button_Click(self, sender, args):
WinForms.MessageBox.Show(Thisbutton was clicked-on!)
>>> WinForms.Application.Run(SimpleApp())
-
7/31/2019 Mixed Language Programming
42/43
The Boxing Problem An example:
points = System.Array.CreateInstance(Point, 3)
points[0].X = 1 # Wont work as you expect!
In C#, the compiler knows that Point is a valuetype and change the value in place.
On Python, the setattr (.X = 1) changes the stateof the boxed value, not the original unboxedvalue It is still 0!
Handling the boxing problem:
point = points[0] point.X = 1
points[0] = point
The rule in Python is: "the result of any attribute
or item access is a boxed value.
-
7/31/2019 Mixed Language Programming
43/43
The Alternative: IronPython
An implementation of Python running on .NET. Same syntax as Python For .NET.
Pros:
Managed code -> no resource leaks.
Better .NET interoperability.
Has (unofficial) support from Microsoft.
Cons:
Cant use neither Python modules written in C (likectypes), nor P/Invoke (directly).
Not every Python module is implemented (os) or
behave the same (re)