README: ctypes code generation

This document describes the ctypes code generator. The generator converts declarations in C header files into executable Python code: enums, structs, unions, function declarations, com interfaces, and preprocessor definitions.

The code generator is brand new and still experimental, although hopefully better than in the 0.9.5 release.

Contents

Overview

Creating wrappers for C header file(s) is a two step process. First, the h2xml.py script runs GCC-XML over the include file(s), creating a XML file containing declarations and definitions. Second, the xml2py.py script parses the XML files and creates Python code containing all or a subset of these definitions.

Python code is generated for C enums, structure, unions, typedefs and function declarations. Also included are preprocessor definitions, as far as they can be converted to valid Python code.

It is possible to filter the output so that only a subset of the definitions are converted, so it is either possible to generate a complete Python module wrapping a shared library, or the generator can be used to create snippets of Python code which can be pasted into a manually written module.

Requirements

GCC-XML (from http://www.gccxml.org/) is required to parse C header files into an XML description. Unfortunately the latest official version 0.6.0 does not fulfill the ctypes code generator requirements.

For windows, you can download a prebuild GCC-XML windows installer from the ctypes download page (this is an unofficial release). Please note that even the installation of GCC-XML requires that MSVC 6, 7, or 7.1 is installed on your system, because it needs the system header files.

For other platforms you must get the development version from CVS and build GCC-XML from source.

Changes in the ctypes 0.9.6 release

  • h2xml.py doesn't try to find preprocessor definitions by default anymore. Can be changed with the new -c or --cpp-symbols flag.
  • Added -k option to h2xml.py to not delete temporary files.
  • Better error output if the compilation fails.
  • h2xml.py can load an optional local configuration file from the current directory
  • several bugs in the gccxml installer plus one bug in gccxml has been fixed, and the gccxml installer snapshot has been rebuilt.

Usage examples

Here are several examples that show how to call the h2xml.py and xml2py.py scripts, and the code they generate. The output has sometimes been cleaned up a bit, for clarity. Generally, the codegenerator creates a lot of comments pointing to the C header file. This is useful to quickly look up the C source code.

Parse the windows header files (this may take a while), and include preprocessor definitions in the xml file:

C:\>python h2xml.py windows.h -o windows.xml -q -c
C:\>

Generate code for the RECT structure:

C:\>xml2py.py windows.xml -s RECT
from ctypes import *
LONG = c_long
class tagRECT(Structure):
    pass
RECT = tagRECT
tagRECT._fields_ = [
    ('left', c_long),
    ('top', c_long),
    ('right', c_long),
    ('bottom', c_long),
]
assert sizeof(tagRECT) == 16, sizeof(tagRECT)
assert alignment(tagRECT) == 4, alignment(tagRECT)
C:\>

Generate the MB_xxx constants, which are flags used for the MessageBox function - since these are #define'd symbols this works only when the xml file includes preprocessor definitions:

c:\>python xml2py.py windows.xml -r MB_.*
# generated by 'xml2py.py'
# flags 'windows.xml -r MB_.*'
MB_USERICON = 128
MB_DEFBUTTON3 = 512
MB_USEGLYPHCHARS = 4
MB_ABORTRETRYIGNORE = 2
MB_ICONASTERISK = 64
MB_ICONINFORMATION = MB_ICONASTERISK
MB_ICONHAND = 16
MB_ICONERROR = MB_ICONHAND
MB_ICONEXCLAMATION = 48
MB_ICONWARNING = MB_ICONEXCLAMATION
MB_RIGHT = 524288
MB_SYSTEMMODAL = 4096
MB_ICONQUESTION = 32
MB_APPLMODAL = 0
MB_OK = 0
MB_TYPEMASK = 15
MB_MODEMASK = 12288
MB_TASKMODAL = 8192
MB_OKCANCEL = 1
MB_RETRYCANCEL = 5
MB_DEFAULT_DESKTOP_ONLY = 131072
MB_RTLREADING = 1048576
MB_PRECOMPOSED = 1
MB_DEFBUTTON1 = 0
MB_DEFMASK = 3840
MB_DEFBUTTON2 = 256
MB_YESNOCANCEL = 3
MB_CANCELTRYCONTINUE = 6
MB_HELP = 16384
MB_ICONMASK = 240
MB_SETFOREGROUND = 65536
MB_TOPMOST = 262144
MB_COMPOSITE = 2
MB_DEFBUTTON4 = 768
MB_YESNO = 4
MB_ERR_INVALID_CHARS = 8
MB_NOFOCUS = 32768
MB_ICONSTOP = MB_ICONHAND
MB_MISCMASK = 49152
C:\>

Generate code for the RegisterClass function, note how this pulls in a lot of types (if you want code compatible with Python 2.3 don't use the -d flag):

C:\>python xml2py.py windows.xml -w -s RegisterClass -d
# generated by 'xml2py'
# flags 'windows.xml -w -s RegisterClass'
from ctypes import *
from ctypes import decorators

WORD = c_ushort
ATOM = WORD
class tagWNDCLASSA(Structure):
    pass
WNDCLASSA = tagWNDCLASSA

@ decorators.stdcall(ATOM, 'user32', [POINTER(WNDCLASSA)])
def RegisterClassA(p1):
    return RegisterClassA._api_(p1)

RegisterClass = RegisterClassA
UINT = c_uint
LONG_PTR = c_long
LRESULT = LONG_PTR
WNDPROC = WINFUNCTYPE(LRESULT, c_void_p, c_uint, c_uint, c_long)
PVOID = c_void_p
HANDLE = PVOID
HINSTANCE = HANDLE
HICON = HANDLE
HCURSOR = HICON
HBRUSH = HANDLE
CHAR = c_char
LPCSTR = POINTER(CHAR)
tagWNDCLASSA._fields_ = [
    ('style', UINT),
    ('lpfnWndProc', WNDPROC),
    ('cbClsExtra', c_int),
    ('cbWndExtra', c_int),
    ('hInstance', HINSTANCE),
    ('hIcon', HICON),
    ('hCursor', HCURSOR),
    ('hbrBackground', HBRUSH),
    ('lpszMenuName', LPCSTR),
    ('lpszClassName', LPCSTR),
]
assert sizeof(tagWNDCLASSA) == 40, sizeof(tagWNDCLASSA)
assert alignment(tagWNDCLASSA) == 4, alignment(tagWNDCLASSA)

Generate code for the ICreateErrorInfo com interface. This example uses the -m command line flag, and the generated code imports several symbols from the specified ctypes.com module. Note that the _iid_ member of the interface cannot be created by the code generator, but the generated comments allows to quickly locate the header file and patch this manually:

C:\sf\ctypes\ctypes\wrap>xml2py windows.xml -s ICreateErrorInfo -m ctypes.com
# generated by 'xml2py'
# flags 'windows.xml -s ICreateErrorInfo -m ctypes.com'
from ctypes import *
from ctypes.com import IUnknown
from ctypes.com import GUID
class ICreateErrorInfo(IUnknown):
    _iid_ = GUID('{}') # please look up iid and fill in!
    # C:/Programme/gccxml/bin/Vc71/PlatformSDK/oaidl.h 5366
    pass
from ctypes.com import HRESULT
from ctypes.com import GUID
WCHAR = c_wchar
OLECHAR = WCHAR
LPOLESTR = POINTER(OLECHAR)
from ctypes.com import DWORD
from ctypes.com import STDMETHOD
ICreateErrorInfo._methods_ = IUnknown._methods + [
# C:/Programme/gccxml/bin/Vc71/PlatformSDK/oaidl.h 5366
    STDMETHOD(HRESULT, 'SetGUID', [POINTER(GUID)]),
    STDMETHOD(HRESULT, 'SetSource', [LPOLESTR]),
    STDMETHOD(HRESULT, 'SetDescription', [LPOLESTR]),
    STDMETHOD(HRESULT, 'SetHelpFile', [LPOLESTR]),
    STDMETHOD(HRESULT, 'SetHelpContext', [DWORD]),
]

Create a Python module win_lean.py containing wrappers for the 'lean' windows api, this creates a fairly large file:

C:\>python h2xml.py windows.h -D WIN32_LEAN_AND_MEAN -D NO_STRICT -o win_lean.xml -q
C:\>python xml2py.py win_lean.xml -w -o win_lean.py

Create a Python module SDL.py containing wrappers for the SDL library. To work around a problem GCC-XML has with a certain construct in the SDL header files on Windows, you must define the SDLCALL macro to an empty value:

C:\>python h2xml.py SDL.h -I SDL\include -D SDLCALL= -o SDL.xml -q
C:\>python xml2py.py SDL.xml -o SDL.py -l SDL.dll
C:\>

On linux, the SDL.py module could be created in this way - both the location of the include file and the name of the shared library is different:

thomas@linux:~/ctypes/wrap> locate SDL.h
/usr/include/SDL/SDL.h
thomas@linux:~/ctypes/wrap> python h2xml.py SDL/SDL.h -o SDL.xml -q
thomas@linux:~/ctypes/wrap> python xml2py.py SDL.xml -o SDL.py -l libSDL.so
thomas@linux:~/ctypes/wrap> ls -l SDL.py
-rw-r--r--    1 thomas   users       95772 2004-10-26 02:23 SDL.py
thomas@linux:~/ctypes/wrap> 

The h2xml.py script

h2xml.py lets you specify the names of the header files to parse, the name of the XML output file, and a few options which are passed to GCC-XML. The -D, -E, and -I flags may occur several times.

-h, --help

Print a short usage summary and exit.

-q, --quiet

Run in quiet mode. The default mode is verbose, h2xml.py prints what it is currently doing.

-D name[=value]

This flag defines a preprocessor name.

-U name

This flag undefines a preprocessor name.

-I directory

This flag defines an additional include directory.

-o xmlfile

This flag specifies name and path of the XML output file.

-c, --cpp-symbols

Run additional magic to include #define symbols in the xml output. Depending on the madness in the header files this may or may not work.

-k

Do not delete temporary files created - this may be useful to find compilation problems.

The h2xml.cfg configuration file

If you enable preprocessor definition processing with the -c flag, it is possible that you get compiler errors on certain symbols. In this case, it is needed to exclude those symbols by writing in local configuration file named h2xml.cfg in the current directory. The configuration file sections are named after sys.platform, and it allows to specify the symbols to exclude by names or by regular expressions:

# h2xml.cfg
#
# config file for h2xml script
#
# sections should be named after 'sys.platform'
# options supported are:
# 'excluded' - names of preprocessor symbols to exclude,
#         separated by whitespace
# 'excluded_re' - regular expressions, separated by whitespace, matching
#         cpp symbols to exclude
# See the documentation of the standard Python ConfigParser module
# for maore information on the file format.

[win32]
excluded = foo bar
excluded_re = spam.* .*spam

[cygwin]
excluded =
excluded_re =

[linux2]
excluded =
excluded_re =

[darwin]
excluded =
excluded_re =

[freebsd5]
excluded =
excluded_re =

The xml2py.py script

-h, --help

Print a short usage summary and exit.

-d

Use Python 2.4 decorators for wrapped functions.

-k[d][e][f][m][s][t]

Specifies the kind of types to include in the output:

d - simple preprocessor definitions: #define <identifier> <identifier>

e - enumerations

f - function declarations

m - preprocessor macros taking parameters: #define <ident>(parameters) something

s - structures and unions

t - typedefs

-l sharedlib

specify shared library to search for exported functions.

-m module

specifies a Python module containing symbols that will be imported instead of generated.

-o outputfile

name of the output file containing Python code. If not specified, the code is printed to standard output.

-r regular_expression

regular expression specifying names of symbols to include.

-s symbol

name of symbol to include.

-v

verbose mode: prints a short summary of types generated.

-w

windows only: add all standard windows dlls to the list of shared libraries searched for functions.

Note that specifying the -k, -s, and -r flags create a start set of type declarations, the generated code, however, may also contain other types. If, for example, the code for an external function is generated, code for the argument and return types is also needed.

Also note that code for function declarations is only created when xml2py.py can locate the function in one of the shared libraries specified with -l or -w.

Getting Help

If you have questions or need assistance with ctypes or the code generator, please post a message to the ctypes-users mailing list.