python development with eclipse and ant
TRANSCRIPT
4/8/12 Python development with Eclipse and Ant
1/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
Sign in (or register)English
Technical topics Evaluation software Community Events
Python development with Eclipse and AntPython development using the Eclipse IDE and Apache Ant build tool
Ron Smith ([email protected]), Principal, RPS Technologies, Inc
Summary: Python is a flexible and powerful dynamic scripting language with full object-oriented features. Its many proponents say that the Python language
allows them to express their intentions more directly and efficiently than other languages. But newcomers to Python from the Java technology™ or Microsoft®
.NET world may find themselves missing their feature-rich and polished IDEs and development tools. Those developers can look to familiar Java tools for a
solution. This article highlights the use of the popular Java technology-based Eclipse and Apache Ant development tools for Python development.
Date: 15 Jun 2004
Level: Intermediate
Activity: 128919 views
Comments: 2 (View | Add comment - Sign in)
Average rating (107 votes)
Rate this article
Introduction
There has been a good deal of cross pollination between the Java language and Python camps over the years. Probably the most prominent example of this is
Jython, the pure Java implementation of the Python runtime. In that tradition, you will explore the use of the Eclipse IDE and the Ant build and deployment tool for
use in Python development. Eclipse and Ant are extremely popular, feature-rich, extensible, and open source; qualities shared with Python. What makes it possible
to use these Java tools for Python development are the PyDev and PyAntTasks extensions to Eclipse and Ant, respectively. This article will start with pointers ondownloading and installing the required tools and extensions. I'll use a working example of Python code which reads RSS feeds in order to demonstrate the use of
Eclipse and Ant for Python development.
This article won't get into the details of Eclipse, Ant, or Python. See the links in the Resources section for in-depth coverage of these.
Flavors of Python supported
The software covered in this article has been tested against CPython 2.3. With some exceptions, it should also work with Jython. Specifically, the PyDev debugger
does not currently support Jython. Another difference is that scripts executed via Jython go into interactive mode after being run from within PyDev, forcing you to
have to kill them manually. The PyDev editor works for Jython source code, and the Python Ant tasks work with Jython with the exception of the py-doc task.
Using Eclipse for Python development
Overview of Eclipse
Eclipse is a Java technology integrated development environment that was developed and open sourced by IBM. It is the basis for IBM's commercial WebSphere
Application Development environment, and various other tools. Eclipse has an extremely active community of developers who develop Eclipse itself and the large
number of plug-ins available for it. See the Resources section for links to the Eclipse and Eclipse plug-ins Web sites. Although traditionally a tool for Java
development, several plug-ins exist for developing with other languages within Eclipse, including C/C++, Python, and Perl.
Within Eclipse, source code is organized into projects. Projects may be loaded, unloaded, and imported. The Eclipse user interface is structured into views and
editors. Examples of views and editors include the source code outline view, Java source editor, Python source editor, and filesystem navigator view. One of the
key metaphors within the Eclipse user interface is the perspective. Perspectives are an organization of views that would typically be used together when carrying
out some type of activity. The perspectives that are standard with Eclipse are: Debug, Java Browsing, Java, Java Type Hierarchy, Plug-in Development, CVSRepository Exploring, Resource, and the Install/Update perspective. A separate Python perspective does not currently exist. When doing Python development, I
typically use the Resource perspective and Debug perspective.
Installing PyDev
To get started, download and install Eclipse from the Eclipse Web site (see link in Resources section), following the installation instructions appropriate for your
platform.
The Eclipse update facility makes it easy to install the PyDev plug-in. From within Eclipse, select Help > Software Updates > Update Manager to start the
Install/Update perspective. In the Feature Updates view at the bottom left, add the PyDev plug-in update site as a new Site Bookmark under the folder "Sites to
4/8/12 Python development with Eclipse and Ant
2/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
Visit". The Eclipse update site URL for PyDev is http://pydev.sf.net/updates/. "PyDev" should now be displayed as a feature in the Feature Updates editor. In the
Feature Updates editor, expand PyDev > Other, and select the PyDev feature which is displayed (should be at least 0.4.1). Then select "Install Now" to install the
feature. Eclipse will download the PyDev plug-in and install it into Eclipse.
Importing the sample project
To access the sample code used within this project, download the zip file (see Resources section), expand the zip file into your filesystem, and then import the
contained project into Eclipse. To import the project, switch into the Resource perspective, select File > Import, select "Existing Project into Workspace", and
select the location where you expanded the zip file. The feedParserTest project should now appear in the Navigator view.
For the sample project, I've included the Feed Parser universal feed parser library, which is distributed under the Python open source license. See the Resourcessection for a link to the Feed Parser project Web site.
A tour of PyDev
You'll now learn how to use the imported project to explore PyDev's features. PyDev is a work in progress, but is already a very effective development
environment for Python development. The current list of PyDev features includes:
Python editor including Python syntax highlighting
Python syntax analysis, highlighting errors in the Python editor and in the Tasks View
Option to convert Tabs into spacesOutline view showing imports, classes, and functions
Python stack traces in the console view are hyperlinked to the source code
Hyperlinking within the source code;Imports and function calls in the same module can be navigated via hyperlinks
Ability to run a Python script from the Navigator view
Debugger supporting breakpoints, stepping through code, and displaying variables
PyDev preferences window
The PyDev preferences can be reached via Window > Preferences and selecting Pydev (see Figure 1). The first set of preferences lets you change settings on
how PyDev treats tabs within source code, and to change syntax coloring.
Figure 1. PyDev preferences window
Setting the Python interpreter
The PyDev Debug preferences let you choose the Python interpreter that PyDev will use to execute your Python code. If PyDev is not able to find your Python
interpreter, or if you want to use a different one, configure it here (see Figure 2).
Figure 2. PyDev Debug Preferences
4/8/12 Python development with Eclipse and Ant
3/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
Working with source code
I do most of my Python work within the Resource perspective. Switch into the Resource perspective, and in the Navigator view at the top left, double click on the
file feedParserTest/src/feedparserTest/FeedparserTest.py. The Python editor opens the file, parsing the Python syntax for syntax coloring and syntax checking (see
Figure 3).
Figure 3. Python editor
If any errors are found in the source, they are displayed in the Tasks view at the bottom right. Double clicking on an error in the Tasks view will take you to the
offending line of code.
The Outline view at the bottom left displays a browsable structure of the file currently being edited. Imports, classes, and functions are displayed, and may be
navigated to by clicking on the item in the Outline view. PyDev re-parses the Python file as it is being edited, and updates the Outline view, performs syntax
checking, and syntax coloring.
Editor features
The PyDev 0.4 release added a function and import hyperlink feature to the Python source editor. If you hold down the control key while hovering over an import
or a function call that is on the PYTHONPATH, PyDev will display a hyperlink, allowing you to navigate to the source of the import or function. Note that in order for
this to work for your source code across modules (linking from one module to another), you'll have to change your PYTHONPATH environment variable to include
those modules so that PyDev can find them.
Some nice-to-have source editing features have started being added to PyDev in the latest release, including block comment/uncomment, and shift code right/left
(see Figure 4).
Figure 4. PyDev editor additional features
4/8/12 Python development with Eclipse and Ant
4/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
Running a Python script
An IDE isn't very useful if you're not able to execute code. To execute Python code, select the feedparser.py file within the Navigator view, right click, and select
Python > Run. The Python launch configuration window will be displayed (see Figure 5).
Figure 5. Python launch configuration
The Python launch configuration window gives you the ability to specify the current working directory from which the script will be executed, arguments to pass to
the script, and what Python interpreter should be used to run the script. feedparser.py takes an RSS URL as an argument, so fill one in the arguments field, such as
http://www-106.ibm.com/developerworks/news/dw_dwtp.rss. The rest of the default options are fine, so click Run.
The script is executed and the output is displayed in the Console window. If any errors occurred, a stack trace is displayed in the Console window and the lines
within the stack trace are hyperlinks to the Python source code.
Python debugger
A Python debugger was recently added to the PyDev plug-in. To use it, set a breakpoint by clicking to the left of the line of code you'd like to break on within the
Python editor. In Figure 6, I set a breakpoint at line 1830 of feedparser.py. Then select the Python module within the navigator view, right click and select "Python
> Debug..." A launch configuration similar to before will be displayed. Click Debug to enter the Debug perspective and to launch the debugger.
Figure 6. Python debugger
4/8/12 Python development with Eclipse and Ant
5/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
The Debug view in the top left shows the processes and threads that are currently executing, the Variables view in the top right displays variables that are currently
in scope, the Python editor shows at which line of code the debugger is currently stopped, and any output is displayed in the Console view. The debugger can be
controlled via the buttons at the top of the Debug view or via the Run menu.
Other Eclipse features
Eclipse and its plug-ins have many other features which are applicable to Python development, including XML editors, UML editors (although most are Java code-
centric), and source control plug-ins. Currently, almost 500 plug-ins are listed on the Eclipse plug-ins site (see link in the Resources section). Without going into
detail, I'll highlight one plug-in in particular that would be useful to many Python developers: the CVS plug-in which is included as a part of the Eclipse distribution.
Eclipse includes a very feature-rich integration with CVS:
Supports the SSH, pserver, and ext connection methods
Basic CVS commands support: checking out projects, committing changes, updating, adding files/patterns to .cvsignore
File merge view
Compare differences in source control
Project synchronization with a repository showing outgoing and incoming changes
Support for other source control systems such as ClearCase, Subversion, and Visual SourceSafe is provided by other plug-ins.
Using the Python interactive shell within Eclipse
The Python interpreter supports interactive execution of Python code. This can be very handy for trying out pieces of code without putting them into a Python script
and executing the script. And the Python interpreter interactive mode can be very easily integrated into Eclipse.
To add support for interactive execution of Python, add an External Tool launcher via Run > External Tools > External Tools. This will bring up the External
Tools launcher configuration window. Create a new configuration by selecting "Program" in the Configurations list, and click "New". Name the configuration
something like "pythonInteractive", set the Location to point to your Python interpreter, and pass "-i" as the single argument (see Figure 7).
Under the Common tab, check the box to display this configuration in the External Tools favorites menu.
Figure 7. Python interactive mode configuration
4/8/12 Python development with Eclipse and Ant
6/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
To run the launcher you just created from within Eclipse, select Run > External Tools > pythonInterpreter. The Python interpreter output is displayed in the
Console view. Python commands may be typed into the Console and executed just as they are when executing Python from the command line. In order to import
and use your modules within Python in interactive mode, you'll need to add the location of your modules to the PYTHONPATH environment variable.
The one difference in executing Python within the Eclipse Console from when executing on the command line is that the command history feature (accessible via the
up arrow and down arrow) isn't enabled because the Eclipse Console interprets these keys itself.
Using Ant for Python development
Python automatically compiles modules when it needs to. This means that Python developers don't typically have to explicitly compile modules. Nonetheless, there
are cases when it’s useful to manually compile Python code, and there are many other aspects of the build and deployment process that could be automated. This iswhat build tools are for.
I'll highlight a build tool from the Java programming world, Apache Ant, which is largely applicable to Python development. Apache Ant is the de-facto standardbuild tool within the Java programming world. It was created as a more portable and Java technology-friendly alternative to other build tools. Ant will run on any
platform that supports Java programming language. Although most of what was required was already provided by Ant, there were some key Python-specificfeatures that were needed if Ant was going to serve as a Python build tool. I've developed some custom Ant plug-ins (tasks in Ant lingo) which provide Python-
specific features needed to do Python builds.
Ant uses XML as its format for describing builds. A build file is organized into targets to be carried out. Each target may depend on other targets. Ant will execute
whatever targets necessary based on the target you request be executed, and the set of depended-upon targets. Each target may contain any number of Ant taskswhich actually carry out the work of the target. Ant has many built-in tasks for things such as compiling Java code, generating documentation, manipulating files and
directories, and there are many add-on tasks provided by third parties.
Installing the Python Ant library
I'll introduce the basics of an Ant build script and the Python Ant tasks by working through creating a build script for the feedparser test project. In order to use thePython Ant tasks, you'll need to download and install the Java library that contains these tasks. First, download the Python Ant tasks library (pyAntTasks.jar) from
the URL listed in the Resources section. Then, copy the JAR file into the lib directory of the Ant plug-in within Eclipse. Under your Eclipse install directory, itshould be in a directory like plugins/org.apache.ant_1.5.3.
Once you've copied in the Python Ant tasks library, it must be enabled within Eclipse. Select Window > Preferences, and thenAnt > Runtime. Expand out theAnt Home Entries to see the list of libraries (JAR files) that Eclipse uses. Add the Python Ant JAR file you just copied by selecting "Add JARs" and selecting thePython Ant JAR file within the Eclipse Ant plug-ins lib directory (see Figure 8).
Figure 8. Adding Python Ant tasks to classpath
4/8/12 Python development with Eclipse and Ant
7/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
You should now be able to create and run Ant build scripts that contain the Python tasks. So onto creating the build script!
Creating the build script
I'll step through the creation of a simple Python build script (see Listing 1). The complete build script, build.xml, can be found in the top directory of thefeedParserTest project.
Listing 1. Build script fragment for compiling Python source code
<project name="feedParserTest" default="compile">
<taskdef resource="pyAntTasks.properties"/>
<property name="src.dir" value="src"/>
<target name="compile"> <py-compile dir="${src.dir}" pythonpath="${src.dir}" optimize="0"/> </target>
</project>
Let's start with a build script that just compiles the example Python code. The <project> tag is always the root tag of the build script. The <taskdef> tag declaresthe Python tasks you'll be using throughout the build script. Near the bottom of the build script, you will define the compile target. Inside of the target element are
the tasks that are carried out when compile is run. In particular, the py-compile task is used to compile all Python code starting in the src directory. The task willrecursively go into any subdirectories and compile any Python modules. Rather than hard-coding the path to the src directory everywhere it will be needed, you've
defined it as a property called src.dir within the build script. Then, wherever it is needed, it can be referred to via ${src.dir}.
To run the build script, open the build script within Eclipse. Eclipse has built-in support for editing and navigating Ant build scripts. The Outline view should display
the structure of the build script. In the Navigator view, select the build script, right click and select "Run Ant...". Select the compile target, and click on "Run". Theoutput of the build script execution should be displayed on the Console view, showing a successful run.
Python script execution task
Next, you'll add a target to the build script to execute a Python script (see Listing 2). In this case, you'll execute the feedparser.py script, which takes an RSS URL
as an argument.
4/8/12 Python development with Eclipse and Ant
8/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
Listing 2. Running the feedparser script build script fragment
<target name="run.feedparser" depends="compile"> <py-run script="src/feedparser/feedparser.py" pythonpath="${src.dir}" optimize="0"> <arg value="http://www-106.ibm.com/developerworks/news/dw_dwtp.rss"> </py-run> </target>
The above target executes the feedparser.py script, passing the RSS URL as the single argument. The target is declared to depend on the compile target, so it willbe executed first. This actually isn't really necessary because Python will automatically compile the source code if needed. If you execute the run.feedparser
target, the feedparser.py script should run, outputting the RSS content to the Console.
Python documentation task
Python has an API documentation facility similar to Java technology’s JavaDoc system, called PyDoc. The following snippet of XML in Listing 3 added to yourbuild script will generate PyDocs for all of your Python modules.
Listing 3. PyDoc generation build script fragment
1: <property name="pydoc.dir" value="pydoc"/> 2: 3: <target name="init"> 4: <mkdir dir="${pydoc.dir}"/> 5: </target> 6: 7: <target name="pydoc" depends="init,compile"> 8: <py-doc pythonpath="${src.dir}" destdir="${pydoc.dir}"> 9: <fileset dir="${src.dir}">10: <include name="**/*"/>11: </fileset>12: </py-doc>13: </target>
Deconstructing the pydoc target above, line 7 declares the target name, and indicates it depends on the init and compile targets. This means that prior torunning the pydoc target, Ant will make sure that the init and compile targets have already been run, and will run them if not.
The init target, which the pydoc target depends on, is defined in lines 3-5. The init target simply creates a directory to hold PyDoc API documentation files.As before, you've defined a property, named pydoc.dir, for the location of the generated documentation.
Line 8 is the start of the py-doc task. As before, you pass the PYTHONPATH you want to use in generating the pydocs. The destdir attribute tells the py-doc taskwhere to output the generated HTML documentation.
Lines 9-11 specify what Python source files should be processed when generating documentation. The fileset is a common construct in Ant scripts, used to specifya collection of files to operate on. This is a powerful feature allowing you to use naming patterns, boolean logic, and file attributes to select what files to operate on.
This is fully documented in the Ant documentation. In this case, you're recursively selecting all files starting with the 'src' directory.
Python unit test task
Python comes standard with a unit testing framework (as of Python 2.3. In Python 2.2, it was an optional module), much like the Java jUnit framework. Test casesare structured in the same manner as jUnit. Each class/module to be tested would typically have its own test class. The test class contains test fixtures, which areinitialized in a setUp function. Each test is written as a separate test function within the test class. The unittest framework will cycle through the test functions, calling
setUp, the test function, then tearDown for each test function. See Listing 4 for an example.
Listing 4. Python unit test module
import unittestfrom pprint import pprint
import feedparser
class FeedparserTest(unittest.TestCase): """ A test class for the feedparser module. """ def setUp(self): """
4/8/12 Python development with Eclipse and Ant
9/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
set up data used in the tests. setUp is called before each test function execution. """ self.developerWorksUrl = "testData/developerworks.rss"
def testParse09Rss(self): """ Test a successful run of the parse function for a 0.91 RSS feed. """ print "FeedparserTest.testParse09RSS()" result = feedparser.parse(self.developerWorksUrl) pprint(result)
self.assertEqual(0, result['bozo']) self.assert_(result is not None) channel = result['channel'] self.assert_(channel is not None) chanDesc = channel['description'] self.assertEqual(u'The latest content from IBM developerWorks', chanDesc) items = result['items'] self.assert_(items is not None) self.assert_(len(items)> 3) firstItem = items[0] title = firstItem['title'] self.assertEqual(u'Build installation packages with solution installation and deployment technologies', title) def tearDown(self): """ tear down any data used in tests tearDown is called after each test function execution. """ pass if __name__ == '__main__': unittest.main()
The above listing is a test class which performs basic tests on the feedparser module. The complete test class can be found in the feedParserTest project under
src/feedparserTest/FeedparserTest.py. The setUp function prepares the test fixtures used throughout the tests, in this case just the path to the test RSS file you'llparse in the test function. testParse09Rss is the actual test function. This function calls the feedparser.parse function with the test RSS file, outputs the parsed
results, and performs some basic checks via the TestCase class assert functions. If any assertion does not evaluate to true, or if any exceptions are thrown as a partof processing, unittest will report it as a test failure or error. Finally, the two lines at the bottom allow you to run the tests within this test class by simply running this
module.
To run this test class by itself, run the FeedparserTest.py module in the same manner as before. Select the FeedparserTest.py module in the Eclipse Navigator
view, and run via Python > Run. The launch configuration window will be displayed. The default values are fine except for the Base directory. The Base directoryhas to be the feedParserTest project directory, so that the RSS file (testData/developerworks.rss) can be found from the current directory. Change the base
directory setting and click 'Run'. The output will be displayed on the Console.
You'd like any unit tests we write to be executed automatically as a part of the build. The following build fragment in Listing 5, added to your build script, will dojust that.
Listing 5. Unit test execution build script fragment
1: <target name="tests" depends="compile">2: <py-test pythonpath="${src.dir}" dir=".">3: <fileset dir="${src.dir}">4: <include name="**/*Test.py"/>5: </fileset>6: </py-test>7: </target>
The first line is the target declaration, the same as the others. Lines 2 through 6 invoke the py-test task. These lines will run any tests found anywhere under the 'src'
directory, looking for any files that end in 'Test.py'. The PYTHONPATH will be set to 'src', and the tests will be executed with the current working directory set to thecurrent directory ('.').
4/8/12 Python development with Eclipse and Ant
10/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
To run the target, run the build script, and select the 'tests' target to execute. The target will run all test cases that end with 'Test.py', in this case just
FeedparserTest.py.
Summary
The combination of Eclipse and the PyDev plug-in, and Apache Ant along with the Python Ant tasks provide a complete integrated development environment and
build/deployment tool for Python development. These tools are undergoing development, so check for updates periodically, and if you feel an itch for a featureyou'd like to see, consider rolling up your sleeves and pitching in.
Download
Name Size Download methodos-ecant/feedparsertest.zip HTTP
Information about download methods
Resources
Learn
Visit Eclipse.org for Eclipse downloads and documentation.
The PyDev SourceForge project site provides downloads, documentation, and user forums.
The Python Ant tasks Web site contains the latest version of the Python Ant tasks.
Feed Parser, the universal feed parser implemented in Python, can be found at the Feed Parser Web site.
Python in a Nutshell (O'Reilly & Associates; 2003), by Alex Martelli, is an excellent reference on the Python language.
Find more articles for Eclipse users.
Visit the developerWorks Open source zone for extensive how-to information, tools, and project updates to help you develop with open source
technologies and use them with IBM's products.
Get products and technologies
The Eclipse Plug-ins Web site contains hundreds of commercial and open source plug-ins that you can install into Eclipse.
The Apache Ant Web site has Ant downloads and documentation on Ant.
For downloads and information on Python, see the Python Web site.
See the latest Eclipse technology downloads on alphaWorks.
Innovate your next open source development project with IBM trial software, available for download or on DVD.
Discuss
Get involved in the developerWorks community by participating in developerWorks blogs.
About the author
Ron Smith is founder of RPS Technologies, Inc, a software development and software consulting company based in the Chicago area. Ron Smith consults for
clients developing J2EE-based enterprise applications and develops software products within RPS Technologies. You can contact Ron [email protected].
Close [x]
4/8/12 Python development with Eclipse and Ant
11/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
developerWorks: Sign in
IBM ID:
Need an IBM ID?Forgot your IBM ID?
Password:
Forgot your password?Change your password
Keep me signed in.
By clicking Submit, you agree to the developerWorks terms of use.
Submit Cancel
The first time you sign into developerWorks, a profile is created for you. Select information in your developerWorks profile is displayed to the public, but
you may edit the information at any time. Your first name, last name (unless you choose to hide them), and display name will accompany the content that youpost.
All information submitted is secure.
Close [x]
Choose your display nameThe first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you
post on developerWorks.
Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email
address for privacy reasons.
Display name: (Must be between 3 – 31 characters.)
By clicking Submit, you agree to the developerWorks terms of use.
Submit Cancel
All information submitted is secure.
Average rating (107 votes)
1 star 1 star
2 stars 2 stars
3 stars 3 stars
4 stars 4 stars
5 stars 5 stars
Submit
Add comment:
Sign in or register to leave a comment.
Note: HTML elements are not supported within comments.
Notify me when a comment is added1000 characters left
4/8/12 Python development with Eclipse and Ant
12/12www.ibm.com/developerworks/library/os-ecant/?ca=drs-tp2604
Print this page Share this page Follow developerWorks
About
Help
Contact us
Submit content
Feeds and apps
Newsletters
Report abuse
Terms of use
IBM privacy
IBM accessibility
Faculty
Students
Business Partners
Post
Total comments (2)
very usefull
Posted by frankly_wang on 09 September 2011
Report abuse
Great tutorial - it never ceases to amaze me how versatile Eclipse really is.
One small amendment to this tutorial though - PyDev has moved to http://pydev.org/ and the update site to http://pydev.org/updates/. I have just tested the update
site link and it works nicely under Helios (Service Release 1).
Posted by aaronnewton on 13 November 2010
Report abuse