Thursday, July 5, 2012

Python package/library source code browsing.


Recently I moved to new project where the programming language was Python. I often get confused about what module do I need to import ? what is the signature of the method/function? etc. I also tried to consult the documentation provided by the third party library but many a times I didn't get the clear idea. 
However, Java provides javadoc which helps to figure out the type of package that needs to be included in a particular class. It also describes the function signature. This is possible because of good editors(IDE: Eclipse) with integrated javadoc. On the other hand Python editors lack these features. Most of the Python IDEs neither help to auto-complete nor provide any help for the method/function signature. This makes life more difficult and frustrating. Python third party library/module provides their documentation but sometimes much more is needed which can sufficed through access to source code. This helps to understand how third party library functions and gives a very first hand experience on how the code is organized, what modules are needed to be included in the project in order to access certain objects, methods, or classes, what is the method signature etc. If you have access to such information your life becomes easier while coding in Python. 
Bitten by source code browsing bug I started looking for tool which can help me to browse python library source code and I stumbled upon pydoc.net and Nullege.com.  First we will talk about  pydoc.net and then will talk about Nullege(Don't get confused with pydoc module present in python). Let's see what pydoc.net says about itself in its own words:

"PyDoc.net is an online open source package library for Python language. It aims to provide ease of navigation through their source code in a better and organized way. It is still on its experimental stages."


 I have been using this tool for quite sometime and liked it. It provides very limited functionality e.g indexing, searching and source code.  If you are too geeky and know the technology well then in that case it might be useful. But if you are a newbie then this is not what you are looking for. If pydoc.net had provided similar functionality  as grepcode has provided for java source code, then pydoc.net would be definitely the first choice for Python developers . If you are wondering about grepcode and it functionality, then you may be interested to check this post.  Since grepcode doesn't provide python module source code, I have to settle with pydoc. Given that pydoc.net is experimental project and handled by single person, I would give this site 3.5 out of 5.
Enough of talking, lets check its functionality. This site provides search service, where you can search by keyword, package/module name and will list the related packages and modules. Checkout the  picture below. It also lists the packages download count wise. There is one more list which shows the packages/modules with their respective tags as marked 2 in the picture below. 
Pydoc: search result for package or modules
module search result
If you click the link "index package" which is marked 4  in the above picture, it lists all the packages in alphanumeric wise which you can see in the shown below. This gives you idea about the similar packages and the gamut of package who share common prefix/name.

pydoc: alphanumerically indexed python packages
Module/package index
Ok found the package name so what? Well this was the first step on the source code browsing. On selecting the package it shows the meta-data of the package/module as shown in the picture below. This also shows how to install the package, example-code and the documentation link. In some cases it also shows the link from where the source code can be downloaded.
pydoc: Python module/package details, example, how to install etc.
Package/module details
As you may have noticed there is a directory hierarchy which shows how modules are organized and what are the source files present under any particular directory. If you click the file name you will see the source code which will look similar to picture as shown below.
pydoc: source code of module

Nullege also has the similar interface as that of pydoc. You can see the user interface in picture below. I noticed few differences between these two websites. Nullege doesn't give any meta information about the package and doesn't show the documentation provided by the python third party library/package. But Nullege has the feature to search the keywords in the source code while pydoc doesn't have.

Nullege: user interface, with package structure and source code.
User may not like one thing about Nullege i.e the advertisements are shown below the source code hierarchy which restrict the complete view of directory hierarchy. If the advertisements were shown somewhere else, it would have given more view area to the directory/file hierarchy and hence would have provided good user experience. If asked, I would give 3 out of 5 to this website.

How to download source code:

If you are interested in downloading the source code, even that is possible with the help of "pip". Pip is a python tool which installs the python packages and it's replacement of easy_install. If you system doesn't have the pip installed then run the following command.
 
$python easy_install pip

For downloading source code of particular package run the following command.
 
 $pip install -src= . <package-name>   #This will download the src in the current directory.

Happy source code browsing!!!

Related topics:
Android/java source code browsing

Your valuable comments are always welcomed. It will help to improve my post and understanding.

"By three methods we may learn wisdom: First, by reflection, which is noblest; Second, by imitation, which is easiest; and third by experience, which is the bitterest."
By : Confucius