Statically checking python

With python, type checking is usually done at runtime.  Python will tell you of type problems when you run the code.  At runtime Python will tell you if there is a problem with the program 1 + 'a'. Since the types int and string can not be added together with the + operator.  This is why Python is often considered only a dynamically typed language, not a statically typed language.

However there are now tools to type check your python programs before you run them. Unfortunately it is not a widely known that you can type check python programs before you run them!

But what exactly are the benefits of static type checking anyway?  If you have 100% unit test coverage of your code, shouldn't that catch all the same bugs? Yes, it probably would. However, not all teams unit test all of their code.  Also, sometimes it might be dangerous to run the code to check for errors, or maybe the code takes a long time to run before it encounters errors.

You can check that:
  • interface methods are implemented
  • compatible operators are used on variables, eg "asdf" + 2
  • the correct number of arguments are passed into function calls
  • there are no unused variables, or modules
  • that variables aren't assigned to and never used
 Type inference also allows you to:
  • lookup the type(s) of a variable
  • find usages of functions
  • refactor, and type check the changes
There's hundreds more things you can check for with tools like pylint and some of the IDEs.  There are also field list markers in doc strings, where you can specify argument and return types. These can be used by tools like IDEs (eg pycharm) and other static checkers. Full program type inference is now doable for large parts of python. See the shedskin, and RPython restricted subsets for two implementations.

Now I will go over some problems you can check before runtime. I will show which tools work, and which tools do not work for these problems.

Want to download the code? You can follow along at this github link: https://github.com/illume/static_checking_python

The main tools I cover are:

Implements an interface

An interface is a way to specify what public interface implementations should present.

Can we check if an implementation of an interface in python has all of the required methods?  Below we have a Birds interface. We also have the Duck and Pidgeon implementations of the Birds interface.

implementsinterface1.py


import abc

class Birds(object):
    """docstring for Birds"""

    __metaclass__ = abc.ABCMeta

    def __init__(self, arg):
        self.arg = arg

    @abc.abstractmethod
    def noise(self):
        """docstring for noise"""
        pass

    @abc.abstractmethod
    def move(self):
        """docstring for move"""
        pass

class Duck(Birds):
    """docstring for Duck"""
    __implements__ = (Birds, )
    def __init__(self, arg):
        super(Duck, self).__init__(arg)

    def noises(self):
        """docstring for noise"""
        print self.arg

    def moves(self):
        """docstring for move"""
        print self.arg

class Pidgeon(Birds):
    """docstring for Pidgeon"""
    __implements__ = (Birds, )
    def __init__(self, arg):
        super(Pidgeon, self).__init__(arg)

    def noises(self):
        """docstring for noise"""
        print self.arg


    def moves(self):
        """docstring for move"""
        print self.arg 
 
 
 
 
(anenv)shit-robot:staticchecking rene$ pylint implementsinterface1.py 
No config file found, using default configuration
************* Module implementsinterface1
C:  1, 0: Missing module docstring (missing-docstring)
W: 23, 0: Method 'move' is abstract in class 'Birds' but is not overridden (abstract-method)
W: 23, 0: Method 'noise' is abstract in class 'Birds' but is not overridden (abstract-method)
W: 38, 0: Method 'move' is abstract in class 'Birds' but is not overridden (abstract-method)
W: 38, 0: Method 'noise' is abstract in class 'Birds' but is not overridden (abstract-method)

Here we see that pylint detects that we have not implemented the noise and move methods... because we made a typo.

implementsinterface2.py

In this file I correct the typo, and rerun pylint.

Here pylint can find that we are correctly implementing the required methods.

"""docstring for the module
"""
import abc

class Birds(object):
    """docstring for Birds"""

    __metaclass__ = abc.ABCMeta

    def __init__(self, arg):
        self.arg = arg

    @abc.abstractmethod
    def noise(self):
        """docstring for noise"""
        raise NotImplementedError

    @abc.abstractmethod
    def move(self):
        """docstring for move"""
        raise NotImplementedError

class Duck(Birds):
    """docstring for Duck"""
    __implements__ = (Birds, )
    def __init__(self, arg):
        super(Duck, self).__init__(arg)

    def noise(self):
        """docstring for noise"""
        # This will give a TypeError: cannot concatenate 'str' and 'int' objects
        # Pylint does not find this.
        print "a duck quacks this many times:" + 2

    def move(self):
        """docstring for move"""
        print self.arg

class Pidgeon(Birds):
    """docstring for Pidgeon"""
    __implements__ = (Birds, )
    def __init__(self, arg):
        super(Pidgeon, self).__init__(arg)

    def noise(self):
        """docstring for noise"""
        print self.arg


    def move(self):
        """docstring for move"""
        print self.arg
 

Detecting TypeErrors

Can you check for type errors? Let us test it out.

typeerror1.py


def doduck():
    # This will give a TypeError: cannot concatenate 'str' and 'int' objects
    # Pylint does not find this, However PyCharm does find it.
    return "The duck quacked this many times:" + 2

doduck()

First we try something simple... adding a string to a number.

This will give a TypeError: cannot concatenate 'str' and 'int' objects
Pylint does not find this error.

However, PyCharm shows the error. It says "Expected type 'str | unicode', got 'int' instead."
Note that although pylint did not find this error, it found these problems with the code:

(anenv)shit-robot:staticchecking rene$ pylint typeerror2.py 
No config file found, using default configuration
************* Module typeerror2
C:  7, 0: Final newline missing (missing-final-newline)
C:  1, 0: Missing module docstring (missing-docstring)
C:  1, 0: Invalid constant name "a" (invalid-name)
C:  3, 0: Missing function docstring (missing-docstring)
 

typeerror2.py

a = 2

def doduck():
    return "The duck quacked this many times:" + a

doduck()
 
We make it a bit more complicated for PyCharm, by putting the number into a variable.

Luckily, PyCharm still finds the error.

pylint still can not find this error.

typeerror3.py

import typeerrorsupport

def doduck():
    # This will give a TypeError: cannot concatenate 'str' and 'int' objects
    # Neither PyCharm, or pylint find this.
    return "The duck quacked this many times:" + typeerrorsupport.a

doduck()

Now we move the variable into a separate module.

This is where PyCharm does not find the error.

However pysonar2 can find the return type of the doduck() function is either a string or an int.
pysonar2 is not actually a tool for checking types, but only does the type inference in a library. It is meant for integrating into IDEs and such. It does advanced type inference in python.

Here it guesses that it could either return a string or an int type.

The command I used to generate some html output from all the files. I had installed and compiled
  java -jar pysonar2/target/pysonar-2.0-SNAPSHOT.jar . ./html

Follow the install instructions at the pysonar2 github page: https://github.com/yinwang0/pysonar2

typeerror4.py

import typeerrorsupport

def doduck():
    # This will give a TypeError: cannot concatenate 'str' and 'int' objects
    # Neither PyCharm, or pylint find this.
    return "The duck quacked this many times:{}".format(typeerrorsupport.a)

doduck()

This is correctly using the format method of string to put the int into the string.

Here pysonar2 correctly sees that an int is returned by the doduck() function.

typeerror5.py

import typeerrorsupport

def doduck():
    # This will give a TypeError: cannot concatenate 'str' and 'int' objects
    # Neither PyCharm, or pylint find this.
    return "The duck quacked this many times:{}" + typeerrorsupport.number_of_quacks()

doduck()

By adding the return type into the doc string of the typeerrorsupport.number_of_quacks() function we see that PyCharm can detect the TypeError.

If you follow the reST field lists, PyCharm (and other tools) can use that type information.

typeerrorsupport.py

a = 2

def number_of_quacks():
    """
    :rtype : int
    """
    return 2

The docstring tells it, that it would return an int type, and PyCharm detected this across module boundaries.

Note that pylint does not currently detect the TypeError even though we have told it the return type is int via the doctstring field list.

Conclusion

Much static type checking can be done in a dynamically typed language like python with modern tools.

Whilst none of the tools are perfect by themselves, we can see that by using a combination of the tools we can detect all of the problems we set out to detect.

You can use either the Abstract Base Class in the abc package which comes with python, or zope.interfaces to implement interfaces which can check that interfaces are implemented for you. I haven't gone into much detail here, just showed some basic interface checking with pylint.

PyCharm combined with appropriately written and typed doc strings can detect many problems. PyCharm also has many refactoring tools built in, and things like code completion for a dynamically typed language. Note, that this is not the only IDE or tool for python that can do this, but just the one I'm using for illustration purposes.

pysonar2 shows that types can be inferred without even specifying the types in docstrings. This has already been shown with tools like shedskin, and RPython(by pypy) which are statically inferring types from a subset of python. This style of type inference could be used for Ahead of Time (AOT) compilation, and within IDEs for better type inference. Better type inference allows better type checking, refactoring, and type inspection.

Comments

René Dudfield said…
Discussion on setting up project to mark types.

http://grokbase.com/t/python/code-quality/13b596zpw9/python-skeletons-type-hinting-for-third-party-libraries/oldest
Surbhi Sharma said…
My Printer is a utility software that allows you to access and easily change the settings of your printer such as the paper source. ij.start.canon Installation of My Printer is optional.Simply install the Canon Printer Drivers application on your PC, canon.com/ijsetup start it and it will automatically start detecting any plugged printers, identifying its model number, and presenting you with the matching up-to-date driver. canon.com/ijsetup With a few simple clicks, you can install all the driver software your printer needs and make your printer ready for accepting printing commands. ij.start.canon anon is one of the well-known brands for gadgets like printers, cameras, and many more devices. canon.com/ijsetup Setting up a canon printer and installing it on your computer/laptop and mac is not an easy task if you don't have proper instructions that help you install the driver of the canon printer. ij.start.canon Visit canon printer official website: and get to know the simple and quick steps to download and connect your canon printer software with your operating device. ij.start.canon Canon Printer Software for printing and scanning any type of documents is the best decision one can make and invest in a printing device. ij.start.canon Canon Printer Setup delivers amazing printing quality.
Surbhi Sharma said…

Canon Printer Setup at “ij.start.canon” is surprisingly convenient and fast. canon.com/ijsetup All you need to know is your Printer’s Model Number and this platform will enable you to perform the setup smoothly. ij.start.canon This procedure involves three distinct phases. However, before that, you must launch and open the required webpage. ij.start.canon Download and Setup printer on your device and experience the best quality printing services. canon.com/ijsetup Visit and download the printer software. ij.start.canon is the most trusted printer software available in the market because of its outstanding printing, and copying services. ij.start.canon Visit and get this software. is a website to download Canon printer drivers, you can also visit website for same. canon.com/ijsetup Canon Printers are one of the best Printer Brand available in the market which produces versatile and high-quality printing services. ij.start.canon Canon Pixma MG3600 is the most preferred, reliable and trustworthy because of its easy to use and setup properties.
Surbhi Sharma said…

Go to and create your Amazon account, if you already have an account then sign in to your account. amazon.com/mytv To watch online prime videos on devices like computers, laptops, smartphones, and smart tv. amazon.com/mytv You need to enter the 6 digit amazon activation code at or to. Amazon Prime Video is one of the most popular video-streaming services that contain an immense collection of popular movies. amazon.com/mytv It includes not just movies, but TV shows too. To watch your favorite TV shows and movies on Amazon Prime Video, one needs to create an Amazon Prime Video account. amazon.com/mytv Visit the website to ease the process.This gives you unlimited access to tons of movies and TV shows, both for yourself and your loved ones. amazon.com/mytv the activation process is so simple and straightforward that everyone can activate Amazon Prime Video on On your mobile device or computer, go to Sign in to your Amazon account. amazon.com/mytv Enter the code shown on your TV in to the Amazon website. Amazon Prime Video is a very similar video component as are services like Netflix and Hulu. To watch online prime videos on devices like computers, laptops, smartphones, and smart tv. amazon.com/mytv You need to enter the 6 digit amazon activation code at or to. Amazon Prime Video is one of the most popular video-streaming services that contain an immense collection of popular movies. amazon.com/mytv It includes not just movies, but TV shows too. is link to Sign in and Add Prime Video to your favorite device such as Smart TV, Laptop, Mobile, Tablet etc. amazon.com/mytv With this official link you can manage your Amazon Prime Video Subscription.
Surbhi Sharma said…
McAfee security programming offers a standout mcafee.com/activate amongst the best malware insurance among the various security suites in the worldwide market. mcafee.com/activate As per the most recent reports, there has been noteworthy development in digital assaults in the course of recent years. mcafee.com/activate McAfee Activate is a certain shot way you can deal with the issues with the initiation procedure. mcafee.com/activate
Our McAfee group endeavors to acquire the best tech help for your benefit. mcafee.com/activate mcafee antivirus is widely used aantivirus helps to detect and neutralize computer virus, the mail worms,the trojan programs, mcafee.com/activate
and also helps your system free of virus and other malware is quite a daily challenge. for more details visit here. mcafee.com/activate McAfee helps organizations orchestrate a truly integrated cyber environment that simultaneously protects, identifies and enhances security threats.
Surbhi Sharma said…

On the off chance that you need to reinstall office or fix it, office.com/setup you can reverify your item key on setup page and can download more established form of office. office.com/setup Download and introduce your Office applications on your work area for your requirement. office.com/setup
Incorporates a free preliminary of Word, Excel, PowerPoint, Outlook, and more Install Office on your cell phone, and set up Outlook to work with your new Office 365 letter drop. office setup is the best programming which is broadly utilized in globe .office.com/setup It is an amazing administration that causes you release your best thoughts, complete things, and remain associated on the go.for more subtleties visit: office.com/setup today. office setup is the best software which is widely used in globe .office.com/setup It is a powerful service that helps you unleash your best ideas , get things done, and stay connected on the go.for more details visit: office.com/setup On the off chance that you need to reinstall office or fix it, you can reverify your item key on setup page and can download more established form of office.

selena gomez said…

Thank you to provide us this useful information.for more information visit to my disney site so you can get more useful and valuable information about disney.
Disneyplus.com/begin
Disneyplus.com begin
disneyplus.com login/begin
Disneyplus.com/begin
Disneyplus.com begin
Disneyplus.com/begin
buttler6987 said…
This is very good blog post, I like the way you pointed all stuff, must follow below useful links.
plex.tv/link

ij.start.canon

microsoft365.com/setup

Norton.com/setup

Foxnews.com/connect

foxnews.com breaking news

JacobHarman said…
For the effective finishing of a re-appropriating organization, it's essential to have an agreement set up and work intently close by your supplier to guarantee every one of gatherings' necessities are reliably met.
While deciding to enlist cloud specialists with experience in Purplish blue, it is prescribed to utilize a dependable supplier with a demonstrated history. Mobilunity is an accomplished supplier of IT experts to organizations all over the planet. With us, you can productively employ arrangement planner up-and-comers with experience in Purplish blue. We approach an enormous pool of ability and proposition a successful cycle that guarantees a smooth organization>> important source

Popular posts from this blog

Draft 3 of, ^Let's write a unit test!^

Is PostgreSQL good enough?

post modern C tooling - draft 6