Tracing through Python functions

Python is a very efficient tool to have at hand. Its dynamic nature allows developers to quickly code systems that otherwise would have taken longer in rigid, statically-typed languages. However, this comes at a cost. Every now and then, something will crash and you, the developer, will be left wondering why that happened and how on earth that function got called with that parameter.

Even worse, most of the time you won't even have logging in place for that particular situation (otherwise you took it into account and the crash wouldn't have happened in the first place). Fortunately, there are things you can do. The best example that comes to mind is Sentry, a 3rd party logging handler that annotates stack traces with some context.

sentry-demo

However, you cannot always rely on such a 3rd party tool. Maybe you had so many errors that you just got rate limited, or even if not, you want quick feedback in diagnosing the problem since downtime most likely costs you money. You'd rather not wait until your data was sent to a 3rd party and got processed.

Having all this in mind, I decided to build a tool that would allow Python developers to examine every step of a function's execution, just like they would be able to using a debugger, but without the complexity of deploying a 3rd party integration or attaching a remote debugger.

Enter execution-trace, a Python library that allows you to do just that:

demo-animation

You choose what function you are interested in and it records its execution(s), together with the local state. You can then step through any execution, both forwards and backwards. All you need to do is pip install the library, decorate the target function and run your code as you normally do.

How to

Install the lib, as you would install any other Python package:

pip install execution-trace  

Annotate the function you want to trace.

from execution_trace.record import record


@record()  # <--- this guy
def foo(x, y):  
    # complex stuff here

You can also record multiple executions of the function, by passing in a parameter:

@record(42)

Run your code as you normally would. You'll get a message such as:

INFO:execution_trace.record:Will record execution of foo in /tmp/record_a0nQs5.json . Use `view_trace /tmp/record_a0nQs5.json` to view it.  

View the trace using the supplied viewer:

view_trace /tmp/record_a0nQs5.json  

Go to http://127.0.0.1:5000/. Use the up/down arrow keys or the vertical scroll to step through the program's execution. Change
between function executions using the number input on the right.

There's more information on the project's page about features, performance and caveats. Make sure to check it out.

Future work

While the current release covers the primary need, there is still a lot to be done. On one hand, the UI definitely needs some love; all that data loses its value if it is not properly presented. On the other hand, the library itself can be enhanced to support other features such as:

  • multiple functions at the same time
  • Python 3
  • multi-threaded programs
  • pluggable storage backends (e.g. a database instead of a JSON file)

.. and more, obviously. The code is open source, so please don't hesitate to contribute!

Mihnea Dobrescu-Balaur

Building @Hootsuite Analytics. Interested in Open Source, programming languages and the Open Web. Big fan of basketball and photography. Tweet @mihneadb.

Bucharest, Romania

Subscribe to Mihnea DB

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!