Sessions at EuroPython 2011 about Python on Wednesday 22nd June

This talk will give on overview over some common problems related to profiling and debugging CPython applications (especially desktop-based ones).

The following subjects will be covered:

• Debugging of memory/resource leaks (circular references, __del__, weakref, garbage collector, etc.). We will show in details how the memory management of Python objects work and we will show a few tricks to track memory leaks
• Python code profiling (profile/hotshots/cProfile, design of small tests, etc.). We will show how to measure, profile, and analyse an application to spot performance problems and solve them.
• Post-mortem step-by-step debugging of C/C++ extension under Windows. We will show how to setup Visual Studio for debugging, how to see a traceback in case of an unexpected segfault, and how to further debug and solve the crash.

This talk is aimed at intermediate Python programmers, who have already developed non-trivial Python applications but have never "jumped through" the interpreter abstraction layer. If you feel a little lost when you see a segfault from a Python program or you don't have a clear idea how memory is collected in Python, this talk is perfect for you.

PythonCAD is a CAD package written, surprisingly enough, in Python using the PyQt interface. The PythonCAD project aims to produce a scriptable, open-source, easy to use CAD package for Linux (Cross Platforms).

I start developing PythonCAD in my spare time and now python is the programming language that I mostly use.
I think that will be very interesting to tell my story and how I learn python pyqt ,sqlite, sympy... and how python and pycon and OpenERP change my job.

The goal will be to tell at the new python programmer how easy to use is python even for complex problem.
I can explain why I chose pyqt .. what kind of trouble I have found and how I solve it.
Starting from python going to pyqt and at the end how to deploy a python application with pyInstaller ...

I also explain my actual project OpenPLM an OpenERP module to manage PLM data into OpenERP

see you soon
Matteo

In Python 101, you learned basic Python syntax, what its flow control mechanisms and basic data types are and how they work. You learned how to write functions and developed executable Python scripts that actually work! You probably also learned how to create files, how to open, read from or write to them, and close them. Perhaps you've even learned a little bit of object-oriented programming, developed a couple of Python classes, most with user-defined methods, and have no problems creating instances to use in your applications.

In Python 102 (or equivalent in experience), you explored further, using default values and variable arguments for functions, discovered how to catch exceptions and write handlers for them... perhaps you've even created your own exceptions. You have found some useful Python standard library modules and using them actively in your own applications. You've quite comfortable with OOP, creating classes and using instances regularly in your programs. In fact, you've been coding Python for 6 months to a couple of years now. You're much more serious about Python now because you're no longer a "beginner." You've even taken notice at the growing number of jobs requesting or requiring Python skills.

As an aspiring Python developer, you are starting to be more aware of the entire ecosystem around you, and think you may be ready for "prime-time" and feel able to take on a full-time position as a Python programmer. However, if you have experienced one or more of the below questions or problems, this talk may be for you:

- Throughout this time, you've experienced strange bugs in your code. In particular, you notice that things don't always behave the way you expect and have spent a good amount of time debugging various parts of your software that you thought were actually correct -- you work around them but are disturbed and don't have the time nor committment to fully investigate.

- You've created classes and objects just fine but wish that you could use some of Python's operators (like +, in, len(), etc.) with your objects, which feel like they're "2nd-class" citizens compared to the standard data types.

- Do you know what functional evaluation strategy means? Have you been asked or considered whether Python is "call-by-value" or "call-by-reference"? It is important to you, and can you clearly explain your answer?

- What does "mutability" mean? What is the difference between mutable and immutable objects? Which Python objects are mutable and which aren't?

- Can you clearly explain both the output in the two code snippets below, and even more importantly, *why* the output is the way it is?

SNIPPET A
x = 42
y = x
x += 1
print x
print y

SNIPPET B
x = [1, 2, 3]
y = x
x[0] = 4
print x
print y

- Have you seen odd behavior in container or collection objects like lists or dictionaries? For example, you may have copied those types of objects and discovered weird things happening to both the original and the copy, or that you seen something not quite right if you loop through it and remove items from it.

This is what Python 103 is for... to fill in all the missing gaps, to answer all the questions (including those above) that do not seem to have easy-to-find answers on Google, but only if you have the desire to learn more about the interpreter to take your Python skills to the next level.

This training session will introduce the Python scientific stack to beginner or intermediate-level Python programmers. The basics of scientific programming with Python will be presented:
- creation of arrays and structured arrays using numpy
- fast, loopless manipulation of numpy arrays through fancy indexing and vectorized functions
- convenient saving/loading of array variables using numpy
- improved interactive use through ipython
- data analysis using various scipy modules (signal analysis, image classification, etc)
- plotting large time series, histograms, scatterplots, images etc. using matplotlib
- saving/loading large datasets in structured scientific formats such as netCDF, HDF (depending on interest)

These concepts will be used in coding exercises, in the programming environnement provided by the python(x,y) distribution, which is freely downloadable and includes recent versions of Python, numpy and matplotlib. The Python(x,y) distribution runs on Windows, which will be the OS of choice for this session.

No prior knowledge of scientific programmation using Python is required. A minimum understanding of Python programmation is required.

Attendees should bring their own laptops to the session. They should download and install the Python(x,y) distribution prior to the session, even though they might be able to do so during the session itself.

Lo sviluppo di un emulatore di videogiochi è tra le cose più divertenti che un programmatore può fare davanti ad un monitor.

Durante questo talk mostrerò come emulare un sistema relativamente semplice, il Chip-8, realizzando tutto l'emulatore in Python ed appoggiandomi solo a pygame come libreria grafica.

Al termine del talk avrete imparato quali sono i concetti base necessari per scrivere un emulatore, e non vedrete l'ora di realizzarne uno per il vostro sistema preferito.

The primary goal of this talk is twofold: to evaluate the need of data mining and introduce some very cool, simple yet powerful machine learning techniques to audience such as classification, clustering, collaborative filtering, recommendation etc in your Python web applications. This talk will conclude with some explanation and limitations of machine learning algorithms.

Basic knowledge of Python is sufficient. However some experience with Django, meshups, machine learning or data hunger is encouraged. All talk material and django apps will be available after talk.

Ever wondered how web maps are created? Ever wondered if you
could build something like Google Maps over a weekend? You
probably can't, but this talk will show you the basics of
what you need to know, such as importing data, rendering
maps and even building simple routes. And all of this in
Python!

Abstract:
* Learning how OSM data looks
* Parsing and importing the data
* Rendering maps with Mapnik
* Bits of code required to build geocoder
* Building simple router with Python and PostGIS
* And anything else I forgot to mention in this abstract but will talk about

Definitions:
OpenStreetMap -- the so-called "Wikipedia of maps" project, with thousands of contributors who edit the map data of the whole world. Unlike similar projects, the map data is completely free (both as in beer and as in speech) and thus anyone can make use of it.
Mapnik -- rendering framework, created specifically for OpenStreetMap, written in C++ and Python.
PostGIS -- an extension of PostgreSQL database, with support for many useful GIS features.

Grid computing was all the buzz in the beginning of the millennium and
still has serious attention in different forms although many of the
original grand promises were never delivered. The general level of
ambitions have instead slowly but steadily degraded to those of the
latest buzz word, Cloud.

We as a project have proven that most of the original promises *can*
actually be delivered and we have done so using Python almost solely as
the implementation language.
The choice of Python provided us with a stable and versatile base for
quickly getting this far and it significantly eases extending and
maintaining our middleware in the future.
MiG is currently about 50000 lines of source code but it still offers
more features than competing grid systems with millions of lines of
code.

Apart from introducing the open source MiG middleware and summarizing how we got here, this talk will outline some of the core technologies used to reach
that goal and underline why it can make a lot of sense to choose Python for
complex HPC projects like MiG, too.
Talk keywords include Network Programming, Open Source Python projects, Science and Math and Web-based Systems. There's no special intended audience, but a certain level of Python knowledge and experience may be an advantage.
Please refer to http://code.google.com/p/migrid/ for further MiG information.

If you run on CPython, it's not possible to get a single-process, multithreaded Python program to use more than one CPU core at a time because of the Global Interpreter Lock (GIL). A common way of dealing with this is to run one instance of a Python program for each core a machine has and spread the load amongst those processes. This forces developers to write simple, stateless programs that naturally scale out to many many servers when needed, while also not having to think about things like locking and thread scheduling.

I'll discuss some tools and methods Spotify's backend uses for managing multiple identical server processes as well as load balancing with DNS, proxy servers, and using hashing to send repeated requests to the same process. I will also talk about the difficulties that arise when you really need to share data or state between processes, and how they can be dealt with.

Il nostro gruppo di lavoro, composto da varie figure professionali, da anni lavora su più progetti in parallelo, e con team di persone che si incontrano solo virtualmente.

Non è sufficiente dire: facciamolo!

Per non perdere la testa, gestire in modo agile i progetti e vivere felici affiniamo continuamente le nostre tecniche di gestione e gli strumenti che ci permettono di tenere sotto controllo la situazione: Extreme Management tool, Poi Tracker, Buildout sono solo alcuni degli strumenti che rendono il nostro lavoro fluido ed efficace.

Presenterò le nostre modalità operative, dimostrando che lavorare con un team composto da persone contemporaneamente a Milano, Napoli, Torino e Ancona è possibile. Non solo: è il futuro!

The talk will focus on how to distribute multi-platform proprietary Python applications, using PyInstaller:
http://www.pyinstaller.org

PyInstaller takes a program written in Python and turns it into a static executable (binary) that does not depend upon an existing Python installation, nor third party library. The executable can then be packed and shipped in many different ways (through installer wizards, deb/rpm packages, etc.).

PyInstaller is a good multi-platform alternative to tools like py2exe and py2app, but it found troubles to gain acceptance in a market that seems strongly tied to older tools. Instead, PyInstaller puts forward important new features: for instance, it works on the most important desktop platforms (Windows, Linux/UNIX, Mac), and it has got an automatic and fully transparent support for all the workarounds needed to make applications using complex libraries like Matplotlib or PyQt work within a static executable.

In detail, the following points will be discussed:

Distribution of a priopriertary software: initial requests
PyInstaller: how it works (basic tutorial)
The secrets of PyInstaller: technical details
Import hooks: effectively handling third party extensions
Flexibility with PyInstaller: what goes in and what stays out
Linux: static binaries for a single distribution or cross-distributions
Mac: creation of bundles for graphic applications

CUDA technology permits to exploit the power of modern NVIDIA GPUs. In this talk, after a brief introduction to GPU architecture, we will focus on how CUDA got inside Python through libraries like PyCUDA and others…

By some examples we will show the main concepts and techniques for good GPU programming.

This talk targets anyone who wants to know how to exploit this technology from Python, the suitable use cases, the using techniques and the do-not-using techniques to get the best from his own GPU

We will look at the limitations of available rule engines when it comes to complex pricing rules. Rob has been part of a Python team writing a pricing engine based on network graphs. We will show:

(1) Why pricing is not easy
(2) Rule engine anti-patterns, e.g. to change one price requires a
week's development time, and has to be fitted into a release cycle
(3) How to make pricing rules data driven, usable by the business
(4) How the use of graphs and sets reduces the complexity

Technologies used:
python-graph, NetworkX, Redis

We had the privilege of working on a mobile web application for ETH Zurich, funded by ETH North-South Centre, using Django/Postgis, jQuery mobile and Openlayers technology.

In this talk, I would like to share what we have learnt technically and the experiences and fun we had with polygon creation on a map and saving polygon vertices and related data into our spatial database via geodjango.

This mobile web app is developed based on research by Dr Koh of ETH Zurich, specializing on Matrix-calibrated and edge-corrected species-area models that allow users to evaluate the biodiversity consequence and trade-offs of land-use decisions. The development and description of these models were published in Conserv. Biol. 24, 994 and J. Appl. Ecol. 47, 1063.

As a long-time R&D consultant I'm often working to make slow, experimental code run faster for tasks like physics simulation, flood modeling and natural language processing. Python allows a smooth progression from rough-and-ready (but slow) algorithms through to finely tuned tasks that efficiently use as much CPU power as you can bring to bear. Speed-ups of 10-500* can be expected for the Mandelbrot code we'll use.

In this talk I'll cover a set of libraries that make CPU-bound tasks run much faster. We'll begin with a look at profiling using RunSnakeRun and line_profiler to identify our bottleneck. We'll take a look at slow algorithms in Python and how they can run faster using numpy and numexpr.

Next we'll cover the use of multiprocessing to utilise multiple CPU cores along with Cython or ShedSkin to easily use C code in a friendly Python wrapper. Multiprocessing on a quad-core system can often provide a 4* speed-up for the right tasks. Next parallelpython will let us run our code on a network of machines.

Finally we'll look at pyCUDA to utilise an NVIDIA GPU. CUDA can give the best improvements for mathematical problems (over 100* on the right tasks) but works on a narrower set of problems.

How it'll work:
The tutorial will be hands on, you'll be converting example files from normal Python to faster variants using the tools below. All of it is optional, you'll get the most benefit by having everything installed. We'll work in groups and open discussion is encouraged.

NOTE - you are expected to have all these tools installed *before* the tutorial (if you don't, you might find it hard to follow what's going on!).

I'll be using Python 2.7.1 on a Macbook (Snow Leopard). All of these tools run on Windows and Linux, as long as your versions are fairly recent everything should run just fine.

My versions (roughly ordered by importance):
Python 2.7.1
RunSnakeRun 2.0.1b6 (with wxPython 2.8.12.0 Unicode)
line_profiler (1.0b2)
Cython 0.14.1
ShedSkin 0.7.1
numpy 1.5.1
numexpr 1.4.2
ParallelPython 1.6.1
pyCUDA HEAD from git as of 14th June 2011 (with CUDA 4.0 drivers)
PyPy 1.5

Some background reading:
http://ianozsvald.com/2010/07/14...
http://ianozsvald.com/2008/11/17...

The CPython interpreter always compiles your source code to bytecode, usually stored in .pyc files. This bytecode is then loaded and executed in the CPython virtual machine.

This talk will explore the bytecode from the outside in. Starting with how to read a .pyc file, following the steps the interpreter takes to arrive and a usable python module. It then dives into the structure of the bytecode itself and the principles of the virtual machine, detailing how the VM executes this bytecode to do useful work.

Having seen all these details you should have a good idea of the various innards of CPython and how to manipulate these to create weird, wonderful, dangerous and occasional useful hacks.

Structured as an adaptation of Paulo Coelho's teachings for the striving for excellence member of programming community, this talk will cover values- and principles-based foundations for building good patterns and other technical and social aspects of being a good programmer.
With examples in Python.

Camelot is a Python framework that leverages SQLAlchemy and QT to build rich desktop applications. The model - view definition used was inspired by the Django Admin interface. Some see it as a replacement for MS Access, but it's underlying libraries allow much more advanced features.

In this talk we will focus on the changes and new features that were introduced in Camelot and more importantly, how they can be used in your applications. Those features range from Matplotlib chart integration and new types of actions to displaying custom SQLAlchemy queries in a table view.

We will share our experiences with large scale deployment of Python and Camelot applications to the desktop of the user. In a number of case studies we will point out the mistakes we made and how they were corrected.

Celery is an open source task queueing system based on distributed message passing.

I will talk about the tools that Celery offers for task distribution and how to monitor and manage the system using a Django web interface.
This talk will also focus on how we use Celery at Jamendo and our real solutions to some common issues you may encounter when developing a back-office based on Celery.

The talk will cover the following topics:

-- A brief overview of Celery and the AMPQ protocol.
AMPQ protocol overview
Celery introduction
Celery, RabbitMQ code examples

-- The impact of Celery on the Jamendo work-flow; examples with real tasks.
Here I will talk about the Jamendo back-office infrastructure and some of our common tasks. I will discuss the improvements made by introducing a new back-office system based on Celery.
I will show some code snippets and go over some real scenarios.

-- Overview of the Django Celery admin interface and some Jamendo extensions
Let's talk about the Django-Celery interface that allows one to monitor or schedule tasks directly from the Django admin.
I will explain which common additional features are necessary and how to add them.

-- Common "gotchas" we encountered while working with Celery and how we solved them.
- Global task locks
- Centralized logging: be able to read all the logs of all celery workers on different servers and filter them for real-time debugging

Cloudkick is a server management and monitoring software as a service used by thousands of different companies all across the globe.

The service is built in and powered by many different programming languages and technologies, but the web application and majority of the backend services are written in Python.

In this talk I will present how and where we use Python (Django, Twisted, txamqp, Piston, …), different problems we have encountered while building our service and how we worked around them, good practices and real-world advices on building effective Twisted services and much more.

Teaser for http://lanyrd.com/2011/europytho...

Web development is a complexity challenge nowadays. Growing number of functionalities results in customer expectations increase which makes project design more difficult. Using proper tools that suite your customer needs is essential.

In this talk I would like to present two successful stories using closely together Pyramid and Plone. Basing on these examples I wished to highlight the main reasons for using Plone as a CMS only and letting Pyramid do the rest (vertical application). Moreover, I will underscore good and bad practices during integration process and how to make farsighted architectural decisions in a right moment.

Prerequisites: be familiarized with common python web framework concepts.

Build tools are not very popular in Python environment. In this talk, I'd like to show how useful it is to have one, even if you do just simple web applications. A nice, single point of entry instead of hairy shell scripts, fabric fabfiles and django admin commands all over the place.

Written in the neat, friendly Python instead of some Anty XML jumbo, you are about to have a refreshing cleanup.

Teaser for http://lanyrd.com/2011/europytho...

Python has a well-thought out system for classes. Beginners can use it without understanding it all. Experts can use it to produce code that is both elegant and powerful (such as models and class-based views in Django). Python classes can be used for many purposes.

This tutorial arises from the author's experience in using classes to solve problems, such as:
A. Add custom methods and attributes to objects that are, at root, just an integer.
B. Construct dictionaries whose values are functions.
C. Construct classes on-the-fly.
D. Implement JavaScript object semantics in Python (advanced topic).

The outline syllabus is:
1. Review of the class statement in Python (and decorators).
2. Subclassing built-in types such as int and tuple.
3. How to define classes without using a class statement.
4. How to define a dispatch dictionary using a class statement.
5. Metaclasses as syntactic sugar for class construction.
6. Metaclasses to provide new class semantics.
7. Review of JavaScript object semantics.
8. Using Python classes to implement JavaScript object semantics.

For Guido on the history of Python classes see
http://python-history.blogspot.c...
http://python-history.blogspot.c...
http://python-history.blogspot.c...

This tutorial is for Intermediate or Advanced Python programmers. (Beginners will find it very hard going.) The aim of the course is to explain exactly what happens when a class in constructed, and to learn ways of using this knowledge.

Teaser for http://lanyrd.com/2011/europytho...