IPython is becoming a very popular platform for scientific methods development, providing tools for documentation, presentation and distributed computing. A key part of the platform is the IPython "notebook" which integrates with IPython parallel. IPython parallel is a very intuitive and (importantly) simple framework for distributing python functions as tasks in a queue - similar in some ways to parallel-python or Celery.
This talk is focused on introducing the IPython.parallel platform and will cover:
1) How to identify if a problem is distributable.
2) The architecture of the IPython parallel framework.
3) How to quickly start a cluster of "workers" using the web interface and command line.
4) How to push tasks to workers and collate the results using "direct" and "load balanced" views.
The talk will provide examples of embarrassingly parallel tasks as notebooks with a focus on meteorological data sets.
Nathan is a forecast guidance scientist at the Bureau of Meteorology with a background in computer vision. He is actively involved in scientific software development and the open source. Community.