petl.interactive - Optimisations for Use in Interactive ModeΒΆ

New in version 0.5.

The module petl.interactive provides all of the functions present in the root petl module, but with a couple of optimisations for use within an interactive session.

The main optimisation is that some caching is done behind the scenes, so that by default the first 100 rows of any table are cached in memory the first time they are requested. This usually provides a better experience when building up a transformation pipeline one step at a time, where you are examining the outputs of each intermediate step as its written via look() or see(). I.e., as each new step is added and the output examined, as long as less than 100 rows are requested, only that new step will actually be executed, and none of the upstream transformations will be repeated, because the outputs from previous steps will have been cached.

The default cache size can be changed by setting petl.interactive.cachesize to an integer value.

Also, by default, the look() function is used to generate a representation of tables. So you don’t need to type, e.g., >>> look(mytable), you can just type >>> mytable. The default representation function can be changed by setting petl.interactive.representation, e.g., petl.interactive.representation = petl.see, or petl.interactive.representation = None to disable this behaviour.

Finally, this module extends petl.fluent so you can use the fluent style if you wish, e.g.:

>>> from petl.interactive import etl
>>> l = [['foo', 'bar'], ['a', 1], ['b', 3]]
>>> etl(l)
+-------+-------+
| 'foo' | 'bar' |
+=======+=======+
| 'a'   | 1     |
+-------+-------+
| 'b'   | 3     |
+-------+-------+

>>> etl(l).cut('foo')
+-------+
| 'foo' |
+=======+
| 'a'   |
+-------+
| 'b'   |
+-------+

>>> etl(l).tocsv('test.csv')
>>> etl().fromcsv('test.csv')
+-------+-------+
| 'foo' | 'bar' |
+=======+=======+
| 'a'   | '1'   |
+-------+-------+
| 'b'   | '3'   |
+-------+-------+

Project Versions

Previous topic

petl.fluent - Alternative notation for combining transformations

Next topic

petl.push - Branching Pipelines

This Page