petl.fluent - Alternative notation for combining transformations

New in version 0.6.

The module petl.fluent provides all of the functions present in the root petl module, but with modifications to allow them to be used in a fluent style. E.g.:

>>> from petl.fluent import *
>>> t0 = dummytable()
>>> t0.look()
+-------+-----------+----------------------+
| 'foo' | 'bar'     | 'baz'                |
+=======+===========+======================+
| 61    | 'oranges' | 0.41684297441746143  |
+-------+-----------+----------------------+
| 42    | 'bananas' | 0.5424838757229734   |
+-------+-----------+----------------------+
| 55    | 'pears'   | 0.044730394239418825 |
+-------+-----------+----------------------+
| 63    | 'apples'  | 0.6553751878324998   |
+-------+-----------+----------------------+
| 57    | 'pears'   | 0.33151097448517963  |
+-------+-----------+----------------------+
| 57    | 'apples'  | 0.2152565282912028   |
+-------+-----------+----------------------+
| 45    | 'bananas' | 0.1478840303008977   |
+-------+-----------+----------------------+
| 79    | 'pears'   | 0.14301990499723238  |
+-------+-----------+----------------------+
| 11    | 'pears'   | 0.16801320344526383  |
+-------+-----------+----------------------+
| 96    | 'oranges' | 0.3004187573856759   |
+-------+-----------+----------------------+

>>> t1 = t0.convert('bar', 'upper').extend('quux', 42).extend('spong', expr('{foo} * {quux}')).selecteq('bar', 'APPLES')
>>> t1.look()
+-------+----------+----------------------+--------+---------+
| 'foo' | 'bar'    | 'baz'                | 'quux' | 'spong' |
+=======+==========+======================+========+=========+
| 63    | 'APPLES' | 0.6553751878324998   | 42     | 2646    |
+-------+----------+----------------------+--------+---------+
| 57    | 'APPLES' | 0.2152565282912028   | 42     | 2394    |
+-------+----------+----------------------+--------+---------+
| 87    | 'APPLES' | 0.9045902500660937   | 42     | 3654    |
+-------+----------+----------------------+--------+---------+
| 5     | 'APPLES' | 0.6915135568859515   | 42     | 210     |
+-------+----------+----------------------+--------+---------+
| 28    | 'APPLES' | 0.8440288073976338   | 42     | 1176    |
+-------+----------+----------------------+--------+---------+
| 32    | 'APPLES' | 0.047452310539432774 | 42     | 1344    |
+-------+----------+----------------------+--------+---------+
| 93    | 'APPLES' | 0.8100969279893147   | 42     | 3906    |
+-------+----------+----------------------+--------+---------+
| 94    | 'APPLES' | 0.8216793407511486   | 42     | 3948    |
+-------+----------+----------------------+--------+---------+
| 94    | 'APPLES' | 0.7911584363109934   | 42     | 3948    |
+-------+----------+----------------------+--------+---------+
| 34    | 'APPLES' | 0.18846546302867728  | 42     | 1428    |
+-------+----------+----------------------+--------+---------+

Alternatively, if you don’t want to import all petl function names into the root namespace, you can just import “etl”, e.g.:

>>> from petl.fluent import etl
>>> l = [['foo', 'bar'], ['a', 1], ['b', 3]]
>>> etl(l).look()
+-------+-------+
| 'foo' | 'bar' |
+=======+=======+
| 'a'   | 1     |
+-------+-------+
| 'b'   | 3     |
+-------+-------+

>>> etl(l).cut('foo').look()
+-------+
| 'foo' |
+=======+
| 'a'   |
+-------+
| 'b'   |
+-------+

>>> etl(l).tocsv('test.csv')
>>> etl().fromcsv('test.csv').look()
+-------+-------+
| 'foo' | 'bar' |
+=======+=======+
| 'a'   | '1'   |
+-------+-------+
| 'b'   | '3'   |
+-------+-------+

petl executable

New in version 0.10.

Also included in the petl distribution is a script to execute simple transformation pipelines directly from the operating system shell. E.g.:

$ virtualenv petl
$ . petl/bin/activate
$ pip install petl
$ petl "dummytable().tocsv()" > dummy.csv
$ cat dummy.csv | petl "fromcsv().cut('foo', 'baz').selectgt('baz', 0.5).head().data().totsv()"

The petl script is extremely simple, it expects a single positional argument, which is evaluated as Python code but with all of the petl.fluent functions imported.

Project Versions

Table Of Contents

Previous topic

petl - Extract, Transform and Load (Tables of Data)

Next topic

petl.interactive - Optimisations for Use in Interactive Mode

This Page