pipecut - slides from presentation at meetbsd california 2014

Note: About these slides

• Original slides from the presentation follow

– The slides are light on content, because I present without reading the slides. You probably want to watch the video instead of reading these. The slides were originally made in Inkscape / Jessyink

• Past the end, I’ve included a few additional slides with details about pipecut that were demonstrated interactively

(Switch to a shell and show example commands)

Additional information about pipecut

• The previous slide was the end of the deck used for the presentation at MeetBSD, it was followed by a demonstration of the tool.

• The code for pipecut will be hosted here:

• https://code.google.com/p/pipecut/

• Project homepage:

• http://www.pipecut.org/

What is pipecut?

• A tool for developing Unix pipeline command lines more quickly and easily

• An interactive, vi-like curses interface that lets you see the data and edit the command pipeline on the same screen (no need to run and quit more/less over and over again)

• A way to save useful commands and recall them later, or share them with others

How does pipecut make commandlinedevelopment faster & easier? 1/2

• Single-keypress hotkeys for adding common Unix commands

• Real-time highlighting of matches while typing regular expressions

• The ability to see the data transformation applied instantly as you add a ‘blade’ on the command line. See just one screenful, or process the entire data file

• The ability to browse the state of the data at any stage by using the cursor keys (left and right) to ‘scrub’ through the ‘blades’

How does pipecut make commandlinedevelopment faster & easier? 2/2

• Works with every existing Unix command by using pipe(2) and treating it as a black-box – just works better with built-in commands

• Optionally, start with your familiar environment on the command line, and run history | pipecut to take your pipeline into the pipecut environment

• Give a ‘toolset’ and name and description, and it gets saved in your SQLite toolbox for use again later

• With one keypress, generate a shell script that executes your toolset.

What else will pipecut do in the future? 1/3

• Generate code that runs your toolset in other target languages.

• Generate its own regression tests by comparing internal/external command implementations and generated code results

• Provide accelerated implementations for command combinations such as ‘sort | uniq –c | sort –nr’ where a counted hashmap can be more efficient than standalone commands without context of how the data will be processed next (executing in pipecut, or in generated code)

What else will pipecut do in the future? 2/3

• Add interactive support for more utilities’ arguments (Regex, field-references, byte-references)

• Static analysis of regular expressions (e.g. highlight syntactical and semantic errors)

• Optimization of pipeline display to users’ tastes (e.g. egrep –v ‘foo’ | egrep –v ‘bar’ vs egrep –v ‘foo|bar’ )

• Remappable keyboard commands

What else will pipecut do in the future? 3/3

• Interactive command help

• Possess a deep understanding of command line options on each utility

• Allow user to force Regex highlighting mode while typing black-box commands

Pipecut

• The code for pipecut will be hosted here:

• https://code.google.com/p/pipecut/

• Project homepage:

• http://www.pipecut.org/

• Target for first release – Dec 24th, 2014