addisdev meetup ii: golang and flow-based programming
TRANSCRIPT
Go(lang), Flow based programming, and some:
Flow-based programming experiments in Go(lang)
AddisDev Meetup #2, Addis AbabaNov 11, 2014
(meetup.com/addisdev)
Samuel Lampa@smllmp // +samuellampa // saml.rilspace.org // bionics.it
Developer atwww.farmbio.uu.se // bils.se // rilpartner.com
The Go programming language
#golang
Interest increasing every month
Invented at Google by Robert Griesemer, Rob Pike and Ken Thompson
Go(lang) characteristics● Open source● Compiled (like C, C++, Pascal, etc)● Garbage collected (Like Java)● Statically typed (Like C++, Java, C# etc)● Statically linked
(includes dependent code in the executable when compiling, not just linking to it when running)● Has primitives for writing concurrent code built into the language (channels and go-routines)● Multiplexes go-routines on operating system threads
(rather than start one thread per go-routine).● Has a C-like syntax● Has a small set of keywords● Has explicit error handling (no Exception handling)● Supports Reflection● Has no Generic programming● Has, for its age, a rather extensive standard library (Less need to reinvent the wheel)● Is supported by Google, Canonical and a few other companies.
Go(lang) benefits● Easy Concurrency
… due to in-built primitives, multi-plexing, and in-built garbage collector.● Good at handling many concurrent connections
… due to concurrency primitives and the multiplexing.● Generally good performance, especially compared to scripting languages
… due to static typing, being compiled and good concurrency primitives.● Easy to learn
… due to small language, and similarity with existing C-like languages.● Very easy to deploy to production (servers)
… due to static linking (Just copy the executable to the destination server!)● Predictable code
… due to the explicit error handling (see also downside for same reason)● Getting a lot of media attention
… and with that, a fast growing community) due to big backers like Google.
Go(lang) potential drawbacks
● The lack of Generic programming means that some redundancy might be needed at times, when implementing the same algorithm for slightly varying data types.
● Somewhat verbose code might be needed at times due to the explicit error handling (see also upside for same reason)
● Performance can be affected (to the worse) by the pauses incurred by the garbage collector (but being worked on!)
● Might not produce as optimized code as some other languages, due mostly to the young age of the language (although that improves with each release!)
Concurrency is not parallelism
Parallellism
Concurrency
Concurrency + Parallelism
“Pipeline” concurrency
Based on idea of “Communicating Sequential Processes”
(CSP)
Especially suited for concurrent apps that utilize multiple CPU cores
Go-routines
Channels
Some Go(lang) links
● golang.org● tour.golang.org● groups.google.com/forum/#!forum/golang-nuts● gplus.to/golang● talks.golang.org/2012/concurrency.slide#1
Generator Function in Go
More advanced generator function
Connecting Generators together...
Using it ...
Flow-based programming
http://en.wikipedia.org/wiki/Flow-based_programming
What is this?
Right answer: Function calls, or a “call graph”
What is this?
What if we could take control of this...…and define the graph declaratively? Wow! :)
FBP Inventor J. Paul Morrison
q
Invented at the 60's at IBM
The flow based programming book- A must read
→ tinyurl.com/fbpbook
“In computer programming, flowbased programming (FBP) is a programming paradigm that defines applications as networks of "black box" processes, which exchange data across predefined connections by message passing, where the connections are specified externally to the processes. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. FBP is thus naturally componentoriented”
Wikipedia
Core FBP Concepts
● “Black box” processes (comp. to functions)● Named in / outports as interfaces to processes● Channels with bounded buffers● Separate network definition
Some FBP network examples...
Some FBP network examples...
An array of benefits ...● Change of connection wiring
without rewriting components
● Inherently concurrent - suited for the multi-core CPU world!
● Testing, monitoring and logging very easy: Just plug in a mock-, logging- or debugging component.
● Etc.
Many similar tools: Unix pipes
Many similar tools: LabView
Many similar tools: Yahoo Pipes
Many similar tools: Apple Quartz Composer
Mitov Software (Video effects)
Implementations in many languages● Java (JavaFBP) - github.com/jpaulm/javafbp● C# (C#FBP) - github.com/jpaulm/csharpfbp● C++
– CppFBP - github.com/jpaulm/cppfbp
– Dspatch - www.flowbasedprogramming.com/DSPatch/
– Blockie.io – blockie.io
● Javascript – NoFlo - noflojs.org
– Node Red – nodered.org
● Go (GoFlow) - github.com/trustmaster/goflow● Python (PaPy) - papy.org● Common lisp - ● Shell script (net2sh) - sam.nipl.net/code/net2sh/net2sh● D (Dendrite) - bitbucket.org/eris0xaa/dendrite ● …?
Resurgence in FBP interest with NoFlo
NoFlo creator Henri Bergius
NoFlo summary
● Based on Node.js● Written in coffeescript● 237 reusable components● Successful (116% of 100k USD) Kickstarter for
in-browser UI● FBP definition format in JSON● Proof-of concept:
Re-implementation of Jekyll Static site generator: 16 000 LOC 107 components→(4 custom ones of ~500 LOC, rest re-usable!)
The UI for NoFlo: FlowHub
Book on Data Flow, FBP and Reactive Programming
Node-RED, by IBM Emerging tech
Blockie.io (Swedish project)
My adventures in GoFlow
My adventures in GoFlow: Network
My adventures in GoFlow: Component
My adventures in GoFlow: Aspiring bioinformatics library :)
Framework-less Flow-based programming in Go!
A few example components...
A few example components...
A few example components...
A few example components...
Some links
● Main website:
www.jpaulmorrison.com/fbp● Mailing list:
tinyurl.com/fbplist● Google+:
gplus.to/flowbased
Bonus slides
What's trending● Stream based processing● “New” (old) paradigms for multi-core programming
– Flow-based programming
– Actor-based programming● Erlang / Elixir● Akka
● Distributed programming (apps running across many computers)– Erlang / Elixir
– GoCircuit
● Reactive programming● New compiled languages
– Go (Google...)
– Rust (Mozilla)
– D ...
Some recommended news sources
● InfoQ – infoq.com● StrangeLoop talks at InfoQ – infoq.com/strange_loop● Prismatic – getprismatic.com
– Follow topics:● Programming● Distributed programming, etc...
● StatusCode Newsletter - statuscode.org● Hacker News – news.ycombinator.com● Hacker Newsletter – hackernewsletter.com● My twitter - twitter.com/smllmp ;)