jedi mind tricks for git
TRANSCRIPT
Jedi Mind Tricks in GitHelp me Obi-Git, you’re my only hope
Who are we?@jankrag - @randomsort
Git Trainers
“Oh my. Software development sound rather perilous. I can assure you they will never get me
on one of those dreadful Computers”
- C3PO
this -is a command
This is the console output of a commandIt can be coloured as wellthis completed successfully.
#!/foo/barThis is the content of a script
[config]Or the content of a config filehello=git merge
Conventions
Working with git hooks
What are git hooks?
We can jam a hook into Git’s normal flow
Git hook control flow
CompleteFinalizeWrite MsgCommitStage
precommitPrepare commit msg
Commit msg
postcommit
NotificationActionable
Yes - but how do I get started?git init
Making Git hooksls .git/hooks
applypatch-msg.sample commit-msg.sample post-update.sample pre-commit.sample prepare-commit-msg.sample pre-rebase.sample pre-applypatch.sample pre-push.sample update.sample
Client side vs server side hooks
precommitprepare-commit-msgcommit-msgpost-commitpost-checkoutpre-rebase
pre-receive update post-receive
The truth is out there...
Specifically on master
pre-commit hook
●First thing that happens after git commit
●Able to abandon the commit
●We have not got any arguments
Where are we at?git symbolic-ref --short HEAD
master
pre-commit hook#!/bin/bash# Check where our HEAD is atif [ "$(git symbolic-ref --short HEAD)" == "master" ]; then
echo "This is not the branch you are looking for"exit 1
fiexit 0
pre-commit hook#!/bin/bash# Check where our HEAD is atif [ "$(git symbolic-ref --short HEAD)" == "master" ]; then
echo "This is not the branch you are looking for"exit 1
fiexit 0
pre-commit hook#!/bin/bash# Check where our HEAD is atif [ "$(git symbolic-ref --short HEAD)" == "master" ]; then
echo "This is not the branch you are looking for"exit 1
fiexit 0
cp pre-commit .git/hooks/pre-commitgit checkout mastergit commit -am “I want to commit this”
This is not the branch you’re looking for
Well, I should be able to commit on mastergit checkout mastergit commit -n -am “I want to commit this”
[master 5d5308b] I want to commit this 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 foo.md
The mission
● I want to reference an issue in my commit messages
● If I do not reference an issue in my repository - abandon my commit
Where are our issues? Git remote show origin
* remote origin Fetch URL: [email protected]:RandomSort/hooks Push URL: [email protected]:RandomSort/hooks HEAD branch: master Remote branch: master tracked Local branch configured for 'git pull': master merges with remote master Local ref configured for 'git push': master pushes to master (up to date)
I <3 APIScurl -s https://api.github.com/repos/$repo/issues/$issue_number
{ "url": "https://api.github.com/repos/RandomSort/hooks/issues/1", "repository_url": "https://api.github.com/repos/RandomSort/hooks", "labels_url": "https://api.github.com/repos/RandomSort/hooks/issues/1/labels{/name}", ....TONS OF JSON :D
commit-msg hook#!/bin/bash# $1 is the temp file containing the commit messageissue_number=`cat $1 | xargs | sed 's/.*#\([0-9]*\).*/\1/'`repo="`git remote show origin | grep "Push" | xargs | sed 's/.*:\(.*\)/\1/'`"curl -s https://api.github.com/repos/$repo/issues/$issue_number |\ grep -q "\"Not Found\"" > /dev/nullexit $((1 -$?))
commit-msg hook#!/bin/bash# $1 is the temp file containing the commit messageissue_number=`cat $1 | xargs | sed 's/.*#\([0-9]*\).*/\1/'`repo="`git remote show origin | grep "Push" | xargs | sed \ 's/.*:\(.*\)/\1/'`"curl -s https://api.github.com/repos/$repo/issues/$issue_number |\ grep -q "\"Not Found\"" > /dev/nullexit $((1 -$?))
commit-msg hook#!/bin/bash# $1 is the temp file containing the commit messageissue_number=`cat $1 | xargs | sed 's/.*#\([0-9]*\).*/\1/'`repo="`git remote show origin | grep "Push" | xargs | sed 's/.*:\(.*\)/\1/'`"curl -s https://api.github.com/repos/$repo/issues/$issue_number |\ grep -q "\"Not Found\"" > /dev/nullexit $((1 -$?))
commit-msg hook#!/bin/bash# $1 is the temp file containing the commit messageissue_number=`cat $1 | xargs | sed 's/.*#\([0-9]*\).*/\1/'`repo="`git remote show origin | grep "Push" | xargs | sed 's/.*:\(.*\)/\1/'`"curl -s https://api.github.com/repos/$repo/issues/$issue_number |\ grep -q "\"Not Found\"" > /dev/nullexit $((1 -$?))
So now we’ve done something sane ..
●Git hooks can do useful stuff
●Everything in moderation
● (bust the myth, then go all in)
Trust your workflow"Fear is the path to the dark side"
Branch based delivery
C1 C2 C3 C4
ready/feature
master
HEAD
git push origin feature:ready/feature
#serverless
These are not the servers you are looking for
Going native
The mission
● I want to work on a feature branch never on master
● I want to commit work referencing issues
● I want to be able to run my Continuous Integration setup
● I deliver through ready branches
●Only allow integration if
○ Tests succeed
○ it’s a fast forward merge
Where are we at (again)?git symbolic-ref --short HEAD
ready/foo
git rev-parse --show-toplevel
/home/randomsort/repos/hooks
.. And where’s our stuff?
We only want to do fast forward merges
● It’s prettiest
●We prefer to have our automation engine only do trivial merges
C1 C2 C3 C4
Feature#79
master
HEAD
We only want to do fast forward merges
● It’s prettiest
●We prefer to have our automation engine only do trivial merges
C1 C2 C3 C4
Feature#79
master
HEAD
How fast can we go?
git rev-list --maxcount 1 master..ready/foo
C1 C2 C3 C4
ready/foo
master
HEAD
Putting it together#!bin/bashif [ not on ready branch ] do nothingfiif [ not fast forwardable ] cleanupfirun testsif [ successful ] merge cleanupelse cleanup
Putting it together#!bin/bashif [ not on ready branch ] do nothingfiif [ not fast forwardable ] cleanupfirun testsif [ successful ] merge cleanupelse cleanup
Putting it together#!bin/bashif [ not on ready branch ] do nothingfiif [ not fast forwardable ] cleanupfirun testsif [ successful ] merge cleanupelse cleanup
Cleaning up fails#We can't do a fast forward merge so let's abort this and move some stuff aroundfailbranch=`echo $branch | xargs | sed 's/ready/failed/'`echo "Unable to fast forward master to $branch leaving state on \ $failbranch"git checkout -b $failbranchgit branch -D $branch
Cleaning up fails#We can't do a fast forward merge so let's abort this and move some stuff aroundfailbranch=`echo $branch | xargs | sed 's/ready/failed/'`echo "Unable to fast forward master to $branch leaving state on \ $failbranch"git checkout -b $failbranchgit branch -D $branch
Runninggit checkout -b ready/feature
Switched to a new branch 'ready/feature'Running testsTesting has failed, leaving state on failed/featureSwitched to a new branch 'failed/feature'Deleted branch ready/feature (was 462b135).
Runninggit checkout -b ready/feature
Switched to a new branch 'ready/feature'Running testsTesting has failed, leaving state on failed/featureSwitched to a new branch 'failed/feature'Deleted branch ready/feature (was 462b135).
Runninggit checkout -b ready/feature
Switched to a new branch 'ready/feature'Running testsTesting has failed, leaving state on failed/featureSwitched to a new branch 'failed/feature'Deleted branch ready/feature (was 462b135).
Runninggit checkout -b ready/feature
Switched to a new branch 'ready/feature'Running testsTesting has failed, leaving state on failed/featureSwitched to a new branch 'failed/feature'Deleted branch ready/feature (was 462b135).
Running with successful testsgit checkout -b ready/feature
Switched to a new branch 'ready/feature1'Running testsSwitched to branch 'master'Updating e28500e..06a16b1Fast-forward temp.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 temp.mdDeleted branch ready/feature1 (was 06a16b1).
It works on my machine"I find your lack of faith disturbing"
TatooineJan
TATOOINE - LARS OWENS HOMESTEAD
OWEN:
I have no need for a protocol droid.
C3P0:
Sir -- not in an environment such as this -- that's why I've also been programmed for over thirty secondary functions that...
OWEN:
What I really need is a droid that understands the binary language of moisture vaporators.
... secondary functions"I have no need for a protocol droid."
Binary is annoying...
git diff wordfile.docx
diff --git a/wordfile.docx b/wordfile.docxindex d24c436..3c4aa59 100644Binary files a/wordfile.docx and b/wordfile.docx differ
No idea what changed...
Teaching git new tricks
Combining two features:
Git attributes
Assign behaviour per file or path
Custom drivers
Define new behaviour
Git attributes
Each line in an attributes definition is of form:
pattern attr1 attr2 ...
Where a attribute can be set, unset or assigned a value:
*.txt text*.jpg -text*.sh text eol=lf
Git attributes - global
Default location: $XDG_CONFIG_HOME/git/attributes. Fallback:$HOME/.config/git/attributes
If you want to set the path you can use:
git config --global core.attributesfile <path>
git config --global core.attributesfile ~/.gitattributes
e.g.
Git attributes - local
Just add definitions to a local .gitattributes file.
Can be set in root folder and in subfolders
Git attributes
commonly used for changing (force'ing):
CRLF behaviour
text / binary behaviour
But that is not Jedi enough ...... so we will skip those
Diff driver
Set as a config parameter like diff.foo
[diff "hexify"] binary = true textconv = hexdump -v -C
Using the new driver is easy
In .gitattributes:*.bin diff=hexify
Note - we need both:
Driver definition in config
Path attribute that tells diff to use it
But hexdump is still pretty useless 00000000 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 |.PNG........IHDR|00000010 00 00 03 20 00 00 02 80 08 02 00 00 00 eb 79 8b |... ..........y.|00000020 65 00 00 00 09 70 48 59 73 00 00 0b 13 00 00 0b |e....pHYs.......|00000030 13 01 00 9a 9c 18 00 00 00 06 62 4b 47 44 00 ff |..........bKGD..|00000040 00 ff 00 ff a0 bd a7 93 00 00 8a b5 49 44 41 54 |............IDAT|00000050 78 da ec bd 09 5b 53 57 f7 fe df f7 55 32 9d cc |x....[SW....U2..|00000060 33 09 99 e7 79 20 09 19 50 91 41 90 41 26 41 44 |3...y ..P.A.A&AD|00000070 44 14 11 41 04 04 ad b5 b5 b6 b5 b5 56 ed ef 85 |D..A........V...|00000080 fd 17 a5 7f bf 3e 4a 4e 42 38 c9 39 27 e7 be ae |.....>JNB8.9'...|00000090 cf d5 cb c7 47 25 39 67 ef b5 ee bd f7 da f7 fa |....G%9g........|000000a0 ae a7 e7 7b 00 00 00 00 00 c0 21 df e1 11 00 00 |...{......!.....|000000b0 00 00 00 40 60 01 00 00 00 00 40 60 01 00 00 00 |...@`.....@`....|000000c0 00 40 60 01 00 00 00 00 00 08 2c 00 00 00 00 00 |.@`.......,.....|000000d0 08 2c 00 00 00 00 00 08 2c 00 00 00 00 00 00 81 |.,......,.......|000000e0 05 00 00 00 00 00 81 05 00 00 00 00 00 81 05 00 |................|000000f0 00 00 00 00 20 b0 00 00 00 00 00 20 b0 00 00 00 |.... ...... ....|00000100 00 00 20 b0 00 00 00 00 00 20 b0 00 00 00 00 00 |.. ...... ......|00000110 00 04 16 00 00 00 00 00 04 16 00 00 00 00 00 04 |................|<pages and pages without end>
Recipe for success
For each binary type:
1. Find a tool that extracts useful info from a file
2. Set up a diff driver to use it
3. Profit
4. Don't give in to the dark side
Word files
My brain parses markdown quite fine...
*.docx diff=pandoc2md
[diff "pandoc2md"]textconv=pandoc --to=markdown
brew install pandoc
For reference:
.gitattributes
config
git diff wordfile.docx
diff --git a/wordfile.docx b/wordfile.docxindex d24c436..48249e3 100644--- a/wordfile.docx+++ b/wordfile.docx@@ -1,8 +1,8 @@-Headline+Hi Git Merge This is some simple text -*and another important paragraph in italic*+*and another* **important** *paragraph in italic*
Word to MarkDown
git diff --word-diff=color wordfile.docx
diff --git a/wordfile.docx b/wordfile.docxindex d24c436..48249e3 100644--- a/wordfile.docx+++ b/wordfile.docx@@ -1,8 +1,8 @@HeadlineHi Git Merge
This is some simple text
*and another* **important** *paragraph in italic*
Word to MarkDown
Other ideas?
PDF files*.pdf diff=pdfconv[diff "pdfconv"]
textconv=pdftohtml -stdout
brew install pdftohtml
<BODY bgcolor="#A0A0A0" vlink="blue" link="blue"> <A name=1></a>Headline<br> This is some simple text<br>-and another paragraph<br>-and stuff<br>+and another important paragraph in italic<br>+stuff goes here<br>+Greetings from Denmark<br> <hr> </BODY>
Maybe even for non-binaries?
Syntax highlighting
*.py diff=color
pip install Pygments
For reference:
.gitattributes
config
[diff "color"]textconv=pygmentize
git diff --cached primes.py
...@@ -0,0 +1,22 @@+def primes(n):+ """+ Compute the sequence of the first n primes.+ """+ p = []+ if n > 0:+ p = [2, ]+ for x in range(1, n):+ q = p[-1] + 1+ while True:+ for y in p:+ if not (q % y):+ break+ else:+ p.append(q)
Syntax highlighted code in diff'sMy brain parses coloured python better
Snippet from:https://github.com/cbcunc/primer (GPL)
Reformatting
*.md diff=wrap
brew install fmt
For reference:
.gitattributes
config
[diff "wrap"]textconv=fmt
git diff autostash.md
>"Why can't I pull when I have a dirty workspace, when Mercurial can do this out of the box?" -I gave the immediate answer that this is just Git's way of protecting the user from possibly harmful and, more importantly, irreversible changes. Git by default takes a very paranoid approach to any operations that change dirty files in your file system, when Git itself can't get you out of those changes again. _This is normally considered a feature_. The known "workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and then unstash them again (`git stash pop`) when done. It seems obvious that it should be easy to automate this with a git alias, but it turns out that this isn't trivial, as git stash doesn't fail gracefully when there are no local changes.+I gave the immediate answer that this is just Git's way of protecting the user from possibly harmful and, more importantly, irreversible changes. Git by default takes a very paranoid approach to any operations that change dirty files in your file system, when **Git** itself can't get you out of those changes again. _This is normally considered a feature_. The known "workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and then unstash them again (`git stash pop`) when done. It seems obvious that it should be easy to automate this with a git alias but it turns out that this isn't trivial. Git stash does not fail gracefully when there are no local changes.
.md - Before reformatting
git diff autostash.md
@@ -13,13 +13,13 @@ can do this out of the box?" I gave the immediate answer that this is just Git's way of protecting the user from possibly harmful and, more importantly, irreversible changes. Git by default takes a very paranoid approach to any-operations that change dirty files in your file system, when Git+operations that change dirty files in your file system, when **Git** itself can't get you out of those changes again. _This is normally considered a feature_. The known "workaround", or possible workflow, is to stash any changes before doing a pull (with `git stash save`, and then unstash them again (`git stash pop`) when done. It seems-obvious that it should be easy to automate this with a git alias,-but it turns out that this isn't trivial, as git stash doesn't fail+obvious that it should be easy to automate this with a git alias+but it turns out that this isn't trivial. Git stash does not fail gracefully when there are no local changes.
.md - after formatting with fmt
Images
Extract meta data
Define a useful driver:[diff "exif"]
textconv=exiftool
*.jpg diff=exif*.jpeg diff=exif*.png diff=exif*.gif diff=exif
https://openclipart.org/detail/227445/yoda
Let's resize Yoda
git diff Yoda-800px.png
diff --git a/Yoda-800px.png b/Yoda-800px.pngindex fc8ee0f..8ed738c 100644--- a/Yoda-800px.png+++ b/Yoda-800px.png@@ -1,24 +1,26 @@-File Size : 35 kB-File Modification Date/Time : 2017:01:27 17:29:06+01:00+File Size : 85 kB+File Modification Date/Time : 2017:01:30 18:52:20+01:00 File Type : PNG-Image Width : 800-Image Height : 640+Image Width : 600+Image Height : 480 Bit Depth : 8-Color Type : RGB+Color Type : RGB with Alpha
Much more useful
Note: manually abbreviated for slide
But do I know what changed?
Can we see the difference?
cachetextconv caches the result
[diff "jpg"]textconv=jp2a --width=80cachetextconv = true
*.jpg diff=jpg*.jpeg diff=jpg
Can't quite do holograms yet, but...
brew install jp2a
git add Yoda.jpg git diff --cacheddiff --git a/Yoda.jpg b/Yoda.jpgnew file mode 100644index 0000000..cf0300a--- /dev/null+++ b/Yoda.jpg@@ -0,0 +1,32 @@+................................................................................+......................................''''......................................+................................,:ldxkkOOkkxdl:,................................+.............................,lxOOdlllxOOxllldOkxc'.............................+....',:ccllllollc:,'.......'lkOxllllllxOOxllllllkOkl'.......',:cllooollcc:,'....+..;okOOkccccclllodxkxoc;'.:xOOOkkxddddkOOkodddxkkOOOx;.';coxkxdollccccclkOOko;..+....':dOxl;;:::::;;;:ldx:oOxolllloddxxkOOkxxddollllokklcxdl:;;;:::::;:okkd:'....+.......;xOkd:;::::::::':xOxoxOOOOOOOOOOOOOOOOOOOOOOxokOx;,::::::::;:xOOx;.......+........'okOOd;;:::::,okOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOkl,:::::;;dOOkl'........+..........;dOOkl;;:;:xOOOOOOOkkOOOOOOOOOOOOOOOOOOkkOOOOOOOx;;:;;okOkd,..........+............;okOkd,ckOOOOOkc,..,oOOOOOOOOOOOOOkl,..,lkOOOOOk;;dkOkl,............+..............';lo;kOOOOOOl......xOOOOOOOOOOOOd......oOOOOOOk;dl;...............+..................lOOOOOOOk;...'ckOOOOOOOOOOOOk:'...:kOOOOOOOc..................+..................xOOOOOOOOOxddkOOOOOOOOOOOOOOOOkddkOOOOOOOOOo..................
https://openclipart.org/detail/227445/yoda
Let's give him eyes
diff --git a/Yoda.jpg b/Yoda.jpgindex cf0300a..c16b7de 100644--- a/Yoda.jpg+++ b/Yoda.jpg@@ -9,8 +9,8 @@ ........'okOOd;;:::::,okOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOkl,:::::;;dOOkl'........ ..........;dOOkl;;:;:xOOOOOOOkkOOOOOOOOOOOOOOOOOOkkOOOOOOOx;;:;;okOkd,.......... ............;okOkd,ckOOOOOkc,..,oOOOOOOOOOOOOOkl,..,lkOOOOOk;;dkOkl,............-..............';lo;kOOOOOOl......xOOOOOOOOOOOOd......oOOOOOOk;dl;...............-..................lOOOOOOOk;...'ckOOOOOOOOOOOOk:'...:kOOOOOOOc..................+..............';lo;kOOOOOOl'oxc..xOOOOOOOOOOOOd,dd:..oOOOOOOk;dl;...............+..................lOOOOOOOk:odc'ckOOOOOOOOOOOOkldd:.:kOOOOOOOc.................. ..................xOOOOOOOOOxddkOOOOOOOOOOOOOOOOkddkOOOOOOOOOo.................. ..................dOOOOOOOOOOOOOkookOOOOOOOOOdldOOOOOOOOOOOOOl.................. ..................,kOOOOOOOOOOkcc:oxkOOOOOOkxocccxOOOOOOOOOOx'..................
git diff Yoda.jpg
Even in gitk
Other quick onesMP3 file meta data: exiftool can do those too, or try mp3info:
[diff "mp3"]textconv=mp3info -x
Excel files:[diff "xlsconv"]
textconv=xls2csv
Zip files:[diff "zipshow"]
textconv=unzip -c -a[diff "ziplist"]
textconv=unzip -l
Length Date Time Name --------- ---------- ----- ---- 0 01-25-2017 12:52 stuff/ 0 01-25-2017 12:53 stuff/foo/ 7 01-25-2017 12:53 stuff/foo/bar 12 01-25-2017 12:52 stuff/hello.world- 9 01-25-2017 12:52 stuff/hi.txt --------- -------- 28 5 files+ 19 4 files
git diff stuff.zip
And we haven't even left Tatooine yet!
Recap - Git objects
https://github.com/pluralsight/git-internals-pdf
Filter driversProcess blobs on save or checkout
[filter "name"]clean =
<command>smudge =
<command>
Let's have some fun with them too...
There is no try...
Yapf = Yet Another Python Formatter( pip install --user yapf )
[filter "cleanpython"]clean = yapf smudge = cat
*.py filter=cleanpython
ugly.py x = { 'a':37,'b':42,
'c':927}
y = 'hello ''world'z = 'hello '+'world'a = 'hello {}'.format('world' )class foo ( object ): def f (self ): return 37*-+2 def g(self, x,y=42): return ydef f ( a ) : return 37+-+a[42-x : y**3]
git add ugly.pygit commit ....git push
... and behold.
Caveat
That was a conceptual demo - probably not production ready.
Git diff gets a bit confused:
git status now shows ugly.py as modified
git add - and it disappears :-)
“Your eyes can deceive you. Don’t trust them.” – Obi-Wan
For demo: simple rot13 'encryption' - only affects letters
[filter "secret"]clean = ruby -ne 'print $_.tr( \"A-Za-z\", \"N-ZA-Mn-za-
m\") ' smudge = perl -pe 'tr/N-ZA-Mn-za-m/A-Za-z/'
*.java filter=secret
public static void sort(int[] numbers) { int n = numbers.length; int temp = 0;
for (int i = 0; i < n; i++) { for (int j = 1; j < (n - i); j++) {
if (numbers[j - 1] > numbers[j]) { //comment temp = numbers[j - 1]; numbers[j - 1] = numbers[j]; numbers[j] = temp; } } }}
sort.java
git ls-tree HEAD
100644 blob cf0300a887b362db6e26891afb6ad86757e00f72 Yoda.jpg100644 blob 60f95e16c59e6e0ef7dc5542cc0f5c15a35e9df4 primes.py100644 blob 887177cdd3976c695ec5303e6a6d2c1832b458a9 sort.java100644 blob 809ec4f968232287368681257a0b5ad5c726c08a ugly.py
git show 887177cdd
choyvp fgngvp ibvq fbeg(vag[] ahzoref) { vag a = ahzoref.yratgu; vag grzc = 0;
sbe (vag v = 0; v < a; v++) { sbe (vag w = 1; w < (a - v); w++) {
vs (ahzoref[w - 1] > ahzoref[w]) { grzc = ahzoref[w - 1]; ahzoref[w - 1] = ahzoref[w]; ahzoref[w] = grzc; }
} }}
And on GitHub too, obviously
Now just imagine we had used
GPG or AES-256 with private key
*.cred filter=secret
Publish your private content to public repos :-)
The End"Already know you that which you need." – Yoda