anthony grimes clojail

8/3/2019 Anthony Grimes Clojail

1/123

ClojailLife in the Clojure Prison

Thursday, November 10, 11


2/123

Me

Clojure programmer for around 3 years. Wrote http://tryclj.com (Try Clojure). Is writing a book called Meet Clojure.

Is the youngest person in this room. Is a Stuart Sierra groupie.

http://tryclj.com/http://tryclj.com/http://tryclj.com/


3/123



4/123

If code werent dangerous, you wouldnt beable to

read/write to the file system. talk to the internet.

do pretty much anything useful at all. Dangerous is good.

Code is dangerous!



5/123

You are your codes sandbox.

You control everything that happens. Its why you dont usually care about

sandboxing.

We almost never need or want to allowpeople to evaluate code on our machines...

How often do you think of it like

that?



6/123

But Clojure makes it so easy

user>(eval(read-string"(+ 3 3)"))

6



7/123

But what about this?

[13:05:31] &(+ 3 3)

[13:05:32]

6[13:05:34] &(System/exit 0)

[13:05:35] * lazybot has left IRC (you've killed him)



8/123

It gets worse

[13:15:40] &(map #(.delete %) (file-seq (System/getProperty "user.home")))

[13:15:41] deletin' ur datas...

We need to be more cautious!



9/123Thursday, November 10, 11


10/123

We need a sandbox tokeep us safe



11/123

I/O, such as

Interaction with the file system. Interaction with the internet.

The execution of arbitrary programs.

Destruction of the JVM.

A sandbox can prevent...



12/123

Step 1: The JVM



13/123

is thorough. has been around since before I learned to

walk.

prevents I/O. denies access to certain methods and classes.

is customizable.

Basically...

The JVM sandbox



14/123

It stops this from

happening

user=>(System/exit0)

[cake]errorconnectingtosocket



15/123

And this stuff

Among other things...

user=>(slurp"http://hackmycreditcard.com")

"processing payments"

user=>(->"user.home" System/getProperty

(java.io.File.".emacs")

.delete)

true



16/123

It saves your computerfrom evil...



17/123

Unfortunately, it isnt

enough for every use-case.



18/123

Step 2: Clojure



19/123

The hardest part of

sandboxing issandboxing the state of

Clojure



20/123

user=>(def+-)#'repl-1/+

user=>(+1010)

0

What if they rebind things?



21/123

What about infinite loops?

user=>(loop[](recur))



22/123

Try Clojure is a Clojure REPL in yourbrowser. 4Clojure is Clojure koans for your browser.

Lazybot is a Clojure REPL in your IRCchannel.

Facing these problems



23/123

So, what do figure outwhat we need to do!



24/123

Rip apart code

Look at

namespaces symbols classes

packages vars

etc



25/123

def

Need to keep defs from being abusedbecause they can rebind things in the namespace.

be used to abuse memory.



26/123

Loops

Timeouts!



27/123

Threads

They can be used to avoid timeouts. They must die.



28/123

The dot (.) special form

Is evil because you can abuse Clojures Javaclasses.

Observe: Cannot be gotten rid of.

Cannot be rebound because it is a special

form.

Must be replaced entirely.

user=>(.intern*ns*'+)#'user/+



29/123

Thats what is needed to

make it safe, but whystop there?



30/123

Extensibility

Safety isnt the only concern.

Customized evaluation contexts. Allow users to block whatever they want.

4Clojure is built on this.



31/123

Clojail



32/123

An all you can eatsandboxing library



33/123

written by this guy

Anthony GrimesThursday, November 10, 11


34/123

and this guyAlan Malloy



35/123

inspired by ideas from this guyHeinz N. Gies



36/123

which were inspired by an IRC bot fromthis guy

Kevin DowneThursday, November 10, 11


37/123

for you guys.Well, mostly for me, but you guys can use it too!



38/123

It can...

take advantage of the JVM sandbox.

sandbox the Clojure side of things. do very selective blacklisting.

basically do everything weve talked about.



39/123

Using it



40/123

Or



41/123

(defprojectscared"0.1.0"

:description"Feeling vulnerable"

:dependencies[[clojail"0.5.0"]])

Easy enough, right?



42/123

user=>(use'clojail.core)

user=>(defsb(sandbox#{}))

#'user/sb

user=>(sb'(+33))

6

user=>(sb'(System/exit0))

AccessControlExceptionaccessdenied...

user=>(defsb(sandbox#{}:jvmfalse))

#'user/sb


Open For Business



43/123



#'user/sb

user=>(sb'(+33))

6




#'user/sb


Open For Business



44/123



#'user/sb

user=>(sb'(+33))

6




#'user/sb


Open For Business



45/123

Blocking Clojure

sandbox takes a set of things like Javaclasses, packages, symbols, sets, namespaces.

These are called testers.



46/123

user=>(defsb(sandbox#{'+'-Math}))

#'user/sb

user=>(sb'(+33))

SecurityExceptionYoutrippedthealarm!+isbad!

user=>(sb'(-42))

SecurityExceptionYoutrippedthealarm!-isbad!

user=>(sb'(Math/cos10.3))

SecurityExceptionYoutrippedthealarm!classjava.lang.Math

isbad!

Math Can Be Difficult



47/123

user=>(defsb(sandbox#{'+'-Math}))

#'user/sb

user=>(sb'(+33))

SecurityExceptionYoutrippedthealarm!+isbad!

user=>(sb'(-42))

SecurityExceptionYoutrippedthealarm!-isbad!

user=>(sb'(Math/cos10.3))

SecurityExceptionYoutrippedthealarm!classjava.lang.Math

isbad!

Math Can Be Difficult



48/123

user=>(use'clojail.testers)

nil

user=>(defsb(sandboxsecure-tester))#'user/sb

user=>(sb'(do(future(range))nil))

SecurityExceptionYoutrippedthealarm!future-callisbad!

Security Blanket



49/123


nil

user=>(defsb(sandboxsecure-tester))#'user/sb


SecurityExceptionYoutrippedthealarm!future-callisbad!

Security Blanket


' l j il


50/123


nil

user=>(defsb(sandboxsecure-tester))

#'user/sb


SecurityExceptionYoutrippedthealarm!future-callisbad



51/123

user=>(defsb(sandbox*))

#'user/sb

user=>(sb'(+33)secure-tester)

6user=>(defsb(sandbox secure-tester))

#'user/sb

user=>(sb'(+33))

6



52/123

user=>(defsb(sandbox*))

#'user/sb

user=>(sb'(+33)secure-tester)

6user=>(defsb(sandbox secure-tester))

#'user/sb

user=>(sb'(+33))

6



53/123

user=>(sb'(loop[](recur)))

TimeoutExceptionExecutiontimedout. clojail.core/thunk-timeout

(core.clj:57)

user=>(defsb(sandboxsecure-tester:timeout5000))#'user/sb



(core.clj:57)



54/123



(core.clj:57)

user=>(defsb(sandboxsecure-tester:timeout5000))#'user/sb



(core.clj:57)

Wait for it...


Definitely


55/123

user=>(defsb(sandboxsecure-tester-without-def))

#'user/sb

user=>(doseq[name'[abcdef]]

(sb`(def~name0)))

nil

user=>(sb'e)user=>CompilerExceptionjava.lang.RuntimeException:Unableto

resolvesymbol:ainthiscontext,compiling:(NO_SOURCE_PATH:0)

user=> (sb'f)

0

user=>(sb(cons'do(map#(list'def%0)

'[abcdef])))#'sandbox207/f

user=>(sb'f)

user=>CompilerExceptionjava.lang.RuntimeException:Unableto

resolvesymbol:finthiscontext,compiling:(NO_SOURCE_PATH:0)

Definitely


Definitely


56/123


#'user/sb


(sb`(def~name0)))

nil



user=> (sb'f)

0



user=>(sb'f)



Definitely


Definitely


57/123


#'user/sb


(sb`(def~name0)))

nil



user=> (sb'f)

0



user=>(sb'f)



Definitely


A Littl S


58/123


#'user/sb

user=>(defsb-two(sandboxsecure-tester))

#'user/sb-two

user=>(sb'*ns*)

#

user=>(sb-two'*ns*)

#

user=>(defsb(sandboxsecure-tester

:namespace'my-ns))

#'user/sb

user=>(sb'*ns*)

#

A Little Space


A Littl S


59/123


#'user/sb

user=>(defsb-two(sandboxsecure-tester))

#'user/sb-two

user=>(sb'*ns*)

#

user=>(sb-two'*ns*)

#


:namespace'my-ns))

#'user/sb

user=>(sb'*ns*)

#

A Little Space


S Di ti


60/123

user=>(use'clojail.jvm)nil

user=>(defcon

(->(java.io.FilePermission."foo"

"read,write")

permissions

domain

context))

#'user/con

user=>(defsb(sandboxsecure-tester:contextcon))

#'user/sb

user=>(sb'(do(spit"foo""Hi!")(slurp"foo")))

"Hi!"

user=>(jvm-sandbox#(do(spit"foo""Hi!")(slurp"foo"))con)"Hi!"

Secure Dispensations


S Di ti


61/123


user=>(defcon


"read,write")

permissions

domain

context))

#'user/con


#'user/sb


"Hi!"




S Di ti


62/123


user=>(defcon


"read,write")

permissions

domain

context))

#'user/con


#'user/sb


"Hi!"





63/123


:init'(deffoo"foo")))

#'user/sb

user=>(sb'foo)

"foo"

user=>(definit'(require'[clojure.string:asstring]))

#'user/init

user=>(defsb(sandboxsecure-tester:initinit))

#'user/sb

user=>(sb'(string/join[\a\b\c]))

"abc"

Laying The Foundation



64/123


:init'(deffoo"foo")))

#'user/sb

user=>(sb'foo)

"foo"

user=>(definit'(require'[clojure.string:asstring]))

#'user/init

user=>(defsb(sandboxsecure-tester:initinit))

#'user/sb

user=>(sb'(string/join[\a\b\c]))

"abc"

Laying The Foundation


A S l B d S ll


65/123

user=>(let[writer(java.io.StringWriter.)]

(sb'(println"Hello, world!"){#'*out*writer})

(println(strwriter)))

Hello,world!

A Simple Binding Spell



66/123

So thats Clojail.



67/123

Mmmm, implementation details!



68/123

Lets look at theindividual pieces that

make up the sandbox,starting with check-

form



69/123

(defncheck-form[formtesternspace] (sometester(separateformnspace)))

Border Patrol


Check ALL The Things!


70/123

(defn-separate [snspace]

(set (flatten-all

(map#(if(symbol?%)

(let[resolved-s(safe-resolve%nspace)

s-meta(metaresolved-s)]

(ifs-meta

[resolved-s

((juxt(compsymbolstr:ns):ns:name)

s-meta)]

(let[[bottom](mapsymbol(.split(str%)"/"))

resolved-s(safe-resolvebottomnspace)]

(if(class?resolved-s)

[resolved-s%]

%)))) %)

(flatten-all(collify(macroexpand-mosts)))))))





71/123


(set

(flatten-all

(map#(if(symbol?%)



(ifs-meta

[resolved-s


s-meta)]




[resolved-s%]

%)))) %)






72/123


(set

(flatten-all

(map#(if(symbol?%)



(ifs-meta

[resolved-s


s-meta)]




[resolved-s%]

%)))) %)






73/123


(set

(flatten-all

(map#(if(symbol?%)



(ifs-meta

[resolved-s


s-meta)]




[resolved-s%]

%)))) %)






74/123


(set

(flatten-all

(map#(if(symbol?%)



(ifs-meta

[resolved-s


s-meta)]




[resolved-s%]

%)))) %)






75/123


(set

(flatten-all

(map#(if(symbol?%)



(ifs-meta

[resolved-s


s-meta)]




[resolved-s%]

%)))) %)





76/123

So thats how your non-

interop code is handled.But what about your

Java interop code?



77/123

Clojail sandboxes in two stages.

1. It checks the code before evaluation.

2. Modifies the code so that it can sandbox

things that couldnt be checked/it couldhave missed before evaluation

We will replace the . special form with ourspecialized dot macro.

This is just a simple recursive walk. Its whatdot does that is interesting.


Th I P l


78/123

(defn-make-dot[tester-str]

`(defmacro~'dot[object#method#&args#]

`(let[~'tester-obj#(binding[*read-eval*true]

(read-string~~tester-str))

~'obj#~object#

~'obj-class#(class~'obj#)] (if-let[~'bad#(some~'tester-obj#

[~'obj-class#

~'obj#

(.getPackage~'obj-class#)])]

(security-exception~'bad#)

(.~object#~method#~@args#)))))

The Interop Police


Th I P li


79/123

(defn-make-dot[tester-str]

`(defmacro~'dot[object#method#&args#]

`(let[~'tester-obj#(binding[*read-eval*true]

(read-string~~tester-str))

~'obj#~object#

~'obj-class#(class~'obj#)] (if-let[~'bad#(some~'tester-obj#

[~'obj-class#

~'obj#

(.getPackage~'obj-class#)])]

(security-exception~'bad#)

(.~object#~method#~@args#)))))

The Interop Police



80/123

And thats how dot is

handled. But what abouttimeouts?


Patience...


81/123

(defnthunk-timeout

...

([thunktimeunittg]

(let[task(FutureTask.thunk)

thr(iftg(Thread.tgtask)(Thread.task))]

(try

(.startthr)

(.gettasktime(or(uglify-time-unitunit)unit))

(catchTimeoutExceptione (.canceltasktrue)

(.stopthr)

(throw(TimeoutException."Execution timed out.")))

(catchExceptione

(.canceltasktrue)

(.stopthr)(throwe))

(finally(whentg(.stoptg)))))))

Patience...


Patience...


82/123

(defnthunk-timeout

...

([thunktimeunittg]



(try

(.startthr)



(.stopthr)


(catchExceptione

(.canceltasktrue)

(.stopthr)(throwe))


Patience...


Patience...


83/123

(defnthunk-timeout

...

([thunktimeunittg]



(try

(.startthr)



(.stopthr)


(catchExceptione

(.canceltasktrue)

(.stopthr)(throwe))


Patience...


Patience...


84/123

(defnthunk-timeout

...

([thunktimeunittg]



(try

(.startthr)



(.stopthr)


(catchExceptione

(.canceltasktrue)

(.stopthr)(throwe))


at e ce...



85/123

thunk-timeout can handle typical (Thread. ...)threads.

Do not allow anything that uses threadpools.

secure-tester tries to do this.

Sandboxing threads



86/123

The moment youve all

been waiting for.The evaluator!


Finally An Eval!


87/123

(defn-evaluator[codetester-strcontextnspacebindings]

(fn[]

(binding[*ns*nspace

*read-eval*false]

(let[bindings(orbindings{}) code`(do~(make-dottester-str)

~(dotify(macroexpand-most code)))]

(with-bindingsbindings

(jvm-sandbox#(evalcode)context))))))

Finally, An Eval!


Finally An Eval!


88/123

(defn-evaluator[codetester-strcontextnspacebindings]

(fn[]

(binding[*ns*nspace

*read-eval*false]

(let[bindings(orbindings{}) code`(do~(make-dottester-str)

~(dotify(macroexpand-most code)))]

(with-bindingsbindings

(jvm-sandbox#(evalcode)context))))))

Finally, An Eval!



89/123

That is a lot to take in. How does clojailput it all together?

The sandbox* function!



90/123

Shield your eyes, its areally dense function.


Make It So


91/123

(defnsandbox* [&{:keys[timeoutnamespacecontextjvm

initns-initmax-defsrefer-clojure]

:or{timeout10000 namespace(gensym"sandbox")

context(->(permissions)domaincontext)

jvmtrue

refer-clojuretrue

max-defs5}}]

(let[nspace(create-nsnamespace)]

(binding[*ns*nspace]

(whenrefer-clojure(clojure.core/refer-clojure)) (evalinit))

(let[init-defs(conj(user-defsnspace)'dot)]

(fn[codetester&[bindings]]

(let[tester-str(read-testertester)

old-defs(user-defsnspace)]

(whenjvm(set-security-manager(SecurityManager.)))

(try

(let[result(if-let[problem(check-formcodetesternspace)](security-exceptionproblem)

(thunk-timeout

(evaluatorcodetester-strcontextnspacebindings)

timeout:ms

(ThreadGroup."sandbox")))]

result)

(finally(wipe-defsinit-defsold-defsmax-defsnspace))))))))



92/123

And that was without the docstring!


Make It So


93/123

(defnsandbox*[&{:keys[timeoutnamespacecontextjvm




jvmtrue

refer-clojuretrue

max-defs5}}]


(binding[*ns*nspace] (whenrefer-clojure(clojure.core/refer-clojure))

(evalinit))






(try (if-let[problem(check-formcodetesternspace)]

(security-exceptionproblem)

(thunk-timeout


timeout:ms

(ThreadGroup."sandbox")))



Make It So


94/123





jvmtrue

refer-clojuretrue

max-defs5}}]



(evalinit))








(thunk-timeout


timeout:ms




Make It So


95/123





jvmtrue

refer-clojuretrue

max-defs5}}]



(evalinit))








(thunk-timeout


timeout:ms




Make It So


96/123





jvmtrue

refer-clojuretrue

max-defs5}}]



(evalinit))








(thunk-timeout


timeout:ms




Make It So


97/123





jvmtrue

refer-clojuretrue

max-defs5}}]



(evalinit))








(thunk-timeout


timeout:ms




Make It So


98/123





jvmtrue

refer-clojuretrue

max-defs5}}]



(evalinit))








(thunk-timeout


timeout:ms




Make It So


99/123





jvmtrue

refer-clojuretrue

max-defs5}}]



(evalinit))








(thunk-timeout


timeout:ms





100/123

The result of all that, my good friends, is a Clojure sandbox.Its awesome and all, but we need to think about a few things.



101/123

If being safe is important, you should takeevery possible precaution imaginable.

The JVM sandbox is mature and thorough, butthat doesnt mean it is invincible.

Run your code in its own user account.



102/123

Youre okay as long as you use the JVM sandbox.

Allowing everyone to safely evaluate code in thesame namespace is clojails goal.

We are limited by not being Rich Hickey.


Luckily Clojurians are drawn to holes in


103/123

Luckily, Clojurians are drawn to holes inclojail like moths to a flame.



104/123

People (mostly me) trust

Clojail enough to use it intheir own projects.


T Cl j


105/123

Try Clojure

An interactive tutorial website for Clojurewith a Clojail-powered REPL.

Similar in nature to the other TryLanguagewebsites, particularly TryHaskell. Has a space in the name, unlike the other

sites. This makes it cooler.

Built on Chris Grangers awesome Noir webframework.

Runs on Heroku. Also makes it cooler.



106/123


A h


107/123

Approach

def is allowed. Each user has his own namespace. Timeouts happen very fast.

Tries to emulate a REPL as closely as possible.



108/123

(defnmake-sandbox[]

(sandboxtry-clojure-tester

:timeout2000 :init'(future(Thread/sleep600000)

(->*ns*.getNameremove-ns))))

(defnfind-sb[old]

(if-let[sb(getold"sb")]

old

(assocold"sb"(make-sandbox))))

(defneval-request[expr]

(try

(eval-stringexpr(get(update-session!find-sb)"sb"))

(catchTimeoutException_

{:errortrue:message"Execution Timed Out!"})

(catchExceptione

{:errortrue:message(str(root-causee))})))



109/123

(defnmake-sandbox[]

(sandboxtry-clojure-tester

:timeout2000 :init'(future(Thread/sleep600000)

(->*ns*.getNameremove-ns))))

(defnfind-sb[old]

(if-let[sb(getold"sb")]

old

(assocold"sb"(make-sandbox))))

(defneval-request[expr]

(try

(eval-stringexpr(get(update-session!find-sb)"sb"))

(catchTimeoutException_

{:errortrue:message"Execution Timed Out!"})

(catchExceptione

{:errortrue:message(str(root-causee))})))


Credits


110/123

Andrew Gwozdziewycz (apgwoz)

Design the whole thing.

Chris Done

Awesome jquery-console used for theREPL interface.

Awesome design on TryHaskell that we

took inspiration from.

Allen Johnson (mefesto)

Wrote the interactive tutorial stuff.


4Clojure


111/123

j

Perhaps the most interesting Clojail use-case.

Solve koan-like Clojure problems/tasks in yourbrowser.

Has a long list of problems of variable difficulity,ranging from easy to very hard.

Wonderful as a companion to any Clojure learningmaterial, and is a great learning experience evenfor veteran Clojurians.



112/123


Approach


113/123

Approach

Relies on dynamic sandboxing.

If a problem calls for the reimplementation ofa core function, the core function or similarfunctions can be blacklisted to prevent

cheating.



114/123

(for[testtests]

(try

(when-not(->>user-forms

(s/replacetest"__")

read-string-safely first

(sbsb-tester))

"You failed the unit tests")

(catchThrowablet(.getMessaget))))


Credits


115/123

Credits

David Byrne and Alan Malloy (project leads) Alex McNamara (top contributor) Carin Meier (frontend)


Lazybot


116/123

Lazybot

An IRC bot written in Clojure. Extensible via plugins.

Totally dynamic and can be run/manipulatedfrom a repl.

Has a Clojure evaluation plugin.

Can be found in #clojure, stealing peoplescodez.



117/123


Approach


118/123

Approach

def is not allowed. Everybody uses the same namespace in all

channels.



119/123

(defnexecute-text[box?bot-nameusertxtpre] (try

(with-open[writer(StringWriter.)]

;; I am aware of the existence of with-out-str.

;; Look closer -- it wont work here.

(let[bindings{#'*out*writer}

res(ifbox?

(sb(safe-readtxt)bindings)

(pr-str(no-box(read-stringtxt)bindings)))

replaced(string/replace(strwriter)"\n"" ")

result(strreplaced(when(=last\space)" ")res)]

(str(orpre"\u21D2 ")(trimbot-nameusertxtresult))))

(catchTimeoutException_"Execution Timed Out!")

(catchExceptione(str(root-causee)))))


Credits


120/123

Credits

Alan Malloy

A zillion contributors whose names wont allfit in this slide (or talk). You know who youare.


Guidelines for using Clojail


121/123

in your own code The JVM sandbox is your

friend. Always use it.

Follow Clojails releasecycle closely and update atevery convenient chance.

Report any and everyissue you find with it.

Dont be paranoid.Remember that the JVMsandbox will protect youfrom real danger.

If you avoid sharing thesame namespace witheverybody, it is less likelythat one person will blow

away the state of thewhole thing foreverybody.

Dont allow def and giveeveryone the samenamespace. Thats askingfor it.


Further reading


122/123

Further reading

https://github.com/flatland/clojail/wiki


Thanks
https://github.com/flatland/clojail/wikihttps://github.com/flatland/clojail/wiki


123/123

Thanks

The internet:

For all of the adorablecat pictures.

Baishampayan Ghose

For having the longestname Ive ever had totype.

For the 10 conjpictures.

My Geni co-workers(Alan, Lance, Justin):

For listening to thistalk and reviewing it.

Helping me prepare.

Alan Malloy

For turning myinsane ideas intogood ones.

anthony grimes clojail

Documents