can we run the whole web on apache sling?
TRANSCRIPT
![Page 1: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/1.jpg)
APACHE SLING & FRIENDS TECH MEETUP BERLIN, 26-28 SEPTEMBER 2016
Can we run the whole Web on Apache Sling? Bertrand Delacretaz & Chetan Mehrotra
@bdelacretaz - @chetanmeh Sling committers and PMC members CQ/AEM core team members, Adobe
slides revision 2016-09-26
![Page 2: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/2.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
The Whole Web ??2
Are you guys crazy?(yes, but not that much)
![Page 3: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/3.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 3
Where’s Sling going? from
SERVERS to
SYSTEMS
![Page 4: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/4.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 4
httpd
mod_lua script
mod_proxy
haproxy
etcd -> config
MongoDB
processor selector
more pools…more pools…
fileserver pool
slingshot pool
default pool
more pools…
etcd
monitoring / metrics
REDDR - Resource Driven Dynamic Routing Sling resource/script resolution drives routing
1 2
34
1
2
3
4
Request to Sling Processor Selector
Response with a single word: fileserver
+Header: Sling-Processor-Role:fileserver
haproxy routes to fileserver pool
Rebuild haproxy config on changesSling instances announce themselves to etcd:-IP address-Port-RolePools are actually haproxy “backends”
fun!
clientrequest
port 80
port 81
Sling-Processor-Role
![Page 5: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/5.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 5
REDDR REsource-Driven Dynamic Routing
![Page 6: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/6.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 6
REDDR routing to fileserver instances pool
httpd
mod_lua script
mod_proxy
haproxy
etcd -> config
processor selector
fileserver pool1 2
34
0
1
2
3
4
Request to Sling Processor Selector
Response with a single word: fileserver
+Header: Sling-Processor-Role:fileserver
haproxy routes to fileserver pool
0 Client request
.routing script engine + select based on resource type and parent types.Default servlets and scripts disabled.
Select backend pool based on Sling-Processor-Roleheader, using haproxy “acl” generated from etcd
![Page 7: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/7.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 7
Dynamic Registration of Sling Instancesregenerate haproxy config based on etc data
![Page 8: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/8.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 8
haproxy
etcd -> config
more pools…more pools…
fileserver pool
slingshot pool
default pool
more pools…
etcd
haproxy config is rebuilt via event-driven confd + reload.sh script
Sling instances announce themselves to etcd:-IP address-Port-Role
Dynamic wiring of Sling instances in the haproxy pools
1
2
12
![Page 9: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/9.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 9
Building customised Sling Docker imageswith just a provisioning model + trivial Dockerfile
![Page 10: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/10.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 10
base/pom.xml base/src/main/docker/Dockerfile base/src/main/docker/slingroot/announce.sh base/src/main/docker/slingroot/start.sh base/src/main/docker/slingroot/wait-for-it.sh
default-processor/pom.xml default-processor/src/main/docker/Dockerfile default-processor/src/main/provisioning/composum-browser.txt default-processor/src/main/provisioning/default-processor.txt default-processor/src/main/provisioning/launchpad.txt default-processor/src/main/provisioning/…
Base and default-processor images
FROM ch.x42.at16.base COPY ${project.build.finalName}.jar /opt/sling/launchpad.jar
Dockerfile for default-processor
Custom Sling Docker images
![Page 11: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/11.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 11
DYNAMIC CLUSTER DEMO A Big Sling Cluster…on my laptop
![Page 12: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/12.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 12
httpd
mod_lua script
mod_proxy
haproxy
etcd -> config
MongoDB
processor selector
more pools…more pools…
fileserver pool
slingshot pool
default pool
more pools…
etcd
monitoring / metrics
REDDR - Resource Driven Dynamic Routing Sling resource/script resolution drives routing
1 2
34
1
2
3
4
Request to Sling Processor Selector
Response with a single word: fileserver
+Header: Sling-Processor-Role:fileserver
haproxy routes to fileserver pool
Rebuild haproxy config on changesSling instances announce themselves to etcd:-IP address-Port-RolePools are actually haproxy “backends”
![Page 13: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/13.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 13
“Background Worker” Sling instance minimal, disposable, focused
![Page 14: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/14.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Asset worker pool
default pool
MongoD
B
ActiveM
Q
5
2
1
3
4
1
2
3
4
Request to Sling Frontend server
Write to NodeStore
Add job to message queue
Pick up of job by one of the worker
5 Write back of Job result
Worker Profile
• Tailor made Sling instance • Min required bundles • Low footprint • JCR API used mostly for CRUD • No observation • Disposable • Take post processing load off
frontend servers
![Page 15: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/15.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 15
Repository Etiquette be precise while talking to repository and thou shall not be troubled!
![Page 16: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/16.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Choose the right nodetype
Avoid nt:unstructured
• Orderable children • Limits writes
Prefer oak:Unstructured
OAK-657
+ dashboard (nt:unstructured) - role_observer - projects-outdoors - role_editor - projects-editor + gadgets (nt:unstructured) + team + asset
+ dashboard (oak:Unstructured) - role_observer - projects-outdoors - role_editor - projects-editor + gadgets (oak:Unstructured) + team + asset
![Page 17: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/17.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Choose the right nodetype
Avoid nt:resource with nt:file
• Referenceable • 1 nt:resource = 1 entry in uuid index • 1M Files = 1M entry in uuid index
Prefer oak:Resource
+ book.jpg (nt:file) - jcr:createdBy - admin + jcr:content (nt:resource) - jcr:lastModifiedBy - admin - jcr:mimeType - image/jpeg - jcr:uuid - "dafe0c9c-1872-4397"
OAK-4567
+ book.jpg (nt:file) - jcr:createdBy - admin + jcr:content (oak:Resource) - jcr:lastModifiedBy - admin - jcr:mimeType - image/jpeg
![Page 18: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/18.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Query Precisely
• Use specific nodetype
• Relativize property names wrt root of micro tree
• Include path restrictions
• Use union if nodetypes differ
SELECT * FROM [nt:base] AS a WHERE a.[type] = 'image'
SELECT * FROM [dam:Asset] AS a WHERE ISDESCENDANTNODE([/content/dam]) AND a.[jcr:content/metadata/type] = 'image'
![Page 19: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/19.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Observe Precisely
▪ Observation costs resources
▪ Use JackrabbitEventFilter to filter on ▪ Multiple paths ▪ Nodetypes
▪ In works* filtering based on ▪ Property names ▪ Node names
*OAK-4796
JackrabbitEventFilter eventFilter = new JackrabbitEventFilter() .setAbsPath(paths[0]) .setNodeTypes(new String[]{"dam:Asset"}) .setEventTypes(Event.NODE_ADDED) .setIsDeep(true);
eventFilter.setAdditionalPaths("/content/en"); JackrabbitObservationManager om = (JackrabbitObservationManager)session.getWorkspace() .getObservationManager(); om.addEventListener(this, eventFilter);
![Page 20: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/20.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Define your own types
▪ Nodetype/mixin are content annotation
▪ Repository uses them to
▪ Enforce structure ▪ Determine index rules ▪ Filter observation events ▪ Bundle Nodes* (new stuff!)
*OAK-1312
![Page 21: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/21.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Prefer Lucene Property Indexes
▪ Property Indexes ▪ Cause conflict in index data ▪ Causes commit root to be ‘/’ ▪ Stored as nodes ▪ Good for sparse and full sync case
foo=bar bar=/a/b
Commit Root
![Page 22: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/22.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Prefer Lucene Property Indexes …
▪ Lucene Property Indexes ▪ Async indexing ▪ No impact on commit root ▪ Compact storage ▪ Multi restriction evaluation
▪ Are not they async? ▪ In works near real time indexing*
*OAK-4412
foo=bar
Commit Root
![Page 23: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/23.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
FUTURE PERFORMANCE-RELATED Oak FEATURES work in progress!
![Page 24: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/24.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Mongo
Segment NSDocument NS
Sling
Segment NSDocument NS
Sling
Segment NS as Local Secondary Store▪ Segment NS act a local copy
of remote repository like a local git repo
▪ Updated via Observation
▪ Handles read call for “not so recently modified” Nodes
▪ Reduces read latency
▪ Configured to store certain path. Defaults to ‘/’ *OAK-4180
![Page 25: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/25.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016
Bundle Multiple JCR Node in one Document
*OAK-1312
+ book.jpg (nt:file) + jcr:content - jcr:data
Bundling Pattern + jcr:system/documentstore/bundlor + nt:file - pattern - [jcr:content]
{ "_id": "2:/test/book.jpg", "_commitRoot": {"r1560bfe1650-0-1": "0"}, ":pattern": {"r1560bfe1650-0-1": "[\"str:jcr:content\"]"}, "jcr:primaryType": {"r1560bfe1650-0-1": "\"nt:file\""} "jcr:content/jcr:data": {"r1560bfe1650-0-1": "\"bar\""}, }
▪ Store subtree aggregates of specific nodes in same NodeDocument
▪ JCR Node to Mongo mapping ▪ 1 JCR Node = 1 Mongo Doc ▪ 1 dam:Asset ~ 20 JCR Node ▪ 1M Assets = 20M Mongo Doc
▪ Nodetype as content annotation hint to optimize storage
▪ Benefits ▪ Lower size of _id index ▪ Less number of queries to read the micro
tree
Bundled NodeDocument
![Page 26: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/26.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 26
CODA The Whole Web, really?
![Page 27: Can we run the Whole Web on Apache Sling?](https://reader035.vdocument.in/reader035/viewer/2022062503/58f165d61a28ab456a8b4633/html5/thumbnails/27.jpg)
Can we run the whole Web on Apache Sling? Bertrand Delacretaz, Chetan Mehrotra, adaptTo() 2016 27
CODA
Thank you for attending! Chetan Mehrotra (@chetanmeh) Bertrand Delacretaz (@bdelacretaz)
REDDR enables (extreme) scaling driven by Sling resource/script resolution.
Building customized Sling instances is easy: provisioning model + trivial Dockerfile.
From servers to systems! Precise and focused repository types and operations improve performance.