Download - Prophet - Beijing Perl Workshop
![Page 2: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/2.jpg)
You may know me from...
RT (Request Tracker)JiftySVKHiveminderPerl 6T-shirts
![Page 3: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/3.jpg)
I’ve been hacking on an open source database
called “Prophet”
![Page 4: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/4.jpg)
It has an API like Amazon SimpleDB or Google App Engine’s...
![Page 5: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/5.jpg)
It’s designed for “team-scale” apps
![Page 6: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/6.jpg)
It’s built for P2P replication and
disconnected use
![Page 7: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/7.jpg)
But first, a brief digression...
![Page 8: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/8.jpg)
...about cloud computing
☁☔
![Page 9: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/9.jpg)
Living in the cloud =
sharecropping(佃农)
![Page 10: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/10.jpg)
(That’s bad)
☹
![Page 11: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/11.jpg)
The bad old days:
![Page 12: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/12.jpg)
Pic of sharecroppers
![Page 13: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/13.jpg)
You farmed land you didn’t own...
![Page 14: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/14.jpg)
...with tools you couldn’t really afford
![Page 15: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/15.jpg)
You paid for it with part of your harvest...
![Page 16: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/16.jpg)
It sounded like apretty sweet deal...
![Page 17: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/17.jpg)
...until things got bad
![Page 18: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/18.jpg)
(Things always got bad)
![Page 19: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/19.jpg)
![Page 20: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/20.jpg)
(Internet)
![Page 21: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/21.jpg)
(Cloud)
(Internet)
![Page 22: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/22.jpg)
In a bad year, you got further in debt tothe land owner
![Page 23: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/23.jpg)
![Page 24: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/24.jpg)
So, what does this have to do with software?
![Page 25: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/25.jpg)
The (more recent)bad old days:
![Page 26: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/26.jpg)
pic of mainframes
![Page 27: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/27.jpg)
You ran code you didn’t own on hardware you
didn’t own
![Page 28: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/28.jpg)
Things started to get better in the 1980s
![Page 29: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/29.jpg)
![Page 30: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/30.jpg)
Pic of PCs
![Page 31: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/31.jpg)
Users started to be able to make choices about computing...
![Page 32: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/32.jpg)
They weren’t all rosy
![Page 33: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/33.jpg)
Pic of BSOD
![Page 34: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/34.jpg)
Sometimes new versions of software
broke things...
![Page 35: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/35.jpg)
...leaving you locked in to old versions
![Page 36: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/36.jpg)
pic of win 31?
![Page 37: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/37.jpg)
Things got ‘better’
![Page 38: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/38.jpg)
rmsche
![Page 39: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/39.jpg)
Now, things are getting worse again...
![Page 40: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/40.jpg)
![Page 41: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/41.jpg)
What happens when your favorite service
goes down?
![Page 42: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/42.jpg)
pic of twitter being down
![Page 43: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/43.jpg)
...or stops accepting new signups?
![Page 44: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/44.jpg)
![Page 45: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/45.jpg)
...or starts making arbitrary choices about what’s ‘safe’ content?
![Page 46: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/46.jpg)
![Page 47: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/47.jpg)
...or breaks
![Page 48: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/48.jpg)
![Page 49: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/49.jpg)
You don’t own the services you use
![Page 50: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/50.jpg)
You probably don’t even have a contract
![Page 51: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/51.jpg)
When a service provider cuts you off,
you lose
![Page 52: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/52.jpg)
Not so secret shame:
I still need the cloud
![Page 53: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/53.jpg)
My calendar lives at google.com
![Page 54: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/54.jpg)
![Page 55: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/55.jpg)
I make a Web 2.0 todo list service called Hiveminder.com
![Page 56: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/56.jpg)
pic of hiveminder
![Page 57: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/57.jpg)
☣ Using hosted appsis going to hurt you! ☣
![Page 58: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/58.jpg)
What about Google Gears, Adobe Air, etc?
![Page 59: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/59.jpg)
Great. now you can use your word processer while you’re offline!
![Page 60: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/60.jpg)
Pic of wordperfect
![Page 61: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/61.jpg)
Real offline appsshould not need servers
![Page 62: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/62.jpg)
Real offline appsshould sync like you do
![Page 63: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/63.jpg)
Back to that database thing...
![Page 64: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/64.jpg)
Jesse Vincent
![Page 65: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/65.jpg)
Chia-liang Kao
![Page 66: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/66.jpg)
We work together
![Page 67: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/67.jpg)
CL lives in TaipeiJesse lives in Boston
![Page 68: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/68.jpg)
Sometimes we needto work face to face
![Page 69: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/69.jpg)
TPE - BOS:TPE - HNL:BOS - HNL:
9410 mi5,095 mi5,069 mi
![Page 70: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/70.jpg)
Step 1: Go to Hawaii for “work”Step 2: ???Step 3: Prophet!
Our Plan
![Page 71: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/71.jpg)
The Plan Backfired
We were there for 8 days
We wrote 8000 lines of Perl
We figured out step 2
![Page 72: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/72.jpg)
Step 2:
Build a Disconnected Syncable Database
![Page 73: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/73.jpg)
Fallacies of Distributed Computing
1. The network is reliable.
2. Latency is zero.
3. Bandwidth is infinite.
4. The network is secure.
5. Topology doesn't change.
6. There is one administrator.
7. Transport cost is zero.
8. The network is homogeneous.
![Page 74: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/74.jpg)
A grounded, peer to peer replicated, disconnected, versioned, property database with self-healing conflict resolution
Prophet
![Page 75: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/75.jpg)
What do all thosebuzzwords mean?
![Page 76: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/76.jpg)
grounded
Runs here
![Page 77: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/77.jpg)
grounded
Not here
![Page 78: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/78.jpg)
grounded
Runs at the edge
Doesn’t need to run in the cloud
Syncs with services you already use
(Adaptors talk to “Foreign Replicas”)
![Page 79: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/79.jpg)
Update any replica
Pull from any replica
Push to any replica
Publish a replica
Changes will propagate
peer-to-peer replicated
![Page 80: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/80.jpg)
Real-time replication is hard to scale
It only “works” with constant connectivity
I don’t have constant connectivity
Neither do you
Prophet sync can happen whenever
disconnected
![Page 81: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/81.jpg)
Every update is recorded as a change set
Change sets don’t lose any data
(so you can use them to go backwards)
All history is introspectable
Replication just replays changesets
versioned
![Page 82: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/82.jpg)
Atomic operations
CREATE, READ, UPDATE, DELETE, SEARCH
Record types can have optional validation and canonicalization
Records of the same type do not need to have the same properties
Add and remove properties at will
property database
![Page 83: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/83.jpg)
Remembers all conflict resolutions
Syncs all resolutions with your peers
Detects identical conflicts
Uses your peers’ resolutions to “vote” for the winner of a conflict
self-healing conflict resolution
![Page 84: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/84.jpg)
Working with Prophet
![Page 85: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/85.jpg)
RESTy API
GET /records.json
GET /records/Cars.json
GET /records/Cars/716499-5F9-4AC4-827.json
GET /records/Cars/716499-5F9-4AC4-827/wheels.json
POST /records/Cars.json
POST /records/Cars/716499-5F9-4AC4-827.json
POST /records/Cars/716499-5F9-4AC4-827/wheels.json
![Page 86: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/86.jpg)
RESTy API
Yes, we should be using PUT and DELETE
(or just provide an OpenRESTY adaptor)
Yes, you can have a commit bit and help us fix it :)
![Page 87: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/87.jpg)
Native API(Yes, the core is Perl.)
my $cli = Prophet::CLI->new();
my $cxn = $cli->app_handle->handle;
my $record = Prophet::Record->new( handle => $cxn, type => 'Person' );
my $uuid = $record->create( props => { name => 'Jesse', age => 31 } );
$record->set_prop( name => 'age', value => 32 );
my $people = Prophet::Collection->new( handle => $cxn, type => 'Person' );
$people->matching( sub { shift->prop('species') ne 'cat' } );
![Page 88: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/88.jpg)
What could you build with Prophet?
![Page 89: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/89.jpg)
A bug tracker: “simple defects”
• id. Status, Summary
• (Arbitrary other properties too)
•History
•Comments
•Attachments
sd
![Page 90: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/90.jpg)
Initialize
./bin/sd init
![Page 91: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/91.jpg)
./bin/sd ticket create -- summary="Can't sync sd with Google Code" status=new
Created ticket 5 (93BF979E-08C1-11DD-94C3-D4B1FCEE7EC4)
Create
![Page 92: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/92.jpg)
./bin/sd ticket search --regex publish
29 } new the online help doesn't describe publish
34 } new publish a static html view of records
35 } new publish should create a static rss file
List and Search
![Page 93: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/93.jpg)
./bin/sd ticket update --uuid 93BF979E-08C1-11DD-94C3-D4B1FCEE7EC4 -- status=resolved
Updates
![Page 94: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/94.jpg)
Bugs on my laptop aren’t interesting.
![Page 95: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/95.jpg)
Jesse
sd publish --to fsck.com:public_html/sd/
CL
sd clone --from http://my.com/~jesse/sd
Sync!
![Page 96: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/96.jpg)
Jesse
sd server
CL
sd pull --local
Hackathon mode!
![Page 97: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/97.jpg)
My project has a bug tracker
![Page 98: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/98.jpg)
Actually, mine use two:
• RT
• hiveminder.com
My project has a bug tracker
![Page 99: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/99.jpg)
Foreign Replicas
Prophet makes Foreign Replicas easy
SD gets them "for free"
![Page 100: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/100.jpg)
(Using only the public REST API)
It took an afternoon
Mirror an RT instance into SD
Share it with your peers using prophet
Sync changes back from your peers to RT
Supports Comments and Attachments
Wrote an RT Replica for SD
![Page 101: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/101.jpg)
(Using only the public REST API)
...and one for Hiveminder
![Page 102: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/102.jpg)
I can sync my bugs with RT or Hiveminder
![Page 103: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/103.jpg)
Actually, it’s better
![Page 104: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/104.jpg)
I can sync between RT and Hiveminder
![Page 105: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/105.jpg)
I can sync between two different RTs, too
![Page 106: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/106.jpg)
• Trac
• Launchpad
• Google Code
• SourceForge
• Bugzilla
• Jira
• GForge
• debbugs
• GNATS
• todo.txt
• Lighthouse
• Redmine
• FogBugz
• What else?
We need more replica definitions:
![Page 107: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/107.jpg)
What else can you use Prophet for?
![Page 108: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/108.jpg)
•CRM
•Bug tracking
•Sales orders
•Phone book
•Blog
•Trading Card Database
•Ideas?
All the databases you want while offline.
![Page 109: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/109.jpg)
How about a P2P BBS?
Prophet doesn’t need a server.
You can sync over sneakernet.
“Private” Social Networks
![Page 110: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/110.jpg)
A look inside Prophet
![Page 111: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/111.jpg)
Anatomy of a Prophet Replica
![Page 112: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/112.jpg)
The bits and pieces
Database UUID
Replica UUID
Record Store
Changeset Store
Resolution Database
Configuration metadata
![Page 113: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/113.jpg)
The Record Store
Stores individual records by type
Not guaranteed to have all old versions
![Page 114: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/114.jpg)
The Changeset Store
Stores every change to a set of records
Guaranteed to have all old changesets
Replaying all changesets will create an exact clone of the replica
![Page 115: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/115.jpg)
Replica Backends
![Page 116: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/116.jpg)
Filesystem
Readable
Flat files
Compact
Fast
(Not yet fully atomic)
![Page 117: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/117.jpg)
HTTP
Designed to let you “publish” databases
Flat-files, Currently read-only.
Same format as the filesystem replica type.
![Page 118: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/118.jpg)
Backends are pluggable!
The filesystem is cheap and easy
The filesystem is portable
Help us write new backends:
CouchDB, Postgres, SQLite, MySQL, S3, AppEngine, $YOUR_FAVORITE_DB
![Page 119: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/119.jpg)
Prophet is designed to sync with “other” databases and systems
They don’t need to support all of Prophet’s features - Prophet knows how to interpret mumbo-jumbo from the Cloud
Foreign Replicas will usually be app specific
All current examples are for SD
Foreign Replicas
![Page 120: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/120.jpg)
Synchronization
![Page 121: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/121.jpg)
Publish
Serialize and export all of a replica's resolutions and changesets
![Page 122: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/122.jpg)
Pull
Integrate unseen resolutions and then unseen changesets from a replica
![Page 123: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/123.jpg)
Push
Integrate new resolutions and changesets into a replica
![Page 124: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/124.jpg)
Conflicts
![Page 125: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/125.jpg)
Figures out the best resolution
“Nullifies” the conflict so the changeset can be cleanly integrated
Integrates the conflicting changeset
Records the resolution as a new changeset
Records the resolution decision in the resolution database
Resolving Conflicts
![Page 126: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/126.jpg)
Prophet has clever ways to figure out the best resolution.
If there are previous resolutions for the same conflict and a majority agree, use that
If the merger has specified a “prefer this side” choice, use that
Prompt the user to make a decision, giving them info about previous decisions for this conflict
“The Best Resolution”
![Page 127: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/127.jpg)
Scaling
![Page 128: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/128.jpg)
Scaling to giant clusters is boring
(Can I play the “They’re not Green” card here?)
Scales to many weakly connected peers
You are not Google
Does anyone here work for Google?
Current target is databases of O(50k) records
How does it scale?
![Page 129: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/129.jpg)
We have a political agenda.
Cloud computing is not Open.
APIs for “export” are not good enough.
You should always have full control.
You probably don’t need to store 10 billion records in one database.
Why not, then?
![Page 130: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/130.jpg)
Do you have 10 billion bugs, customer contacts
or sales orders?
![Page 131: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/131.jpg)
Can I come work for you?
![Page 132: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/132.jpg)
We would love a scalable, high performance Prophet backend
![Page 133: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/133.jpg)
Getting Involved
![Page 134: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/134.jpg)
Project Status
Simple, well-defined Perl API
RESTy web API (with microserver)
Fast, lightweight backend
Small, active dev community
Great test coverage
...less than great documentation coverage
![Page 135: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/135.jpg)
Better ergonomics
Improved search and indexing
(Including full-text indexing)
Client libraries for other languages
Proper security model
More apps
Our Plans
![Page 136: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/136.jpg)
Prophet
8225 lines of code and doc
2120 lines of tests
sd
2751 lines of code and doc
1121 lines of tests
Codebase
![Page 137: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/137.jpg)
Prophet is very young
Prophet designed in April
Prophet core implemented in April
SD designed in April
SD built in June and July
![Page 138: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/138.jpg)
We need your help!
Kick-ass functional and text indexing
Backend data store improvements
Slick GUIs for syncing
More Foreign Replicas for SD
Documentation improvements
A clever logo
New applications
![Page 139: Prophet - Beijing Perl Workshop](https://reader038.vdocument.in/reader038/viewer/2022103016/55583227d8b42ac6078b460e/html5/thumbnails/139.jpg)
Prophet
http://syncwith.us/prophet/download
SD
http://syncwith.us/sd/download
Getting Prophet