apache geode - the first six months
TRANSCRIPT
The First Six Months
(incubating)
About
Anthony Baker (@metatype)
Engineering leader for Pivotal GemFire, committer on Apache Geode
How we transformed a hard-core commercial engineering team into an open source community-driven
powerhouse*
* forward-looking statement
What the government doesn’t want you to know about open source
6 reasons to be addicted to open source
7 things Lady Gaga has in common with open source
8 unbelievable things you never knew about open source
Why you should give up sex and devote your life to open source
11 ways investing in open source can make you a millionaire
http://www.contentrow.com/tools/link-bait-title-generator
Alternative titles?
(just kidding)
Topics• Introducing Geode
• Preparing for incubation
• Getting started
• Adopting The Apache Way
• Growing a community
Introducing Geode
Apache Geode is…
"…an in-memory, distributed database
with strong consistency
built to support low latency
transactional applications
at extreme scale.”
2004 2008 2014
• Massive increase in data volumes
• Falling margins per transaction
• Increasing cost of IT maintenance
• Need for elasticity in systems
• Financial Services Providers (every major Wall Street bank)
• Department of Defense
• Real Time response needs • Time to market constraints • Need for flexible data
models across enterprise • Distributed development • Persistence + In-memory
• Global data visibility needs • Fast Ingest needs for data • Need to allow devices to
hook into enterprise data • Always on
• Largest travel Portal • Airlines • Trade clearing • Online gambling
• Largest Telcos • Large mfrers • Largest Payroll processor • Auto insurance giants • Largest rail systems on
earth
China RailwayCorporation
5,700 train stations 4.5 million tickets per day
20 million daily users 1.4 billion page views per day
40,000 visits per second
*http://pivotal.io/big-data/pivotal-gemfire
Indian Railways
7,000 stations 72,000 miles of track
23 million passengers daily 120,000 concurrent users
10,000 transactions per minute
World: ~7,349,000,000
~36% of the world population
Population: 1,251,695,6161,401,586,609
China RailwayCorporation
Indian Railways
Application patterns• Caching for speed and scale using read-through,
write-through, and write-behind
• OLTP system of record with in-memory for speed, on disk for durability
• Parallel compute grid
• Real-time analytics
Preparing for Incubation
Some context
• 1M+ LOC, 1000’s of customers running business critical systems in production
• More than 12 years in development
• Large, multi-geo engineering team
• Established development practices
Why OSS? Why ASF?• Open source is fundamentally changing software
buying patterns
• Customers avoid vendor lock-in and get transparency, co-development of features
• It’s the community that matters
• ASF provides a framework for open source
Find a champion
(Roman)
and mentors too!
A rose by any other name…
Legal & biz stuff
• Source code clearance
• License compatibility, attribution
• Commercial features?
Cleaning up the source
• Get rid of internal dependencies
• Make sure the build is easy and fast
• Remove embarrassing source comments :-)
• Make testing easy
Writing the proposal
• Use a prior project as a template (lots of options)
• Identify background, rational, status, comparables, risks
• Who are the committers?
Submitting the proposal
• Is your source available? Not necessary, but helpful!
• Discussion will occur on [email protected]
• Wait for consensus and vote :-)
Getting Started
Setting up infrastructure
Review Board
https://issues.apache.org/jira/browse/INFRA-9536
Getting social
http://what’s important• Brief product description
• Community coordinates
• Email lists (dev, user), JIRA, wiki
• Getting started (one page)
• Building and running the product
• How to obtain the source (no binaries before a release)
• Roadmap, things to work on
Adopting The Apache Way
https://en.wikipedia.org/wiki/The_Scream#/media/File:The_Scream.jpg
http://theapacheway.com
“The Apache Way is sort of like Zen. It's something that's difficult to explain, has many
interpretations, and the best way to learn it is to do it.”
–Shane Curcuru, ASF
“Community, Merit, and Openness, backed up by Pragmatism and Charity.”
The Apache Way
• Community over code
• If it didn’t happen on the mailing list, it didn’t happen
• Rough consensus and working code: do-ocracy
• Decision model: +1, 0, -1 (also lazy)
Let it go!(sing along with me)
Apache stuff
• License headers (duh)
• Packages: org.apache.*
• Version: *-incubating
• Incubation reports
Policies• RTC or CTR?
• Branching (gitflow)
• Versioning (semver)
• Role of mentors (advisors or participants)
• Commit messages (GEODE-XXX: …)
• Design reviews
Committing> git flow feature start GEODE-‐nn
> rbt post -‐-‐repository geode
(wait for +1)
> git checkout develop > git pull -‐-‐rebase > git flow feature finish -‐-‐no-‐ff GEODE-‐nn
> git push
Pull Requestscontributor:
fork, clone, <write code>, create PR, wait for review
committer:
> git remote add github https://github.com/apache/incubator-‐geode
> git fetch github pull/n/head:feature/GEODE-‐nn > git checkout feature/GEODE-‐nn
> git checkout develop > git pull -‐-‐rebase > git flow feature finish -‐-‐no-‐ff GEODE-‐nn
> git push
Releasing
Growing a Community
Engage!https://hu.wikipedia.org/wiki/Jean-Luc_Picard#/media/File:Jean-Luc_Picard_2.jpg
Project roles• Contributor
• Lurking on mailing lists :-)
• Asking questions
• Answering questions
• Submitting bugs
• Submitting patches
• Committer
• PMC
Include your customers• Increase their opportunity to interact directly with
committers
• Do they want to contribute existing tooling?
• Questions they will ask:
• Are you still committed to the product?
• Why should I pay?
Community activities• ApacheCon! (and other conferences)
• OSCon, SpringOne2GX, QCon, …
• Local meetup groups
• Portland, Toronto, Palo Alto, San Francisco, London, Cork, Pune
• Virtual meetups (Geode Clubhouse) 2x / month
• Blogs, twitter, hackathon
Virtual meetups• Two formats
• Technical deep-dive on a specific subject
• Open mike, like a “standup” where participants get to bring up topics for discussion
• Note: any decisions must still be published on the dev list!
Be responsive
• More responsive and interactive communities have better engagement and retention
• On a temporary basis, find community activists to ensure questions, PR’s, and bugs are addressed quickly
• Soon this behavior becomes automatic
“Open source is where ecosystems are built.”
Ecosystem
YCSB
Metrics
• Committers: > 60
• Commits: 194 on /develop
• JIRA’s: 219 (open), 164 (resolved)
• Emails: 1658 (dev), 212 (user)
What’s next? TLP of course!
https://issues.apache.org/jira/browse/GEODE-1
