cvpr2010: learnings from founding a computer vision startup: chapter 8: software engineering: doing...
TRANSCRIPT
![Page 1: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/1.jpg)
8. Software Engineering
![Page 2: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/2.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Research vs Developing a Product
![Page 3: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/3.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Challenge 1: Quality
Stability
Modularity
Robustness
Flexibility
Flickr: sbisson
![Page 4: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/4.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Challenge 2: Time to market
Flickr: Kristian M
Effort Estimation
Effort vs Reward
Priorization
Often in direct conflict with quality
![Page 5: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/5.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Challenge 3: Changing requirements
Flickr: 416style
A new competitor
Customer feedback
A new technology
![Page 6: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/6.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Challenge 4: User Experience
Flickr: geoffeg
![Page 7: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/7.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Challenge 5: Efficient Teamwork
Flickr: royskeane
![Page 8: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/8.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Summary Challenges
QualityTime To Market
Changing Requirements
User Experience
Efficient Teamwork
![Page 9: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/9.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Running the project
QualityTime To Market
Changing Requirements
User Experience
Efficient Teamwork
Needed: a process that is
- Flexible (Changing Requirements)
- Quickly leads to results (Time to Market)
- Little overhead (Efficient teamwork)
- Shows problems early on (Time to Market and Quality)
![Page 10: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/10.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Traditional way of running software projects
waterfall model (image source: Wikipedia)
What if requirements change halfway through project?
What if unexpected issues arise?
![Page 11: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/11.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
A better way to run software projects
Iteration 1
Iteration 2
Iteration 3
Do short iterations with all phases: iterative process
Examples:
- Scrum
- RUP (Rational Unified Process)
- ...
![Page 12: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/12.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
http://www.youtube.com/watch?v=IyNPeTn8fpo
Scrum et al. (Google Tech Talk)Ken Schwaber
![Page 13: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/13.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Scrum
![Page 14: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/14.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Backlogs
Requirements written as “User Stories”
Example: “iPhone app user can log-in with username and password, in order to log history of requests”
Effort for each story is estimated (how, see later)
Prioritized list of stories in the Product Backlog. Re-prioritize regularly.
For each sprint, just select top-n stories to Sprint Backlog
![Page 15: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/15.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
SprintsSprint = Iterations (2-4 weeks)
Meetings
Sprint Planning Meeting-> Team decides how much it can do in the Sprint (not the manager).Set a goal.
Daily Scrum15 minutes max (“timeboxed”). Each team member: What I am working on now? / what’s next? / problems?Check progress on burndown chart. If needed drop tasks.
Sprint Review Meeting
![Page 16: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/16.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
NO INTERRUPTIONS DURING SPRINT
“People need uninterrupted time to get things done”
why??
http://gettingreal.37signals.com/ch07_Alone_Time.php
http://www.joelonsoftware.com/articles/fog0000000068.html
With programmers, it's especially hard. Productivity depends on being able to juggle a lot of little details in short term memory all at once
![Page 17: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/17.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Estimations
Product Backlog: Stories are estimated roughlyUnit: Story points. A measure of “size” / ”complexity”Ex 1: 0, ½, 1, 2, 3, 5, 8, 13, 20, 40, 100Ex 2: 0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89
Team estimates using Planning Poker
Sprint Backlog: Each story gets assigned detailed tasks (in hours) at Sprint Planning
Longterm planning: velocity = story points / sprint“we need about 3 sprints to finish this module at our current velocity”
![Page 18: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/18.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Why a process at all? Why Scrum?
You need some process, otherwise you get lost. At the latest with more than 5 people more than 5 customers.
Scrum doesn’t get in the way. Simple. Lightweight. “Advanced common sense” *
Perfect for startups
* David Allen, in order to describe “Getting Things Done”
![Page 19: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/19.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Scrum is part of Agile
http://agilemanifesto.org/
![Page 20: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/20.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
More Agile Practices (or: more common sense)
QualityTime To Market
Changing Requirements
User Experience
Efficient Teamwork
Testing / Test Driven Development
Code refactoring
Continuous Integration
Pair Programming
![Page 21: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/21.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
If you don’t like Agile ...
http://www.joelonsoftware.com/articles/fog0000000043.html
![Page 22: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/22.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Consolidated must-haves
Source control (no-brainer) Subversion: http://subversion.tigris.org/Git: http://git-scm.com/
Automated TestingGoogle Test (http://code.google.com/p/googletest/)Often part of good frameworks (e.g. Rails)
Continuous IntegrationAutomated daily builds
CDash http://www.cdash.org/Cruise Control http://cruisecontrolrb.thoughtworks.com/ (Ruby only)
Hallway Usability TestingJust give your product to a few random people (in person) and observe them using it (before release)
![Page 23: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/23.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
On Deployment- We have 20 million items indexed. Need multiple machines to keep index- What if we have many requests, so we need to replicate the whole system
- Buying and maintaining so much hardware is expensive
Amazon Web Services (“Cloud Computing”) to the rescue
![Page 24: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/24.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
On Outsourcing Work
Do not outsource core competence
Overhead from time difference, communication, project management over distance, travel
Graphic design, administration etc. are things to outsource first (www.elance.com)
![Page 25: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/25.jpg)
What is special about Vision?In Terms of Engineering
![Page 26: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/26.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
What’s special about Vision (Consumer)
Software is a lot harder to do than e.g. a web-community in PHP (means you need more money also)
Lot’s of uncertainty and complexity
Still early stage of technology
Hard to find good specialists
![Page 27: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/27.jpg)
How we did it
![Page 28: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/28.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
How we did it
- Two Scrum-Teams in parallel - Use of Story points- 3-week sprints- 2-day break between sprints
Ec2, Trac + Agilo, C++, Rails, Googletest, SVN, CDash, Reviewboard
In-house: Vision, Rails, iPhoneOutsourced: Android, Symbian etc.
![Page 29: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/29.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Challenges we still face
User stories for system related topics
Uncertainty in Computer Vision related stories/tasks
Testing can still be improved
![Page 30: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/30.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Polar Rose: How we did it
- Two teams (infrastructure and vision) - Sys admin & dedicated Q/A tester- 2-3 week sprints- daily and weekly standup Skype meetings
Ec2, S3, C++, Python Java, Jira, Confluence, WordPressSVN, Bamboo
In-house: Vision, Infrastructure, Web, iPhone/iPad, Androidoutsourced: Symbian, small projects/prototypes
![Page 31: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/31.jpg)
Q & A
![Page 32: CVPR2010: Learnings from founding a computer vision startup: Chapter 8: Software Engineering: doing research and delivering a product](https://reader033.vdocument.in/reader033/viewer/2022052523/555a768fd8b42a972b8b5332/html5/thumbnails/32.jpg)
Lear
ning
s fr
om fo
undi
ng a
Com
pute
r Vis
ion
Star
tup
Resources IIKen Schwaber Scrum http://www.youtube.com/watch?v=IyNPeTn8fpo
Agile Manifesto http://agilemanifesto.org/
The Joel Testhttp://www.joelonsoftware.com/articles/fog0000000043.html
Wikipedia: Scrum http://en.wikipedia.org/wiki/Scrum_(development)
Wikipedia: Planning Poker http://en.wikipedia.org/wiki/Planning_poker