from renamer plugin to polyglot ide

From Renamer Plugin to Polyglot IDE

Dmitry JemerovCTO, JetBrains

Monday, September 9, 13

IntelliJ IDEA: 2000-2013

• Started as a plugin for JBuilder

• Currently a product line of 8 IDEs, a compiler, a DSL workbench and a server-side code browser

• $xxK initial investment, $yyyM total revenue

• HEAD is a usable IDE every single day

Agenda

• IntelliJ IDEA over the years

• IntelliJ IDEA development practices

IntelliJ IDEA:Over the Years

• February 1st - company birthday

• Founded by Sergey Dmitriev, Eugene Belyaev and Valentin Kipiatkov from TogetherSoft

• IntelliJ Renamer, IntelliJ CodeSearch

Vista 1.0, Jan 2001

• PSI, VFS, commands

• Saved 2 months by not having plugin API

• Mentioned by Martin Fowler onhttp://refactoring.com/

Stella2.0, June 2001

• First external developers

• JSP, CVS, Ant, formatter, live templates

Pandora2.5, Dec 2001

• 13 new refactorings

• JUnit integration, one unit test in code

• Released 1 month after Eclipse 1.0

IDEA 2.6June 2002

• Company renamed to JetBrains

• JOLT Award in April 2002

Ariadna3.0, Nov 2002

• OpenAPI,

• 2 plugins (open-source): Starteam, Tomcat

• XML

• Real tests

• Oldest version available for download

Fabrique

• Framework and set of components for developing Web applications

Fabrique

• Visual IDE based on top of IntelliJ IDEA

Fabrique

• Drove a lot of platform API changes

Fabrique

• Project view, structure view, extensions

Fabrique

• Project view, structure view, extensions

• Canceled in 2005 before reaching 1.0

ReSharper

• Started in mid-2003

• Implemented in C#, di!erent architecture

• Initially used some parser/PSI technology from IntelliJ IDEA

Aurora4.0, Feb 2004

• Multiple-module projects

• On-the-fly inspections

• UI Designer

Pallada4.5, July 2004

• J2EE

• First two community-developed plugins

• Inspection Gadgets

• Intention PowerPack

Irida5.0, Aug 2005

• Custom language API

• JavaScript, Python

• Perforce + Subversion, open-source plugins

• 1M LOC

• 10 developers, no QA engineers

Demetra6.0, Oct 2006

• Core and Enterprise subteams

• TeamCity 1.0

• First plugin contest

Selena7.0, Oct 2007

• New caching VFS implementation

• Facets

• Ruby, Groovy

Diana8.0, Nov 2008

• Java-independent IntelliJ Platform extracted

• RubyMine 1.0 in April 2009

• Language-independent indices

• Language-independent debugger

Community EditionOct 2009

• Moved to git

• ~ 60% of codebase open-sourced

• Expected 30% drop in sales, got small gain

• Couple dozen external contributors

Maia9.0, Dec 2009

• Background indexing

• Artifacts

• PHP

Idea X, Xena10.0, Dec 2010; 10.5; Feb 2011

• Autopopup completion

• Android in Community Edition

• PhpStorm (May 2010), PyCharm (Oct 2010)

Nika11.0, Dec 2011

• UI redesign

• "core" package for Kotlin compiler

• AppCode (Oct 2011)

Leda12.0, Dec 2012

• Darcula

• External make

• UpSource, headless indexing framework

Android StudioMay 2013

• Built by Google with support by JetBrains

• Apache 2.0 licensed, no contracts and no money involved

• 500K downloads in first 3 weeks

CardeaVersion 13, in development

• ~ 25 developers

• ~ 5.6M LOC, ~3M LOC open-source

• ~ 25 developers

• ~ 100K daily active users, 72% Ultimate

• ~ 25 developers

• ~ 100K daily active users, 72% Ultimate

• 50% Windows, 30% Mac, 20% Linux

IntelliJ IDEA:Development Practices

Break the Rules

• No detailed planning

• No unit tests

• No QA

• No code comments or internal docs

• No API compatibility

• Many wheels reinvented

Release Planning

• Management sets only high-level goals and target date

Release Planning

• Each developer responsible for detailed planning of their subsystem(s)

Release Planning

• No iteration planning

Release Planning

• No iteration planning

• No feature specifications

Standup Meetings

• Daily, broken into sub-teams

• Over video conference between St.Petersburg, Munich and Prague

• Allow management, QA and writers to stay on top of dev activity

Development Flow

• Everything done in master

• Almost no long-lived feature branches

• Branches used only for releases

• Refactorings in incremental steps

• New features side by side with existing code, turned on by system property

Automated Testing

• Mostly data-driven acceptance tests

• Code before, action to perform, code after

• Few pure unit tests, little usage of mocks

• Test framework agnostic (JUnit, TestNG, Cucumber)

Continuous Integration

• Used TeamCity since day 0, CruiseControl before that

• Remote run not mandatory but recommended

Testing: Upsides

• Easy to write

• Often just copy code example from bug report

• Very little fragility when impl changes

• Tests written 8 years ago still valuable

Testing: Downsides

• Whole test suite (37K tests) takes 7 hours to run

• Multiple commits per test run

• Failures di"cult to debug

• Especially async code (indexing, UI)

• Tests stay red for weeks

Testers

• Had no testers until 2006

• Focus on manual testing and usability

• No "quality assurance" as such, relying more on CI tests and user feedback

update.bat

• Every developer starts their day with building IntelliJ IDEA from latest sources

• "Do not update" emails if something badly broken

• Core Java stu! always works

Early Access Preview

• Public free-to-use builds released every 1-2 weeks

• Broad community testing for features we don't use internally

• A few thousand active EAP users

• Licenses for most helpful participants

Public Issue Tracker

• Initially ITN, then JIRA, then YouTrack

• ~ 50 new issues per day

• Low noise but many duplicates

• Triaging incoming issues – almost full-time job

• Imbalance between developers

Exception Analyzer

• Separate from issue tracker

• Reports grouped into problems

• Semiautomatic merging of duplicates

• Exception duty rotated between developers, takes a few hours per day

Code Review

• Previously used manual review (mostly face-to-face) for merging into release branches

• Now reviewing all platform changes

• Using Crucible and hating it

• Need tools to see context of change

Support

• Until recently one support engineer (Serge Baranov) was covering all IntelliJ Platform-based IDEs

Support

• Best Technical Support (small to medium-sized business, SD Magazine Reader's Choice 2005)

Support

• Best Technical Support (small to medium-sized business, SD Magazine Reader's Choice 2005)

• Developers actively involved

Internal Docs

• XP's belief: comments are a code smell

Internal Docs

• Most 3rd party plugins are open-source, good examples

Internal Docs

• Improving docs does not increase average plugin quality

Internal Docs

• Improving docs does not increase average plugin quality

• Investing into docs to promote IntelliJ Platform to companies

Plugin API

• No separate facade, plugins access internal IDE classes directly

Plugin API

• No separate facade, plugins access internal IDE classes directly

• No way to repurpose IDE the way we did without breaking API compatibility

Community Plugins

• Large part of feature set

• Always by agreement with author

• Usually reworked at JetBrains

• Usually kept open-source

• Rewards through plugin contest

• Sometimes hiring authors

Build System

• Not using Maven or Gradle

• Still storing dependency .jar files in VCS

• JPS: tool that builds IntelliJ IDEA project from command line

• Same as external make in IDE

• Gant scripts to generate distribution

Extensibility

• Not using OSGi, Guice or Spring

• PicoContainer for dependency injection

• Home-grown extension point system

• Components and extensions to load specified in .xml files

Non-Java Languages

• Some Groovy for tests

• Live Edit's Chrome extension written in Kotlin and translated to JavaScript

• Scala plugin written in Scala

• Clojure plugin uses some Clojure

Summary

• Codebase repurposed far beyond original goals through relentless refactoring

Summary

• Full-time dogfooding is essential for maintaining quality and usability

Summary

• Full-time dogfooding is essential for maintaining quality and usability

• Lightweight process is enough for product development with no external stakeholders

yole@jetbrains.com@intelliyole

Develop with Pleasure!

from renamer plugin to polyglot ide

agenda intellij idea

intellij codesearch

idea x

loc opensource

fabrique framework

cardea version

togethersoft intellij

web applications visual

Technology

polyglot heaven

kató lomb - polyglot

renamer by den4b

tesla - jfokus...automatic local snapshot purging split...

be a polyglot programmer

polyglot metadata for hadoop

the polyglot project

Информатички практикум 1 ·...

web polyglot programming

management of polyglot persistent integrations with virtual...

polyglot programming @ jax.de 2010

polyglot persistence in azure

the polyglot - issue 3

polyglot mule transformers

hypno polyglot slide

polyglot - february 2009

polyglot - june 2009

polyglot poly paradigm

polyglot - february 2010

the polyglot - issue 2