Picking the right architecture, sticking to it and evolving it

Petter Holmström, Vaadin Architectpetter@vaadin.com

Even when you are doing agile development, the architecture is something you want to design up front, at least to some extent. It is also something you want to get right from the beginning.

Do your homework!

What is the project scope?

● What is the customer expecting?○ Does the customer really know that?

● Changing or growing scope a common source of problems● Red flag: “nothing fancy, just a few screens”

What are the Non-Functional Requirements (NFRs)?

● Make sure the customer and you are on the same page● It is really easy for a customer to crank up the NFRs without thinking about the

consequences in terms of budget and schedule● NFRs form the boundaries of your architecture● Can be difficult and expensive to fix afterwards if you miss any of them or they

change during the project● Examples: Availability, Performance, Consistency, Testability, Security, Safety,

etc.

Initial design time!

Play it safe or experiment?

● Know your tools and make sure you have more than one● Do some research, are there any new patterns or technologies worth taking a

closer look at?● Can you try out a new approach in this project?

○ Yes, there are customers out there who are open to this. They are rare, but they exist.

To prototype or not to prototype

● High-risk features● Nontrivial things● Stuff you’ve never done before● Remember: a prototype is a PROTOTYPE - no more, no less

Think about the future… but not too much

● You don’t want to overdesign● You also don’t want to underdesign and end up refactoring everything later● How could the scope and the NFRs change during the next e.g. five years?● Is there any low hanging fruit?● Are there any high-risk features you may or may not end up needing?

Hey - what could go wrong?

● Your system becomes too slow when the number of features and/or users increases

● Your system becomes harder to maintain and extend as new features are added and existing ones are refactored

● Your multi-threaded features start to behave strangely when the load increases

● Your system or database does not scale well enough or does not scale at all● Integrations with existing systems turn out to be more complicated than

anticipated● You start to experience data integrity problems as your data model becomes

more complex

More things that can go wrong

● Problems in other systems cascade to your system or vice versa● Your monolithic system must be turned into a distributed one or vice versa● You have to replace the entire data storage system● You have to replace the authentication and/or authorization system● You have to replace the remoting protocol● Remember Murphy’s Law

My opinion

● It is better to over design a little and end up with stuff you don’t need, than to under design and end up refactoring everything later○ You do not necessarily have to implement everything from the start, but design in a way that

makes it easy to add afterwards

● Still, try to KISS as much as you can● Low coupling and high cohesion will take you far

Document the architecture

● An iterative process● Remember who you are writing for!● Keep it short and to the point● My preference: a wiki, bullet points and diagrams

What to include in the documentation?

Design decisions

● Document all your design decisions● If you had more than one alternative, list the others as well● Developers will ask why you did this instead of that● We have a short memory

NFRs

● Explanation of the NFR● Metrics and how to measure them● Tactics● Priority● Risk

Conceptual Design

● Helicopter view● System boundaries● External systems● Initial sub systems● Architectural styles

Integration Design

● All external systems you have to integrate with● How to integrate● Importance and risks● PoC?

Technology Stack

● Application/web servers● Database servers● Middleware● Frameworks and libraries● Licensing and versions● Customer approval

Technical Design

● How to implement your conceptual design using the technology stack● Deployment design● Concurrency design● Data design● Security design● List explicitly how each NFR is addressed by your technical design

Implementation time!

Set up a testing environment!

● All components should be present○ Remember: None, One, Many

● Replicated components should have at least two replicas● Will you be running performance tests?● Old hardware, RaspberryPi:s● Cloud● VirtualBox

Encryption

● “Add later when needed”● Generate certificates and keys● Distribute them● Try it out!● Yes, you can still change it later

Create a skeleton application

● Does not have to do anything but hum to itself● Most critical components and interfaces should be present● Run in your testing environment● Try out with encryption enabled

Think about the developers

● Make sure you can easily run the application on your workstation○ git clone; mvn clean install; run

● Mock external services if needed● Make sure the developers have access to the testing environment

Set up an initial testing harness

● Unit tests?● Integration tests?● Acceptance tests?● Performance tests?

Convince the rest of the team!

Teach and explain

● Crucial that the rest of the team understands and accepts the architecture● A complex architecture may seem like overkill in the beginning● Encourage people to challenge your designs!

○ Preferably already during the initial design phase○ You may have forgotten something○ They may know something you don’t○ Forces you to make valid arguments for each design decision

Strict or loose?

● Trade-off between developer productivity, consistency and future maintainability

● Strict○ Everything is always done in the same way○ Easy for junior developer or maintenance developers who don’t know the codebase too well○ Many code changes may be needed to implement a simple feature○ Developer productivity may suffer

● Loose○ Similar problems are solved in different ways depending on the context○ The developers must understand what they are doing○ Maintenance can become more difficult○ Scalability may suffer○ Increased developer productivity, at least in the beginning

Some examples

● Can you contact your persistence layer directly from the UI when retrieving reference data or do you have to go through the service layer?

● Do you have to use DTOs in your UI or is it OK for your service layer to return entities?

● Do you have to use Java interfaces for everything or can you let an object access another object via its class?

● Can a facade/boundary invoke another facade/boundary or does it have to invoke the service/control objects directly?

Stick to your architecture!

Talk with your colleagues!

● How are they planning to design and implement a certain feature?● Have they run into something in the architecture that makes their everyday

lives harder?● Or maybe they are even happy about the architecture!?!● Avoid a “my way or the highway” or “whatever I don’t care” attitude

○ (Can be hard at times, especially if you are tired and keep getting your designs challenged by

the rest of the team members)

● You want your team members to come and talk to you!● Keep an open mind - the architecture is not your baby● The success of the project and the team is all that matters

Perform code reviews

● Especially important in the beginning● Make sure the architecture is being utilized as you intended● Correct misunderstandings or shortcomings in your design early● Small changesets and quick reviews● Don’t let this become a bottleneck!

Pay attention to the tests

● How much time is spent on writing tests vs. writing production code?● What is the quality of the produced tests?● Are the tests affecting the quality of the design in some way?● Are they using introspection to access private members instead of using

package visibility?

Use static analysis tools

● You can’t review everything● Red flags:

○ Big classes○ Long methods○ Classes and packages with circular dependencies

Test!

● Test early● Test often● Get the future users to try out the system ASAP● The earlier you find a problem, the easier it is to fix it

Something’s come up...

The real world just called...

● Your architecture should make it easy to implement 90% of the features● The remaining 10% should be doable but may require more work● You have a problem if:

○ Your architecture makes it completely impossible to implement some of the features○ Your architecture makes it increasingly difficult to add new features to the system○ Your system fails to meet any of the NFRs

If you are lucky...

● You managed to identify possible problem spots in the beginning● You kept an eye on them and saw the problem coming● You already have a few cards up your sleeve● Your intended solution might turn out not to work, but you still have time to

come up with another

If you are unlucky...

● The problem took you by surprise● Lack of experience● Misunderstandings● Something that you thought would work didn’t● The requirements changed

Key questions to ask

● What caused the problem?● Why did you end up in this situation (the customer is going to ask you this)?● Do you think you know how to fix the problem?

○ The perfect solution might not be a viable alternative at this stage in the project. Be prepared to

compromise!

● Will you need help or can you fix the problem yourself?● Will the rest of the team be able to continue working while you are fixing the

problem?● How long will it take to fix the problem and will it work?● How will the budget and schedule be affected?

Finally

● If you are lucky, the required changes are small and can be performed without fuzz○ The architecture is evolving

● If you are unlucky, you have to have The Talk with the customer○ The architecture requires a redesign

● Many smaller changes is better than one major change● Getting it right is difficult! Remember that every failure is also a learning

experience!○ Although I prefer learning from other people’s mistakes… ;-)

That’s all folks!