Testbed Removal

Slinger & Jaap

Removing the Testbed

• Original plan– Create a testbed to compare “equal” solutions in

problem situation– Do measurements on

• Capacity• Performance• …

– Problem: define “equal”• Problem situations are unique, some patterns may fit better

than others, situational• Patterns are unique, some have different concerns

Evaluation Criteria

• Scalability• Evolvability• Mainainability• Extensibility• Autonomity• Composability• “Usability”• Implementation costs• Configurability• Domain specificity

• Interchangeability• Security/Securability• Robustness• Tailorability• Etc.

Replaced by…

• Full Qualitative Evaluation– Interviews with developers– Observation of pattern in use

• Next deliverable– Qualitative Evaluation Framework– Validate through interviews with pattern experts

at pattern conference

Also

• One pattern author is not enough– Come up with method for developing multiple

patterns for the pattern catalogue– Other researchers?– Students?

• Msc projects• Capita selecta

– Existing pattern literature

• How (often) are patterns used in practice?