science & scientific method. daimihenrik bærbak christensen2 literature [wikipedia, 2005]...
Post on 21-Dec-2015
Embed Size (px)
Science &Scientific Method
DAIMI Henrik Bærbak Christensen 2
[Wikipedia, 2005]– Scientific Method.
[Carter, 1996]– The Scientific Method.
[Zobel, 1997]– Writing for Computer Science, Chap 8. Springer-
DAIMI Henrik Bærbak Christensen 3
Science is both a process of gaining knowledge, and the organized body of knowledge gained by this process.
The scientific process is the systematic acquisition of new knowledge about a system. This systematic acquisition is generally the scientific method, and the system is generally nature.
Science is also the scientific knowledge that has been systematically acquired by this scientific process.
DAIMI Henrik Bærbak Christensen 4
Science is the “opposite” of faith or belief. Often observations of the world can lead to false conclusions, or people generalize too quickly. People mistake truth for belief.
Classical trick: Which of the two horizontal lines is the longest?
DAIMI Henrik Bærbak Christensen 5
For centuries people believed in spontaneous generation. This was the idea that non-living objects can give rise to living organisms. It was common “knowledge” that simple organisms like worms, beetles, frogs, and salamanders could come from dust, mud, etc., and food left out, quickly “swarmed” with life.
For example: Observation: Every year in the spring, the Nile River flooded areas
of Egypt along the river, leaving behind nutrient-rich mud that enabled the people to grow that year’s crop of food. However, along with the muddy soil, large numbers of frogs appeared that weren’t around in drier times.
“Conclusion”: It was perfectly obvious to people back then that muddy soil gave rise to the frogs.
DAIMI Henrik Bærbak Christensen 6
Microscope in the 1600s: observation of bacteria growth in soup seemed to support spontaneous generation.
But – what exactly goes on?
“Life force” present in the molecules of all inorganic matter, including air and the oxygen in it, that could cause spontaneous generation to occur?
DAIMI Henrik Bærbak Christensen 7
Question: – The scientist then raises a question about what (s)he
sees going on. The question raised must have a “simple,” concrete answer that can be obtained by performing an experiment.
– For example, “How many students came to school today?” could be answered by counting the students present on campus, but “Why did you come to school today?” couldn’t really be answered by doing an experiment.
DAIMI Henrik Bærbak Christensen 8
Question: – Is there indeed a “life force” present in air (or oxygen)
that can cause bacteria to develop by spontaneous generation? Is there a means of allowing air to enter a container, thus any life force, if such does exist, but not the bacteria that are present in that air?
DAIMI Henrik Bærbak Christensen 9
This is a tentative answer to the question: a testable explanation for what was observed. The scientist tries to explain what caused what was observed.
DAIMI Henrik Bærbak Christensen 10
Hypothesis: • There is no such life force in air, and a container of
sterilized broth will remain sterile, even if exposed to the air, as long as bacteria cannot enter the flask.
DAIMI Henrik Bærbak Christensen 11
Aspects of Hypotheses
In a cause and effect relationship, what you observe is the effect, and hypotheses are possible causes. A generalization based on inductive reasoning is not a hypothesis. An hypothesis is not an observation, rather, a tentative explanation for the observation. – For example, I might observe the effect that my throat
is sore. Then I might form hypotheses as to the cause of that sore throat, including a bacterial infection, a viral infection, or screaming too much at a ball game.
DAIMI Henrik Bærbak Christensen 12
Aspects of Hypotheses
Hypotheses reflect past experience with similar questions (“educated propositions” about cause) and the work of others. Hypotheses are based on previous knowledge, facts, and general principles. Your answer to the question of what caused the observed effect will be based on your previous knowledge of what causes similar effects in similar situations. – For example, I know that colds are contagious, I don’t
know anyone with a cold, I was at the ball game yesterday, and I was doing a lot of yelling while I was there, so I think that caused my sore throat.
DAIMI Henrik Bærbak Christensen 13
Aspects of Hypotheses
Multiple hypotheses should be proposed whenever possible. One should think of alternative causes that could explain the observation (the correct one may not even be one that was thought of!) – For example, maybe somebody sitting near me at the
ball game had a sore throat and passed it on to me.
DAIMI Henrik Bærbak Christensen 14
Aspects of Hypotheses
Hypotheses should be testable by experimentation and deductive reasoning. – For example, throat culture and other tests yielded no
signs of a bacterial or viral infection, I have no fever or other signs/symptoms, and the doctor says my vocal cords are “swollen” in a way that would indicate overuse.
DAIMI Henrik Bærbak Christensen 15
Aspects of Hypotheses
Hypotheses can be proven wrong/incorrect, but can never be proven or confirmed with absolute certainty. It is impossible to test all given conditions, and someone with more knowledge, sometime in the future, may find a condition under which the hypothesis does not hold true.
– Example: Newtonian physics does not hold at relativist speeds…
DAIMI Henrik Bærbak Christensen 16
Aspects of hypothesis
That a hypothesis cannot be proven to be correct only incorrect has some consequences.
First, you should actually try to falsify the hypothesis and if this fails repeatedly only then should your confidence in it grow.
It is generally difficult for people to be brutally critical about their own work!
DAIMI Henrik Bærbak Christensen 17
Types of reasoning
– Inductive reasoning goes from a set of specific observations to general conclusions: I observed cells in x, y, and z organisms, therefore all animals have cells.
– Deductive reasoning flows from general to specific. From general premises, a scientist would extrapolate to specific results: if all organisms have cells and humans are organisms, then humans should have cells. This is a prediction about a specific case based on the general premises.
DAIMI Henrik Bærbak Christensen 18
Prediction:– Next, the experimenter uses deductive reasoning to test the
– If there is no life force, broth in swan-neck flasks should remain sterile, even if exposed to air, because any bacteria in the air will settle on the walls of the initial portion of the neck. Broth in flasks plugged with cotton should remain sterile because the cotton is able to filter bacteria out of the air.
DAIMI Henrik Bærbak Christensen 19
Testing: – Then, the scientist performs the experiment to see if
the predicted results are obtained. If the expected results are obtained, they support (but does not prove) the hypothesis.
– In science when testing, when doing the experiment, it must be a controlled experiment. The scientist must contrast an “experimental group” with a “control group”. The two groups are treated EXACTLY alike except for the ONE variable being tested.
DAIMI Henrik Bærbak Christensen 20
Testing: – Wide-mouth jars each containing a piece of meat
were subjected to several variations of “openness” while all other variables were kept the same.control group — These jars of meat were set out without lids so the meat would be exposed to whatever it might be in the butcher shop.experimental group(s) — One group of jars were sealed with lids, and another group of jars had gauze placed over them.
DAIMI Henrik Bærbak Christensen 21
A theory is a generalization based on many observations and experiments; a well-tested, verified hypothesis that fits existing data and explains how processes or events are thought to occur. It is a basis for predicting future events or discoveries. Theories may be modified as new information is gained. – This definition of a theory is in sharp contrast to
colloquial usage, where people say something is “just a theory,” thereby intending to imply a great deal of uncertainty.
DAIMI Henrik Bærbak Christensen 23
The essential elements of the scientific method are iterations and recursions of the following four steps:– Characterization – Hypothesis (a theoretical, hypothetical explanation) – Prediction (logical deduction from the hypothesis) – Experiment (test of all of the above)
DAIMI Henrik Bærbak Christensen 24
The scientific method depends upon the careful characterization of the subject of the investigation.
Observation demands careful measurement and the use of operational definitions of relevant concepts.
– When the terms used are formally defining, they acquire exact meanings which do not necessarily correspond with their use in natural language:
– for example, mass and weight are quite distinct concepts. On the surface of the Earth, the distinction may not be obvious, but in orbit around Earth, we are weightless and not massless.
– Isaac Newton: "I do not define time, space, place and motion, as being well known to all"
DAIMI Henrik Bærbak Christensen 25
A hypothesis includes a suggested explanation of the subject. It will generally provide a causal explanation or propose some correlation.
Observations have the general form of existential statements, stating that some particular instance of the phenomena has some characteristic. Causal explanations have the general form of universal statements, stating that every instance of the phenomena has a particular characteristic. It is not deductively valid to infer a universal statement from any series of particular observations.
DAIMI Henrik Bærbak Christensen 26
Prediction from the hypothesis
A useful hypothesis will enable predictions to be made, by deductive reasoning, that can be experimentally assessed. If results contradictory to the predictions are found, the hypothesis under test is incorrect or incomplete, requiring either revision or abandonment. If results confirming the hypothesis are found, the hypothesis might be correct, but is always subject to further test.
DAIMI Henrik Bærbak Christensen 27
Once a prediction is made, an experiment is designed to test it. The experiment may seek either confirmation or falsification of the hypothesis. Yet an experiment is not an absolute requirement. In observation-based fields of science, like astronomy, actual experiments are never performed, and control experiments are impossible.
Scientists assume an attitude of openness and accountability on the part of those conducting an experiment. Detailed recordkeeping is essential, to aid in recording and reporting on the experimental results, and providing evidence of the effectiveness and integrity of the procedure. They will also assist in reproducing the experimental results.
DAIMI Henrik Bærbak Christensen 28
Evaluation and Iteration
The scientific process is iterative.– Verification
• Science is a social enterprise, and scientific work will become accepted by the community only if they can be verified.
• Scientific journals use a process of peer review, in which scientists' papers describing experimental results and their conclusions are submitted to a panel of fellow scientists for evaluation
– Reproducibility • Your experiments should be reproducible so other scientists
can validate your hypothesis and conclusions.
– Reevaluation• All scientific knowledge is in a state of flux, for at any time
new evidence could be presented that contradicts a long-held hypothesis.
DAIMI Henrik Bærbak Christensen 29
Evidence and assumptions
Evidence comes in different forms and quality, mostly due to underlying assumptions.
An underlying assumption that 'objects heavier than air fall to the ground when dropped' is not likely to incite much disagreement. An underlying assumption like 'aliens abduct humans' however is an extraordinary claim which requires solid proof. Most extraordinary claims also do not survive Occam's razor.
Scientifc Process case
Debugging as science
DAIMI Henrik Bærbak Christensen 31
The scientific method may sound heavy, but often you can use the
observe hypothesis prediction experiment– and detailed record keeping
...in fast iteration to analyze precisely
One good example is debugging...
DAIMI Henrik Bærbak Christensen 32
Case: Haken-Blostein algoritm
Haken-Blostein is a complex algoritm for spacing sheet music.
Basically it treats the graphical objects like a system of connected springs with rods.
DAIMI Henrik Bærbak Christensen 33
Observation: Each invokation of the algoritm spaced the sheet music differently. Something gets accumulated or changed between invokations?
Hypothesis: The observed behaviour is due to some kind of accumulating defect in the code. Some variable(s) change value during invocation
But - Too vague to be useful...
DAIMI Henrik Bærbak Christensen 34
Parameters:– external algorithm parameters: a
– spring constants ci, rod sizes Rj
Hypothesis 1:– The algorithm parameter, a, changes value
Experiment:– Inspect the value of a before and after invokation– Easy: there is a dialogue box where it can be
Result: Hypothesis false.
DAIMI Henrik Bærbak Christensen 35
Hypothesis 2:– Spring constants changes
Experiment:– Inspect the values of a before and after invocation– Difficult:
• There are a lot of springs and no user interface. • Inserting debug output into code.• Manually comparing long lists of double values.
Result: Hypothesis false.
DAIMI Henrik Bærbak Christensen 36
Hypothesis 3:– Rod sizes changes
Experiment:– Inspect the values of a before and after invocation– Even more Difficult:
• There are a hundreds of rods and no user interface. • Inserting debug output into code.• Using diff-tool to compare very long lists of double values.
Result: Hypothesis accepted.– a few rod sizes somehow accumulate width…
… and computer science ?Software Engineering ?
DAIMI Henrik Bærbak Christensen 38
Computer science / software engineering has some twists…
Computer science deals more with artifacts than nature itself.
Thus the material we make hypotheses about and experiment with are constructed objects.– i.e. there exists a relation between hypothesis and the
material in the experiment that we often do not see in physics or biology…
– gives problems with reproducibility• who else builds systems like ours?
– scope of hypothesis (our system versus all systems?)
DAIMI Henrik Bærbak Christensen 39
Computer science/software engineering is a creative discipline just as much as a descriptive discipline.– You do not invent nature in physics or biology– but software engineers do!
Part of computer science is inventing stuff– meaning there is room for presenting hypotheses
without experimentation, only convincing argumentation.
Example: Inheritance in Simula– started object-orientation
DAIMI Henrik Bærbak Christensen 40
Another research tradition: Mathematics– No experiments but proofs.– [but proofs are subject to errors as well ]
Science in the small…
DAIMI Henrik Bærbak Christensen 42
Science in the small
From the studieordning:
– Ved udarbejdelsen af hovedopgaven skal den studerende demonstrere fortrolighed med almindelige principper for videnskabelig metode og færdighed i at anvende metoder og teorier til selvstændigt at afgrænse og behandle problemstillinger inden for området softwarekonstruktion.
DAIMI Henrik Bærbak Christensen 43
This is the yardstick that we will use to evaluate your thesis!– softwarekonstruktion: the focus is on constructed
software objects (not necessarily constructing them!)
– selvstændigt at afgrænse og behandle problemstillinger: you must define your focus yourself.
– videnskabelig metode, færdighed i at anvende metoder og teorier : adherence to the process is an important aspect.
DAIMI Henrik Bærbak Christensen 44
Note an important implication:– building application X or Y is not interesting in
The focus is on – A) the scientific process– B) the product is knowledge
This is usually in contrast to engineering degrees where the product is the focus…
DAIMI Henrik Bærbak Christensen 45
Now – you are not supposed to become world leading scientists in software engineering :
– we do no expect ground-breaking new hypothesis
– comprehensive, large sample, double-blind experiments
– large software artifacts being build
DAIMI Henrik Bærbak Christensen 46
But – we do expect– Characterization
• that you master the terminology and precision and clarity in statements and writing
• that you provide clear definitions in new concepts or ideas you present or introduce
– Assumptions and scope• that you state assumptions• that you know what “variables” you control and what remain
fixed• that you clearly scope your work (what do you not consider
part of the work, and argue how this influence your conclusions).
DAIMI Henrik Bærbak Christensen 47
But – we do expect– Scientific method
• that you either– try hard to formulate your problem as a hypothesis to be
evaluated (prediction and experiment)
– perform an experiment to falsify/confirm an existing hypothesis/claim in a ‘novel’ environment
• that software construction techniques are used as part of the process
DAIMI Henrik Bærbak Christensen 48
But – we do expect– Scientific progress
• that you not replicate well known facts in well known situations
• that you consult the (scientific) literature for inspiration and relations to your work (including web, products, etc).
– Scientific experiment• “Detailed recordkeeping is essential, to aid in recording
and reporting on the experimental results”• that you document both experiments and process in a clear
and unambiguous way so your observations are easy to understand and reproduce
• record ‘open issues’ as you go along!
DAIMI Henrik Bærbak Christensen 49
But – we do expect
– Scientific evaluation and iteration• that you present and discuss your work with your fellow
‘researchers’• Short papers and presentations at seminars
– include as part of your final report
• Peer reviewing groups• Consulting senior researchers (course teachers )
DAIMI Henrik Bærbak Christensen 50
If you do an interview– you analyse and conclude on the interview– ... but the interview raw data must be present also
Not only the result but also the steps in achieving the result is important (you may make an error that a fellow scientist will correct!)
DAIMI Henrik Bærbak Christensen 51
’we build a system type’
hypothesis: ”using our tool is better and easier”– better: ”will achieve result in half the used time”
how to handle this:– describe the current situation using a couple of
concrete scenarios• inductive reasoning
– describe same output using the tool • deductive reasoning confirms hypothesis
DAIMI Henrik Bærbak Christensen 52
Every time you make a decision you potentially invalidate your analysis (because you may make the wrong choice!)
Thus every decision you make must be supplemented with a rationale. Why did you choose as you did?
Designing a project
… or finding a problem…
DAIMI Henrik Bærbak Christensen 54
At least two ways to go…
Question– observer some interesting aspect
• from the work place; open source system; old project…
– turn the scientific machine• formulate hypothesis; make predictions; experiment
Evaluation– that is reproduce someone else’s results:– take existing hypothesis/technique– make predictions and experiment
DAIMI Henrik Bærbak Christensen 55
Process of making hypothesis
Hypothesis making is difficult ! Example:
– General idea:• “I would like to write something about vegetation in Africa”
– Specific topic• “Trees in Africa”
– Problem • “Imbalance between felling and re-growth”
– Hypothesis• “Urban wood consumption is more harmful to tree cover than
DAIMI Henrik Bærbak Christensen 56
– General idea:• “I would like to work with frameworks”
– Specific topic• “Frameworks for classic board games”
– Problem • “Hot spots required to cover large set of board games”
– Hypothesis• “It is possible to construct a framework for classic board
games with 90% code reuse that covers backgammon, chess, and checkers”.
It is only the latter that we can objectively measure…
DAIMI Henrik Bærbak Christensen 57
– General idea:• “We want to build software to plot weather data from the
school’s weather station”
– Specific topic• “Plotting software for weather data (time series)”
– Problem • “Adding hot spots to existing plot framework for weather
– Hypothesis• “It is possible to extend framework X with relevant hotspots
using patterns A, B and C to provide hot spots for weather data manipulation”.
Less “objectively measurable” but adds a lot of precision.
DAIMI Henrik Bærbak Christensen 58
Even small questions take a lot of time! – Better to answer small, specific, questions and add a
few more questions than– … drown in work on a very broad hypothesis.
Problem X turns out to be– problem A– problem B– problem C– problem D– ...
DAIMI Henrik Bærbak Christensen 59
Important to be explicit about assumptions and limitations:
Problem X consists of subproblems A..D. We will here delimit us to subproblem C.
DAIMI Henrik Bærbak Christensen 60
Health care scenario– [Nurse showing graph on TV using PDA]
Subproblems:– locating TV to use– combining ctrl and viewer for PDA and TV– steady-state interaction between PDA and TV– MFC combines ctrl and viewer in concrete code
... but most of these are independent problems and can be abstracted away...
DAIMI Henrik Bærbak Christensen 61
’Open issues’ is an important product!!!
Scientific process leads to– results (typically only on a single subproblem)– open issues
• that is: all the unsolved subproblems that we encountered during the process of trying to produce the result!
Scientists do not solve all the problems! The usually identify more problems than they
solve! (We don’t want to become unemployed)
DAIMI Henrik Bærbak Christensen 62
Example: Open Issues
Health care scenario– [Nurse showing graph on TV using PDA]
Experiment– .NET on Laptop and PDA
• Laptop keeps model and viewer; PDA keeps controller
Open issue– MFC’s architecture merge ctrl and view making
experiment really troublesome– analyze and propose alternative implementation.
DAIMI Henrik Bærbak Christensen 63
Another path is to evaluate existing hypotheses/techniques and report results– Using Quality Attribute workshops to assess qualities
in an architecture– Report experience on architectural documentation
using multiple view – Use architectural prototyping to assess buildability of
DAIMI Henrik Bærbak Christensen 64
Parts of the presentation is work by J. Stein Carter.
Parts are taken from Wiki pedia