Psychology and Science: Answers to practice questions session 1
1. What is the relation between the behaviorist John Watson and (Logical)
Watson wanted to change psychology into a natural science, based entirely
on objective observations ('facts'). This corresponds exactly with the view of
positivism. According to positivism, the only real knowledge we can obtain
consists of objective facts based on observations. As a result, theoretical
ideas that are not directly linked to observations are not scientific. In the
same way, psychology should be based only on observable relations between
stimuli and responses, according to Watson. Furthermore, he rejects the
introspective methods of Wundt and Titchener, and the psychoanalysis of
Freud. Each of these methods and ideas consist of unobservable theoretical
concepts like consciousness (Wundt, Titchener) or unconscious motives
(Freud). They are unscientific because they are unobservable, according to
Watson. So, Watson is in full agreement with positivism.
2. What was the outcome of the replication studies performed by Aarts et al. (2015)?
What would be the effect of this outcome on the reputation of psychology as a science?
Aarts et al. (2015) performed replication studies for 100 psychological
studies published in one year in the best journals on experimental
psychology. They found that most of them (in between 50% to 70%) could
not be replicated. That is, the significant effects as reported in these studies
were not found. They also found that in all successful replications (around
36% in total), the reported effect size was about half in size compared to the
effects reported in the original studies.
This outcome has a very negative effect on the reputation of psychology as a
science. A science should be able to replicate its findings, because that
makes these findings, and thus the science, reliable. This is needed so that
other people (scientists, professionals or other people) can trust the results
obtained by this science. That is, they can use it, e.g. in their own work.
3. What is the problem with performing many studies and/or many data analyses and
using 'statistical significance' as a measure of reliability?
Statistical significance gives the probability that a result obtained in an
experiment (e.g. the difference found between the experimental condition
and the control condition) is due to chance instead of to a real difference
between these conditions. If this probability is low (say < 5%) you accept
that the outcome is the effect of a real difference between these conditions,
instead of due to chance (in which case the result is a false positive result).
However, there is of course a 5% probability that the result is indeed due to
chance after all. The problem with doing more experiments and/or more data
analyses is that the probability increases that at least one of these produces
a significant result that is due to chance. In other words, the probability
increases that you do get a false positive result. The more analyses you do,
the higher this probability becomes. So, the results become less reliable. 4. The committees that reported on the fraud case of Diederik Stapel concluded that
there was a culture of 'sloppy science’. Describe in your words what they meant with
The term sloppy science refers to the fact that the people who worked with
Stapel did not conduct as scientists should do. They were uncritical of their
own work and methods and were basically interested only in getting the
results they wanted or expected. So, they were only interested in
'verifications' and not in studies that could have shown a negative outcome.
To achieve the result they wanted, they designed the studies in such a way
that a negative result could in fact not occur. Or, they continued to do the
same study until it gave the desired results. Or, they eliminated subjects or
parts of the data that did not fit with the desired result. Hence, they made
their hypotheses 'immune to the facts'.