Module 7 Fact or Fallacy Editable Notes
Module 7 Fact or Fallacy Editable Notes
Outcomes
A student:
› selects and processes appropriate qualitative and quantitative data and information using a range
of appropriate media INS11/12-4
› analyses and evaluates primary and secondary data and information INS11/12-5
› solves scientific problems using primary and secondary data, critical thinking skills and scientific
processes INS11/12-6
› communicates scientific understanding using suitable language and terminology for a specific
audience or purpose INS11/12-7
› uses evidence-based analysis in a scientific investigation to support or refute a hypothesis
INS12-14
Content Focus
The scientific process is the most powerful tool available for generating knowledge about the world. It
uses evidence and measurement to find truth and highlight misinterpretations and misrepresentations.
Science as a human endeavour is subject to human failings, which can contribute to fallacies,
misinterpretations and, on occasion, fraud. For this reason, scientific processes attempt to
compensate for human failings by questioning evidence, re-testing ideas, replicating results and
engaging with peer review in order to evaluate research.
Students investigate claims through conducting practical and secondary-sourced investigations and
evaluate these based on scientific evidence. They explore examples of scientific claims made in the
media and investigate the benefits of peer review.
Working Scientifically
In this module, students focus on selecting, processing, analysing and evaluating primary and
secondary data and information sources. Students communicate scientific understanding and
information about factual or fallacious claims. Students should be provided with opportunities to
engage with all Working Scientifically skills throughout the course.
Content
Testing Claims
Inquiry question: How can a claim be tested?
Students:
● plan and conduct an investigation based on testing a claim, and consider:
– validity of the experimental design
– reliability of the data obtained
– accuracy of the procedure, including random and systematic error
● using examples, evaluate the impact that sample selection and sample sizes can have on the
results of an investigation
● compare emotive advertising with evidence-based claims, including but not limited to:
– health claims on food packaging
– claims about the efficacy of a product
1.1 Plan and conduct an investigation based on testing a claim, and consider:
validity of the experimental design
reliability of the data obtained
accuracy of the procedure, including random and systematic error
Claim: There is 40% chocolate in each of Arnott’s choc chip cookies.
Context/Background information: Arnott’s claims that there is 40% chocolate in each
choc chip cookie. The cookie is composed of chocolate and the cookie dough. Prior to
opening each packet of choc chip cookies, there is a likely chance that crumbs or
chocolate of the cookie already fell off or broke apart such that the percentage of
chocolate in the cookie has been changed. However, the experiment is performed on
multiple packets of the same product such that the uncontrolled variable’s impact on
the results can be minimised.
Aim: To identify if the choc chip cookie is 40% chocolate.
Hypothesis: If the proportion of the mass of chocolate to the mass of the cookie is
40:60, there is 40% chocolate in the cookie.
Risk assessment:
Identify Rating Minimisation
Sharp tools may cut a person and cause bleeding. Medium Avoid holding
the sharp end towards people and pass tools by allowing people to hold the handle
instead of the blade. When someone is bleeding, apply bandage on the bleeding
area.
Glassware can fall off the table and shatter, causing a risk of cuts to skin and
bleeding, Medium Keep all glassware away from the edges of the table. If
glassware shatters, let the supervisor know and clean up the glass without direct
contact by using a dustpan. Wear in closed shoes whilst performing the experiment.
Variables:
Independent: the selected choc chip cookie
Dependent: the calculated percentage of chocolate in the cookie
Controlled: same technique of cutting the chocolate out, same spatula, same mass
scale, same 50mL beaker, same product, same brush
Equipment:
Spatula/chisel
Brush
Cookie (stick to one brand e.g. Arnott’s)
A 50mL beaker
Mass scale
Method:
Set up the experiment as seen in the diagram above.
Weigh out the mass of the cookie as a whole.
Weigh out the 50mL beaker prior to filling it with cookie crumbs and
chocolate.
Separate the chocolate from the cookie dough into its beaker by using the
brush and spatula.
Weigh out the mass of the beaker with chocolate.
Calculate the mass of the chocolate by the formula, mass of filled beaker –
mass of empty beaker = mass of the chocolate.
Calculate the percentage of the chocolate in the cookie by mass of chocolate ÷
mass of cookie as a whole × 100%.
Record the mass and percentage of chocolate in the cookie as a whole.
Repeat at least 3 times.
Results:
Trial Mass of cookie (g) Mass of chocolate (g) Percentage of chocolate in
each cookie (%) (2d.p.)
1 17.67 7.68 43.46
2 15.31 5.95 38.86
3 16.57 6.74 40.68
4 23.14 10.58 45.72
5 22.41 7.58 33.82
6 17.08 8.23 48.19
7 17.51 7.58 43.29
Discussion
Claims of ‘truth’
● Everyday confronted by claims, propositions and statements that various sources such as
media, educational institutions and our community understand to be true.
● However, when we examine and compare these claims by various individuals and
institutions in detail, there are usually different interpretation or contradictions.
● Informed decision making can difficult because some information can reflect reality or be a
misguided belief
Scientific research is undertaken with the explicit goal of solving a problem or developing a
technology, and along that path, new knowledge is constructed
Scientific method process is the best tool we have to test and evaluate claims on the truth.
However
Science can cause debate and discussion as scientists investigate phenomena
Outcomes of this discourse is that it allows individuals, communities, institutions and governments
to make decisions on the most accurate information available in order to promote progress and
benefit society
Sample size
● Determines reliability
● Problems occur when sample size is too small to eliminate random errors from swaying the
results e.g. flipping coin should be 50/50
● The greater the sample size, the more reliable the results are likely to be:
o Collecting results for large samples is extremely time consuming and often costly
o Scientists predict which sample size will be large enough to produce reliable results
while eliminating unnecessary sampling that would only confirm the data already
obtained.
o Determining optimum sample size depends on confidence level and confidence
interval
Confidence interval – a value, with an associated margin of error, that indicates the range in which a
random sample form the population is likely to fall into for a given measure or treatment
Determining optimum sample size typically depends on confidence level and confidence interval
Truth in advertising
● Science and advertising claims can contradict each other
● Primary reason of contradiction: both have different purposes i.e.:
o Purpose of science is to develop an understanding of the truth of the world and
empower people to make informed decisions and can be argued
o Advertising focuses on persuading people to purchase goods or services
o Advertisers employ a variety of different techniques, some take advantage of innate
(inborn, natural) human biases and vulnerabilities, to persuade customers to
purchase their products.
o Testing advertiser’s claims using the scientific process, it is possible to become
aware of biases and vulnerabilities and determine the real qualities of a product
o Scientific process helps to test the claims to become aware of real merits of the
claims.
Impacts on Investigations
Inquiry question: What factors can affect the way
data can be interpreted, analysed and
understood?
Students:
● using examples, justify the use of placebos, double-blind trials and control groups in order to
draw valid conclusions
Control groups
Control group – the group in an investigation that is used as benchmark in comparison to a second
that receives a prescribed treatment.
● It helps to determine whether a change in dependent variable is due to the change in the
independent variable or is it due to random circumstances.
● It is the benchmark in comparison
● Enable a scientist to determine whether a particular treatment or process applied to a
sample group actually has an effect or whether a change in a variable is the result of random
circumstances.
● Researcher studies the differences between the two groups: 1. treatment variable 2. no
treatment to determine effect of the treatment when compared to the normal conditions
● Used in an investigation that seeks to determine the efficacy
● E.g. 2 groups of petri dishes with colonies of the same bacterial disease growing on them
and then applying antibiotics on one colony while leaving the control group petri dish
untreated. After incubating both groups of petri dishes under the same conditions,
comparing them would determine the efficacy of the antibiotic
Green dot is the filter paper soaked in antibiotic treatment.
Purple dot is the control- filter paper soaked in distilled water
Placebos
● A medicine or procedure that has no therapeutic effect
● Inert (inactive) or pharmacologically (the characteristics and the properties of a drug that
make it medically effective) mild ‘medicine ‘that had a positive impact on a patient’s health
and wellbeing
● Prescribing a medication could increase a patient’s expectations of feeling better
● Subsequent physiological effects that helped the patient’s overall health even though
medicine was inert
● Commonly used in trials for medical treatments
● Impossible to make a person unaware the are participating in an experimental trial, they will
have the tendency to expect some sort of effect. This effect can be overreported or
sometimes can result on physiological changes even when no active treatment is given
● To account for this effect scientists give one group the true treatment and a second control
group a placebo and compare the reported effects
● If the true treatment group measured above the false positives/‘noise’ of the control
placebo group then the scientist can be reasonably confident that the treatment is
producing the effects observed
Double-blind Trials
● An experimental technique in which neither the participants nor the researcher in contact
with them are aware of who is receiving the treatment and who gets the control.
● When a researcher is conducting research on humans it can create bias when they are
communicating because the researcher may have expectations about the investigation
outcome
● Own expectations can bias results – recording results reactions, body language could give
away expectation to participant
● Therefore, double-blind trials must be conducted – the administrator is unaware of the
treatment a particular participant may receive. Gold standard in experimental research
usually used in medical research
● Can be used to conduct taste tests in the food industry by removing the labels from food or
beverage product and having a research collaborator unaware of which product is which
provide the research participants with food. This process helps determine which food
product customers prefer based on taste alone
● evaluate the impact of societal and economic influences on the collection and interpretation of
data, including but not limited to:
– predicting variations in climate
– suggesting remedies for health conditions
– manipulating statistical data
Introducing Bias
Scientists can be swayed by:
● Personal beliefs and motives such as earning a living
● Findings of their own investigations
● Social, cultural, economic factors that influence their lives
● Can lead to subsequent conclusions and generalisations
Bias in medicine
● Humans naturally tend to identify objects and people collectively rather than individually.
E.g. different species of trees with different shapes as ‘trees’
● Process of classifying is called stereotyping – classifying based on superficial characteristics
and generalising to all individuals that fit within the bounds of that classification.
o This stereotyping poses problems for medical professionals who have to make
recommendations for health conditions affecting a wide range of people.
o Effect- racial minorities and women → delayed diagnosis and treatment → higher
mortality rates despite relatively equal prevalence of a disease in both populations
Reasons for bias in medicine
● Observer-expectancy effect: an effect in which an individual making a judgement on the
presented data is biased by their preconceptions of the subject to be judged.
o Can have a negative effect in medicine
o E.g. Robert Rosenthal asked students to rate the performance of rats in a maze.
There was no difference in the rats; however, before running the maze, the students
were told one group of rats were smarter than the others. Later the students
consistently rated the so-called smarter rats better than the other rats when in
reality there was no noticeable difference between both groups
● Survivor selection: Research phenomenon where not all participants who start the study
complete it. Incomplete data on participants is removed to maintain the integrity of study
o In a random group of patients, those that survive longer are likely to receive trial
treatment and have their treatment assessed.
o Those that do not complete the treatment do not usually receive the full treatment
and are not assessed
o This can bias the outcomes and results in treatment being recommended for future
patients, even though it has minimal effectiveness
Bias in Statistics
Collection
Calibrated – to have correlated the readings of an instrument against that of a known standard to
ensure the accuracy of the instrument
Ideology – a set of beliefs and ideas that people have in relation to the way the world works
Analysis
● techniques used to analyse the data depends upon what information researcher needs to
find out
Presentation
● Data interpreted in a table with a lot of variables can be complex to read and important
information can be missed.
● Hence researchers often present data graphically but they can be misleading too.
− 1991 study that linked hormone replacement therapy to coronary heart disease
− the Mozart Effect on child development
Evidence-based analysis
● If two or more variables are correlated, it does not mean that one necessarily causes the
other to happen
● Correlation does not mean causation
o Correlation: connection between two or more things, or a measure of how things
are related
o Causation: where a change in one variable affects the other variable also referred to
as cause and effect
Terminology
● Common language can change from generation to generation or can be adapted to be used
in different contexts.
● Terminology is typically precise and narrow meanings that are applicable to a given field but
not wider community
● Scientific terminology – Specialised terms within a particular field –allow people to
communicate ideas and concepts easily and effectively and ensures universal understanding.
(without the need to premise it with detailed definitions)
● Common and scientific language is different – common is fluid and changes from one
generation to the next and scientific is more static to maintain consistency
● Can result in words having more than one meaning causing confusion – science literate - can
understand what a particular term means from that field
● analyse how conflicts of interest can result in scientific evidence being suppressed, misinterpreted
or misrepresented and discuss measures to counteract such conflicts, including but not limited to:
Conflict of Interest
● Science can be appropriated to further the interests of different social, political and
economic groups due to conflict of interest
● It happens when two groups have incompatible objectives.
● If these two groups have unequal power, then one group can use their power for personal
benefit.
● There are many industries that are well-established and have profitable practice, but
scientific findings are advising that they should be stopped or modified.
● E.g. fishing industry advised that they are overfishing in a particular area and that if they
continue fishing there will be not enough fish to restock the are and the industry will
collapse. However, this has happened and the industry increased fishing to make as much
short-term profit as possible before the collapse
● Industries continue a given practice despite the warning signs because:
o It could be very expensive to change business model or might go out of business if
they cannot adapt quickly
o Corporate model demands to get maximum profit for shareholders. But funding of
changing business model can reduce that dividend and probable reduction in share
price.
● Hence Corporations:
o suppress scientific research that conflicts with their practice
o engage in campaigns to misrepresent the findings
o create confusion in public to delay the effective implementation
● If scientists are working in the same industry in which the investigation is being conducted,
then the findings can be skewed or biased as their employment depends on the results that
are favourable to the profit of that company.
DDT levels
o I n humans DDT is carcinogenic (causing cancer) linked to increase risk of breast
cancer in women exposed to DDT prior to puberty
· D espite the known risks of DDT and its damage to the environment, the World Health
Organisation still allows DDT to be used to control mosquito-borne malaria even with proven
effectiveness of techniques such as drugs and community involvement
· Pesticide controversies still prevalent today. 2015 – International Agency for Research on
Cancer classified glyphosate, an active ingredient in common herbicides, as “probably
carcinogenic. Representatives that manufacturer herbicides said evidence in inconclusive
● describe the halo effect and, using examples, explain how the influence of positive perceptions
can result in the rejection of valid alternative perspectives, including but not limited to:
– celebrities endorsing products or viewpoints
– popular brand companies making misleading advertising claims
Buying a feeling
· Halo effect – psychological phenomenon where a person is perceived more
favourably/positively because he/she is well presented, has honest reputation, is trusted and
well regarded in society and people are more likely to follow their recommendations without
checking the validity of their claims.
· T ends to bias a person’s decision in favour of the speaker
· C an be problematic if the speaker is a public figure and use their status to support a
product/service/opinion that they have no knowledge or expertise in that field about
· E .g. brands can use advertisements that use positive emotional imagery and language to
make the consumer feel good whenever they see the brands to buy their products. Does not
provide factual information or scientific data as a means of persuading consumers
· T echnique often used to sway public opinion and is misleading when issues centre on
interests of individuals and the greater community
● using examples, analyse a pseudo-scientific claim and how scientific language and processes
can be manipulated to sway public opinion, including but not limited to:
– astrology
– numerology
– iridology
· practices that claim a definitive connection between variables when in reality, no such
connections have ever been conclusively determined by measurement or observation
· t hat claim to be following scientific processes but there is no valid link between independent
and dependent variable can be found and their cause and effect claim has no evidence.
· U se the ignorance of people that have low scientific literacy and are easily swayed by
explanations. Use complex language, terminology and processes to confuse and confound
followers
proposal but the institution must ensure that it is viable and will not cause any harm
to anyone
· A pproval procedure makes scientist accountable for the method that can only be
§ P
aper is found to be fraudulent and retracted
results and the old theory in mind, while doing their best to gather more evidence.
With such surprising results, they had an even greater responsibility to complete
thorough and careful testing to support their results and eliminate the possibility of
experimental error.
· However, there was new pressure to publish quickly since they were competing
with Jones’ group. It’s common for two or more groups to investigate the same
problem at the same time, and so science has a rule for assigning credit that the first
group to publish gets the credit for a new discovery. Thus, if either Jones or the
Pons/Fleischmann team spent too much time doing additional tests before
publishing, they ran the risk of missing out on the scientific credit.
· A dditionally, Pons and Fleischmann’s results suggested the possibility of profiting
through applications for power generation and so they were also concerned about
patent rights. The standards for scientific conduct and the time required for
thorough testing conflicted with the limited time caused by other concerns.
· J ones generously proposed that both groups submit their papers to the same
journal at the same time so that the credit could be shared. The proposed date of
submission was too early for Pons and Fleischmann to complete their testing.
Although this severely cut down their time, Pons and Fleischmann felt they had no
choice and agreed to the joint paper submission. They returned to the lab,
deter-mined to collect as much evidence as possible in the remaining time.
· D espite their agreement, Pons and Fleischmann submitted a journal article on
their own before Jones, breaking the standards for scientific behaviour by failing to
uphold the ethical standards set by the scientific community by breaking their
agreement and failing to sufficiently expose their ideas to testing. In their rush to
publish, they failed to perform some simple and obvious experiments, the results of
which would have provided key evidence about whether or not their cold fusion
hypothesis was correct.
For example, they could have:
o U sed a control by running their fusion cell with regular water in place
used a technique in which gases were allowed to escape the fusion cell
and then the amount of heat carried away by these gases was
estimated. If they had used a different technique in which no gases
escaped, they would have obtained more accurate results.
o S ought expert advice on their search for neutrons and other nuclear
products. Detecting these particles is not easy, and Pons had no
previous experience in this area. Moreover, the equipment Pons used
was not very sensitive. More sensitive equipment and more
experience operating it would have added credibility to their claims
Pons and Fleischmann submitted their paper, whose editor felt that
the weight of their potential discovery merited special treatment.
· I nstead of waiting for the scientific community to evaluate Pons and
Fleischmann’s radical claims or for the paper to be published, the University of Utah
held a press conference to announce the success of cold fusion to the world. Very
little information was given, but the two scientists and university officials repeatedly
emphasised the amount of energy that they thought their fusion cells could produce
in the future if the cells were made bigger and better. This gave the public a highly
optimistic view of cold fusion and aroused much excitement about the possibilities,
all before the scientific community determined if cold fusion was real
Review
· T he editor put the article through an abbreviated form of peer review—the
system science has in place to make sure journal articles meet good scientific
standards. Peer review can catch a variety of shortcomings in articles before they get
published e.g. peer reviewers normally notice when the evidence is insufficient to
support the authors’ claims and suggest that additional evidence should be collected
before publication. Reviewers also look for potential flaws in reasoning and
experimental design. Adequate peer review might have caught a serious flaw in their
logic—they had incorrectly calculated the magnitudes of the forces acting on
deuterium while inside palladium. The correct calculation revealed forces much,
much smaller—too small to push deuterium atoms close enough together to fuse.
However, this and other shortcomings in Pons and Fleischmann’s article slipped
through the rushed review. The reviewers had little time (one week) to scrutinize
the paper and did not review the changes the authors made in the second draft. This
short review period skipped some of the checks set up in the process of science, and
would eventually contribute to unnecessary confusion, and wasted time, energy and
money.
· M any scientists later criticized their lack of thoroughness as well as the quality of
their work. Pons and Fleischmann had not performed the experiments or the
analysis very carefully, and a month after the paper appeared, they had to publish a
list of corrections two pages long that included important modifications to their
data. However, before the scientific community got their chance to evaluate Pons
and Fleischmann’s ideas about cold fusion, the two brought their claims to the public
at large
· s cientists also tried to verify the work in other ways, scrutinizing the cold fusion
paper for potential sources of error. Many of the problems they noticed would likely
have been caught in a thorough peer review, and some mistakes were surprisingly
simple. For example, scientists noted that Pons and Fleischmann hadn’t stirred the
heavy water inside their fusion cells. Just as not stirring a pot of soup on the stove is
likely to leave some parts cold and others burnt, not stirring the water in a fusion
cell leads to uneven heat distribution and inaccurate temperature measurements.
Replicate
· E arly publicity, combined with curtailed peer review, caused some problems in
this case as the scientific community was in an uproar after the press conference.
Pons and Fleischmann had made extraordinary claims, but because the paper was
not yet available, the scientific community had no way to evaluate the work
presented in the paper—try to replicate it. While the process of science doesn’t
require that every experiment be replicated, with results as surprising as Pons and
Fleischmann’s—results that contradict a well-supported theory—it is mandatory.
Science aims to uncover the unchanging rules by which the universe operates,
meaning that a phenomenon should be replicated
· T heir paper was still several weeks away from publication, but scientists didn’t
let that stop them. Unauthorized copies of the article began to circulate within the
scientific community by fax—but when other scientists tried to set up the same
experiment, they found that the paper did not describe all the relevant details, as
many procedures are complex, and fully describing them would be too long. In these
cases, the authors are expected to furnish the relevant details upon request but Pons
and Fleischmann refused to provide these details. University of Utah officials later
revealed that they had instructed Pons and Fleischmann not to give away too many
details before a patent was filed. Withholding such information obstructs the
scientific process by preventing ideas from testing, but the scientific community
were not stopped
· S cientists continued to try to replicate the findings by trying out many different
experimental combinations, hoping to hit on the one used by Pons and Fleischmann.
Initial results were mixed. While most research groups reported seeing no evidence
for fusion, a few groups did claim to observe excess heat and/or neutrons coming
from their fusion cells. However, these groups conflicted with each other on the
conditions needed for fusion. For example, some found that months were needed for
the nuclear reactions to begin, others noted results in just a few hours. And often,
these groups couldn’t even replicate their own results. Some of the results were
simply mistakes. Several confirmations of Pons and Fleischmann’s results had to be
retracted due to errors e.g. forgetting to connect a key wire in the experimental set
up. Other discrepancies were due to differences in data analysis. Scientists collect
“raw” data—which must be analysed and interpreted before it can say anything
meaningful about the test e.g. many of the cold fusion scientists, including Pons and
Fleischmann, tried to gauge whether fusion was happening by measuring the heat
produced by the cell, which is not simple. The cell exchanges heat with its
surroundings, and some heat is carried away by escaping gases. The impact of these
factors must be carefully estimated and taken into account in the data analysis. If
two groups handle these adjustments differently in their analysis, they can come
conclude differently about the experimental results. Scientists can also make
different interpretations of the same analysed data. One group was able to show that
Pons and Fleischmann had misinterpreted the data from their neutron search. At
first glance, the data seemed to show clear evidence of neutrons—but neutrons
would lead to a series of reactions with the water surrounding the cell—and their
data was missing evidence of the last link in that chain of reactions. Further
investigation revealed problems with the equipment used to gather the neutron
data. Thus, it seems that Pons and Fleischmann’s data would have been more
reasonably interpreted as evidence of equipment error, not as evidence in favour of
the cold fusion hypothesis
· S cientists brought the most sophisticated and sensitive experiments to bear on
the questions of cold fusion, but were unable to find any evidence in support of it.
However, there was still the possibility that the finding couldn’t be replicated
because other scientists weren’t matching the conditions of the original experiment
exactly. Perhaps Pons and Fleischmann’s specific method in their experiment was
not revealed or not realised by themselves, and led to cold fusion. The best way to
test this would be to have independent experts search for fusion products coming
from Pons and Fleischmann’s fusion cells. Many scientists offered to collaborate, but
their offers were declined. Pons and Fleischmann were actively standing in the way
of tests that could have shed light on whether or not their hypothesis was correct.
With no resolution as to whether cold fusion was real, the scientific community
began insisting that these tests be done. There is no governing body of science that
could have forced Pons and Fleischmann to perform the follow-up tests but the
scientific community can apply pressure to uphold the standards of good science by
withholding esteem, funding, or jobs, and by being particularly sceptical of research
performed with lax standards. Only after significant pressure from the scientific
community did Pons and Fleischmann finally agree to perform the tests.
· O ne follow-up study involved searching for helium-4, one of the products of the
fusion reaction. Perhaps the searches for neutrons had come up empty because the
helium was stuck in the palladium rods and was not releasing its excess energy as
neutrons, but in another way. Pons and a group of other scientists decided to test for
helium in five palladium rods, only one of which had been used in Pons and
Fleischmann’s fusion cell. If fusion had indeed occurred, then only the fusion rod
should have elevated helium levels. To reduce the possibility of bias influencing
results, they decided on a “double-blind” study design. Pons would give the rods to
an intermediary, who would distribute segments of all five rods to six different
laboratories. Neither the intermediary nor the testing labs would know which rod
was which, and Pons wouldn’t be able to unintentionally tip off the laboratories
about it when he gave them the rods
· T he six labs tested each rod segment for helium and gave their results back to the
intermediary, who met with Pons to exchange the results and the rod information.
Pons had kept the detail of which rod had been used in a fusion cell at this time to
himself. He reviewed the helium data and saw that the fusion rod did not elevate
helium levels. Thus, the study did not support cold fusion.
· W hile these results might seem valid, Pons cast doubt on them when they were
finally publicized. He explained that the particular fusion rod he’d submitted for
helium analysis had not produced as much heat as he’d claimed at recent scientific
conferences. This was problematic on several levels. If the rod hadn’t had much
fusion going on in it, then that would explain why it didn’t have elevated helium
levels. But then Pons sabotaged the helium study by providing a bad rod and
reported such high levels of heat for his original fusion experiment. He may have
been manipulating the data.
Validate
· T here was no proper validation
· I n a last-ditch effort to validate the cold fusion results, fellow University of Utah
professor Michael Salamon was allowed into Pons’ lab to conduct experiments
searching for neutrons coming from Pons and Fleischmann’s own fusion cells, the
only experiment that could be sure to replicate the conditions of the original. During
his test, Salamon was unable to detect any neutrons.
· P ons tried to cast doubt on these results by claiming that the cells were not
producing excess heat (and hence, that fusion was not going on) during the
experiment except during a two-hour period that happened to coincide with a
power outage. However, one of Salamon’s instruments was still able to collect data
on neutrons during the outage. As expected, no spike in neutrons was observed.
Pons attempted to censure Salamon’s data by threatening legal action if Salamon did
not voluntarily retract his report. Such attempts to control information severely
violated scientific ethics and presented an obstacle to scientific progress. Despite all
the evidence against them—conflict with established theory, problems with the
original experiments, multiple failed replication attempts, and even tests suggesting
that the original experiments had produced no fusion—Pons and Fleischmann
refused to adjust their hypothesis about fusion occurring in palladium and broke
with standards for good scientific behaviour. Though scientists are expected to be
open-minded about new ideas, when multiple lines of evidence accumulate against
them, even the most intriguing hypotheses must be abandoned.
· T he scientific process had finally been able to sort through the evidence
regarding cold fusion. Few groups had found support for the hypothesis, and those
few had inconsistent results and could not reliably reproduce their findings. This
lack of replicable evidence was a major blow for cold fusion. If cold fusion works in
one laboratory under a certain set of conditions, we’d expect it to work in other
laboratories at other times under the same conditions. Hence, lack of reproducibility
is a serious problem for any scientific finding, casting doubt on the validity of the
original result and suggesting that there’s been a misinterpretation of what is going
on. In Pons and Fleischmann’s case, lack of reproducibility indicated that whatever it
was they had originally detected, it probably wasn’t cold fusion. This interpretation
is also supported by the fact that independent scientists couldn’t find any evidence
that Pons and Fleischmann’s own cells had actually produced fusion. In light of all
this evidence, most scientists consider Pons and Fleischmann’s results to be an
experimental error.
· A n error like this would normally be detected before it caused an uproar in the
scientific and broader communities. However, in the case of cold fusion, the checks
inherent in the process of science were weakened when Pons, Fleischmann, and
others caught up in the excitement broke with norms for good scientific conduct.
While the process of science is resilient to a single, or even a few divergences from
best practices, the convergence of multiple infractions can hinder the process. The
journal editor who allowed the original article to be published with minimal peer
review did not adhere to the standards science had set for such publications. Pons
and Fleischmann withheld experimental details from the community and tried to
shield their ideas from testing. They and the other scientists who “reproduced” cold
fusion, only to later retract their results, failed to perform adequate tests to evaluate
their ideas. And, of course, Pons’ behaviour during the helium experiment, as well as
the broken publication agreement with Jones, smacked of dishonesty. It’s important
to note that even with such unscientific behaviour, the process of science still
worked.
· T he scientific community had investigated Pons and Fleischmann’s claims and
come to the consensus that what had been observed wasn’t really cold fusion.
However, there was still a price to pay for this misconduct: time, energy, and
upwards of 100 million tax dollars were squandered on cold fusion. Pons and
Fleischmann also did damage that is harder to quantify. Perhaps most worrying is
the effect that this trouble had on the public’s perception of science. Pons and
Fleischmann’s unclear statements at the press conference, which emphasized only
the future benefits of cold fusion and not the early stage of the investigation,
contributed to the media hype and raised society’s expectations without warrant.
These unmet expectations coupled with accusations of fraud and dishonesty
damaged the public’s trust in science. Because science is so deeply intertwined with
the broader community, scientific misbehaviour has implications far beyond the
group of physicists and chemists who study cold fusion.
· D espite all this, some scientists continue to investigate the possibility of cold
fusion. Science doesn’t give up on ideas that have merit, even if they experience
setbacks. All scientific knowledge is tentative. So, though there is every reason to
think that what Pons and Fleischmann observed was not cold fusion, some a small
minority of scientists of the physics community continue to investigate whether or
not cold fusion is possible. But to convince the rest of the physics community, they’ll
need to find many lines of solid evidence to support their views
This case study highlights these aspects of the nature of science:
· T he scientific community is responsible for checking the work of community
· T hey refused to let other scientists to research for fusion products in their cell
Even journals that attempt to maintain high standards have problems with things like
fraudulent peer review rings and lack of proofreading, but predatory journals will
basically publish anything, and that’s damaging. Many reporters and scientists like
Smolyanitsky have intentionally caught journals in the act by submitting phony
papers to expose how lax the standards are. And this tactic has (unfortunately)
become an important tool for ferreting out scam journals.
The text of the Simpson, et al. paper wasn’t even coherent. Smolyanitsky just used a
bunch of jargon from the text generator SCIgen. Come on. Smolyanitsky told Vox, “I
wanted first and foremost to come up with something that gives out the fake
immediately,” he said. “My only regret is that the second author isn’t Ralph
Wiggum.”
Ever-increasing volumes
· Therefore, increase in the sheer numbers of research investigations that are submitted and
Scientific and wider community expect all research papers to be peer reviewed before
publication, but scientist needs to be reputable and already published and cited to be requested
to do peer reviews
· Increase of journal articles can also be a result of the increase ‘open access’ (journals that
allows researchers to pay to have their papers published) or ‘pay-to-publish’ – some have
reputation of predatory publications and unclear or accountable standards that claim to be peer
reviewed
· People have deliberately published openly fraudulent papers to show the lack of rigour and
inexperienced research scientists wanting their paper to be published, but not follow the
standard cross-checking processes such as peer review to ensure scientific rigour
· Scientists expected to write a review on the work being peer reviewed highlighting issues
· Creates pressure for scientists to verify the work of their peers by conducting replication
cross-checked by reproducing the investigation and/or than insufficient replication is being done
publication – survey of 1500 scientists 52%, agree there is a significant crisis
· Most trusted means of cross-checking and verifying the authenticity of research papers
before publication
· Established protocols around peer review require that a scientist’s ideas and work be judged