BiG Tips: With so much information available to us, I know it’s easy to get overwhelmed and knowing what’s true, worth spending time and energy on, and not. The exponential growth in information over the past decades, that many organisations are springing up to help us understand and manage information, disinformation, misinformation and BS.
These resources, I hope, will help you decide what to trust, believe, or not.
How to detect crap and BS
The Centre for an Informed Public (CIP), USA, University of Washington, they says is: “an interdisciplinary research initiative at the University of Washington dedicated to resisting strategic misinformation, promoting an informed society and strengthening democratic discourse.”
Created in 2019, it’s is a terrific body of research, resources and education.
Especially its Calling Bullshit, Data Reasoning in a Digital World. Here are their brilliant, 10 x hour long video lectures Calling Bullshit, Spring 2017 [Enjoy, go nuts!]:
“These lectures were recorded using multiple cameras and edited to form a video series. We have divided up every lecture into a set of a shorter segments; each segment should more or less stand alone on its own merits. The full playlist of all course videos is available on the UW Information School’s YouTube channel.
Lecture 1: An Introduction to Bullshit
1.1 Introduction to Bullshit.
Bullshit is everywhere, and we’ve had enough. We want to teach people to detect and defuse bullshit where ever it may arise.
1.2 Calling Bullshit on Ourselves.
Jevin uses data graphics to boast about explosive growth at our website callingbullshit.org — and Carl calls bullshit. Old-school bullshit versus new-school bullshit.
1.3 Brandolini’s Bullshit Asymmetry Principle.
Lecture 1.3 “The amount of effort necessary to refute bullshit is an order of magnitude bigger than to produce it.”
1.4 Classroom Discussion.
Students discuss: What is bullshit anyway?
1.5 The Philosophy of Bullshit.
How do we define bullshit? Does intention matter? Calling bullshit as a speech act.
Lecture 2: Spotting Bullshit
2.1 Spotting Bullshit
Discusses some ways to spot bullshit and challenges students to tell whether four nuggets of wisdom from the internet are true or bullshit.
2.2 Sounds Too Good to be True.
If a claim seems too good — or too bad — to be true, it probably is. An example involving recommendation letters, and the perils of confirmation bias.
2.3 Entertain Multiple Hypotheses.
The importance of generating and considering multiple alternative hypotheses. As an example, we consider why men cite themselves more than women do.
2.4 Fermi Estimation.
Using Fermi estimation to check the plausibility of claims, with an example of food stamp fraud. This example is treated in further detail in one of our case studies.
2.5 Unfair Comparisons.
In this segment on unfair comparisons, Carl explains why St. Louis and Detroit are not quite as bad as clickbait “most dangerous cities” lists portray them to be, and looks at the silly arguments over attendance at Trump’s inauguration. Also: how to call bullshit on algorithms and statistics without a PhD in machine learning or statistics.
2.6 Assignment: Bullshit Inventory.
In our first assignment, we ask students to take a week-long bullshit inventory of the bullshit they encounter, create, and debunk.
Lecture 3: Correlation and Causation
3.1 Correlation and Causation
Correlations are often used to make claims about causation. Be careful about the direction in which causality goes. For example: do food stamps cause poverty?
3.2 What are Correlations? Jevin providers an informal introduction to linear correlations.
3.3 Spurious Correlations?
We look at Tyler Vigen’s silly examples of quantities appear to be correlated over time, and note that scientific studies may accidentally pick up on similarly meaningless relationships.
3.4 Correlation Exercise
When is correlation all you need, and causation is beside the point? Can you figure out which way causality goes for each of several correlations?
3.5 Common Causes
We explain how common causes can generate correlations between otherwise unrelated variables, and look at the correlational evidence that storks bring babies. We look at the need to think about multiple contributing causes. The fallacy of post hoc propter ergo hoc: the mistaken belief that if two events happen sequentially, the first must have caused the second.
3.6 Manipulative Experiments
We look at how manipulative experiments can be used to work out the direction of causation in correlated variables, and sum up the questions one should ask when presented with a correlation.
Lecture 4: Statistical Traps and Trickery
4.1 Right Censoring
We look at a graph of age at death for musicians in different genres, and use this to illustrate the problem of right-censored data. We consider this issue in further detail in one of our case studies.
4.2 Means and Medians
Simple as it may sound, the difference between mean and median values offers fertile ground for cooking up misleading statistics.
4.3 p-Values and the Prosecutor’s Fallacy
Carl presents what he thinks may be one of the most important segments in the whole course: a discussion of the prosecutor’s fallacy. This logical fallacy is not limited to the courtroom: it underlies a very common misinterpretation of the p values associated with scientific experiments.
4.4 The Will Rogers Effect
Will Rogers purportedly quipped that when the Okies left Oklahoma for California, they raised the average intelligence in both states. The same phenomenon can arise in epidemiology and a host of other areas.
4.5 Jevin’s Turn
Jevin goes looking for bullshit and finds it — in Carl’s textbook. Jevin calls bullshit on Carl’s use of track and field records by age to illustrate senescence, and Carl tries to explain himself. This example is described further in another of our case studies
Lecture 5: Big Data
5.1 Big Data Introduction
We briefly introduce big data and provide a few the cautionary tales surrounding this recent phenomenon. Beware of those ponies…
5.2 Garbage In, Garbage Out
You don’t need a PhD in statistics or machine learning to call bullshit on big data. Simply by focusing on the input data and the results is often sufficient to refute a claim.
5.3 Big Data Hubris
We discuss the Google Flu Trends project and how it moved from being a poster child for big data to a providing an important precautionary tale.
We examine overfitting, the Achilles heel of machine learning. We illustrate overfitting visually, and consider and what to look out for.
5.5 Criminal Machine Learning
A recent paper claims that machine learning can determine whether or not you are a criminal from a photograph of your face. That’s bullshit. This example is described further in one of our case studies.
5.6 Algorithmic Ethics
We discuss gender and racial biases inherent to many of the machine learning algorithms and recommender systems prevalent in today’s technology, and encourage others to call bullshit on machine injustice.
Lecture 6: Data Visualization
6.1 Dataviz in the Popular Media
Until recently, the popular media made minimal use of sophisticated data visualization. People have not necessarily had time to hone their bullshit detectors for application to data graphics.
6.2 Misleading Axes
One of the most common abuses of data visualization involves the inappropriate ranges on the dependent variable (y) axis. Carl looks at a series of example, and explain why bar charts should include zero whereas line graphs need not — and often should not — do so. This example is treated in further detail in one of our callingbullshit.org articles.
6.3 Manipulating Bin Sizes
By binning data in different ways, bar charts can be made to tell very different stories. Here we consider an example from the Wall Street Journal.
6.4 Dataviz Ducks
Edward Tufte uses the term “ducks” to refer to data graphics that put style ahead of substance. We explain why, and explore a number of examples.
6.5 Glass Slippers
We propose the term “glass slipper” to describe to data visualizations in which the designer has taken a beautiful data design intended for very specific situations, and tried to shoehorn entirely inappropriate types of data into it. Carl considers examples including a periodic table of data science, a subway map of corporate acquisitions, a phylogenetic tree of internet marketing, and numerous Venn diagrams.
6.6 The Principle of Proportional Ink
Our principle of proportional ink states that when a shaded region is used to represent a numerical value, the area of that shaded region should be directly proportional to the corresponding value. We look at graphs that violate this principle and discuss how such violations can be misleading. This example is treated in further detail in one of our callingbullshit.org articles.
Lecture 7: Publication bias
7.1 Duck hunting
For last week’s homework assignment, students searched for examples of “duck” and “glass slipper” data visualizations. Carl and Jevin look at a few of the best finds.
7.2 Science is amazing, but…
Science is probably the greatest human invention of all time, but that doesn’t mean it doesn’t come with its share of bullshit..
Jevin discusses how spreadsheet errors reversed the conclusions of a high-profile paper that was used to justify austerity measures.
7.4 A Replication Crisis
Scientists have difficulty reproducing a surprisingly large fraction of the published literature. What is going on?
7.5 Publication Bias
Journals prefer to publish positive results and scientists prefer to submit successful experiments. This can be misleading given that we typically can look only at the published literature.
7.6 Science is not Bullshit
The subject matter of today’s lecture notwithstanding, science generally works pretty darn well. We can build airplanes and iPhones and save lives with antibiotics and vaccines, after all. Carl looks at five reasons why this is true.
Lecture 8: Scholarly publishing and predatory publishers
8.1 What Motivates Scientists?
Scientists are not purely seeking knowledge; like anyone else, they are also pursuing fame and fortune. If we can understand the incentives that scientists face, we can better understand why they do what they do.
8.2 An Overview of Scholarly Publishing
Levin talks about how scholarly publishing has become a big business, and describes the rise of the open access publishing model.
8.3 Predatory Publishing
We introduce the world of so-called predatory or otherwise questionable scientific publishers, and consider the reasons that authors publish in them anyway.
8.4 Reputable or Questionable?
We challenge our audience to distinguish papers published in reputable journals from papers published in journals from so-called predatory publishers.
8.5 Journal Spam
Carl explores the scourge of journal spam and some of the humorous ways that academics have fought back.
Lecture 9: Fake News
9.1 The Spreading of Fake News
We investigate some of the top fake news stories from the last year, and discuss how our digital environments facilitate the spreading of this information.
9.2 Fake News Definitions and Examples
We present examples and definitions of fake news. We explore how this white noise is being shared millions of times from both sides of the political aisle, making people lots of money, and fooling our search engines, which can make it difficult for democracy to run effectively.
9.3 The Ecology of Fake News
Fake news is not a new thing. It has been around for a long time — but social media, bots, and highly partisan environments are spreading fake news ever more readily.
9.4 Sharing as Social Signaling
We are all publishers. Often we share information as a signal of our group membership. Click-driven publishing models are facilitating the spread of this fake news.
9.5 Stamping out Fake News
Resources exist for checking and stopping fake news. Remember to question a source of information and do you best not to spread articles you haven’t checked. Consider supporting high quality journalism with a subscription.
Lecture 10: Refuting Bullshit
10.1 Four Rules of Calling Bullshit
We consider four rules for calling bullshit: Be correct; Be charitable; Be clear; Admit fault.
10.2 Reductio ad Absurdum
The method of reductio ad absurdum, in which an argument’s methods are shown to lead to ridiculous conclusions, is extremely powerful for refuting bullshit claims. We examine statistical projections of gold medal 100 meter times and the cognitive-emotional responses of a dead salmon.
10.3 Debunking Myths
Levin introduces Cook and Lewandowsky’s Debunking Handbook, and suggests a number of rules for how to successfully change opinions rather than reinforcing erroneous beliefs.
10.4 Deploying Null Models and Tracing the Origin of Falsehoods
First, models can sometimes be used to show that the evidence someone presents does not require the process for which they use it as evidence. Second, people are more readily convinced of a falsehood when they are shown where in the communication chain the falsehood arose.
10.5 Counterexamples and analogies
A single counterexample can demolish an elaborate argument, and a well-chosen analogy can draw out the fallacious reasoning underlying an argument.
10.6 Walk away; Conclusions
Carl stresses the important of being willing to walk away, and offers a few parting thoughts.”
RMIT ABC Fact Check They say: “Assesses the accuracy of claims by politicians, public figures, advocacy groups and institutions engaged in public debate. A collaboration between the ABC and RMIT University. Australian focus.”
Great Fact Checking Library Guides on Fake News, Fact checking toolkit and more.
How to spot fake news online, Australian Government
How Fake Science Can Fool You (YouTube) Wisecrack (USA). They say “Pseudoscience: It’s easy to mock it, and even easier to fall for it.”
Idea Labs and Echo Chambers – Tim Urban [USA] Wait But Why
Watch – The Psychology of Conspiracy Theories, BuzzFeed. They say: “People are susceptible to conspiracy theories by nature, and periods of uncertainty and heightened anxiety increase that susceptibility.”
An AI tool can distinguish between a conspiracy theory and a true conspiracy – it comes down to how easily the story falls apart, Timothy R. Tangherlini, in The Conversation
To combat conspiracy theories teach critical thinking – and community values, Thomas Roulet, The Conversation
How to spot fake news online Australian Government
University of Washington iSchool, Calling Bullshit lecture Series (You Tube):
The Spreading of Fake News
Fake News Definitions and Examples
The Ecology of Fake News
Sharing as Social Signalling
Stamping Out Fake NewsHow Fake Science Can Fool You (YouTube) Wisecrack. They say “Pseudoscience: It’s easy to mock it, and even easier to fall for it.”
Russell Brand especially in Russell Brand Blasts MSM Lies in Mindblowing Interview
MediaBias/Fact Check (USA) [FB Page] They say: “(MBFC), founded in 2015, is an independent online media outlet. MBFC is dedicated to educating the public on media bias and deceptive news practices. MBFC’s aim is to inspire action and a rejection of overtly biased media. We want to return to an era of straight forward news reporting.
*CHECK THIS OUT! Plug into their ‘search’ any media source WORLDWIDE for their bias rating: Left Wing – Right Wing and in between.
Search Australia and you get a comprehensive list of Australian media outlets and their bias rating. Their News Facts Networks claims “Verified Factual News from Media Bias Fact Check” NFN will only publish news summaries that are certified to be factual according to the editor of Media Bias Fact Check. While a source may be rated Mostly Factual by MBFC, we will ensure that the article is factual before publication. In the unlikely event that something is not factual or circumstances change we will immediately issue a correction.
See to RMIT/ABC Fact Check How large is Rupert Murdoch’s reach through News Corp in Australian media, old and new?
The Truth About Unbiased News (watch- You Tube) WisecrackMedicine and Health
Money and Finance
AFCA The Australian Financial Complaints Authority “considers complaints that previously would have been handled by the Financial Ombudsman Service, the Credit and Investments Ombudsman and the Superannuation Complaints Tribunal. We are the dispute resolution scheme for financial services. We consider complaints about:
- credit, finance and loans
- banking deposits and payments
- investments and financial advice
MoneySmart’s (ASIC) list of companies not licensed to give financial advice, or their Online stockbroking scams list. This website they say: “is for all Australians – young or old, rich or poor, investing or paying off debt. We offer free, independent guidance so you can make the best choices for your money. We are not selling you anything. And when life puts your finances under stress, MoneySmart is here to help you ride the storm. We have smart tips on dealing with the ups and downs of life: losing your job, having a baby, divorce or separation, buying a home, losing your partner and many more.” Here’s their warning list of common tricks:
- COVID-19 phishing scams Directing you to a fake website
- Stopping you pulling out of the deal
- Threatening legal action
- Using social media to approach you or your friends
- Artificially inflating the share price
- Passing your call along the line
- Calling or emailing you persistently
- Operating from overseas
ASIC suggests you ask questions and request information such as:
- What is your name and what company do you represent?
- Who owns your company?
- Does your company have an AFS licence and what is the licence number?
- What is your address?
- Is your investing prospectus registered with ASIC?
Check the legitimacy of the person offering the investment
The best way to protect yourself is through awareness and education. The Little Black Book of Scams is recognised internationally as an important tool for consumers and small businesses to learn about scams including:ACCC
- the most common scams to watch out for
- the different ways scammers can contact you
- the tools scammers use to trick you
- the warning signs
- how to protect yourself, and
- where you can find help.”
Types of Scams:
- Current COVID-19 (coronavirus) scams
- Attempts to gain your personal information
- Buying or selling
- Dating & romance
- Fake charities
- Jobs & employment
- Threats & extortion
- Unexpected money
- Unexpected winnings
- Flubot scams
Choice They say: ‘[is] independent and transparent… [its] one clear mission: to ensure the consumer voice is heard loudly and clearly… [it] empowers consumers to get the most out of all their purchasing decisions by providing a mix of advocacy and advice… the people’s watchdog… constantly looking out for misleading or unhelpful practices that stop you from getting the best deal. The leading consumer advocacy group in Australia… [it]campaigns to protect and assist consumers through changes to laws and industry behaviour… completely self-funded [it] prides itself on delivering expert, unbiased information you can trust.“
Victoria’s Consumer Action Law Centre is a great resource.
Franchises (mediation services), accc.gov.au
Horticulture (mediation services), accc.gov.au