As an example, attention and memory are two of the most studied areas of psychology and cognitive neuroscience. They are the key features of clinical conditions like ADHD and Alzheimer's disease, not to mention they're routinely used as indicators of brand and consumer engagement by behaviouralists and neuromarketers. But do we really know what these words mean? Or to put this another way, when we use these terms do we mean the same thing?
What's the problem?
In 2014 the field of Psychology was somewhat rocked by a paper from the Open Science Collaboration (link here) which suggested that only around 50% of psychological phenomena were reproducible, i.e. every other law of human behaviour doesn't hold up. I say somewhat because anyone that has tried to teach an undergraduate research methods practical has seen even the most robust paradigms fail!
Since the Open Science Collaboration article came to light, academics have identified some of the prime culprits for this failing. Two of these are:
Publication bias
Previously, if ten labs ran an experiment and nine of these failed to find a significant effect, it's only the one successful result that gets published and discussed whilst the other nine studies that fail to find an effect never see the light of day.
The link between publications and career advancement
'Publish or Perish' is still a mantra practised by most academics. This has led to multiple instances of data 'manipulation' in order to get that coveted significant effect that can be published. Now, I'm not referring to fraud here (although that has happened), but instead the practise of tweaking statistics. This includes things like looking back over data to find and remove 'outliers' in order to strengthen your effect, or perhaps re-considering the variables in a statistical model.
I'm pleased to say that the advent of registered reports, data sharing, and a close examination of how statistics are employed in the field have gone a long way to address reproducibility. However, over the last few years I've found myself consumed by another problem that has received much less attention. Are we studying the same thing?
I spent over five years working with autistic individuals studying social cognition, particularly to what extent autistic individuals can understand different points of view - a process often referred to as Theory of Mind. Now, there are lots of ways to test for Theory of Mind ... but what became abundantly clear is that depending what test you use the result can be quite different.
Why is this the case?
Is this another example of publication bias and dodgy statistics? Although there are multiple factors to consider, I believe the biggest problem is that all-encompassing nebulous terms like attention, memory, or Theory of Mind hide a multitude of sins. Whilst many authors claim to be testing the same process, they are actually testing something different, or testing multiple processes at once which can change the outcome. Unfortunately, these subtle differences are hidden because we're using these large ill-defined labels.
Let's continue with the Theory of Mind example. Two of the most popular tests are false belief tests (i.e. the Sally-Anne task) and Reading the mind in the eyes.
The objective of the Sally-Anne task is to understand that someone has a different point of view from yourself, in this case a false belief. In the example above, the correct answer is Sally will look for the marble in her basket because that's where she left it. We all know the marble is in the box but Sally wasn't there to see Anne move it, so she'll think (incorrectly) think that the marble is still in the basket. In the Reading the mind in the eyes test, you have to show you can understand what another person is feeling by choosing which emotion the eyes are conveying. Both of these are popular Theory of Mind tests but they're testing different psychological constructs!
The image below shows data from a meta-analysis of Theory of Mind studies using fMRI - link here. It's clear that these two tasks activate different neural networks.
Why are two Theory of Mind tasks activating different brain networks? It's most likely because the Reading the mind eyes task relies on recognising and understanding emotions whereas the Sally-Anne task does not. This has been supported by studies which show that alexithymia (a condition related to emotion recognition in self and others) rather than autism explains poor performance on the reading the mind in the eyes task - link here. Does Theory of Mind require emotion recognition? To understand what another person is feeling, yes, but not to understand a different preference or point of view. If these two tests measure different things, why are they both called Theory of Mind tests? and which is most accurately measuring Theory of Mind?
What is the solution?
There are probably lots of solutions to this issue, but i'm going to discuss what I think is the best solution. Back in 2018 (weren't things simple back then compared to 2020!), Professor Russ Poldrack gave an interview entitled "Psychology will fail if it keeps using ancient words like attention and memory". In this article, Prof Poldrack points to the field of Physics stating that
"when they [Physicists] describe why things fall because of gravity, they’re not using those terms, they’re using math".
The idea of replacing psychological constructs with mathematical terms has been championed for some time. For example, sub-disciplines such as perceptual decision making, value-based decision making, and computational psychiatry have been around for decades. Each of these fields uses mathematical models to define neural processes, in turn these mechanisms can be combined to create a cascade of processes that leads to complex behavioural outcomes. Let's see an example using a neurocomputational mechanism called a prediction error signal to define Theory of Mind.
Prediction error signals have been investigated for decades using a full array of methods in almost every animal species. This is a highly reproducible brain mechanism signalling the difference between the expected and actual outcomes. Typically, this is used to describe first person learning, i.e. I expected this to happen but it didn't. However, you can shift the frame of reference to describe other people, i.e. I expected them to do/say this, but they didn't. We refer to this as a social prediction error because it's a learning signal about another person. Learning about another person's preferences and mental state can be classified as Theory of Mind, therefore social prediction error signalling might be a more accurate and clearly defined mechanism describing learning about others - you can see a full review I wrote on social prediction errors here.
Unlike the term Theory of Mind, social prediction errors are clearly defined (expected outcome - actual outcome) and highly reproducible.
The additional bonus of using mathematical terms to define behaviour is that they can be compared to see which mathematical model best explains behaviour. So using mathematical models we can answer the earlier question, which of the Theory of Mind tests is the best.
Yes, I've started with a bit of a rant, but this is something very important to me. To summarise:
Psychology will fail if we stick to using ill-defined antiquated terms.
The ability to link brain and behaviour requires biologically-plausible mathematical models.
We will only develop accurate brain-based biomarkers of clinical conditions by using mathematical models to describe clinical symptoms.
Do psychologists, neuroscientists, and behavioural scientists understand each other? They will when they speak the language of maths.
Comments