Everyone likes to think they are unique and nobody likes to be treated otherwise. However, one of the most robust bodies of research evidence in the social sciences shows that it can sometimes be harmful to treat people as special cases in their own right. In many situations where experts are required to make repeated routine judgments, a huge body of research has shown that the judgment of the experts is outperformed by equations. Examples where this research has been done include psychiatric diagnoses, medical diagnoses, college admissions, and parole board hearings. The equations are always based on the analysis of a long series of judgments; once the equation has been devised it can then be applied to new cases and compared with the judgments of the experts.
An equation can take one of two forms, broadly speaking. It can be based on identifying objective relationships between known predictors and an outcome measure; or it can be based on an analysis of how an individual has used information when making previous judgments (this is known as the 'bootstrap' model).
One of the potential objections to the use of such equations instead of experts is that the latter may have information available to them that has not been incorporated into an equation. For example, they may have interviewed an offender at a parole board hearing and got an impression of them that is not part of the predictive equation. However, this kind of situation has also been studied and it turns out that equations continue to outperform experts even when the latter have other information available. The key issue is that people, for various reasons, are inconsistent in the way they use information. Equations never are. But moreover, human judges - when given extra information - tend to treat everyone as a special case.
This important, but neglected, finding was first discovered by Paul Meehl and has subsequently been further investigated by many other people, notably Robyn Dawes. A brief description of some of the research can be found in my book Judgment and decision making: Psychological Perspectives.
We also have a tendency to treat situations as unique when perhaps we shouldn't. Another body of research has been observed that people are unduly optimistic about their ability to meet deadlines when engaged on some kind of project. Even when asked to state a project completion date for a situation when everything goes as badly as it possibly could, many people still fail to complete by that date. It has been suggested that people would be less optimistic if they were asked to think about similar projects that they have undertaken in the past. However, it turns out that this doesn't seem to have much effect, the problem being that people consider (wrongly) the current project to be a unique situation that the previous cases can't be applied to.
None of the above is new, but I got to thinking about these things again after reading Robert Shiller's book on the current economic crisis: The subprime solution. Shiller argues that our current problems stem from the fact that we have been living through a housing 'bubble', a period in which property prices have continued to rise far beyond any real value that those properties have. People continue to purchase houses and lenders continue to lend because everyone thinks that the price trend will continue. Of course, throughout history there have been many other such bubbles, characterised by the inevitable crash. But a point that Shiller makes is that whenever we are in the midst of such a bubble, even when a few critical voices dare to suggest that things can't go on as they are, and pointing to previous examples of bubbles, many 'experts' argue forcefully that we are living in a new and different kind of economic period where the previous rules don't apply.
The President of the European Association for Decision Making has written "It seems that each generation believes in its uniqueness to the extent of wanting to repeat the errors of the past". Trying to forsee the future will always be a difficult business, but perhaps we need to learn a little humility and realise that, when it comes to diagnoses and predictions we should not regard ourselves as somehow special.
An equation can take one of two forms, broadly speaking. It can be based on identifying objective relationships between known predictors and an outcome measure; or it can be based on an analysis of how an individual has used information when making previous judgments (this is known as the 'bootstrap' model).
One of the potential objections to the use of such equations instead of experts is that the latter may have information available to them that has not been incorporated into an equation. For example, they may have interviewed an offender at a parole board hearing and got an impression of them that is not part of the predictive equation. However, this kind of situation has also been studied and it turns out that equations continue to outperform experts even when the latter have other information available. The key issue is that people, for various reasons, are inconsistent in the way they use information. Equations never are. But moreover, human judges - when given extra information - tend to treat everyone as a special case.
This important, but neglected, finding was first discovered by Paul Meehl and has subsequently been further investigated by many other people, notably Robyn Dawes. A brief description of some of the research can be found in my book Judgment and decision making: Psychological Perspectives.
We also have a tendency to treat situations as unique when perhaps we shouldn't. Another body of research has been observed that people are unduly optimistic about their ability to meet deadlines when engaged on some kind of project. Even when asked to state a project completion date for a situation when everything goes as badly as it possibly could, many people still fail to complete by that date. It has been suggested that people would be less optimistic if they were asked to think about similar projects that they have undertaken in the past. However, it turns out that this doesn't seem to have much effect, the problem being that people consider (wrongly) the current project to be a unique situation that the previous cases can't be applied to.
None of the above is new, but I got to thinking about these things again after reading Robert Shiller's book on the current economic crisis: The subprime solution. Shiller argues that our current problems stem from the fact that we have been living through a housing 'bubble', a period in which property prices have continued to rise far beyond any real value that those properties have. People continue to purchase houses and lenders continue to lend because everyone thinks that the price trend will continue. Of course, throughout history there have been many other such bubbles, characterised by the inevitable crash. But a point that Shiller makes is that whenever we are in the midst of such a bubble, even when a few critical voices dare to suggest that things can't go on as they are, and pointing to previous examples of bubbles, many 'experts' argue forcefully that we are living in a new and different kind of economic period where the previous rules don't apply.
The President of the European Association for Decision Making has written "It seems that each generation believes in its uniqueness to the extent of wanting to repeat the errors of the past". Trying to forsee the future will always be a difficult business, but perhaps we need to learn a little humility and realise that, when it comes to diagnoses and predictions we should not regard ourselves as somehow special.
Posts
1 comments:
"Nothing special? " (My background is French)
I only read the first paragraph. What should we do with these "equations," when we understand that it is this "long series of [expert] judgments" behind them that have led our "generalist" species in the evolutionary predicament in which it is now? In my opinion, it is more than time to throw them in the waist basket. Not the "experts" themselves—since there will always be ways to rehabilitate their brain power for our survival, not for the "advancement of knowledge," once we understand that it is the progress that they have brought us that has led our "generalist" species in front of the evolutionary cliff in which we stand now,—but the "equations."
Einstein has told us that we shouldn't do what you are suggesting us to do: use the language that has created the problems, to solve them. This is why my heart stop when I ear Obama says that he understands the problem, and that he wants to throw money at it. It is indeed as if a well intention leading soul of the Black Plague era, would have suggested his people to stay close to the sick, and kiss the dying good buy. That is exactly what Obama and the "experts" advising him are planning to do, "kiss the dying good buy." Believe me, I know that if they do this, we will all die with them. It is only once we had understood the extrinsic cause of the plague that we could cure it. Today, the cause of our problems is "intrinsic" to our own species. That is why it is hard to figure out, since we all take ourselves for granted. We shouldn't. We are an unsound species, which was conceived with the seed of destruction embedded in its mentality, of which we experience the blooming at the moment.
The real problem is that nobody in the whole wide world, except me, maybe, and especially not any "expert," understand the problem, which is global and endemic to our species, and because it is global and endemic to our species. That is why I consider myself as being "special." Indeed, it is paradoxically because of "experts" and in spite of them, that after sixty-five years, I am still a well informed "generalist," as we used to be for millions of years, when our extinction was not a problem.
I am presently setting myself up to formulate this global problem that is confronting us now, in a way that will be solvable and in a language that will be understood by everybody, even specialists, I hope. If I succeed, there will be hope, since we do possess all the "expertise" we need to solve any problem that is confronting us, once it is defined. Wish "yourself" good luck, since if I don't define the problem, I don't know who will, as it should be, and you'll be affected by our crass ignorance, as everybody else will be. Understand the pressure in which I am? It will be formulated in Part III of. "Quantum Reality" see http://gaudwin.spaces.live.com/blog/cns!75D2857795790980!236.entry
PS If you decide to publish this comment, please edit it, and send a version to gaudwin@rogers.com.
Post a Comment