Liars don't statistify, but statistifiers sometimes do.
ElectricAye
Posts: 4,561
Is Science failing to face the shortcomings of statistics?
I know I've never been able to make head or tail of this stuff.
http://www.sciencenews.org/view/feature/id/57091/title/Odds_Are,_Its_Wrong
I know I've never been able to make head or tail of this stuff.
http://www.sciencenews.org/view/feature/id/57091/title/Odds_Are,_Its_Wrong
Comments
When I was studying experimental psychology, one of our lecturers couldn't replicate any of the experiments on subliminal perception performed by NF Dixon, and repeatedly got non-significant results. The effect was widely accepted at the time, and he couldn't get his papers published - he used to joke about publishing a Journal of Non-Significant Results. He eventually did manage to get some papers published. It's still a controversial area.
Being an outsider to statistics, I've always been fascinated by how researchers pick and chose what to dismiss and what not to dismiss as significant.
Quoting the article:
There is increasing concern, declared epidemiologist John Ioannidis in a highly cited 2005 paper in PLoS Medicine, that in modern research, false findings may be the majority or even the vast majority of published research claims.
Of course, the grand irony of the research done by John Ioannidis is that it, too, came under fire for using faulty statistics and/or reasoning:
Quoting the same article:
"Ioannidis claimed to prove that more than half of published findings are false, but his analysis came under fire for statistical shortcomings of its own. It may be true, but he didnt prove it, says biostatistician Steven Goodman of the Johns Hopkins University School of Public Health. On the other hand, says Goodman, the basic message stands. There are more false claims made in the medical literature than anybody appreciates, he says. Theres no question about that."
Gosh, and if the experts can't get it right... where does that leave little ole me? Could it be said that scientific truths have to hit us in the face before they can be accepted by the general community? Is it possible that subtle effects, phenomena that wiggle around on the statistical edge of things, are always going to be difficult to deal with? I'm just asking.
When Joe Public hears scientists bashing each other over their results and the methods it becomes less and less obvious who you should trust with "the facts" and who not.
Is asprin safe? Is Prozac safe? does it work? Look at the arguments raging around the evidence for global warming.
In the end Joe Public gives up, any quacks remedy is as good as what any doctor says. Any paranormal explanation is as good as anything so called scientists can expound. Creationism has as firmer ground to stand on as evolution.
Science gets undermined, the search for knowledge loses its importance, funding gets cut, we enter another dark age.
An example is the Space Shuttle Challenger disaster. The O-Ring failure can be partially attributed to a term called "normalization of deviance". Designers of the O-Ring began altering the expected success rates of the O-Ring based upon successful use in missions. Even though unexpected defects caused by lower launch temperatures were found in many recovered O-Rings, the fact that they "still performed the purpose" was good enough for design engineers to state that the success rates were higher than original estimates. This is the equivalent of saying that because you drove to work this week without an accident, you won't get in one next week.
The "normalization of deviance" comes into play because the slight deviation from spec of the true performance of the O-Ring soon became the new "spec". The launch temperature the morning of the disaster was much lower than normal, but since the range of acceptability for the O-Rings had been opened up, this did not stop the launch.
I would encourage everyone to read "What Do You Care What Other People Think?: Further Adventures of a Curious Character" by Richard Feynman. The second half of the book is dedicated to his time spent on the Rogers Commission investigating the disaster. Also, Feynman's report submitted to the commission (which was only included as an appendix due to it's "honesty", read the book for more details on that) is a very good read for any engineer. Sorry for the plug, but it is related to my post and I am a major Feynman fan.
What a marvellous expression.
Seems to some up all that has gone wrong with society in the western world since the nineteen sixties:)
Couldn't agree more! Science once enjoyed an elevated position. Now it is the butt of jokes. Numerous acquaintances of mine now disregard all scientific "findings" because of their experience with a few - most especially Global Warming - which has now morphed into Global Climate Change because sometimes temperatures go down. It's fabulous when your pet theory accounts for all observed behavior other than nominal. It's a statistical goldmine.
When I was just starting out on this planet, back in the 50's and 60's the human race had a great deal of optimism, mostly brought on by the wonders of science.
We had only relatively recently harnessed electrical power, invented the radio and TV, learned to fly, created the atomic bomb and the promise of unlimited nuclear fuel. Invented plastics and electronics and computers,given everyone an automobile a washing machine and countless other gadgets. Meanwhile medicine had conquered polio and smallpox. With the crowning achievement of putting man on the moon. Humans, it seemed, could and would be able to anything they wanted in a bright and glorious future brought to us by the wonders of science.
By 2000 this had all turned bad.
Over population, dwindling resources in the face of exponentially growing demand, AIDS, mad cow disease, pollution, climate change, the realization we are stuck on this little globe with an array of ever increasing problems that science seems powerless to solve and in many cases helped create.
Better turn to the church and mystics, these scientists have conned us.
I'm not so concerned about Joe Public. Honestly, Joe Public is going to believe whatever he wants to believe no matter what the evidence might be. History is proof of that. The biggest problem is when trying to make sense of things in the commercial world. Sometimes the PhDs in statistics can crank out numbers but nobody knows what the meaning of those numbers might be. Even when you get down on your hands and knees and beg the PhDs, weeping into their feet, they can't seem to frame the results in any way that helps you determine meaningfulness. Often there's a seemingly cavernous gap between the numbers and any kind of meaning, even to the PhDs, but humans have to act on the meaning of things.
There are 6 billion Joe Publics on this planet. I start to worry very much about what they think say and do because it will have a profound effect on the lives of my children and their children.
How those Joe Publics live, what they eat, what they buy, what they value as important or not, will have a great effect on the use we make of our dwindling resources, a great effect on the survival of pretty much every species, a great effect on pollution etc etc etc.
As you say, science needs to be able to explain itself, to win over the hearts and minds of the Joe Publics.
That's true. There was a lot of optimism back then. But Joe Public was terrified, too. Just look at all of the science fiction movies back then. The Bomb was always hanging over our heads. Bomb testing created monsters, giant spiders, the Incredible Shrinking Man, the 50 foot Woman. There was The Fly. The evil Doctor Linus Pauling told us our bones would glow and don't eat the snow. Duck and Cover. It was Silent Spring. Was it tea time or time for more thalidomide?
How come when we all get older, we always think it was so much better in the "good ole days" that really just sucked same as now? Ah, sweet memories of yesteryear...
Probably because ever since older people started saying it, it was actually true:)
I was born in the same year that the movie "Forbidden Planet" was made. It has been remade, in ever decreasing quality, ever since. Think Star Wars.
I was around just early enough to follow man's trips to the moon and appreciate what was going on. A task we have not been able to perform for some decades now.
I rest my case:)
While the misuse of statistics is a common complaint leading to a distrust of science, the complexity of the issues involved is probably just as big a problem. I remember a conversation with a friend who thought she had educated herself on the issue of stem cell research and when I mentioned the adult stem cell vs. embyonic aspect she just went blank. People make decisions at the ballot box or the grocery store based on a very superficial understanding of scientific issues and there's probably very little that can be done about it. We have so many causes trying to raise our awareness that I think our capacity to have it raised is maxed out.
But still, the average Joe is pragmatic and when he sees that Charlie has regrown his lost foot, he can make a rational decision about that and when monsters from dimension X are pouring through the hole in space-time created by the CERN's newest particle accelerator, he can make a rational decision about that.
I personally am in favor of monsters from dimension X pouring in through that hole. One thing is for certain, there is no stopping them; they will soon be here. Id like to remind them that as a trusted college professor, I can be helpful in rounding up others to toil in their underground sugar caves.
-phar