As an observation, and not meant as an especial dig at Scott, does anyone remember those classic AI vs. the Creationists threads?
It's interesting to me quite how much this 'debate' resembles those threads, and how much the 'dissenting' opinion uses the same kinds of arguments and facts.
I remember that Creationists would attempt to argue that there was no scientific consensus by citing the work of mavericks, for example, and would question very firm, quantifiable evidence by unearthing hoary old Victorian fake fossils or quibbling over scientific terms. I also remember that when they were called out on their science it turned out that they didn't actually know very much about science at all. This happened time and time again.
The truth's absolutely obvious to everyone who hasn't had their opinion already decided for them according to a particular ideological position, rather than the facts on the ground.
It's a study of ocean salinity in question here, published in the world's oldest and most prestigious natural sciences magazine. What's so upsetting about its publication? Is it the study's results? Is it the research methods? Is it the implications if the study's actually correct? Is it the political thinking of the objector, attempting to defend an orthodoxy that produced a particular government's environmental policy?
Yeah, I had observed that in an earlier global warming thread - the frustration I have is dealing with people criticizing science that they don't understand because it conflicts with their personal worldview. And these arguments have the same tendency to keep shifting, where they will launch 5 objections, you beat those down, and then without ever admitting any defeat, they point-shift a couple of the arguments (well, what I mean is....), then add in a new couple they think might hold. And it goes on and on - never admitting they are wrong about anything, and the arguments are so fluid that you cannot pin them down, because if you dam up one tact, they simply divert into some tangent.
I still think it is healthy to argue with them now and again (creationsists as well - I used to be quite active in that area). I think it is important to let others know how hollow the arguments are.
One issue that Scott did bring up is data mining producing results that are meaningless, say drinking more than 5 glasses of wine a day will prevent cancer in women in st. louis. I am also aware of the fact that the oceanographic data is limited in quantity, ie it is impossible, within reason, to measure every location on the surface of the ocean for salinity. One product of this decreased data set may be increasing the error associated with data mining operations and hence produce false or grossly misdirected conclusions. Presumably this will come out in the statistics, but Scott could reasonably argue that without showing those stats the authors leave much evidence out that would ordinarily convince scientist, even unaware ones, of the validity of the conclusions.
But then again, science which directly questions everyday interpretations of life and its seeming direction brings to the table all of human psychology.
My experience is limited with oceanographic studies, i am a biophysicist but I do think that I have the ability to read and understand fields that are divergent from my own without resorting to like comparissons.
I am an oceanographer (albeit a biological oceanographer, not PO like the authors of the Nature paper).
The stats are both in the paper AND (as is usual in Nature) in the assorted references). The authors explicitly stated how and why they chose their ransect line. Could they have potentially looked at hundreds of transect lines, picked the one that supported their view, and then a posteriori created a rationale for their given transect? Sure. I would argue that is true of almost any study (why choose the sampling sites you did for fish abundances? Why use shannon-weaver index instead of some other measure? why did you look at herring and swordfish densities as your measure of fishery CPUE?). The key is that the data are available for everyone to see, and the rational are described in the paper. A certain amount of trust is inherent in any study - when you tell me that you measured the stresses on some cartilaginous component in a jaw, and the mean was X, I have to assume that you are telling the truth (since almost nobody published their raw data, and even if they did, it could be "dry-labbed"). For me to criticize your study by just saying "how do I know you didn't measure a bunch of other jaw components and only publish the one that was significant" is meaningless.
And my apologies if I implied that one cannot crtitique a study outside of their field - my point was, if oyu have very little understanding of a field (as I think Scott has demonstrated here), you need to be a bit careful, and understand that your criticisms may be way off base.
In the light of fishdoc's excellent, detailed explanation of the researchers' methodology, do you still hold your opinion that the stats were sloppy and the study's results are questionable?
what a balm to my spirit to read closely reasoned, coherent prose based on the best available knowledge. Moreover, your lucid explication of the moving target that is faith based reality could serve as critique of half the contention on half the threads in AO. If ever my fish were to fall ill, I would seek your services.
Single Transect. Doesn't matter why they chose it it's still only one out of an ocean full of them. One proves nothing ... ever. Unless all were the same but we have no basis to know that.
Confidence intervals (CIs). Strictly speaking the 95% CIs (for example) tells you what interval 95 out of 100 repeat experiments will fall in. If you are trying to estimate something by averaging 10 trial then the CI tells you what interval 95 out of 100 repeated averages will fall in.
If you want to say something has changed then the better method IMO is to use a freshman stats p test (or something like it). "We measured this in the 60s and that in the 80s and here's the p value that shows they are not the same." If there's no signal in an ocean full of data then you need to set your p value very very low or you're going to think a lot of data is significant when it's not.
You never addressed the error estimate in the temperature. One hundredth of a degree? The more I think about it the more I call bullshit on it. Especially after being average out over this that and the other. Where I work we have a NIST traceable temperature probe and it's only good to a tenth of a degree. Considering all the interpolation that going on in the data from over 40 years ago there's no way the error is that low.
Anyway the easiest thing to do is rip someone else's work apart. I try not to be one of those people. But getting data and stats correct is almost impossible.
Well, I note that the vast majority of your "criticisms" have been rebutted, and you have not even tried to resurrect them here, which is a start. I honestly do not understand your odd indignation at this paper, since you do not seem to have any very significant problems with the methods or results.
I dealt with your comment on the transect, and what you said here added no further comment. They specifically chose a transect with particular properties, which they justified. See my earlier response if you need more.
You are much closer to the correct meaning of CI here, and in fact your second sentence describing it here pretty much gets it on the nose. Contrast this with your earlier use, where you claimed that "5% of your normals will come across as significant". Admittedly a minor point, but one I would hope ot get right if I were criticising others' stats.
As for the error - they told you what the uncertainties were, they told you the software they used to get it, and they told you the database they used. What exactly is it I need to explain? Unless you can prove they are wrong (look at the database itself, look at similar papers using that database to check, or even email the lead author), I don't think that makes any sense. They gave you everything you need to check - if they are wrong, you could well have your own Nature paper right there, waiting for you to publish.
Edited to be clearer here - the point is, you cannot just say "well, I just cannot believe that" and consider that a valid criticism. If you really are so disbelieving, I think you should email ms curry, and ask her how come her data don't fit with your perceptions. I am sure she could explain it to you, and most authors enjoy getting emails and letters from the public asking about their work (I know I do). It is part of our job as scientists to communicate our work to the public who pas for so much of it.
The bottom line, Scott, is that the report you read upset you for some reason, in part it appears due to your misunderstanding (e.g., you thought that the Curry paper talked about cooling in Europe, you thought that a Nobel prize-winner wrote the article, etc), and you railed against your own misunderstanding of a reasonably clear scientific work.
Trust me - there are actual scientists who would LOVE to find major errors in a Nature paper (and do on numerous occasions). These *qualified* scientists would have taken Curry et al to task if they were making such huge mistakes.
Give it up. You're arguing marine biology with a marine biologist. I mean, I enjoy a good pistol whipping as much as the next guy, but this is getting just.... depressing.
Here's another problem I have. They average the data over a column of water. Normally (no pun) when you average data you assume that the noise in the data is zero mean and normally distributed about the average. But if the temperature varies as a function of depth you can't assume that the variation in the data is normally distributed. Each sample will (could) have a different underlying unknown average. Same problem with the transect.
Now sure you can average whatever you want and call it a stat. But the typical rules you use to determine the error in that estimate don't apply.
Give it up. You're arguing marine biology with a marine biologist. I mean, I enjoy a good pistol whipping as much as the next guy, but this is getting just.... depressing.
If it were an "argument" I would say have at it. What I am seeing are a series of basic misunderstandings being patiently corrected after which you come up with further misunderstandings. It's not pretty.
Well, it looks like this will be the last time, as I have finally gotten you down to one complaint about the data...
First, read the paper again if you would - much of the study does not involve vertically-averaged T and S measurments. Look at the first figures - the show (and analyze) the data stratified by both depth and density (the figures show, for example, AABW (Antarctic bottom water, northeast atlantic deep water, etc). The vast majority of the paper involves these stratified data.
Now let's talk about the graphs where they DID use integrated means. If you bin the water column (say, in 1 meter averages), then calculate the mean at that longitude, that IS the mean...they are asking questions about the means themselves, because they are asking questions about the *entire water column* all along the transect (it matters a great deal what your "experimental unit" is in cases like this). Still not convinced? It gets better, look at the graphs - the stats, for example in Fig 3c, are about the amount of freshwater required to account for the freshening of the water column. The beauty of this is, using vertically-integrated averages with a conservative property (like salinity), the error structure is unimportant. The mean salinity times the volume of water = total salt. It doesn't matter how that salt is distributed. Cool, eh?
Scott, give it up - this really IS a good paper, and your lack of understanding doesn't make it any worse. Really and truly, if you sincerely believe you have found an Achilles heel, you could have yourself a Nature paper with almost no effort. Better still, if you are confused by the results - email her. I was not kidding when I said that
scientists enjoy the opportunity to educate the public.
Comments
It's interesting to me quite how much this 'debate' resembles those threads, and how much the 'dissenting' opinion uses the same kinds of arguments and facts.
I remember that Creationists would attempt to argue that there was no scientific consensus by citing the work of mavericks, for example, and would question very firm, quantifiable evidence by unearthing hoary old Victorian fake fossils or quibbling over scientific terms. I also remember that when they were called out on their science it turned out that they didn't actually know very much about science at all. This happened time and time again.
The truth's absolutely obvious to everyone who hasn't had their opinion already decided for them according to a particular ideological position, rather than the facts on the ground.
It's a study of ocean salinity in question here, published in the world's oldest and most prestigious natural sciences magazine. What's so upsetting about its publication? Is it the study's results? Is it the research methods? Is it the implications if the study's actually correct? Is it the political thinking of the objector, attempting to defend an orthodoxy that produced a particular government's environmental policy?
I still think it is healthy to argue with them now and again (creationsists as well - I used to be quite active in that area). I think it is important to let others know how hollow the arguments are.
Fish
One issue that Scott did bring up is data mining producing results that are meaningless, say drinking more than 5 glasses of wine a day will prevent cancer in women in st. louis. I am also aware of the fact that the oceanographic data is limited in quantity, ie it is impossible, within reason, to measure every location on the surface of the ocean for salinity. One product of this decreased data set may be increasing the error associated with data mining operations and hence produce false or grossly misdirected conclusions. Presumably this will come out in the statistics, but Scott could reasonably argue that without showing those stats the authors leave much evidence out that would ordinarily convince scientist, even unaware ones, of the validity of the conclusions.
But then again, science which directly questions everyday interpretations of life and its seeming direction brings to the table all of human psychology.
My experience is limited with oceanographic studies, i am a biophysicist but I do think that I have the ability to read and understand fields that are divergent from my own without resorting to like comparissons.
The stats are both in the paper AND (as is usual in Nature) in the assorted references). The authors explicitly stated how and why they chose their ransect line. Could they have potentially looked at hundreds of transect lines, picked the one that supported their view, and then a posteriori created a rationale for their given transect? Sure. I would argue that is true of almost any study (why choose the sampling sites you did for fish abundances? Why use shannon-weaver index instead of some other measure? why did you look at herring and swordfish densities as your measure of fishery CPUE?). The key is that the data are available for everyone to see, and the rational are described in the paper. A certain amount of trust is inherent in any study - when you tell me that you measured the stresses on some cartilaginous component in a jaw, and the mean was X, I have to assume that you are telling the truth (since almost nobody published their raw data, and even if they did, it could be "dry-labbed"). For me to criticize your study by just saying "how do I know you didn't measure a bunch of other jaw components and only publish the one that was significant" is meaningless.
And my apologies if I implied that one cannot crtitique a study outside of their field - my point was, if oyu have very little understanding of a field (as I think Scott has demonstrated here), you need to be a bit careful, and understand that your criticisms may be way off base.
Fish
In the light of fishdoc's excellent, detailed explanation of the researchers' methodology, do you still hold your opinion that the stats were sloppy and the study's results are questionable?
what a balm to my spirit to read closely reasoned, coherent prose based on the best available knowledge. Moreover, your lucid explication of the moving target that is faith based reality could serve as critique of half the contention on half the threads in AO. If ever my fish were to fall ill, I would seek your services.
sadly, most of my patients do not survive. Still, I would be pleased to lend my not-so-healing hands to any sick fishes you happen to have.
F
Originally posted by fishdoc
Addabox,
sadly, most of my patients do not survive. Still, I would be pleased to lend my not-so-healing hands to any sick fishes you happen to have.
F
I have a smoked salmon I would like to swim. Any suggestions?
Originally posted by Anders
I have a smoked salmon I would like to swim. Any suggestions?
Duct tape and a bottle rocket.
Originally posted by fishdoc
I think his best path to swimming again has to be via your gastrointestinal tract....
Single Transect. Doesn't matter why they chose it it's still only one out of an ocean full of them. One proves nothing ... ever. Unless all were the same but we have no basis to know that.
Confidence intervals (CIs). Strictly speaking the 95% CIs (for example) tells you what interval 95 out of 100 repeat experiments will fall in. If you are trying to estimate something by averaging 10 trial then the CI tells you what interval 95 out of 100 repeated averages will fall in.
If you want to say something has changed then the better method IMO is to use a freshman stats p test (or something like it). "We measured this in the 60s and that in the 80s and here's the p value that shows they are not the same." If there's no signal in an ocean full of data then you need to set your p value very very low or you're going to think a lot of data is significant when it's not.
You never addressed the error estimate in the temperature. One hundredth of a degree? The more I think about it the more I call bullshit on it. Especially after being average out over this that and the other. Where I work we have a NIST traceable temperature probe and it's only good to a tenth of a degree. Considering all the interpolation that going on in the data from over 40 years ago there's no way the error is that low.
Anyway the easiest thing to do is rip someone else's work apart. I try not to be one of those people. But getting data and stats correct is almost impossible.
I dealt with your comment on the transect, and what you said here added no further comment. They specifically chose a transect with particular properties, which they justified. See my earlier response if you need more.
You are much closer to the correct meaning of CI here, and in fact your second sentence describing it here pretty much gets it on the nose. Contrast this with your earlier use, where you claimed that "5% of your normals will come across as significant". Admittedly a minor point, but one I would hope ot get right if I were criticising others' stats.
As for the error - they told you what the uncertainties were, they told you the software they used to get it, and they told you the database they used. What exactly is it I need to explain? Unless you can prove they are wrong (look at the database itself, look at similar papers using that database to check, or even email the lead author), I don't think that makes any sense. They gave you everything you need to check - if they are wrong, you could well have your own Nature paper right there, waiting for you to publish.
Edited to be clearer here - the point is, you cannot just say "well, I just cannot believe that" and consider that a valid criticism. If you really are so disbelieving, I think you should email ms curry, and ask her how come her data don't fit with your perceptions. I am sure she could explain it to you, and most authors enjoy getting emails and letters from the public asking about their work (I know I do). It is part of our job as scientists to communicate our work to the public who pas for so much of it.
The bottom line, Scott, is that the report you read upset you for some reason, in part it appears due to your misunderstanding (e.g., you thought that the Curry paper talked about cooling in Europe, you thought that a Nobel prize-winner wrote the article, etc), and you railed against your own misunderstanding of a reasonably clear scientific work.
Trust me - there are actual scientists who would LOVE to find major errors in a Nature paper (and do on numerous occasions). These *qualified* scientists would have taken Curry et al to task if they were making such huge mistakes.
Fish
Give it up. You're arguing marine biology with a marine biologist. I mean, I enjoy a good pistol whipping as much as the next guy, but this is getting just.... depressing.
Now sure you can average whatever you want and call it a stat. But the typical rules you use to determine the error in that estimate don't apply.
Originally posted by addabox
Scotty, buddy, pal, sweetheart....
Give it up. You're arguing marine biology with a marine biologist. I mean, I enjoy a good pistol whipping as much as the next guy, but this is getting just.... depressing.
Actually we are arguing about data analysis.
Originally posted by Scott
Actually we are arguing about data analysis.
If it were an "argument" I would say have at it. What I am seeing are a series of basic misunderstandings being patiently corrected after which you come up with further misunderstandings. It's not pretty.
First, read the paper again if you would - much of the study does not involve vertically-averaged T and S measurments. Look at the first figures - the show (and analyze) the data stratified by both depth and density (the figures show, for example, AABW (Antarctic bottom water, northeast atlantic deep water, etc). The vast majority of the paper involves these stratified data.
Now let's talk about the graphs where they DID use integrated means. If you bin the water column (say, in 1 meter averages), then calculate the mean at that longitude, that IS the mean...they are asking questions about the means themselves, because they are asking questions about the *entire water column* all along the transect (it matters a great deal what your "experimental unit" is in cases like this). Still not convinced? It gets better, look at the graphs - the stats, for example in Fig 3c, are about the amount of freshwater required to account for the freshening of the water column. The beauty of this is, using vertically-integrated averages with a conservative property (like salinity), the error structure is unimportant. The mean salinity times the volume of water = total salt. It doesn't matter how that salt is distributed. Cool, eh?
Scott, give it up - this really IS a good paper, and your lack of understanding doesn't make it any worse. Really and truly, if you sincerely believe you have found an Achilles heel, you could have yourself a Nature paper with almost no effort. Better still, if you are confused by the results - email her. I was not kidding when I said that
scientists enjoy the opportunity to educate the public.
But somehow I suspect this wont happen.....