Robert Koch (1843-1910), German physician and one of the first microbiologistsLast time I said I would attempt to lay out the difference between believing something was true, and being able to prove it. There is a huge difference. A useful framework for understanding that difference can be explored by looking at Koch's Postulates. Robert Koch was a German physician and one of the men who established the germ theory of disease in the 1800s-1900s as the preeminent explanation for many diseases. He also discovered Bacillus anthracis, the bacteria that causes anthrax, Mycobacterium tuberculosis, which causes TB, and other nasty pathogens.
Koch was working at a time when many people did not believe in germs, bacteria, viruses, etc., and when many early laboratorians were beginning to grow all kinds of bacterial cultures, but they did not know what they were growing, or which bacteria did what types of things. Koch put forward four classic postulates that could prove a connection between a particular disease and a particular bacterial pathogen. Since he was writing in German, the English translation of his postulate has been stated in various ways, but essentially these are what they say:
1 - The microorganism must be found in all organisms suffering from the disease, but not in healthy organisms.
2 - The microorganism must be isolated from a diseased organism and grown in pure culture
3 - The cultured microorganism should cause disease when introduced into a healthy organism.
4 - The microorganism must be re isolated from the inoculated, diseased experimental host and identified as being identical to the original specific causative agent.
Ok, so what do they mean in plain English? First, you have to always find the same organism in an animal or person suffering from the disease. You have to grow a pure culture (not mixed with other bacteria). Then, in experiments, inoculating a healthy test subject (usually an animal) with bacteria from the pure culture should reproduce the disease. And finally, you should be able to re-isolate the same bacteria from the diseased test subject. If a scientist could repeatedly do all of these things, he or she could claim to have established the linkage between the disease and the pathogen.
I bring this up to illustrate that there is in science a high burden of proof. In epidemiology there is also a high burden of proof. One of the accusations in the raw milk article that, frankly, set me off was the accusation that epidemiologists just jumped to the conclusion that raw milk caused disease and ignored other explanations. That isn't how we do our work. But what we do is not well known by many people, so I'd like to open the curtain to our methods, and our burden of proof. Forgive me if this turns into a long post.
There are many types of studies and different proof tests, so I will confine my remarks to outbreak epidemiology. This is the most relevant to the topic anyway, and it is the type of field epidemiology I do, so it is what I know the most about. When we are confronted with an outbreak of unknown etiology (an outbreak we recognize but do not know how it is spreading), we first want to interview the first reported cases to understand their recent exposures, and we want to determine what laboratory evidence there is, to help us focus on a cause. We test for bacteria, viruses, parasites, chemical exposures, toxins, nutrient imbalances, or other things depending on the symptoms the patients have. Once we know the pathogen we re-interview the cases to hone in on the exposures during known incubation periods. For example, salmonella has a usual 1-3 day incubation period, staph intoxications just 2-4 hours, norovirus 24-48 hours, E. coli 2-10 days, etc. So depending on the pathogen, we look at longer or shorter exposure periods.
If we determine that several cases share an exposure, for instance, eating Peter Pan peanut butter, then we have a hypothesis to test. We design a case-control study where we develop a systematic questionnaire asking about exposures, and make sure that the particular food item we are interested in is on the questionnaire, along with every other exposure we can identify that could explain the disease. In the case of salmonella, we would make sure that eating raw eggs, mayonnaise or ice cream made at home, raw milk, pet turtles, pet snakes, pet lizards, bird exposures, all kinds of things would be in the questionnaire that other people in previous outbreaks had implicated. We would also ask about restaurants, brand names of food items, grocery store chains they buy food at, etc. on the interview. Every investigation includes hundreds of potential exposures in the study design.
We would administer this lengthy questionnaire to all the case patients we knew about, plus another group of healthy people who were otherwise the same as our ill patients. We may choose the healthy controls in a variety of ways, but usually they are matched on key demographic and geographic variables to the ill persons, and we interview both the ill people and the control group systematically. Then we enter all of their responses into computers and run statistical tests on all the variables.
What we are calculating with these tests are called Odds Ratios. We want to see if there is any exposure in the study that ill people share overwhelmingly, but controls do not. Then we apply other statistical tests to our odds ratios to determine the probability that any association we derive could occur by random chance, and whether the association is toward causing disease, or protecting someone from getting the disease.
Once we have an exposure that is statistically associated with disease, we will try to, in the case of bacteria, culture the bacteria from some of the product we are implicating statistically. This is sometimes impossible, if all the food or beverage item has been consumed. But many times we can locate food samples to test. The goal is to grow the bacteria from both the patient and the implicated food item, if possible.
If we do implicate a food item, before we can publish that finding we typically have to satisfy at least two or three other groups of epidemiologists that our findings are valid. We would have to present our evidence to our peers and superiors inside our agency, and then usually to a similar group in another agency (like the CDC, FDA, Food Safety Inspection Service, USDA, or state health departments). Only if the evidence was found sufficiently strong by several different groups in diverse agencies would we be able to publish findings, initiate product testing, recalls, and trace back activities.
What I am saying here is that I do not believe the accusation by proponents of raw milk that epidemiologists implicating raw milk in particular outbreaks did so recklessly, or that they ignored other explanations. We face a high burden of proof, and there are many internal and external checks and balances in the system.
Similarly, proponents of raw milk are asserting that it has many health benefits over pasteurized milk. I think there is a high burden of proof they should meet before making those claims as a justification for relaxing the laws and regulations to open greater access to raw milk. To get there, we would need to agree on exactly what is the specific health benefit, and then we would need to design a study or studies to meet the burden of proof. At a minimum, proponents of raw milk would need to put forward a health benefit that all people would get from the product; it would need to be measurable; it would need to be reproducible; and benefit would have to be shown not to be caused by something else. Right now proponents of raw milk are asserting benefits that are vague or not measurable, are subjective, and diverse. They also seem to be challenging those of us who do this type of research to disprove their health claims or to verify them. But when someone is advancing a new claim or idea, it is on the claiming party to fund the studies, produce the research, publish the findings, and face peer review. That is how science is done.
5 comments:
Amen!!!!! While I do not purport to know much of anything about epidemiology or the question at issue raw milk, it simply jams to see you or anyone pounding out the basics of scientific work and hopefully "proof" . I am so tired of accusations, implications, and protestations, that are not backed by any rigorous method, held out to be as valid as those issues that have been tested, verified, or at least statistically scrutinized.
Hi apexDBS,
Again, I am only addressing the issues concerning raw milk here. I apologize for not being very interested in epidemiology. It definitely seems like you know and enjoy your field of expertise.
As I said in an earlier post, it is unfortunate that some of us (defenders of the freedom to choose raw milk) have become cynical of “agencies” and “professionals” that have been inaugurated to guard our health. This suspicion was not born of thin air – it is an experiential fact that will materialize any time that you follow the “illness or death from raw milk” myths.
What more could we think of these people when most – if not all – of these claims turn out to be caused by something other than raw milk from grass-fed animals? That’s kind of like blaming the jury for the guy with 3 DUI’s losing his license.
Regarding the last paragraph of your post, I have to say that I am flabbergasted that someone of your obvious intelligence could get something so backward. We don’t have to prove our belief that a food product is healthy to use it. You have to prove that it is deadly to stop us from using it.
Are you seriously saying that you believe drinking raw milk is “a new claim or idea”? Of course you don’t because you drank it when you were a child. The new idea was born in the early 20th century when the industrialists discovered that if they fed the cows garbage instead of grass and confined them to a cement slab instead of a meadow that they could make way more money.
That’s when pasteurization became mandatory as the industrialists were not about to give up their profits just because a bunch of kids were dying.
Thanks for listening,
msenkpiel
I believe that suggesting we can relax pasteurization rules is a new idea, compared with what has been established practice for more than 60 years. And I do think that proponents of raw milk who are claiming its health benefits need to be much more precise in what they are claiming it does for them. I have read claims that it cures asthma, cancer, allergies, west nile virus, anorexia, basically, everything that ails you. Those claims are too broad to be plausible.
As for the repeated assertion that you have proven the milk associated outbreaks were due to something else, the point of this post was specifically to call that assertion into question. Unless you have obtained the datasets from those investigations and can show that exposures were left out of the investigations, or that stronger statistical associations were ignored in favor of the published ones, I can't imagine what research you would have to refute the published findings. As I said in the post, the internal and external burdens of proof before associations can be claimed is very high.
I live in Wilkes County where the local health department supplied information to the local newspaper such that we, the residents of Wilkes, were told that raw milk was definitively linked to camphylobacter illnesses. I'd mainly like to ask a simple question: was the burden of proof you described in this post met in the Wilkes County case?
The health department spoke in VERY strong terms in the Wilkes Co. case. I would find their words egregiously scientifically irresponsible apart from the kind of burden of proof you described. At the same time, how am I to verify that that burden of proof was met? How am I to distinguish epidemiological science (in this particular case) from prejudice (which clearly exists, whether founded on prior science or not)?
The only facts I know, and these may be wrong, are that one or more people drank fresh milk and subsequently became ill. I personally know another local woman that was diagnosed and treated for camphylobacter at the exact same time that didn't drink any milk at all. She said the doctors repeatedly tried to link her case to fresh milk (i.e. approached her case with an agenda/prejudice), even though she absolutely didn't have any such links. I also understand that milk from the dairy in question was tested, and no camphylobacter could be found.
As an epidemiologist, can you assure me that the kind of information that formed the basis of the newspaper article I read wouldn't be published apart from a mere circumstantial connection. In other words, can you assure me that the burden of proof for telling the local newspapers what they did exceeded the mere fact of a small number of people that happened to have drunk the milk having camphylobacter? All the evidence I've been able to gather suggests the burden of proof met in the Wilkes County case (prior to the health department pronouncing its judgment) was pathetic. Maybe I'm wrong, but I know enough about human nature to be skeptical.
In reference to you comment above, the established practice to which you refer was established to protect people from the unhealthy and unnatural conditions in which the animals were raised. This is my point – if animals are raised in those conditions, their products are unsafe, raw or processed. To include raw milk from grass-based animals with the products of industrial dairies is comparing apples and oranges.
Regarding the health benefits, I believe that it is a hindrance toward legalization for raw milk advocates to include this in their argument, but there is no other food product that has to prove its health benefit to be legal. It must only show through testing and experience that it is safe. This is being done on a daily basis by thousands of consumers who insist on this choice, legally or not.
For traditional foods to require scientific proof of health benefits before it is legal is putting the cart before the horse. The scientists get funding for this or that theory of a products’ health benefit while it is legal to consume. It would be quite irregular for the sponsors of such tests to fund testing of an illegal product, especially if they are against its being legal.
Luckily, in some places that are not yet lobotomized by fear and ignorance, testing is on-going (see Inverse association of farm milk consumption with asthma and allergy in rural and suburban populations across Europe). These are the places where the scientific proof will be pursued and published. This is not the task of the lay person, who only knows on an experiential level that their affliction has been alleviated.
As to my repeated assertion that “most – if not all – of these claims turn out to be caused by something other than raw milk from grass-fed animals”, I am confident that I can prove that 99.99% of the accusations of raw milk contaminations do not come from a small, family-owned, organic farm where the animals are grass-fed and not routinely administered unnaturally (e.g. antibiotics or growth hormones). In addition I believe it is clearly obvious that the accusing bodies do not attempt to correct their accusations toward raw milk once proven unfounded.
If all else fails and it is scientifically proven that someone indeed got sick after consuming raw milk from properly raised animals, reality easily confirms that raw milk still has a much better safety record than other legal foods. To logically justify continuing the criminality of raw milk, it must be shown that it is more dangerous than legal foods. Since it is one of the few – if not the only – illegal food, the burden of proof is actually on the opponents to show that it is more dangerous that the foods that are legal.
Thanks,
msenkpiel
Post a Comment