Thursday, March 8, 2018

D is for Devastating: A Statistical Error and the Vitamin D Saga

Statistical errors in research are quite common in research, and not always detected. As economists are well aware, when an error with important policy implications is revealed, it may prompt a media frenzy. I was surprised to learn recently of a major statistical error with potentially huge public health implications, yet with seemingly sparse media coverage when it was revealed.

The error concerns the Recommended Dietary Allowance (RDA) of Vitamin D. A 2014 paper found a statistical error in a study used by the Institute of Medicine (IOM) to determine the RDA, resulting in a recommendation that was about an order of magnitude too low.

I am neither a public health expert nor medically trained, but (following Miles Kimball's lead) have developed an interest in public health, and especially nutrition, research, largely due to its parallels with macroeconomic research. What little press coverage I did find about this Vitamin D study omitted technical discussion of the statistical error--"We'll spare you the gritty mathematical details," said one article. But I wanted these details, and so might you, so I dove in to what turned out to be a fascinating story. You may want to share it with your econometrics students: Correct interpretation of confidence intervals can truly be a matter of life and death.

First, some background. The human body can make Vitamin D (unlike other vitamins) when exposed to sunlight. It can also be attained by nutritional sources and supplements. Upon activation by the liver and kidneys, it acts as a hormone that plays a role in calcium metabolism. Sufficient Vitamin D is critical for bone health and a plethora of other health outcomes (more on that later). Research on the health effects of Vitamin D typically looks at health outcomes associated with different serum 25-hydroxyvitamin D (25(OH)D) levels (a measure of concentration in the blood).

The IOM issues dietary recommendations, including RDAs, for the US and Canada. The RDA is supposed to designate the nutrient intake sufficient to meet the needs of 97.5% of healthy individuals. For Vitamin D, issuing this guideline requires first deciding what 26(OH)D level is desirable, then deciding how much supplemental Vitamin D should be taken so that most people have the desired 26(OH)D level. Based on associations between 26(OH)D levels and various health outcomes, the IOM aimed to recommend an RDA that would result in 25(OH)D levels of 50 nmol/L or more.

The IOM then had to determine how much supplemental Vitamin D to recommend based on this goal. They looked at 10 studies of the dose response relationship of vitamin D intake and 25(OH)D. Some of these studies examined 25(OH)D levels for multiple different doses, so in total there were 32 estimates (the green diamonds in Figure 1). They fitted a dose response relationship curve to these points, with 95% confidence interval. The IOM came up with an RDA for individuals 1 to 70 years of age of 600 IU per day. You can see the vertical line at 600 in Figure 1. It intersects the fitted dose response curve at 63 nmol/L and the lower bound of the 95% confidence interval at 56 nmol/L. Remember, this was the amount that was supposed to achieve 25(OH)D levels of at least 50 nmol/L in at least 97.5% of healthy individuals.

Figure 1. Source: Veugelers and Ekwaru (2014)

In October 2014, Paul J. Veugelers and John Paul Ekwaru explained in a paper in Nutrients that the IOM's interpretation of these confidence intervals was incorrect. They thought that 2.5% of individuals would have serum levels below the lower 95% confidence interval, but in this meta-analysis, the unit of observation was not the individual, but the study average. In the authors' words:
The correct interpretation of the lower prediction limit is that 97.5% of study averages are predicted to have values exceeding this limit. This is essentially different from the IOM’s conclusion that 97.5% of individuals will have values exceeding the lower prediction limit.
Veugelers and Ekwaru returned to the 10 studies, eight of which reported both average and standard deviation serum level for particular doses of Vitamin D. From these statistics, the authors could calculate the 2.5th percentile at each dose. Then they regressed these 2.5 percentile values (the 23 yellow dots in Figure 2) on vitamin D intake, coming up with the red dashed line in Figure 2. The green dashed lines are the confidence intervals from Figure 1, for the sake of comparison.

In the figure, you can see that at 600 IU per day, 97.5% of individuals will have serum levels above around 27 nmol/L, not 50 nmol/L. To get 97.5% of individuals with serum levels above 50 nmol/L, you would actually need a higher dose than any of the studies examined. Out-of-sample extrapolation led them to estimate that 8895 IU of vitamin D per day would actually be required. Veugelers and Ekwaru also pointed to two studies in which 10% or 15% of Canadian subjects had serum 25(OH)D levels of less than 50 nmol/L despite vitamin D supplementation at the RDA level. They wrote, "If the RDA had been adequate, these percentages should not have exceeded 2.5%. Herewith these studies show that the current public health target is not being met."

Figure 2. Source: Veugelers and Ekwaru (2014)

Veugelers and Ekwaru did caution that as t8895 IU of vitamin D per day "is far beyond the range of studied doses, caution is warranted when interpreting this estimate. Regardless, the very high estimate illustrates that the dose is well in excess of the current RDA of 600 IU per day and the tolerable upper intake of 4000 IU per day."

In March 2015, in the same journal, Robert Heaney, Cedric Garland, Carole Baggerly, Christine French, and Edward Gorham published a letter in the same journal that alleviated some of the concern about extrapolating beyond the available data. They presented entirely different data on individuals with daily vitamin D intakes from zero to over 10,000 IU. They came up with an estimate that was slightly lower than Veugelers and Ekwaru's, but confirming the finding that the IOM recommendation was around an order of magnitude too low, and wrote:
Thus, we confirm the findings of these investigators with regard to the published RDA for vitamin and we call for the IOM and all public health authorities concerned with transmitting accurate nutritional information to the public to designate, as the RDA, a value of approximately 7000 IU per day from all sources.
Like Veugelers and Ekwaru, Heaney et al. remarked upon the safety of such a high recommendation, though their take was more optimistic:
The total, all-source intake of 7000 IU/day is below the no observed adverse effect level (NOAEL) of both the IOM and the Endocrine Society, below the tolerable upper intake level (UL) of the Endocrine Society, and well within the safe range delineated by Hathcock et al., who had generated that range using the IOM’s method of hazard identification.
The hormonal role of vitamin D explains why the Endocrine Society also issues guidance about it. Remember, vitamin D is fat-soluble, so excess amounts are stored and can accumulate in body tissues-- hence the concern about safety at higher doses. Overall health benefits may increase with dose up to a point, and then start to decline. Initial guidelines on Vitamin D RDA were based on prevention of rickets. But as scientists have learned more about other health benefits, the cost-benefits calculus of vitamin D recommendations has shifted. This shift, however, was slow to be reflected inn health policy.

"Worldwide reports have highlighted a variety of vitamin D insufficiency and deficiency diseases. Despite many publications and scientific meetings reporting advances in vitamin D science, a disturbing realization is growing that the newer scientific and clinical knowledge is not being translated into better human health," wrote Andrew Norman in a 2008 issue of the American Journal of Clinical Nutrition. A 2007 article in the same journal, by Reinhold Vieth and many coauthors, describes the situation as a "frustrating and regrettable situation for nutrition researchers."

Vieth et al. summarize the many health benefits attributable to adequate vitamin D, and evidence that the tolerable upper limit is around ten times higher than officially-recommended intakes. But they point to an over-cautious and under-nuanced take by the public media that has kept public supplemental intake too low:
Evaluation of most relations of health and disease that involve vitamin D leads to the conclusion that a desirable 25(OH)D concentration is ≥75 nmol/L (30 ng/mL). If a concentration of 75 nmol/L is the goal to be achieved by consumption of vitamin D, then why is it so rare for members of the population to accomplish this? One reason is that almost every time the public media report that vitamin D nutrition status is too low, or that higher vitamin D intakes may improve measures of health, the advice that accompanies the report is outdated and thus misleading. Media reports to the public are typically accompanied by a paragraph that approximates the following: “Current recommendations from the Institute of Medicine call for 200 IU/d from birth through age 50 y, 400 IU for those aged 51–70 y, and 600 IU for those aged >70 y. Some experts say that optimal amounts are closer to 1000 IU daily. Until more is known, it is wise not to overdo it.” The only conclusion that the public can draw from this is to do nothing different from what they have done in the past.
The evidence in favor of higher 25(OH)D concentration and a higher RDA continued to grow in subsequent years. "Despite research on the association between low vitamin D status and many diseases, no consensus has emerged on the optimal serum 25(OH)D concentration. The concern is whether it is safe to maintain serum 25(OH)D concentrations in the range high enough to prevent some types of cancers and coronary heart disease," wrote Garland et al. 2014 in the American Journal of Public Health. In a meta-analysis of serum 25(OH)D and age-adjusted all-cause mortality, they showed that overall age-adjusted hazard ratios for mortality decline steeply with 25(OH)D for serum levels below 30 nmol/L, then gradually level off (Figure 3). The hazard ratio is not statistically different from 1 at 36 nmol/L.

Figure 3. Source: Garland et al. 2014 

In Finland, public health policy was changed in response to widespread low serum 25(OH)D concentration. Vitamin D fortification of certain dairy products and spreads began in 2002, and fortification levels were increased in 2010. This was successful in raising vitamin D intake, and health benefits are already measurable. In July 2017, Dimitrios Papadimitriou noted in the Journal of Preventive Medicine and Public Health the Type I diabetes, which had been on the rise, leveled off and then declined in Finland after the vitamin D fortification policy was implemented. Papadimitriou discusses vitamin D's role as a "powerful nuclear receptor-activating hormone of critical importance, especially to the immune system," and calls for public health authorities worldwide to modify RDAs in line with now quite substantive scientific research.

Papadimitriou titled his article "The Big Vitamin D Mistake," referring in a narrow sense to the IOM's statistical error. But he really discusses a mistake in a broader sense-- mounting evidence has been too slow to be incorporated into policymaking and practice.


  1. Yes,
    US admits math mistake was made in 2010 in estimating Vitamin D, but will not change recommendations – Nov 2017
    Has the details on that study and many others
    Bottom Line
    Needed at least 3,000 IU to meet the 2010 data

  2. Actually, the error is not all that technical. Confusing an average with an individual is pretty dumb.

    There's a bias in the media against taking supplements which accounts for the resistance to recommending such a large increase in the RDA. Unless people spend a lot of time in the sun, which is known to increase the risk of skin cancer and is therefore discouraged by dermatologists it's virtually impossible to get enough vitamin D from food alone. The easy way out is to simply refuse to up the RDA. Also, the toxicity arguments against higher supplemental doses are not based on any evidence and seem only to provide cover for the error and further justify not raising the RDA

    There's a cheap test available for serum 25(OH)D concentration that can be ordered by any doctor. This allows individuals to adjust their supplemental dose to achieve 50 nmol/L or more.

  3. Is there a basic statistical error in this article about basic statistical errors? I think the intervals shown in figure 1 are prediction intervals, not CIs. In the original V&E paper which the figure is taken from they call them "confidence prediction intervals". I've never heard of such a thing.

  4. When I accidentally bought a bottle of 5,000 units per capsule of vitamin D, thinking it was 500 units, I started feeling like I was mildly poisoned, until I noticed the dosage and started taking them once a week instead of daily. Now I make sure I pay better attention.

  5. Very interesting! And that explains why some very decent doctors have recommended taking more than the RDA to me... And it's been helpful. Thanks for making me feel better about it!

  6. Thank you So much for this Important and Useful Information.

  7. Very nice article! Thank you for sharing!

  8. Wonderful article you have published . I admire this post . Keep up sharing .

  9. Great contribution! There is a unit error in your description of figure 3. Figure 3 says ng/ml, in the text you use nmol/L. 40 ng/ml are 100 nmol/L!

  10. Thanks you for sharing the article. The data that you provided in the blog is infromative and effectve. Through you blog I gained so much knowledge. Also check my collection at machine learning online training Blog

  11. It's in reality an excellent and useful piece of information. I'm glad this helpful information you have shared with us. Please keep us up to date like this.
    Clipping Path | Color Correction | Photo Retouching | Product Photo Editing

  12. It is something interesting to note that the reality shows are very popular in India because they combine both the elements of Masala/Drama + Limited episodes !!
    Watch More TV Serial- All TV Serial, TV Serial Today, Daily TV Serials, Mega TV Serial, Desi Serial Online, Live TV Link

  13. Good article you have written . I appreciate your shared .

  14. Thanks for the information keep share this type of article
    multiple word file binder

  15. "Great post, thank you for the useful information. Keep up the good work! FYI, check this out: Ritual Vitamins Review for Women"

  16. Quicken technical support offer excellent customer support on emails and chat. Quicken assist desk direct clients to a consistent group of advisers. these advisers are very expert and skilled to deliver proper Quicken assist
    Quicken support phone number
    Quicken phone number
    Quicken technical support number
    Quicken technical support
    Quicken 2019 support

  17. Nuvigil also known as Armadafinil helping to reduce extreme sleepiness due to sleep disorders, it will help you to stay awake during working hours if you have a work schedule that keeps you from having a normal sleep routine.
    This medicine does not cure your problem. Its only help in reduce symptoms of you disorder. You may not get rid of your sleepiness. Nuvigil or armodafinil does not take the place of proper sleep. It cannot be used for treat tiredness.
    Nuvigil is used to improve wakefulness in adult with excessive sleepiness associated with obstructive sleep apnea(OSA), narcolepsy, shift work disorder (SWD).

    Buy Nuvigil Online,
    Buy Nuvigil 150mg,
    Nuvigil 150mg,
    Cheap Nuvigil 150mg
    Buy Nuvigil 150 mg Online at cheap rate

  18. Very nice article and great content, really helpful .I recommend to read more in below links also .Thanks

  19. Very nice article and great content, really helpful .I recommend to read more in below links also

  20. Get Shifting/Relocation Quotation from ###Packers and Movers Delhi. Packers and Movers Delhi 100% Affordable and Reliable ***Household Shifting Services. Compare Transportation Charges and Save Time, Verified and Trusted Packers and Movers in Delhi, Cheap and Safe Local, Domestic House Shifting @
    Packers And Movers Delhi

  21. Stretch Tents Manufacturers Capetown is the market leaders in design, manufacture and supply of Marquees and Tents in South Africa. Stretch tents south africa provides tents for sale at reasnable price. We help you make your event an awesome event with our range of different Tents to choose from as well as accessories to make your event a memorable one. We have acquired core capability in manufacturing and dealing of high quality fabrics and fabricated made-ups especially those of Stretch Tents.

    Our feature products are :
    tiffany chairs,
    plastic chairs,
    portable toilets for sale

  22. We provide all major photo editing service such as background removal, image masking service, clipping path service, vehicle photo editing service, image shadow service, ecommerce photo editing service etc.

  23. Thank you for your helpful contributions.

  24. It was during my research on HIV/Herpes that I stumbled upon the Hiv/Herpes information; information which is quite easy to find when doing a search for STD on google. I was into conspiracy at the time thought of HIV/Herpes Cured' being a conspiracy was something Ignorance though,I found pretty interesting about herbal medicine. I asked questions about the Herbal cure's on official HIV/Herpes websites and I was banned for doing so by moderators who told me that I was parroting Hiv/Herpes propaganda. This reinforced my belief that there is a cure for Hiv/Herpes Then i found a lady from germany name Achima Abelard Dr Itua Cure her Hiv so I send him a mail about my situation then talk more about it and send me his herbal medicine I drank for two weeks.And today I'm Cured no Hiv/Herpes in my life,I searched for Hiv/Herpes groups to attempt to make contact with people in order to learn more about Hiv/Herpes Herbal Cure's I believed at this time that you with the same disease this information is helpful to you and I wanted to do the best I could to spread this information in the hopes of helping other people.That Dr Itua Herbal Medicine makes me believes there is a hope for people suffering from,Parkinson's,Alzheimer’s disease,Bechet’s disease,Crohn’s disease
    ,Cushing’s disease,Heart failure,Multiple Sclerosis,Hypertension,Colo_Rectal Cancer,Lyme Disease,Blood Cancer,Brain Cancer,Breast Cancer,Lung Cancer,Kidney Cancer,Love Spell,psoriasis,Lottery Spell,disease,Schizophrenia,Cancer,Scoliosis,Fibromyalgia,Fluoroquinolone Toxicity Syndrome Fibrodysplasia Ossificans Progressiva.Infertility,Tach Disease ,Epilepsy ,Diabetes ,Coeliac disease,,Arthritis,Amyotrophic Lateral Sclerosis,Autism,Alzheimer's disease,Adrenocortical carcinoma.Asthma, (measles, tetanus, whooping cough, tuberculosis, polio and diphtheria)Allergic diseases.Parkinson's disease,Schizophrenia,Lung Cancer,Breast Cancer,Colo-Rectal Cancer,Blood Cancer,Prostate Cancer,siva.Fatal Familial Insomnia Factor V Leiden Mutation ,Epilepsy Dupuytren's disease,Desmoplastic small-round-cell tumor Diabetes ,Coeliac disease,Creutzfeldt–Jakob disease,Cerebral Amyloid Angiopathy, Ataxia,Arthritis,Amyotrophic Lateral Scoliosis,Fibromyalgia,Fluoroquinolone Toxicity
    Syndrome Fibrodysplasia Ossificans ProgresSclerosis,Seizures,Alzheimer's disease,Adrenocortical carcinoma.Asthma,Allergic diseases.Hiv_ Aids,Herpe ,Copd,Glaucoma., Cataracts,Macular degeneration,Cardiovascular disease,Lung disease.Enlarged prostate,Osteoporosis.
    Dementia.,Hiv_ Aids,Herpes,Inflammatory bowel disease ,Copd,Diabetes,Hepatitis,I read about him online how he cure Tasha and Tara,Conley,Mckinney and many more suffring from all kind of disease so i contacted him . He's a herbal doctor with a unique heart of God, Contact Emal. ... Phone or whatsapp..+2348149277967.

  25. Great post mate, thank you for the valuable and useful information. Keep up the good work! FYI, please check these depression, stress and anxiety related articles:

    Depression Cure

    Mental Stress in Children

    Depression In College Students

    How To Save Your Relationship

    Depression in Men

    Interesting Depression Facts

    How To Cure Depression

    How To Fight Depression

    30 Ways Stress Can Affect Your Body

    16 Simple Ways to Relieve Stress and Anxiety

    you can also contact me at for link exchange, article exchange or for advertisement.



  26. Wow, great article post.Really looking forward to read more.Thank you so much for your good sharing. Footwear photo editing services


  27. I was scrolling the internet like every day, there I found this article which is related to my interest. The way you covered the knowledge about the subject and the 2 BHK duplex in hoshangabad road was worth to read, it undoubtedly cleared my vision and thoughts towards B 4 BHK Duplex in hoshangabad road. Your writing skills and the way you portrayed the examples are very impressive. The knowledge about 3 BHK duplex in hoshangabad road is well covered. Thank you for putting this highly informative article on the internet which is clearing the vision about top builders in Bhopal and who are making an impact in the real estate sector by building such amazing townships.

  28. this is the Best free dating apps for youth, is the best way to find your perfect partner

  29. This is Really Fantastic And Thanks For Sharing The Valuable Post..Issues With Quicken Software Dial Visit Quicken Support Phone Number +1(888)586-5828 Toll-free !Get Instant Help and Support from Quicken Certified Experts.You Do Not Want To Get Hold Of The Quicken Support Online Or Be Looking.


Comments appreciated!