Saturday, August 15, 2015

Did low fat beat low carb for fat loss, after all?

A new paper, published in Cell Metabolism, by Dr. Kevin Hall and colleagues, entitled: "Calorie for Calorie, Dietary Fat Restriction Results in More Body Fat Loss than Carbohydrate Restriction in People with Obesity"[1] claims (as its self-evident title would imply) that this is precisely the case.

I feel compelled to clarify something for the many people who are already priming themselves to comment below, something to the tune of: "This study does not reflect real world dieting," etc.... The authors know this. In fact, they state it in the paper, and Dr. Hall said the same thing in his WebMD interview[2]:

That's not to suggest this isn't an important consideration. It certainly is. My point is simply that it isn't something the authors neglected to take into account. This study was designed primarily to answer a specific research question. (I do not think they succeeded in doing so - or, at least I suspect their confidence is substantially over-estimated, hence the reason I am writing this post.) Keep that in mind, but we will return to this idea, toward the end of my thought process.

Before getting into the meat of the paper, I would also like to comment that, although I do expect that if carbohydrates were restricted even further, the results of this study (carried out for a longer period of time and with a larger sample) may well have turned out differently, perhaps favoring the carbohydrate-restricted approach, this trial was never intended to be a ketogenic diet study. What would have happened, had they tried to reduce total carbohydrate (CHO) intake further, and "match" it to total fat intake? When the authors did the calculations, the only reasonable way they figured they could keep total Calories and protein intake constant across both study groups was to have the Reduced Carbohydrate (RC) arm eat 140 g of CHO (29% of total kcal) per day, which is exactly what they did. If not, something else would have had to give. It seemed like the only way the researchers could be certain that, if the RC arm reduced adiposity more than the Reduced Fat (RF) arm, it didn't come from an increase in protein, and, conversely, that if the RF arm came out on top, that it wasn't due to a decrease in calories; both of which would be important confounding variables that would have ruined the study and destroyed their chances of being able to discern the mechanistic understanding of weight loss diets they were seeking. In order for them to do what they set out to do, which was to measure potential metabolic changes and inefficiencies under isocaloric conditions - more specifically, to prove that insulin levels need not be low for a "metabolic advantage" to occur in fat mobilization and oxidation - they had no choice but to match Calories and protein, and reduce CHO as low as was possible; which, unfortunately, only turned out to be 29% of total kcal. In their own words:

Ignoring the fact that most people simply cannot resist jumping on the bandwagon and riding out a scandalous message, the main reason I think this trial has already seen so much popularity, in less than 24 hours, is because this was a metabolic ward study; an experiment where people are held in a hospital ward for the duration of the intervention, closely monitored and fed carefully controlled amounts of food, to enforce accurate observations and data point measurements, and avoid contaminating results. As well, this type of controlled experiment avoids the pitfalls of the "free-living" diet study, where participants supposedly eat some kind of diet, but no one can say for certain exactly what that diet might have been, because no one's following each of them around and keeping the would-be-necessary tabs. It is merely presumed that the prescribed diet is adhered to throughout the study. (It's true that metabolic ward studies of this kind may have little practical relevance to the reality of our day-to-day lives, in cases like this where one of the experimental diets is unsustainable outside of the laboratory setting, but that's actually beside the point - the research question (primary outcome) wasn't "Can people adhere to an 8% fat diet, in real life." It was, apparently, "Is Gary Taubes' carbohydrate-insulin hypothesis of obesity correct?")*

*Something that was brought to my attention after I originally published this blog post, and which I think is a fair point, is that, since this was clearly not a "diet trial," -- although, everyone and their grandmother will misinterpret this aspect of the methodology to exploit their personal bias, whether low fat or low carb -- but a human nutritional biochemistry experiment to test the effects of an acute-phase response of insulin on adipocyte metabolism, why not put one group on a 0% CHO diet (for RC) and the opposite (RF) on a 0% fat diet and then make both arms isocaloric, thus, taking the issue of changes in protein right off the table, and fixating exclusively on the question of "What about changes in insulin?" As I say, it is a fair point. After all, what would it really hurt? And if it wasn't a diet trial, but an experimental model of metabolic biochemistry...

Anyway, getting to the point, here...

19 obese, but otherwise seemingly-healthy, subjects (all free from diabetes), between the ages of 18 and 45 years, were selected to complete this trial. 9 women and 10 men were randomized to either a RC or RF arm for 6 days, after a 5 day observation, prior to the intervention, to maintain and document a strict caloric intake:

The inclusion criteria was one of the things about the study that I thought was quite good.

Although, overall, this seems to be a rather small sample, statistically-speaking, it's difficult to conduct a much larger metabolic ward experiment of this type, due to a variety of factors, not the least of which is the expense. However, per their Statistical Analyses, it appears to be adequately powered (at 80%) to detect the changes they're hoping to observe. (As I'm not a statistician by trade, please realize that I'm sort of taking them at their word for this.) As is common, their alpha-level (significance) was set at p < 0.05.

Of the things I enjoyed about this study, one was that the authors controlled for intentional physical activity, by requiring each study participant to partake in the exact same level and intensity of treadmill exercise, throughout the intervention. As someone who tends to pick apart research by the number of possible confounders I can conceive and apply to the data, this one made me smile. They even made sure the female subjects were in the follicular phases of their menstrual cycles, prior to initiating their interventions. Impressive. (And, unless I'm reading them incorrectly, it wasn't just intentional exercise; they also monitored daily walking, with a hip-worn, pager-like accelerometer. This is important, as changes in physical activity, non-exercise activity thermogenesis (NEAT), etc., can add up quickly and could have muddied the energy expenditure data.)

One of the things I was surprised about was that HOMA-IR scores, prior to the intervention periods of the experiment, averaged about 2.7; pretty clearly indicating insulin resistance in these obese persons. Yet, in all the other literature I've seen, thus far, RC diets tend to be more effective for fat loss in IR persons than RF diets do. Clearly, 8% fat is very different, metabolically, than the usual 30% fat (LF) diet. Then again, perhaps it's too premature to anything like this, since subjects spent just 6 days in a chamber, and insulin levels were not significantly different between groups....

The aspect of this study that bothers me - predominantly due to the many silly extrapolations into real world advice to "eat low-fat for fat loss" it has since generated - is that the intervention period was only 6 days. (Now, let's be clear. It was 6 days, per intervention. Let's not forget, this was a cross-over design. In between interventions, each participant underwent a 2- to 4-week "washout" period, before entering the next 6-day intervention.) Still, 6 days is incredibly short, particularly considering the primary outcome they were testing. As many low-carbers have vehemently (and perhaps rightly) argued, 6 days is an insufficient time for one to fully fat-adapt.[3] Keep in mind, once again, this was not a ketogenic diet trial; we aren't talking keto-adaptation, per se. But, it is still quite possible that there are enzymatic changes that may take place, even in the context of a moderately low-carbohydrate diet, such as this one was, which did not have sufficient time to develop and persist. (e.x. Even though fat oxidation did just begin to ramp up in the RC arm, it's likely that these people were still metabolizing glycogen for a few days, all while the RF subjects were starved of almost any incoming fat.)

Sharing the opinion that much more adaptation time may be needed to detect any of the meaningful metabolic changes Hall et al. were looking for - although perhaps related more to transgenerational epigenetic reasons than acute metabolic changes - is respected scientist and renowned islet cell and diabetes researcher, Dr. Jim Johnson (below), who has recently published a paper demonstrating that hyperinsulinemia is a prerequisite for the development of obesity in a genetically-humanized mouse model, and that suppressing it early on can determine the course of that animal's lifetime susceptibility to developing obesity:[4]*

*Don't misinterpret, here, please: I don't mean to suggest that we should extrapolate from rodent data before we do so from human experiments. I'm merely suggesting we ought to be careful extrapolating from a trial of exceptionally short-duration, whose tendentious and yet unreplicated results stand in opposition to mountains of literature regarding historic high-carbohydrate diets, hyperinsulinemia & insulin resistance syndrome and obesity in a variety of experimental, clinical and observational models.

For what it's worth, many of the metabolic ward diet studies I have read have been equally brief. 6-8 days is fairly common for controlled feeding studies. But I don't think it wise to extrapolate widely from those data, either. It's not as if Dr. Hall and his colleagues have committed some egregious misdeed by intentionally undercutting things. I don't think it would be fair to assume this was the case - although, I think it's understandable, to a degree, that some people are left wondering if this wasn't an easy and ill-intentioned way to make the RF arm look better in a shorter amount of time. The troubling thing is when one sees the preliminary results of a 6-day trial and proclaims to the world, "This style of eating works! Low-fat for the win!" This is where I take issue, and where I disagree. (Though, it should be noted that droves of low-carbers do the same thing with preliminary data favoring low carb, which is equally problematic.)

Even if the results of this study prove true, and end up being generalizable, over time - which seems like a long-shot, from my perspective - it doesn't address whether the dietary composition used to achieve 8% of Calories as fat would be healthy. Nor does it suggest whether this eating style is something one could adhere to for longer than 6 days, let alone long-term. It's one thing to force an individual to consume cardboard, while you watch them through a hole in their box to make sure they choke down the last bit, before shutting them back in. It's quite another to expect that they could do it, themselves, in free-living circumstances. (For the record, Dr. Hall and his team already realize this, as well):

Sounds appealing. Sign me up.

8% of Calories as fat (or roughly 17 grams of fat, per day) is equivalent to a little less than 140 grams of lean chicken breast. Are you prepared to count out your walnuts, one by one, and totally avoid otherwise conventionally-healthful fatty foods, like olive oil and avocado, maybe even become a vegan (because it seems the only plausible way to maintain such a low level of total fat intake, on a day-to-day basis)? Good luck with that. Please, report back with your quantified-self results.

This, of course, is only where we stand if we choose to accept these data at face value. I am not ready to do that. There appear to be a few important inconsistencies between the conclusions drawn in the discussion and the raw data.

Firstly, they make us aware that each subject underwent a DEXA scan, to assess fat mass and body fat percentage, but - disregarding that the title of the paper claims that fat loss was superior in the RF arm - no statistically significant differences in either fat mass or body fat percentage could be seen, comparing DEXA scans between the RF and the RC subjects. (p < 0.78 & p < 0.24, respectively; a la Table 3, pictured below.)

As you can see at the bottom of the table, however, "cumulative fat imbalance" differed between groups by 218 grams. (Which, for us Americans, equates to approximately 0.48 lb. of fat.) The alpha-level was far exceeded (<0.0001), in stark favor of the RF arm.*

*Considering that a relatively large number of significance tests were performed, and DEXA was apparently not sensitive enough to detect the ~0.5 lb. fat loss, is it possible that these findings might have been the result of a type I error? I ask only because of the discordance between the cumulative fat imbalance data and the DEXA results. DEXA should have been sensitive enough to pick up a 0.48 lb. change in fat mass, no?

Someone suggested to me that perhaps the DEXA scans were not sensitive enough to pick up the results of said fat imbalance, due, perhaps in part, to some potential concerns with respect to soft-tissue fluid status and the impact of hydration on the intra- and inter-individual variability of DEXA results. Although this sounds plausible, I gather that this hypothesis is predicated on the idea that RC dieters have likely lost some body water due to heightened fluid excretion and depletion of glycogen - as evidenced by the statistically significant changes that were seen in body weight and BMI, which are shown at the top of the table. And I think he's quite right; it's unlikely that participants lost 1.2 lb. of muscle mass in just 6 days; it's probably mostly losses in tissue hydration and glycogenolysis. Unfortunately, all the data I've seen on this topic suggests that fluid overload is more important than fluid excretion, with respect to DEXA screening, and that its impact has more to do with results of lean mass than fat mass[5]. Nevertheless, there is some data suggesting physical mechanisms for why this might be the case, but the concern is minimal, to the point that I'm not sure it makes any practical difference:[6]

Therefore, I am left to think that, since DEXA revealed no significant difference, and because DEXA tends to be the most accurate - albeit not necessarily always the most precise - anthropometric measure we have to assess changes in body fatness that doesn't cost an arm and a leg to implement, it might be the case that this aforementioned claim is false, and that the most any reputable thinker would reason that the conclusions should have read as follows:

Body fatness decreased slightly, concomitant with the total Caloric deficit from baseline, regardless of any changes in the macronutrient composition of the two diets. Between-group DEXA analyses revealed no statistically significant difference in body fatness, after the 6-day intervention periods.

One of the things that concerns me, simply because the data is omitted, and I really think it ought to at least be included in the supplemental materials, is that one of the female participants in the trial - and it was never explained from which arm she belonged - achieved a DEXA result that was a "clear outlier," and was excluded from the statistical analyses. In the supplemental materials, the authors briefly explain that including her results in the analysis would have skewed the distribution - and yet, Dr. Hall's e-mail correspondence with me conflicts with this, as he claimed to have run the computations both including and excluding this subject and that neither significantly changed the results in any meaningful way. But, in a study of 9 women, doesn't every one of their individual data points matter, on some level? Regardless, shouldn't that data be readily available for consumers? And, lastly, what exactly does it mean that her results were "not physiological?" Did she have unexpected lipoplasty before the scan?:

This may come across as accusatory, and I don't intend for it to, but, from what I understand, these data are represented as SEM (or standard error of the mean), as opposed to SD (standard deviation), and, while this may not be the case here, it is widely accepted in statistics that it is easier to emphasize small, but unimportant differences in the data with SEM than with SD, especially with small sample sizes. Since I don't have access to the data necessary to do the appropriate calculations to switch it to standard deviation (because it is not made readily available), I cannot see what the distribution would look like, afterward, but it definitely makes me curious....

Lastly, and the thing that rattles my chain the most, is this:

Someone please explain to me how it is responsible to extrapolate from a 6-day controlled feeding study of 17 subjects, two of which were eating the wrong diet altogether for one of those days? This trial appears seriously flawed on that count alone.

Overall, the experimental design looks good. I found myself impressed with many things about it. And I also think (assuming Dr. Hall and his team can provide some answers to these questions and wouldn't mind relinquishing the rest of the data to absolve the trial of these potential water-marks), it may, if not provide insights into the questionable effects of the immediate impact of insulin secretion on whole-body adipose tissue metabolism directly, serve to ask further questions regarding the nature of the application of this hypothesis to human beings. But, all things considered, it's a big "what if."

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

There is one aspect of this study that I have intentionally omitted from this post. The mathematical models with which Hall, et al have extrapolated weight loss out 6 months from the end of the trial. (The model is described in detail in a separate PDF, in the supplemental materials section.) Although they are intriguing, in their own right, they mean little to me until they have been corroborated with real world data. At this point, they're just mathematical hypotheses, which doesn't interest me. I am only interested in real-time physiological data.

I must admit, I have not tried to go through those numbers to get a more complete understanding of his mathematical model of metabolic regulation, but, I will say this... even if we make the enormously presumptuous leap of faith that Dr. Hall's model is flawless, and he could find subjects who could adhere perfectly to this RF diet for a full 6 months, can you guess how much more weight they will lose, compared to their RC comparators, who get to eat virtually all the fat they want? 3 kg. (or 6.6 lb.). That's roughly 0.25 lb. per week; barely more than intense exercise seems to provide, over the same time interval, which we know is minuscule compared to the benefits carbohydrate restriction seems to provide in free-living experiments. Hardly worth the sacrifice, I think.*

*Some people are bound to jump to the erroneous conclusion that, because an 8% fat diet "worked," any generic fat reducing approach will do the same. Sorry, no cigar. I'd be interested to see what Hall's mathematical models would look like if he assumed even a 2% error in compliance. Let's assume someone complied almost perfectly, and ate, say, 10% dietary fat. How would this alter the extrapolated mathematical data?

Once again, although interesting, this is just a hypothesis, and is predicated on a the results of 17 people after a 6 day trial, which may have gone differently had more adaptation time been given and the trial time extended, or the sample size increased. If we ignore their mathematical extrapolations, which is what I chose to do here, the most a reasonable person might say regarding the actual - as opposed to reported - outcome is something like: "After 6 days of eating RF or RC, participants who ate RF seemed to have an average of 0.4 lb. less body fat than the RC group, but which DEXA was not sensitive enough to detect." And this was a massive and unrealistic reduction in dietary fat, and it was still either equivalent or barely better than the RC group, that ate a comparatively whopping 108 g fat per day. Already, there are hundreds of people saying, "See, fat does make us fat!" I suppose that's one rather silly way to look at it. Then again, both groups lost weight and body fat from baseline, due to the reduction in total Calories, and, even though the RF group ate substantially less fat, they barely had any discernible differences in outcome. Imagine that.

Perhaps the most important and, indeed, most relevant aspect of this study was the authors' attempt to determine whether a reduction in serum insulin could accelerate lipolysis, as predicted by so-called low-carb dogma (specifically Taubes, et al), and whether, leaving insulin elevated, as in the RF arm, would result in an "inability" to mobilize and subsequently burn fat. This was the ultimate research question, and the reduction in serum insulin levels between groups were comparable, and did not reach the statistical significance they were hoping to see (p < 0.48), despite the graphic they used to show a slight reduction in fasting insulin in the RC, as compared to RF arm.

(Table 4):

So, it seems the real answer to the original research question posited by Hall et al. is more like a resounding "We can't really know, based on these data alone," than a clear-cut yes or no, regardless of what some people would like to have you believe.

One 6-day intervention study of 17 participants proves nothing, nullifies nothing and is generalizable to no one. (But, it could possibly provide insights or generate new questions, if it is replicated and these results can be reproduced (a bit longer), and insulin levels actually differ more significantly between comparator arms.)

* * * * *

Important Takeaways (TL;DR):

(1) This experiment was not designed to test diet efficacy for real-world living, it was an attempt to prove the carbohydrate-insulin hypothesis of obesity wrong.

(2) Insulin levels fell only slightly more in the RC arm than in the LF arm, but not enough to be statistically significant or clinically important between groups, thus invalidating the conclusion, from my perspective. (And perhaps only suggesting: "Low carb isn't the only way to reduce circulating insulin or improve insulin sensitivity.")

(3) Controlled feeding studies of this sort may say nothing generalizable to everyday life, or to designing pragmatic nutritional strategies. (Regardless of what the LA Times seems to think.)

(4) The study design was interesting, and quite good. Aspects of the implementation, however, turned me off. (e.g. Feeding 1 RC-er RF on the first day, and 1 RF-er RC on the last day. Mistakes like these can turn out to be study-ruiners, especially when the totality of the time under observation is 6 days per intervention.)

(5) One of the not-so-exceptional aspects of this methodology was brought to my attention by a reader who commented over at Hyperlipid: "...the biggest flaw in this study is that it was not done in a steady state." This calls into question extrapolating from metabolic data collected in an state that has not yet reached an appropriate equilibrium, and speaks back to the lack of an essential adaptation period on the RC diet. This is critical.

(6) DEXA results do not corroborate the message that is being brought to the attention of every media outlet who will listen that the RF arm lost more body fat than RC. Therefore, I'm inclined to think this wasn't actually the case. (If you can't show it, it didn't happen. And, if you have to show it with a microscope, it's not clinically relevant.)

(7) For the sake of argument, let's assume that none of the issues I've brought up are of any real concern. The most accurate thing one can say about these results is not: "More body fat loss with low fat," but something like... "According to these preliminary data, after 6 days in a strictly and artificially controlled environment, a very low-fat diet, compared to an isocaloric, euproteinemic, reduced-carbohydrate diet, appears to have the potential to reduce adiposity in obese, adult human beings by roughly 218 grams (or ~0.22 kg) by day 6. It is unclear whether these results are generalizable to a wider sample of the population, whether they might persist in this manner, as time goes on, whether extrapolating them to free-living conditions is warranted, and whether either dietary model would serve to improve human health over time. More studies are needed to address these questions."* And the issues are concerning.

EDIT: The study protocol for this experiment was finally released, but are, in my opinion, woefully inadequate. Sampling method? Not provided. Intervention? Not provided. All outcome measures? Not provided


[1] Hall, Kevin D., et al. Calorie for Calorie, Dietary Fat Restriction Results in More body Fat Loss than Carbohydrate Restriction in People with Obesity." Cell Metabolism, Available online, 13 August 2015.
[2] Goodman, Brenda. "Low-Carb Diets May Not Work the Way We Think They Do." WebMD, 13 Aug. 2015. Web. 14 Aug. 2015.
[3] Westman, Eric C., et al. "Low-carbohydrate nutrition and metabolism." The American journal of clinical nutrition 86.2 (2007): 276-284.
[4] Templeman, Nicole M., Susanne M. Clee, and James D. Johnson. "Suppression of hyperinsulinaemia in growing female mice provides long-term protection against obesity." Diabetologia (2015): 1-11.
[5] Horber, F. F., et al. "Impact of hydration status on body composition as measured by dual energy X-ray absorptiometry in normal volunteers and patients on haemodialysis*." The British journal of radiology 65.778 (1992): 895-900.
[6] Pietrobelli, Angelo, et al. "Dual-energy X-ray absorptiometry: fat estimation errors due to variation in soft tissue hydration." American Journal of Physiology-Endocrinology And Metabolism 274.5 (1998): E808-E816.