Exploring sources of heterogeneity

Some observers suggest that meta-analysis of observational studies should be abandoned altogether.64 We disagree, but think that the statistical combination of studies should not, in general, be a prominent component of reviews of observational studies. The thorough consideration of possible sources of heterogeneity between observational study results will provide more insights than the mechanistic calculation of an overall measure of effect, which may often be biased. We re-analysed a number of examples from the literature to illustrate this point. Consider diet and breast cancer: the hypothesis from ecological analyses65 that higher intake of saturated fat could increase the risk of breast cancer generated much observational research, often with contradictory results. A comprehensive meta-analysis66 showed an association for case-control but not for cohort studies (odds ratio 1-36 for case-control studies versus rate ratio 0-95 for cohort studies comparing highest with lowest categories of saturated fat intake, P = 0-0002 for difference) (Figure 12.4). This discrepancy was also shown in two separate large collaborative pooled analyses of cohort and case-control studies.67,68 The most likely explanation for this situation is that biases in the recall of dietary items, and in the selection of study participants, have produced a spurious association in the case-control comparisons.68

That differential recall of past exposures may introduce bias is also evident from a meta-analysis of case-control studies of intermittent sunlight exposure and melanoma69 (Figure 12.4). When combining studies in which some degree of blinding to the study hypothesis was achieved, only a small and statistically non-significant effect (odds ratio 1-17, 95% confidence interval 0-98 to 1-39) was evident. Conversely, in studies without blinding, the effect was considerably greater and statistically significant (odds ratio 1-84, 1-52 to 2-25). The difference between these two estimates is unlikely to be a product of chance (P = 0-0004 in our calculation).

The importance of the methods used for exposure assessment is further illustrated by a meta-analysis of cross-sectional data of dietary calcium intake and blood pressure from 23 different studies.70 As shown in Figure 12.5(a), the regression slope describing the change in systolic blood pressure (in mmHg) per 100 mg of calcium intake was reported to be strongly influenced by the approach employed for assessment of the amount of calcium consumed. The association was small with diet histories (slope -0-01) and 24-hour recall (slope -0-06) but large and statistically highly significant when food frequency questionnaires, which assess habitual diet and long-term calcium intake, were used (slope -0-15). The authors argued that "it is conceivable that any 'true' effect of chronic dietary calcium intake on blood pressure or on the development of hypertension could be estimated better by past exposure since it allows for a latency period between exposure and outcome".70 However, it was subsequently pointed out71 that errors had occurred when extracting the data from the original publications. This meant that the weight given to one study72 was about 60 times greater than it should have been and this study erroneously dominated the meta-analysis of diet history trials. Correcting the meta-analysis for this error and several other mistakes leads to a completely different picture (12.5(b)). There is no suggestion that the explanation put forward by the authors for the different findings from studies using different dietary methodologies holds true. This is another demonstration that plausible reasons explaining differences found between groups of trials can easily be generated.73 It also illustrates the fact that the extraction of data from published articles which present data in different, complex formats is prone to error. Such errors can be avoided in collaborative analyses where investigators make their primary data available.74

Fat Intake Screening
Figure 12.4 Examples of heterogeneity in published observational meta-analyses: saturated fat intake and cancer,66 intermittent sunlight and melanoma,69 and formaldehyde exposure and lung cancer.75 SMR: standardised mortality ratio; CI: confidence interval.

Figure 12.5 Relation between dietary calcium and systolic blood pressure by method of dietary assessment. Initial analysis, which was affected by data extraction errors (a),70 and corrected analysis (b).71 CI: confidence interval.

Figure 12.5 Relation between dietary calcium and systolic blood pressure by method of dietary assessment. Initial analysis, which was affected by data extraction errors (a),70 and corrected analysis (b).71 CI: confidence interval.

Analyses based on individual participant data (see also Chapter 6) also allow a more thorough investigation of confounding factors, bias and heterogeneity.

An important criterion supporting causality of associations is the demonstration of a dose-response relationship. In occupational epidemiology, the quest to demonstrate such an association can lead to very different groups of employees being compared. In a meta-analysis examining formaldehyde exposure and cancer, funeral directors and embalmers (high exposure) were compared with anatomists and pathologists (intermediate to high exposure) and industrial workers (low to high exposure, depending on job assignment).75 As shown in Figure 12.4, there is a striking deficit of lung cancer deaths among anatomists and pathologists (standardised mortality ratio [SMR] 33, 95% confidence interval 22 to 47) which is most likely to be due to a lower prevalence of smoking among this group. In this situation few would argue that formaldehyde protects against lung cancer. In other instances such selection bias may be less obvious, however.

In these examples heterogeneity was explored in the framework of sensitivity analysis76 (see Chapter 2) to test the stability of findings across different study designs, different approaches to exposure ascertainment and to selection of study participants. Such sensitivity analyses should alert investigators to inconsistencies and prevent misleading conclusions. Although heterogeneity was noticed, explored and sometimes extensively discussed, the way the situation was interpreted differed considerably. In the analysis examining studies of dietary fat and breast cancer risk, the authors went on to combine case-control and cohort studies and concluded that "higher intake of dietary fat is associated with an increased risk of breast cancer".66 The meta-analysis of sunlight exposure and melanoma risk was exceptional in its thorough examination of possible reasons for heterogeneity and the calculation of a combined estimate was deemed appropriate in one subgroup of population-based studies only.69 Conversely, uninformative and potentially misleading combined estimates were calculated both in the dietary calcium and blood pressure example70 and the meta-analysis of occupational formaldehyde exposure.75 These case studies indicate that the temptation to combine the results of studies is hard to resist.

Diet Tweak System

Diet Tweak System

Trying To Lose Weight Can Be Tough. But... Not Losing Weight and Gaining What You Lost Back, Sucks. If you've ever felt that no matter what you do to lose weight nothing seems to work. If you've ever felt that there has got to be some kind of a system or way to lose weight...but just have not found it yet.

Get My Free Ebook


Post a comment