WEIGHING UP THE EVIDENCE IN OSTEOPOROSIS - ARE YOU SURE YOU'RE SURE?

Management of osteoporosis is commonplace in general practice, both in terms of prevention and treatment of the established disease. With an evidence-based approach now fundamental to disease management, a symposium at the Sixth Bath Conference on Osteoporosis and Bone Mineral Measurement considered how the quality of data on osteoporosis can be assessed and applied to everyday clinical practice to improve patient care.

"When treating patients, we are increasingly being made to be more conscious about the decisions we are making and the evidence on which they are based", said the chairman of the meeting, Dr Anthony Woolf', consultant rheumatologist at the Royal Cornwall Hospital. He noted that although doctors may believe that medicine is an art form based on taking a good history and clinical examination, there is increasing pressure to rationalise medical treatment. "If we do not rationalise the way we treat our patients by rising the evidence sensibly, then I believe that our care will be rationed by others," he argued.

Over recent years, many data on osteoporosis have been generated from large clinical trials and routine clinical practice. This wealth of experience needs to be reviewed in order to establish which data are strong, and which are invalid or irrelevant.

What evidence do we need in osteoporosis?

"We have to realise that evidence has different weights", said Professor John Kanis, professor in human metabolism and clinical biochemistry at the WHO Collaborating Centre for Metabolic Bone Diseases, Sheffield. He emphasised the need to rank evidence, with expert opinion being at the lowest level and meta-analyses of well-designed randomised-controlled trials being at the highest level.

Professor Kanis argued that the logic behind observational studies was fundamentally flawed; "you are trying to infer causality from an observation and clearly the logic behind that is not entirely perfect". Many different forms of bias can be introduced, from informational bias, where the information gathered from people may not be correct, to selection bias, where the group that observations are made on are assumed to be representative of the entire population.

If a trial does not have a control group then it is impossible to tell whether effects observed are due to treatment or just the natural course of the disease. If the trial is not double-blinded, then bias may be introduced from the clinician or the patient, and if the groups are riot randomised, then bias could also be introduced from the patient selection procedures.

The double-blind, randomised, placebo-controlled trial is the gold standard of trial design. However, many additional factors have to be taken into account to establish how good a trial is, such as:

• is the study population appropriate for the target group for the drug?
• is the profile of the original trial population the same as the profile of those who completed the study?
• are there sufficient numbers to detect effects to a significant degree?

The endpoint of the trial also has to be appropriate, and the measurements of that endpoint accurate. In the case of osteoporosis, the most relevant endpoint is fracture. Although bone mineral density is often measured, air increase in bone mineral density does not necessarily correlate with a reduction in fractures, and fractures are the reason for loss of quality of life in patients.

Another point to consider is publication bias; trials that show beneficial effects are not considered as exciting as those showing harmful effects.

"We also need evidence not only of efficacy, but of effectiveness," Professor Kanis concluded. It has to be shown that the treatment will have an impact on the intended population. Finally, however well designed a trial is, the characteristics of patients in the trial population will not necessarily reflect those of patients in the real clinical setting; "there is a leap of strategic thinking to go from the randomised controlled study into developing satisfactory strategies for the osteoporotic population".

What do we know about current therapies for osteoporosis?

According to Dr Peter Selby, a consultant physician at Manchester Royal Infirmary, HRT is generally considered to be the gold standard intervention for osteoporosis. "The problem is that the evidence to support its use is not particularly good." He explained that although there is a substantial amount of evidence for a primary preventive effect on bone mass endpoints, there have only been observational studies looking at fracture endpoints.

"In terms of treating people with established disease, only one, randomised controlled trial(1) has been published, and even that has its problems." The trial looked at the effect of transdermal oestradiol and showed a 50 per cent reduction in vertebral fractures among patients given the drug. Although the trial was well designed, it had very few patients in each treatment group and was conducted over a short period of time. As a result, although the data are consistent with other epidemiological studies, this trial should not be accorded too much weight. "Although we do talk about HRT as being the gold standard, that is based more on faith than evidence", Dr Selby commented.

Trials of calcium and vitamin D, however, are more numerous, and they do show significant reductions in fractures. Criticisms of the studies largely focus on the selective populations used, but due to the number of studies showing similar results, it is generally felt that this can be overlooked.

With regard to calcitriol, Dr Selby said, "There have been several studies that looked at the effect of calcitriol on bone density and I think it is reasonable to say that they tend to point to the fact that over a couple of years it has a relatively modest effect on bone density. It is certainly better than placebo, but is not as potent as HRT or the bisphosphonates."

The data on fractures are a little harder to interpret. Small studies in the USA have had conflicting results and so have been generally dismissed. A trial from New Zealand(2) is regularly cited, however, the most impressive finding was a rise in fractures in the placebo group. "This does not show that calcitriol does not reduce the fracture rate, but I think that until we have some confirmatory evidence, it is difficult to accept it as absolutely hard data that calcitriol does have that effect."

"The agents that have had the greatest impact on the management of osteoporosis, and indeed a lot of other metabolic bone diseases, are the bisphosphonates." There are only two bisphosphonates available for the treatment of osteoporosis in the UK: etidronate was the first, followed by alendronate.

The evidence for etidronate increasing bone mass is good. "There is no doubt that etidronate is capable of producing a significant effect on bone mass in the lumbar spine, a modest effect at the femoral neck and a neutral effect at the wrist." However, the evidence for the drug producing a reduction in fractures is more sparse, with only three studies published.

The USA study(3) showed a 50 per cent reduction in vertebral fractures in patients on etidronate in the first two years. However, in the third year, all patients were switched to active treatment and the fracture rate was lower than the etidronate-treated group in the first two years. According to Dr Selby, this may have been because there were insufficient numbers of patients in the treatment groups. "Although I have no doubt that etidronate does reduce the risk of vertebral fractures, the actual magnitude of risk reduction is still in doubt.

"A study conducted in Denmark(4) suggested efficacy in fracture prevention; however, there was a high dropout rate due to the elderly population involved.

A British study used the General Practice Research Database to look at patients at risk of hip fracture.(5) In the first instance, this may appear to be a robust study design because it examines what happens in the real world. "The problem is because it was an observational study and uncontrolled, there may be factors that we do not know about," said Dr Selby. He explained that the basis of diagnosis may have been different in the different treatment groups, and so although the study showed a reduction in hip fracture with etidronate, we do not have as much confidence in this sort of study as we would have in the randomised controlled trial".

In contrast, there is a good deal of data on alendronate, even though it has been available for a shorter time than etidronate. The best data have come from a randomised controlled trial of 1000 women(6) that showed significant effects on the spine and femoral neck with 10mg alendronate. It also showed a significant reduction in new spinal fractures, although the study was not just conducted at the dosage level of alendronate now used in clinical practice.

This trial was followed by the Fracture Intervention Trial(7) a landmark randomised controlled study involving over 2000 patients with low bone mass and prevalent vertebral fracture. The trial was intended to run for three years, but it was halted early on ethical grounds due to the impressive results in the treatment group. The trial showed a reduction of approximately 50 per cent in fracture at the wrist, hip and in the spine.

How should we put evidence into practice?

The Royal College of Physicians is currently developing guidelines for the treatment of osteoporosis that should be published later this year. Professor Cyrus Cooper, Professor of Rheumatology at Southampton General Hospital, discussed how these guidelines should help and not hinder clinical practice.

"{Guidelines} should in no way be viewed as some kind of external gold standard - an external benchmark against which good or bad practice can be somehow assigned on some universal scale," said Professor Cooper. He explained that guidelines should be enabling in their nature: "they should help us to organise complex series and pieces of information, they should help us to question what we are doing to try and improve patient care and service delivery".

Professor Cooper discussed how guidelines could be used for different purposes, from enhancing clinical decision making, to determining the eligibility of various treatment approaches and assessing the quality of care. "I view guidelines for osteoporosis as a crutch, an enhancement to clinical decision making."

He explained how types of guidance can be very different, ranging from critical pathways to broad frameworks. Their intended uses could also be quite different, from improving the effectiveness of care delivery to maximising

cost-effectiveness or even helping doctors become more confident about their decision making. Guidelines could be based exclusively on the strength of evidence for each intervention; however, 'there is ... for many of us a gap between what we perceive to be the strength of the evidence base and perhaps how we execute that in our clinical practice". So there is variability in the effect that guidelines have, from being followed extensively, to just being a starting point for discussion.

"The stages of guideline development that were outlined by the Agency for Healthcare Policy Research in the United States were: formulation of the problem; identification and interpretation of the evidence; estimation of benefits and hazards; comparison of these benefits and hazards; estimation of the cost-effectiveness of the approach; and then agreement of the set of recommendations."

In osteoporosis, Professor Cooper explained, "we are only just beginning to characterise properly how to formulate the problem and to identify, synthesise, and interpret the evidence base". In developing guidelines, the evidence base has to be assessed for each intervention and ranked according to strength. If cost-effectiveness is the main criterion, then economic comparisons of interventions have to be made.

Professor Cooper cited the example of using bone scans before instigating drug treatment(8) "For a treatment costing £100 a year that is targeted on the basis of bone density to people with the clinical indications, it costs us £366 per averted fracture. Whereas, in contrast, it costs its £1207 per averted fracture when all people with that indication are given the drug."

He explained that for treatments costing less than £50 per year, treating all at-risk groups would save per averted fracture and it would actually cost more to scan patients. However, the greater the cost above £100, the greater the saving if the diagnosis is confirmed. This means that if low-cost drugs are to be used, there is little point in using bone densitometry to confirm the diagnosis. However, when the drugs used are high cost, it is more cost-effective to confirm the diagnosis.

"We have quantified fracture impact, we know that risk prediction is possible in individuals and we know that there are treatments available to reduce bone loss and prevent fracture. These will form the planks of any current approaches to deriving guidelines." He explained that the relative efficacy of different treatments was hard to gauge and that individualised regimens are needed for high-risk patients, although it is hard to say how strict these should be.

Professor Cooper concluded; "to adopt these in clinical practice with a cost-effectiveness basis is a challenge which we still find difficult, but one that, I suppose, our healthcare planning colleagues may insist upon us doing in the difficult, but rewarding, years to come."

Key points

• To rationalise the treatment of osteoporosis, data from well-constructed trials are needed to provide a meaningful evidence base
• The meta-analysis of well-designed randomised-controlled trials is the highest ranking evidence while expert opinion is the lowest
• The most relevant endpoint for osteoporosis trials is fracture rather than bone mineral density
• High-quality evidence for fracture reduction either in prevention or as a treatment for osteoporosis is lacking with HRT
• Trials of calcium and vitamin D show significant reductions in fractures
• The evidence for the beneficial effects of calcitriol on fracture endpoints is unconfirmed
• Evidence on etidronate's effect on fractures, although encouraging, is sparse and open to criticism
• A significant reduction in fractures of the hip, spine and wrist has been shown with alendronate
• Guidelines are to be drawn from the evidence base by ranking each intervention, but their usefulness depends on criteria employed in clinical practice

References

1. Lufkin EG, Wahner HW, O'Fallon WM, et al Treatment of postmenopausal osteoporosis with transdermal estrogen. Ann Intem Med 1992;117:1-9.
2. Tilyard MW, Spears GFS, Thomson J, Dovey S. Treatment Of postmenopausal osteoporosis with calcitriol or calcium. N Eng J Med 1992;326:357-62.
3. Harris ST, Watts NB, Jackson RD, et al. Four-year study of intermittent cyclic etidronate treatment of postmenopausal osteoporosis: three years of blinded therapy followed by one year of open therapy. Am J Med 1993;95:557-67.
4. Storm T, Kollerup G, Thamsborg G, et al Five years of clinical experience with intermittent cyclical etidronate for postmenopausal osteoporosis. J Rheumatol 1996;23:1560-4.
5. Van Staa TP, Abenhaim L, Cooper C. Use of cyclical etidronate and prevention of non-vertebral fractures. Br J Rheumatol 1998;37:87-94.
6. Liberman UA, Weiss SR, Bröll J, et al. Effect of oral alendronate on bone mineral density and the incidence of fractures in postmenopausal osteoporosis. N Eng J Med 1995;333:1437-43.
7. Black DM, Cummings SR, Karpf DB, et al. Randomised trial of effect of alendronate on risk of fracture in women with existing vertebral fractures. Lancet, 1996;348:1535-41.
8. Kanis JA, Delmas P, Burckhardt P, Cooper C, Torgerson D. Guidelines for diagnosis and management of osteoporosis. Osteoporosis International 1997;7:390-406.