Open Access Research

Helping to prioritise interventions for depression and schizophrenia: use of Population Impact Measures

Richard F Heller*, Islay Gemmell and Lesley Patterson

Author Affiliations

Evidence for Population Health Unit, Division of Epidemiology and Health Sciences, University of Manchester, Manchester M13 9PT, UK

For all author emails, please log on.

Clinical Practice and Epidemiology in Mental Health 2006, 2:3  doi:10.1186/1745-0179-2-3

The electronic version of this article is the complete one and can be found online at: http://www.cpementalhealth.com/content/2/1/3


Received:21 October 2005
Accepted:22 March 2006
Published:22 March 2006

© 2006 Heller et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

To demonstrate the potential of Population Impact Measures in helping to prioritise alternative interventions for psychiatry, this paper estimates the number of relapses and hospital readmissions prevented for depression and schizophrenia by adopting best practice recommendations. The results are designed to relate to particular local populations.

Methods

Literature-based estimates of disease prevalence, relapse and re-admission rates, current and best practice treatment rates, levels of adherence with interventions and relative risk reduction associated with different interventions were obtained and calculations made of the Number of Events Prevented in your Population (NEPP).

Results

In a notional population of 100,000 adults, going from current to 'best' practice for different interventions, the number of relapses prevented in the next year for schizophrenia were 6 (increasing adherence to medication), 23 (family intervention), 43 (relapse prevention), and 44 (early intervention); and for depression the number of relapses prevented in the next year were 100 (increasing care management), 227 (continuing treatment with antidepressants), 279 (increasing rate of diagnosis), and 325 (Cognitive Behaviour Therapy). Hospital re-admissions prevented in the next year for schizophrenia were 6 (increasing adherence to medication), 36 (relapse prevention) and 40 (early intervention).

Conclusion

Population Impact measures provide the possibility for a policy-maker to see the impact of a new intervention on the population as a whole, and to compare alternative interventions to best improve psychiatric disease outcomes. The methods are much simpler than others, and have the advantage of being transparent.

Background

There are a number of potential interventions in psychiatry that would help improve patient outcomes. The problem is how to decide which ones to prioritise for service development. While measures, such as Number Needed to Treat (NNT) and Quality Adjusted Life Years (QALY), do provide estimates of the benefit of interventions, they do not allow impact at the population level to be quantified. We have developed a new set of Population Impact Measures (PIMs) to describe the population impact of risks and benefits [1-4]. Population Impact Measures are simple to compute, and contain the elements to which policy-makers would have to pay attention in the commissioning or improvement of services. For describing the population impact of an intervention, the Number of Events Prevented in a Population (NEPP)[2] describes the impact of treatment or other interventions and is defined as "the number of events prevented by the intervention in your population over a defined time period".

The measure NEPP is a population extension of the well-known Number Needed to Treat (NNT), and takes into account the frequency of the condition in the population and the proportion of those with the condition who are actually exposed to the intervention. The measure provides local context to previous measures, allowing policy-makers to identify and prioritise the potential benefits of interventions on their own population. This paper derives NEPP for interventions used in two important psychiatric conditions, and demonstrates the potential for helping to prioritise interventions to make maximal impact on the population.

Methods

We have obtained literature-based estimates of the components required to calculate the NEPP for a number of different interventions for schizophrenia and for depression. There are a number of relevant practice guidelines in the UK [5-7], and in the US [8,9], but none of them give an idea of the benefit to a particular population. The NEPP can be used to estimate the incremental impact of moving from current to best practice and is calculated as shown in Table 1.

Table 1. Formula for calculating Number of Events Prevented in your Population (NEPP)

We chose to examine the population impact of interventions recommended in the NICE guidelines for depression [10] and schizophrenia [11] wherever we could identify data to make the calculations, for a notional population of 100,000 adults. Thus for depression, we examined continuing treatment with antidepressant therapy, screening to increase the rate of diagnosis, and cognitive behaviour therapy (CBT). We also included increasing care management as there are a number of relevant trials. For schizophrenia, we examined intervention by an early intervention team, increasing adherence to drug treatment, and family and relapse prevention interventions. We were unable to find appropriate data for CBT, assertive outreach, crisis resolution or home treatment teams. We chose hospitalisation and relapse as outcomes with different perspectives, although since hospitalisation is rare in depression and not used in most trials, we restricted the use of this outcome to schizophrenia. Table 2 lists the interventions which we included, and those we excluded, in our analysis.

Table 2. Interventions included in the NICE recommendations which were included (or excluded) in this analysis

The data on prevalence and evidence on effectiveness come from a review of a number of papers, some are systematic reviews and some are individual trials. Where differences in estimates are found between publications, an estimate that represents the consensus has been attempted, using median or mean values depending on the data.

Since we have had to use estimates for many of the variables, we have performed a one-way sensitivity analysis [12,13] to explore the differences that would have been seen with the use of different estimates. We calculated the NEPP for the minimum and maximum of each parameter while holding the other parameters constant. For the relative risk reduction we used the lower and upper limits of the 95% confidence interval as the minimum and maximum estimates. We used the minimum of the minimum estimates and the maximum of the maximum estimates as our interval of plausible values for the NEPP.

Results

Depression

The prevalence of major depression was taken from Kessler [14] at 6.6%, although estimates ranged from 5% [15] to 8%[13]. Since only 48% of patients currently have their depression diagnosed (taken from 45%[13] and 51% [14], we have adjusted the prevalence estimates to those currently diagnosed, and used the converse for the estimates of the prevalence of undiagnosed depression. The outcome of 'relapse' was used (1-year wherever possible), since hospitalisation is rare in this condition, and most trials did not report it. Where relapse was not used, but 'remission' reported, the converse of remission was assumed to represent relapse. The baseline risk of relapse ranged from 41%[16] to 65%[13], and we took 50% as our estimate. The proportion of the population currently receiving each intervention was obtained from various sources. We assumed that 50% of patients are currently receiving continuing treatment with anti-depressants, and only 5% currently have increased care management and Cognitive Behaviour Therapy (CBT). Best practice intervention goals are assumed to be 75% for each intervention. Compliance/adherence with each intervention is taken as 82% for continuing treatment with anti-depressants[16], and 60% (from 66% [17] and 51% [18] for increasing care management. We have used 58% for compliance with CBT, based on a summary of the various data sources we could find [19-21].

For increasing the diagnosis of depression, we assumed that we would identify 84% of those previously undiagnosed (being the sensitivity of the screening instrument[13] and that 63% would be compliant with the medication started as a result of the screening diagnosis [22]. Relative Risk Reduction is taken as 70% for continuing treatment with antidepressants[16], 41% for increasing diagnosis (from relative benefits of 1.6 and 1.7 for pharmacotherapy in comparison with placebo [23], 15% increasing care management (taken from Odds Ratios of 1.9 for remission [24], and 50% for CBT (50% relapse rate reduction compared with antidepressants [25] and 66% in recurrent depression [26]). These data are summarised in Table 3.

Table 3. Depression: data used for the calculations. Figures are percentages (range).

Schizophrenia

Prevalence is estimated as 3/1000 population, consistent with data from three reports [27-29], although this includes a variety of clinical states and will vary according to demographic variables. We have used two different outcomes, hospitalisation and relapse, both over 1-year or as close to it as possible. Relapse is defined differently in different studies [30-33], ranged from 44% to 50%, and we have used 48%. Hospitalisation ranged from 39% [34] to 50%[30] and 51% [33], and we have used 45%. The proportion currently receiving the intervention was assumed to be 0 for each of the interventions, with the exception of increasing adherence to medication, for which 66% was the current baseline adherence rate in the intervention arm of trials to increase adherence [35]. The best practice goal for each intervention was estimated to be 75%, apart from increasing adherence to medication, where 76% were adherent after the intervention [35]. Adherence to the intervention was taken from the intervention arm of the trials or summaries of trials, and was 76% for early intervention [33], and estimated as 80% for the other interventions (although not relevant for the increasing adherence to medication intervention where adherence was already taken into account in estimating the best practice goal). Relative Risk Reductions for relapse and hospitalisation respectively were 54% and 52% for early intervention [33], 50% and 44% for relapse prevention [34] and 28% for relapse only for family intervention [31]. Relative Risk Reduction for increasing adherence was estimated as 45%: 64% (Odds Ratio of 2.8 [36] for increasing medication adherence) multiplied by the effect of the medication itself estimated at 70% reduction in relapse rate (and applied also to re-hospitalisation)[37]. These data are summarised in Table 4.

Table 4. Schizophrenia: data used for the calculations. Figures are percentages (range)

Number of events prevented

The NEPP for relapse for both depression and schizophrenia and hospital re-admission for schizophrenia are also shown in Tables 3 and 4. In a notional population of 100,000 adults, going from current to 'best' practice for different interventions, the number of relapses prevented in the next year for schizophrenia were 6 (increasing adherence to medication), 23 (family intervention), 43 (relapse prevention), and 44 (early intervention); and for depression the number of relapses prevented in the next year were 100 (increasing care management), 227 (continuing treatment with antidepressants), 279 (increasing rate of diagnosis), and 325 (Cognitive Behaviour Therapy). Hospital re-admissions prevented in the next year for schizophrenia were 6 (increasing adherence to medication), 36 (relapse prevention) and 40 (early intervention).

They demonstrate a large difference in relapses prevented between depression and schizophrenia, and smaller differences between the interventions for each condition.

Discussion

The choice of conditions to examine in this work was dictated by those interventions for which evidence of benefit was available from a literature review. Some of the interventions included in guideline recommendations do not meet this criterion. In the case of CBT in schizophrenia, a systematic review showed no effect on relapse or re-admission[38], although some individual trials have shown an effect [39]. A review of trials of crisis intervention did show an effect, but this was dependent on only one trial [40]. Where possible, we used systematic reviews, since individual trials often provide conflicting information (as in attempts to increase compliance with therapy [36,41,42]).

One of the most difficult issues in interpreting the results of, and comparing between interventions, in the field of psychiatry, is the decision about which outcome measure to use [43]. Interventions for different diseases or conditions will have different outcomes. Some may result in an improvement in mortality, others in morbidity. This has led to the creation of generic outcome measures, such as the Quality Adjusted Life Year (QALY) and the Disability Adjusted Life Year (DALY), which have the advantage of allowing direct comparison of outcome between different interventions. They rely on derivation of weights and values for different disease outcomes, which have been criticised as being arbitrary, although they have been used extensively in assessments of the cost-effectiveness of interventions in mental health[28,44-46]. The advantage of our measures is that they allow the policy-maker to see exactly which outcomes are being affected. For example, a hospital manager might prioritise the prevention of hospital re-admissions, while a patient or carer might be more interested in symptom relapse. The use of various measures depends on which ones have been included in the studies contained in the evidence base, which vary between study and condition.

Use of Population Impact Measures (PIMs), allows a population perspective to the estimate of risks and benefits. If an intervention is effective in individuals, but the condition it is used to treat is uncommon in your own population, its impact on the population will be more limited than an alternative intervention with lower effectiveness but which can be used on a larger proportion of the population. The findings here demonstrate the different population impact of interventions for conditions of different prevalence in the population. We see from Tables 3 and 4, that the numbers of relapses prevented by each of the interventions for depression are much higher than those for schizophrenia. The differences in population impact between interventions for depression and schizophrenia reflect the difference in prevalence between the two, consistent with estimates by Andrews[44]. Whether there is a real choice to be made between depression and schizophrenia is debatable, since, at least in the UK, the responsibility for those in the population with depression is mainly shouldered by primary care whereas specialist psychiatric services are the agencies mostly responsible for delivery of care for people with schizophrenia[47]. It is also difficult to compare relapse in the two conditions. Since hospitalisation is rare in depression, we cannot compare hospitalisations between the two conditions. Within-condition comparisons between different interventions may allow the policy-maker to prioritise between them. We see large differences in population impact between different interventions although the range of impact indicated by the sensitivity analysis demonstrates how susceptible these measures are to variation in the parameter estimates on which they are based.

Another advantage of the use of Population Impact Measures, is that they provide an estimate in a particular population, of going from current to 'best' practice. The numbers are thus dependent on current practice levels and the treatment goals. Where the goals are close to current practice, the potential population gain will be lower than where the gap is large, due either to poor current practice or ambitious goals. The potential impact thus can be personalised to the individual population into which the intervention is to be introduced. The results we present will thus vary according to local settings. Although the data used here come from a UK setting, the method is fully transportable to other populations.

Although cost data would be needed to make a final prioritisation decision, we have deliberately left the outcomes to be expressed in the terms in which the data were collected (rather than the generic measures used in cost-effectiveness estimates) so that benefit expressed as real outcomes can be identified. A subsequent step would be to prioritise between interventions, taking costs into account, once the population impact can be observed.

The estimates we have made depend on the availability of relevant data. We have used literature-based estimates, which in themselves vary considerably. For example, adherence to drug treatment for depression varied from 30 to 97% in a number of trials, with a median of 63% [22]. We have used a different value for adherence to continuing drug treatment for depression (82%), based on the non-withdrawal rate from trials in a systematic review[16]. The estimates used in this paper for adherence, both to medication and to adherence enhancing interventions, may thus not relate to local practice settings. Similarly, access to non-pharmaceutical interventions will vary considerably. Depending on the definition used, in the UK, there may be 20 times more CBT practitioners per population in the best provided 10% of the population than in the worst 10%, and our estimate of 5% of current exposure to CBT for depression is greater than the 1% national estimate of availability to those who might benefit [48]. The accuracy of the estimates would be increased by better availability of local data from the population to which they would be applied, and the gaps in the data necessary for the preparation of this paper demonstrate a clear need for better relevant data. However, it should be noted that local data may be plagued by small numbers causing under-ascertainment of prevalence and outcomes. The need for good local data should motivate improvements in local data collection systems. If good local data were available, the variability in the NEPP would only be due to variability in measures of risk and RRR and where good meta-analyses exist the RRR estimate is relatively stable. The method could also be adapted to explore the benefits of policy changes such as those designed to prevent suicide, however, this will depend on obtaining some estimate of benefit. A previous paper estimated population attributable fractions for alternative strategies for preventing suicide, and recommended population-based rather than high-risk strategies as a result [49]. This methodology is a fore-runner to ours, which allows 'personalisation' to the local population into which the strategy is to be introduced. We have not explored the impact of different population demographic structures or severity case-mix in this paper, but the implications for a local population will depend on the features that determine prevalence, treatment patterns and baseline risk of disease.

Conclusion

The use of Population Impact Measures provides the possibility for a policy-maker to see the impact of a new intervention on the population as a whole and to compare alternative interventions. Our methods are much simpler than those which use generic outcomes such as the QALY [28,44,45], and have the advantage of being transparent. We believe that not only will their use assist in priority setting, but that their transparency will allow the policy-maker to identify gaps in the availability of local (and trial) data – information which is essential if an evidence base is to be used to prioritise interventions based on improving the health of the population [50].

Competing interests

The author(s) declare that they have no competing interests.

Authors' contributions

RFH conceived the study, RFH and LP reviewed the literature, IG performed the calculations. All were involved in the drafting the manuscript, and have read and approved the final manuscript.

Acknowledgements

We are grateful to Professors Shon Lewis and Max Marshall for helpful comments.

References

  1. Heller RF, Dobson AJ: Disease impact number and population impact number: a population perspective to measures of risk and benefit.

    BMJ 2000, 321:950-952. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  2. Heller RF, Edwards R, McElduff P: Implementing guidelines in primary care: can population impact measures help?

    BMC Public Health 2003, 3:7-7. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  3. Attia J, Page JH, Heller RF, Dobson AJ: Impact numbers in health policy decisions.

    J Epidemiol Community Health 2002, 56:600-605. PubMed Abstract | Publisher Full Text OpenURL

  4. Heller RF, Buchan I, Edwards R, Lyratzopoulos G, McElduff P, St Leger S: Communicating risks at the population level: application of population impact numbers.

    BMJ 2003, 327:1162-1165. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  5. National Institute for Clinical Excellence (Great Britain): [http://www.nice.org.uk/page.aspx?o=289559] webcite

    Schizophrenia: Core interventions in the treatment and management of schizophrenia in primary and secondary care (pre-publication draft). London, National Institute for Clinical Excellence; 2004, 243. OpenURL

  6. National Institute for Clinical Excellence (Great Britain): [http://www.nice.org.uk/page.aspx?o=cg023] webcite

    Depression: the management of depression in primary and secondary care (in progress). London, National Institute for Clinical Excellence; 2004. OpenURL

  7. Association AP: Practice Guideline for the Treatment of Patients With Major Depressive Disorder, Second Edition. [http://www.psych.org/psych_pract/treatg/pg/Depression2e.book.cfm] webcite

    2002.

  8. Network SIG: Psychosocial Interventions in the Management of Schizophrenia. Scottish Intercollegiate Guidelines Network; 1998. OpenURL

  9. Association AP: Practice guideline for the treatment of patients with schizophrenia Second edition. [http://www.psych.org/psych_pract/treatg/pg/SchizPG-Complete-Feb04.pdf] webcite

    2004, http://www.psych.org/psych_pract/treatg/pg/SchizPG-Complete-Feb04.pdf.

  10. Briggs A, Sculpher M, Buxton M: Uncertainty in the economic evaluation of health care technologies: the role of sensitivity analysis.

    Health Econ 1994, 3:95-104. PubMed Abstract OpenURL

  11. Valenstein M, Vijan S, Zeber JE, Boehm K, Buttar A: The cost-utility of screening for depression in primary care.

    Ann Intern Med 2001, 134:345-360. PubMed Abstract | Publisher Full Text OpenURL

  12. Kessler RC, Berglund P, Demler O, Jin R, Koretz D, Merikangas KR, Rush AJ, Walters EE, Wang PS: The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R).

    JAMA 2003, 289:3095-3105. PubMed Abstract | Publisher Full Text OpenURL

  13. Ohayon MM, Priest RG, Guilleminault C, Caulet M: The prevalence of depressive disorders in the United Kingdom.

    Biol Psychiatry 1999, 45:300-307. PubMed Abstract | Publisher Full Text OpenURL

  14. Geddes JR, Carney SM, Davies C, Furukawa TA, Kupfer DJ, Frank E, Goodwin GM: Relapse prevention with antidepressant drug treatment in depressive disorders: a systematic review.

    Lancet 2003, 361:653-661. PubMed Abstract | Publisher Full Text OpenURL

  15. Simon GE, VonKorff M, Rutter C, Wagner E: Randomised trial of monitoring, feedback, and management of care by telephone to improve treatment of depression in primary care.

    BMJ 2000, 320:550-554. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  16. Rost K, Nutting P, Smith JL, Elliott CE, Dickinson M: Managing depression as a chronic disease: a randomised trial of ongoing treatment in primary care.

    BMJ 2002, 325:934. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  17. Proudfoot J, Ryden C, Everitt B, Shapiro DA, Goldberg D, Mann A, Tylee A, Marks I, Gray JA: Clinical efficacy of computerised cognitive-behavioural therapy for anxiety and depression in primary care: Randomised controlled trial. [References].

    British Journal of Psychiatry 2004, 185:46-54. PubMed Abstract | Publisher Full Text OpenURL

  18. Kenwright M, Marks IM: Improving first attendance for cognitive behaviour therapy by a partial booking appointment method: Two randomised controlled trials. [References].

    Journal of Mental Health (UK) 2003, 12:385-392. Publisher Full Text OpenURL

  19. White J, Jones R, McGarry E: Cognitive behavioural computer therapy for the anxiety disorders: A pilot study.

    Journal of Mental Health 2000, 9:505-516. Publisher Full Text OpenURL

  20. Pampallona S, Bollini P, Tibaldi G, Kupelnick B, Munizza C: Patient adherence in the treatment of depression.

    British Journal of Psychiatry 2002, 180:104-109. PubMed Abstract | Publisher Full Text OpenURL

  21. Williams JWJ, Mulrow CD, Chiquette E, Noel PH, Aguilar C, Cornell J: A systematic review of newer pharmacotherapies for depression in adults: evidence report summary.

    Ann Intern Med 2000, 132:743-756. PubMed Abstract | Publisher Full Text OpenURL

  22. Dietrich AJ, Oxman TE, Williams JWJ, Schulberg HC, Bruce ML, Lee PW, Barry S, Raue PJ, Lefever JJ, Heo M, Rost K, Kroenke K, Gerrity M, Nutting PA: Re-engineering systems for the treatment of depression in primary care: cluster randomised controlled trial.

    BMJ 2004, 329:602. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  23. Gloaguen V, Cottraux J, Cucherat M, Blackburn IM: A meta-analysis of the effects of cognitive therapy in depressed patients.

    J Affect Disord 1998, 49:59-72. PubMed Abstract | Publisher Full Text OpenURL

  24. Fava GA, Rafanelli C, Grandi S, Conti S, Belluardo P: Prevention of recurrent depression with cognitive behavioral therapy: preliminary findings.

    Arch Gen Psychiatry 1998, 55:816-820. PubMed Abstract | Publisher Full Text OpenURL

  25. McCreadie RG, Leese M, Tilak-Singh D, Loftus L, MacEwan T, Thornicroft G: Nithsdale, Nunhead and Norwood: similarities and differences in prevalence of schizophrenia and utilisation of services in rural and urban areas.

    Br J Psychiatry 1997, 170:31-36. PubMed Abstract OpenURL

  26. Andrews G, Sanderson K, Corry J, Issakidis C, Lapsley H: Cost-effectiveness of current and optimal treatment for schizophrenia.

    Br J Psychiatry 2003, 183:427-435. PubMed Abstract | Publisher Full Text OpenURL

  27. Goldner EM, Hsu L, Waraich P, Somers JM: Prevalence and incidence studies of schizophrenic disorders: a systematic review of the literature.

    Can J Psychiatry 2002, 47:833-843. PubMed Abstract OpenURL

  28. Weiden PJ, Olfson M: Cost of relapse in schizophrenia.

    Schizophr Bull 1995, 21:419-429. PubMed Abstract OpenURL

  29. Pharoah FM, Rathbone J, Mari JJ, Streiner D: Family intervention for schizophrenia.

    Cochrane Database of Systematic Reviews 2003. OpenURL

  30. Barbato A, D'Avanzo B: Family interventions in schizophrenia and related disorders: a critical review of clinical trials.

    Acta Psychiatr Scand 2000, 102:81-97. PubMed Abstract | Publisher Full Text OpenURL

  31. Craig TK, Garety P, Power P, Rahaman N, Colbert S, Fornells-Ambrojo M, Dunn G: The Lambeth Early Onset (LEO) Team: randomised controlled trial of the effectiveness of specialised care for early psychosis.

    BMJ 2004, 329:1067. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  32. Herz MI, Lamberti JS, Mintz J, Scott R, O'Dell SP, McCartan L, Nix G: A program for relapse prevention in schizophrenia: a controlled study.

    Arch Gen Psychiatry 2000, 57:277-283. PubMed Abstract | Publisher Full Text OpenURL

  33. Dolder CR, Lacro JP, Leckband S, Jeste DV: Interventions to improve antipsychotic medication adherence: review of recent literature.

    J Clin Psychopharmacol 2003, 23:389-399. PubMed Abstract | Publisher Full Text OpenURL

  34. Nose M, Barbui C, Gray R, Tansella M: Clinical interventions for treatment non-adherence in psychosis: meta-analysis.

    Br J Psychiatry 2003, 183:197-206. PubMed Abstract | Publisher Full Text OpenURL

  35. Gilbert PL, Harris MJ, McAdams LA, Jeste DV: Neuroleptic withdrawal in schizophrenic patients. A review of the literature.

    Arch Gen Psychiatry 1995, 52:173-188. PubMed Abstract OpenURL

  36. Jones C, Cormac I, Silveira M, Neto J, Campbell C: Cognitive behaviour therapy for schizophrenia.

    Cochrane Database of Systematic Reviews 2004. OpenURL

  37. Gumley A, O'Grady M, McNay L, Reilly J, Power K, Norrie J: Early intervention for relapse in schizophrenia: results of a 12-month randomized controlled trial of cognitive behavioural therapy.

    Psychol Med 2003, 33:419-431. PubMed Abstract | Publisher Full Text OpenURL

  38. Joy CB, Adams CE, Rice K: Crisis intervention for people with severe mental illnesses.

    Cochrane Database of Systematic Reviews 2004. OpenURL

  39. Kemp R, Hayward P, Applewhaite G, Everitt B, David A: Compliance therapy in psychotic patients: randomised controlled trial.

    BMJ 1996, 312:345-349. PubMed Abstract | Publisher Full Text OpenURL

  40. O'Donnell C, Donohoe G, Sharkey L, Owens N, Migone M, Harries R, Kinsella A, Larkin C, O'Callaghan E: Compliance therapy: a randomised controlled trial in schizophrenia.

    BMJ 2003, 327:834. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  41. Keller MB: Past, present, and future directions for defining optimal treatment outcome in depression: remission and beyond.

    JAMA 2003, 289:3152-3160. PubMed Abstract | Publisher Full Text OpenURL

  42. Andrews G, Issakidis C, Sanderson K, Corry J, Lapsley H: Utilising survey data to inform public policy: comparison of the cost-effectiveness of treatment of ten mental disorders.

    Br J Psychiatry 2004, 184:526-533. PubMed Abstract | Publisher Full Text OpenURL

  43. Haby MM, Carter R, Mihalopoulos C, Magnus A, Sanderson K, Andrews G, Vos T: Assessing cost-effectiveness in mental health: introduction to the study and methods.

    Australia and New Zealand Journal of Psychiatry 2004, 38:569-578. Publisher Full Text OpenURL

  44. Mihalopoulos C, Magnus A, Carter R, Vos T: Assessing cost-effectiveness in mental health: family interventions for schizophrenia and related conditions.

    Australia and New Zealand Journal of Psychiatry 2004, 38:511-519. Publisher Full Text OpenURL

  45. Singleton N, Bumpstead R, O'Brien M, Lee A, Meltzer H: Psychiatric Morbidity among Adults living in Private Households, 2000. London, The Stationary Office; 2001. OpenURL

  46. Shapiro DA, Cavanagh K, Lomas H: Geographic inequity in the availability of cognitive behavioural therapy in England and Wales. [References].

    Behavioural & Cognitive Psychotherapy 2003, 31:185-192. Publisher Full Text OpenURL

  47. Lewis G, Hawton K, Jones P: Strategies for preventing suicide.

    Br J Psychiatry 1997, 171:351-354. PubMed Abstract OpenURL

  48. Heller RF, Page JH: A population perspective to evidence based medicine: "evidence for population health".

    Journal of Epidemiology and Community Health 2002, 56:45-47. Publisher Full Text OpenURL