We need to be mindful of the quality of evidence. Being actively open minded is part of good professional practice, but there is another level of more active critical appraisal. For example, before changing an organisational policy to accord with a new systematic review, we need to be sure of the underlying quality of that review. Are the results valid? Are they reliable? When and how should it be implemented? Critical appraisal has been defined as: 'The process of assessing and interpreting evidence by systematically considering its validity, results, and relevance'. [1]
Sometimes these questions are easy to answer. For example, imagine you are considering surgery for your 82 year old patient. You might find a systematic review of randomised controlled trials examining the outcomes of this surgery. But on closer reading, you note that the authors actually exclude any studies with patients aged 65+. Surgical outcomes could be vastly different between the two different age groups. In such a case it is clear that you cannot automatically generalise the findings to your patient. However the appraisal process can be far more complex in other cases. There are some tools to help.
You can access others' published critical appraisals of studies. Reviewers commonly critically appraise a published research study, and create a structured abstract and commentary on the findings. Two of the most well known freely available databases of such reviews are the Database of Abstracts of Reviews of Effects, and the NHS Economic Evaluations Database, available through the Centre for Reviews and Dissemination. These databases are automatically included when you search The Cochrane Library. There is also the Trip Database, a free database for identifying pre-appraised evidence of a variety of types, including evidence-based synopses. It allows you to limit your search to categories such as systematic reviews or guidelines. Finding such a published appraisal gives you an immediate critique of the quality of a piece of a research, usually by experts in the field.
Another strategy is to check printed or online responses or 'Letters to the Editor' in the subsequent few journal issues that follow the issue publishing an article of interest. These letters can give you an immediate expert opinion of the study.
Other published critical appraisals form the basis of a new type of resource that has emerged over the last few years. These are known as summary journals. Some, such as ACP Journal Club, are free to access. Most, to date, still require a paid subscription, such as Evidence Based Medicine, Evidence Based Nursing etc.
Because different research designs can have different potential sources of bias, critical appraisal tools, such as Critical Appraisal Skills Programme (CASP) Appraisal Tools CASP UK and the Centre for Evidence Based Medicine's CATmaker, have been developed. These provide you with lists of the most important questions which should be asked of different research designs. Palliative care (like other fields) uses a range of study methodologies from qualitative through to quantitative designs, and it is important to match the choice of tool to the study's basic design. Using an appropriate tool to critically appraise research in your field, as a group, may be a useful journal club activity.