Once you have carried out your literature search, as discussed by Watson (2020) in the previous issue of BJN, it is important to have a final check that you have found all possible articles. Look at your search strategies for each database you have used, checking for spelling errors. Have some search terms found few or no references? Consider whether you need to include US spellings or synonyms. It may be helpful to ask a fellow student or colleague to check whether you have missed anything.
Bias in results
Before thinking about other resources, do beware of bias in your results. Three areas of possible bias are database bias, time-lag bias and publication bias. Database bias means relying too much on only one or two databases, so use as many relevant databases to which your workplace or university can provide access. CINAHL is unlikely to be sufficient alone—so consider using other nursing databases, such as the British Nursing Index or EMCARE. Also try specialist databases, such as Midwifery & Infant Care, AMED for complementary therapy items or PsycINFO for psychosocial topics. If you are unfamiliar with searching a particular database, see what assistance is available by way of guides, online tutorials or face-to-face training sessions. Royal College of Nursing (RCN) members can access help at our Support webpages (RCN Library, 2020a).
Time-lag bias refers to the fact that new materials are being published every day. Therefore it is a good idea to set up accounts in any databases you use to save searches so that you can rerun these at a later date. You can also set up alerts via your account to let you know of any new articles. Check for a ‘MyAccount’ or ‘My Research’ function in the database.
You may wonder why you should extend your search beyond journal articles. Publication bias may occur as some studies are more likely to be published in journals than others, and journal articles are only one type of information source.
The term ‘grey literature’ covers a whole area of resources, including government reports, dissertations, conference proceedings, newsletters and news items. Searching for these as well as journal articles will enhance your search results and ultimately your project or assignment. OpenGrey (OpenGrey, 2020) is one source for grey literature and provides free access to technical and research reports, dissertations, official publications and conference papers. Blogs, vlogs, podcasts and tweets from institutions and leading individuals in your subject area can also lead to information on new developments in your topic.
Additional resources
Subject guides produced by libraries are a useful introduction to a topic. The RCN Library has over 50 guides, including not only links to selected books and journals, but also key resources on a topic (RCN Library, 2020b).
If you need to look for clinical trials, Cochrane provides free access to reviews and trials (Cochrane Library, 2020); while the US National Library of Medicine's clinical trials website (ClinicalTrials.gov) (2020) and the World Health Organization's (WHO) International Clinical Trials Registry Platform (WHO, 2020) may be helpful.
It is always worthwhile to check for any available standards if you have a clinical topic. Organisations such as the National Institute for Health and Care Excellence (NICE) and Scottish Intercollegiate Guidelines Network (SIGN) provide pathways, standards and guidance. The British National Formulary (BNF) and British National Formulary for Children (BNFC) provide medication information and guidance. Evidence Search provided by NICE (2020) is a useful resource to check for all types of guidelines and evidence on a topic.
Professional associations and charities are another useful source. Government and health management organisations, such as the Department of Health and Social Care, The King's Fund, Nuffield Trust and National Institute of Health Research will provide links to reports, news items and blogs on current health issues. Don't forget there may be a specialist organisation for your topic, for example the Health and Safety Executive for health issues at work.
Finally, search engines such as Google, Yahoo! and Bing can assist in finding further resources beyond your database results. Be aware when using any search engines that algorithms are used to personalise searches and may limit and change your results. This is called a ‘filter bubble’ as the search engine tries to guess what information you want based upon your previous search histories, online behaviour and location (University College London (UCL) Institute of Education, 2020a). This can help you by not retrieving lots of unnecessary information, but does mean that if you and a colleague did the same search you might not get the same information. It is therefore important to copy and paste any website addresses as you retrieve them for later use.
Evaluating resources
Once you have gathered all your sources, the next step is to evaluate these so you can ensure you include only trustworthy, quality and relevant items in your dissertation, project or workplace study.
There are several tools you can use to evaluate any document, journal article or website by asking questions.
One tool is the 5 Ws, which asks (UCL Institute of Education, 2020b):
Another tool is the CRAAP test (Currency, Relevancy, Authority, Accuracy and Purpose), which asks:
Critical appraisal
Critical appraisal skills allow you to look at a research paper and assess the evidence by weighing up both strengths and weaknesses to judge whether it is relevant. Critical appraisal may appear to be daunting but there are checklists which you follow to make these decisions. It is important in the beginning to identify the type of research, and whether it is based on a quantitative or qualitative method, so as to use the appropriate checklist.
The checklists ask simple questions which guide you through evaluation of an article:
One of the most commonly used checklist is CASP (Critical Appraisal Skills Programme, 2020). the Centre for Evidence Based Medicine (2020) and the Joanna Briggs Institute (2020) have also produced critical appraisal checklists. Your university library or NHS trust may offer training on critical appraisal, and members of the Royal College of Nursing (RCN) can access a subject guide on critical appraisal (RCN Library, 2020c).
Presenting results
Once you have written your report or assignment and cited your references you may need to include your search process. To do this you will need to include all sources used, search terms and any limitations, such as date. The aim is to provide sufficient details so that anyone can reproduce your search. Reporting tools such as PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) allow you to map out the number of records identified, the studies you included—and excluded—and the reasons for exclusion. A flow chart template can be downloaded from the PRISMA website (PRISMA, 2020). You can also use a table or spreadsheet to present your search results—listing your resources, the type of research and bibliographic details.
If you have used reference management software such as Endnote, Zotero, Mendeley or Refworks to download your search results these can assist in producing a bibliography, and will also place your citations in your university or work house style.
Your assignment is now complete and ready to present to your tutor or work colleagues.