Preliminary piloting and validation of a questionnaire identifying basic clinical skills practised by research nurses

Ethical issues

Ethical approval was obtained from the University of West of England ethics committee.

Sample selection

The potential research population included all professionals working under the banner of ‘clinical research nurse’ employed in a clinical research role. There were approximately 120 CRNs employed within the recruiting trust. As a pilot study, 15 participants randomly selected using a random number generator (Stat Trek) (https://tinyurl.com/76rtja8) were invited by email to complete an online survey twice, with a gap of 2 weeks between administrations. According to Connelly (2008), extant literature suggests that a pilot study sample should be 10% of the potential population; 15 people were approached to allow for people to decline the invitation.

Not all participants completed the online questionnaire twice; however, those who did were asked if they were willing to participate in a follow-up interview.

Data collection

There were two phases to the data collection. First, the development and administration of an online questionnaire and, second, interviews. The method of developing the questionnaire is presented below, followed by that of the interview schedule. This mirrors the pattern experienced by the participants.

Developing the questionnaire

The questionnaire was developed using pre-existing knowledge of clinical skills that registered nurses in one NHS trust are known to practise. This was independently assessed by the lead CRN of a large acute NHS trust and by members of the trust training and development department. Demographic data were collected in the first section of the questionnaire (Table 1).

The list of clinical competencies were grouped into four distinct subscales, each relating to the level of expertise required to perform the skill. The four subscales were ‘basic’, ‘intermediate’, ‘advanced’ and ‘clinically specific’. The categorising of the skills was undertaken by the hospital trust's training department and was informed by the trust protocols.

Fourteen skills were categorised as basic, seven as intermediate and two as advanced (Table 2). Clinically specific skills, such as those in oncology (administration of intravenous cytotoxic medication, care of the peripherally inserted central catheters (PICC line), and patient education on the effects of chemotherapy) and those in renal (care of the Hickman line, haemodialysis) were included and respondents were given an opportunity to add other clinically specific skills relevant to their practice.

Respondents were asked to rate their level of confidence, frequency of applying the skill and how essential to their role they considered each skill was. This was done using each skill on standardised formats. The responses for how confident the CRN was in performing the skill included ‘not at all’, ‘moderately’ or ‘very’. To rate how frequently a skill was applied, the choices were ‘weekly’, ‘monthly’, ‘one or two times a year’ or ‘never’; and to establish whether respondents thought that the skill was essential to the role the response options were either ‘not at all’, ‘moderately’ or ‘very’. Responses were assigned value, and each individual's responses within each skill category (basic, intermediate, advanced) were added to result in an overall score for each category.

Development of the interview guide

A semi-structured interview schedule was constructed to focus the discussion on user experience of completing the questionnaire and to ensure the acceptability of the content to the CRNs. The researcher invited respondents who completed the questionnaire twice (n=8) to a face-to-face interview. Five questionnaire respondents agreed to participate in an interview, but owing to clinical commitments interviews with only three CRNs were undertaken.

Questionnaire

SPSS for Windows version 19 was used to analyse the questionnaire responses.

Establishing reliability requires that a questionnaire is both stable and internally consistent (Bryman, 2008). To assess stability, individuals' responses to each administration of the questionnaire were compared using Spearman's rank correlation coefficient (rho) (p); an acceptable level of correlation was set at 0.8 (Oppenheim, 1992). This analysis was undertaken with each skill and for overall scores.

The internal consistency of each of the scales was assessed by means of Cronbach's alpha coefficient.

Interview

Relational content analysis was undertaken with the transcriptions from the three interviews (Kaid, 1989).

Questionnaires

Table 2 shows the frequency of the performance of clinical skills. A test-retest administration of the online questionnaire was conducted with seven complete datasets. The correlations for each skill were calculated for both confidence and essential nature of the skill to the role of the CRN.

The correlations for confidence in basic skills were satisfactory, with only enteral feeding and female catheterisation scoring below 0.7 (Table 2). For intermediate skills, the correlation was good except for male catheterisation, which yielded a negative correlation (-0.218). Spearman's rho for most other basic and intermediate skills was > 0.8. The advanced skills correlated very well, with scores greater than 0.8.

The correlation for the essential nature of the skills revealed similar results, with only enteral feeding and female catheterisation scoring < 0.7 for the basic skills. Administration of oral and intravenous medication scored in the 0.7–0.8 range. With regards to the intermediate skills, male catheterisation revealed a negative correlation, -0.218 again, with venepuncture correlating at 0.753. Spearman's rho for all other basic and intermediate skills was > 0.8. The advanced skills correlated well (> 0.8).

The internal consistency of each subscale and the full scale was assessed by means of Cronbach's alpha coefficient. Coefficients obtained for confidence were 0.979 for the basic skills subscale and 0.904 for the intermediate skills subscale (n=13). Coefficients for the essential nature of skills were 0.976 for basic skills and 0.899 for intermediate skills. The full scale reached a Cronbach's alpha of 0.952.

Scales for respondents' attitudes about confidence and the essential nature of the skills were evaluated. At least 12 respondents were ‘moderately’ or ‘very’ confident about performing basic skills; however, 6 lacked confidence (‘not at all’ response) in some basic skills (Table 3). Similarly, overall, at least 6 respondents were confident about performing intermediate skills (Table 4).

At least 7 respondents considered that many of the basic skills were ‘moderately’ or ‘very’ essential to their role (Table 5). For the intermediate skills, the pattern was more mixed; the majority of respondents reported male catheterisation and care of a central line as ‘not at all’ necessary, with just over half highly valuing the ability to perform venepuncture (Table 5).

Quantative interview

All interviewees reported that the questionnaire questions and content were appropriate to their role as CRNs and that using an electronic format to deliver the questionnaire was positive. However, communication skills were identified as an important skill that should have been included in the questionnaire.

Interviewee 1 did not consider some of the basic skills as essential (‘not at all’ answer) to her role. She confirmed that she was not confident in performing them. However, she did not think that she required any further relevant training because she did not use these skills often enough.

Interviewee 2 reported being ‘moderately’ to ‘very’ confident in all basic skills in the questionnaire. She thought that these skills were ‘moderately’ to ‘very’ essential to her role and she did not require any further training on basic and intermediate skills.

Interviewee 3 demonstrated in his responses that some basic skills are not essential (‘not at all’ response) to his role; he reported not being competent in these skills but did not feel that he required further training.

An advanced skill such as obtaining a medical history was identified by interviewees 1 and 2 as essential to the role of the CRN. They reported having received training in this skill, used this skill regularly and felt confident about performing it. Interviewee 2 added that she was also a non-medical prescriber but did not think that this skill was essential to her role.

Interviewee 1 said it was important to be clinically updated because research nurses may be called upon at any time to support their colleagues in the wards. She expressed her personal concern that she has been off the wards for a long period and would not feel confident at present if asked to work in the clinical environment. Interviewee 3, however, thought that, although he did not feel confident in practising some skills, further training for skills that are not frequently used would not be beneficial.