Research Article, J Addict Behav Ther Rehabil Vol: 9 Issue: 1

Validation of Heroin Craving Questionnaire in Greek Patients under Substitution Treatment with Methadone and Buprenorphine: How to Prevent a Relapse

Christonikos Leventelis^1,2*, Aristidis S. Veskoukis³, Meni Malliori⁴, Michael Koutsilieris⁴, Sophia Zyga¹, Andrea Paola Rojas Gil¹, Antzouletta Kampitsi⁵, Nikolaos Goutzourelas³ and Maria Tsironi¹

¹Department of Nursing, University of Peloponnese, Tripoli, 22100, Greece

²Organization Against Drugs, Athens, 10433, Greece

³Department of Biochemistry and Biotechnology, University of Thessaly, Larissa, 41500, Greece

⁴University of Athens, Athens, 11527, Greece

⁵General Anticancer Hospital “Agios Savvas”, Athens, 11522, Greece

*Corresponding Author : Christonikos Leventelis
Toxicology Scientist, Department of Nursing, University of Peloponnese, Tripoli, 22100, Greece
Tel: +00306944687478
E-mail: clevente@uop.gr

Received: February 09, 2020 Accepted: February 15, 2020 Published: February 25, 2020

Citation: Leventelis C, Veskoukis SA, Malliori M, Koutsilieris M, Zyga S, et al. (2020) Validation of Heroin Craving Questionnaire in Greek Patients under Substitution Treatment with Methadone and Buprenorphine: How to Prevent a Relapse. J Addict Behav Ther Rehabil 9:1. doi: 10.37532/jabtr.2020.9(1).189

Abstract

Keywords: Heroin; Craving questionnaire; Craving; Substitution treatment; Methadone; Buprenorphine; Opioids

Introduction

Craving is an essential feature of substance use disorders and this is evidenced by the fact that it has been included as a diagnostic criterion for those disorders in the Diagnostic and Statistical Manual of Mental Disorders [1]. Craving refers to the desire or urges to re-experience a previously used psychoactive substance after a period of detoxification [2]. It usually continues for months, even years after the cessation of drug use affecting parameters in physical, emotional, cognitive, and behavioral context [3,4]. The majority of the researchers imply that craving is a subjective desire to use an addictive substance [5,6]. To this end it is characterized by three fundamental traits, that is the desire is conscious, is accompanied by its verbal expression and is directed towards the use of a particular substance [7]. Several relevant studies have demonstrated that craving is an important mediator of continuing substance use and, noteworthy, it promotes relapse after abstinence [8-10]. Although there is no clear experimental evidence regarding the association of craving with relapse, it is often considered as an indication of substance dependence [11,12].

The clinical significance of craving measurement is mainly determined by the psychometric capability of the tools used [7]. Given that the evaluation of craving is a rather difficult task, two issues have emerged with the first being the time-frame (i.e., long-time versus instant measurement tools) and the second is the measurement as unidimensional versus multidimensional concept [13]. With regard to the first one, studies have shown that measuring craving with the use of self-reported questionnaires is related to proximal (e.g., within one week) substance use [14-16].

In addition, craving is also influenced by factors such as substancerelated stress or withdrawal symptoms and measurement appears to be a predictive marker for immediate use rather than the time of in-depth use [17]. In relation to the second set of issues, the unidimensional measurement scales are easy to complete, they are highly reliable and the results obtained can be easily interpreted [18]. Nevertheless, they do not cover the various dimensions of craving as well as the different ways in which users wish to describe this desire [10,19].

Regarding the interconnection of craving with individuals under substitution treatment, the available evidence is notable. Interestingly, opioid users who did not follow an opioid maintenance treatment (OMT, with methadone or buprenorphine) reported a higher level of craving to use or relapse [20,21] compared to patients who participated in a substitution program [22]. Similarly, it has been shown that craving for opioids is a prognostic factor for recurrence in individuals with a record in opioid use disorders [23,24]. On the contrary, treatment with substitutes (i.e., methadone, buprenorphine) appears to contribute to the reduction of craving for opioid use [25].

Methadone and buprenorphine are two of the most commonly substances used as Opioid Maintenance Treatment (OMT) for heroin addicts, it has to be mentioned that methadone, as a full MOR agonist, has a long half-life (approximately 15 h-22 h) and causes fewer withdrawal symptoms than heroin [26], whereas buprenorphine is a partial agonist of mu-opioid receptor with a long half-life (20 h-70 h) showing a ceiling effect with less euphoric feelings [27,28].

As it has been previously shown, they can act protectively to patients in terms of confining oxidative stress compared to heroin using individuals [29]. However, taking into consideration the aforementioned findings indicating that craving is an undisputable trait of patients who are under OMT, there are no results with respect to the usage of the Heroin Craving Questionnaire (HCQ) to them. In this context, the aim of our investigation was to assess craving in dependent opioid users under OMT with methadone and buprenorphine validating a multidimensional HCQ.

Materials and Methods

Participants

The present study comprised 258 randomly selected patients under Opioid Maintenance Treatment (OMT) in the therapeutic units of the Organization against Drugs (OKANA), Attica, Greece. All patients were fully informed about the purpose and objectives of the experiment. All necessary information and safeguards were provided to ensure the confidentiality of data, whereas each patient signed a consensus form before participating in the study. According to the prerequisites for inclusion in the study, the patients should be over 20 years of age, they should be long-term users of heroin or other opioid substances, thus suffering from physical and mental dependence. Patients with severe psychopathology and serious pathological problems, which made the monitoring of the program incompatible, were excluded.

Description of the instrument

Heroin Craving Questionnaire (HCQ) has been used as a craving measurement instrument, as previously reported [19]. It consists of 5 items: (1) desire to use heroin, (2) intentions and planning to use heroin, (3) anticipation of positive outcome, (4) relief from withdrawal or dysphoria and (5) lack of control overuse. Each of them includes 9 questions. The 45-item HCQ measures the negative and positive dimensions of craving heroin. The score is calculated with a 7-point Likert scale ranging from 1 (i.e., strongly disagree) to 7 (i.e., strongly agree). The original in English article was translated into Greek. The Greek version of HCQ was then translated back by a different translator. After that, both original and back-translated HCQ were compared and all points of divergence were corrected in order to accurately reflect the intent-accuracy of the item wording.

The final version of Greek HCQ was reviewed by a group of 7 researchers in the fields of psychiatry, nursing, psychology, biochemistry and biostatistics who finally assured that the two questionnaire versions are closely equivalent. The HCQ was then pilot tested with 11 patients under OMT to evaluate whether they understand its items and whether the questionnaire is related to their heroin-using experience. Subsequently, the appropriate revisions about the wording on items made by the patients and the final 45- item HCQ were used in the study. Demographic information regarding gender, age, educational level, nationality, social security, social status, place of residence, area, time in OKANA programs, age of onset and duration of using addictive substances before the OMT was obtained.

Experimental procedure

The study lasted for two weeks and 258 heroin users under OMT were included. The patients completed the HCQ at the unit and shortly thereafter, the substitutes (i.e., methadone and buprenorphine) were administered to them. Ten hours later they refilled the HCQ because the substitutes in question have longer half-life compared to morphine and heroin [30]. Patients with relapse to other addictive substances were excluded from the study. In order to rule out the use of other substances (i.e., opioids, methamphetamine, methadone, benzodiazepines, cannabis, tetrahydrocannabinol, amphetamine, and buprenorphine), all participants underwent weekly urine tests by onestep multidrug test kits during the three-month period of the substitution treatment. All subjects were found negative for substance abuse.

Methadone and buprenorphine administration

Methadone hydrochloride solution (10 mg/ml) and buprenorphine/ buprenorphine-naloxone pills (2 mg-8 mg) were used. The mean daily dose of methadone was 60 mg. According to the relevant literature, daily doses of methadone between 40 mg and 100 mg are effective as opioid maintenance treatments for heroin-addicted patients [29,31,32]. With respect to buprenorphine, the mean daily dose was equal to 16 mg. On the basis of the available data, this dosage regimen is the most commonly used in order buprenorphine to exert its action [29,32].

Ethics

This study was approved by the Nursing Faculty of the University of Peloponnese (Tripoli, Greece) and the Organization Against Drugs (OKANA, Athens, Greece). All participants gave written consent before they completed the HCQ.

Statistics and data analysis

The values of the dependent variables are presented as mean ± Standard Deviation (SD). Qualitative variables are presented as absolute and relative frequencies. A Confirmatory Factor Analysis (CFA) with maximum likelihood procedure was conducted in order to test how well the HCQ model fits the data. The variance of the latent constructs was fixed at one during parameter estimation and the factors were allowed to be correlated. The fit of the CFA model was assessed using the Comparative Fit Index (CFI), the Goodness of Fit Index (GFI) and the Root Mean Square Error of Approximation (RMSEA) [33]. For the CFI and GFI indices, values close to or greater than 0.95 are taken to reflect a good fit to the data [34]. RMSEA values of less than 0.05 indicate a good fit and values as high as 0.08 indicate a reasonable fit [34]. The internal consistency of the questionnaire was analyzed with Cronbach’s α (alpha). Reliability equal to or greater than 0.7 was considered acceptable. Pearson correlations coefficients were used to explore the association among the HCQ subscales. Correlation coefficient values between 0.1 and 0.3 were considered as low, between 0.31 and 0.5 as moderate and those over 0.5 were considered as high. The HCQ subscales were compared pre and post substitute administration using paired t-test. All reported p values are two-tailed. The level of statistical significance was set at 0.05 and analysis was conducted using SPSS and AMOS (SPSS, Chicago, IL, USA) software.

Results

The characteristics of the participants are presented in Table 1. According to them, the majority were men (65.1%), Furthermore, 44.2% of the participants were 31-40 years old whereas 33.7% of them were 41-50 years old. More than half of the participants were middle/ high school graduates (55.8%) and almost all were of Greek nationality (96.5%). Moreover, 52.9% of the patients were unmarried and 91.9% lived in urban areas. The mean age at first heroin use was 18.7 ± 5.8 years and the mean duration of heroin abuse was 17.4 ± 8.1 years. Corrected item-total correlations and Cronbach's alpha (if an item was deleted per factor) are presented in Table 2.

Table 1: The characteristics of the participants.

Table 2: Corrected item-total correlations and internal consistency reliability of the HCQ factors.

Almost all corrected item-total correlations were high. Internal consistency reliability was accepted with Cronbach’s alpha as follows: alpha equal to 0.77 for the desire to use heroin and intention and planning to use heroin, 0.74 for anticipation of positive outcome, 0.71 for anticipation of relief from withdrawal or dysphoria, 0.73 for lack of control over use and 0.90 for the completed questionnaire. As indicated in the results of CFA the 5-dimensional model fitted the data well. The RMSEA, CFI and GFI values were 0.054, 0.953 and 0.939, respectively. None of the item cross-loadings exceeded the item loadings on the intended latent construct. Factor loadings were high and ranged from 0.68 to 0.89 indicating a strong association between the latent factors and their respective items.

The mean values of HCQ factors pre and post substitute administration are presented in Table 3. There were significant decreases in all scales, indicating a decrease in heroin craving after the administration of the substitutes and the sensitivity of the HCQ alteration. The correlation coefficients between HCQ factors revealed significant positive correlations among all factors Table 4. Thus, higher craving for heroin use or intention to use is associated with more intense anticipation of positive outcome and relief from withdrawal or dysphoria. Additionally, higher craving is associated with greater intention to use heroin.

Table 3: Mean values (± SD) of HCQ factors pre and post substitute administration.

Table 4: Correlation coefficients between HCQ factors.

Discussion

The present study made for the first time a validation of the multidimensional HCQ questionnaire previously developed [19] by assessing craving in chronic heroin users who are under OMT pre and post administration of the substitutes (i.e., methadone and buprenorphine). Most of the participants filled the 45-item HCQ in approximately 15 min. On the basis of our findings, the values of Cronbach's alpha coefficient were acceptable for the whole questionnaire items. According to the CFA and the assessment of the CFI, GFI and RMSEA indices, there was a high correlation of the item between them confirming that the questions were reliably and appropriately grouped in the five sub-scales and that this structure fits in Greek. Concerning the construct validity, the correlation coefficients between HCQ sub-scales showed high correlation values with a statistically significant difference, which confirms the construct validity of the questionnaire. Regarding the management of the substitute and the comparison pre and post-administration, a statistically significant decrease was observed in all HCQ factors in post-dose time. This finding is representative of the sensitivity of the HCQ to such changes.

A few previous studies have also presented similar results regarding the validation of HCQ in opioid users. Indeed, the correlation coefficients were also found large indicating a high possibility that HCQ scores represent a comprehensive assessment of craving [35]. In addition, the reliability of HCQ has been assessed in several studies with opioid agonist doses [36-40]. The results of Schmidt et al. [41], Dehghani-Arani et al. [42], Greenwald [37] and Strasser et al. [43] showed decreasing values of HCQ dimension after the therapeutic interventions.

This piece of evidence indicates the ability of the questionnaire to reveal differences in time and its sensitivity in changes, particularly in the treatment of craving in patients under OMT, especially in the factors of anticipation of a positive outcome and intend to use. Furthermore, interesting data have shown that in OMT patient ’ s withdrawal symptoms and heroin craving simultaneously increase, adding extra evidence in the pharmacological validation of the HCQ [36,44].

Multidimensional craving measurement tools contain elements that are synonymous with the desire to include parameters related to intention to use, expectations, or lack of overdose control. Those tools lead to highly accurate results, such as increasing positive mood, reducing negative mood or deprivation expanding the concept of craving beyond the parameter of intense desire [45-47]. In addition, craving as a clinical phenomenon triggering addiction has increased the importance of investigating the potential ways that alterations in craving mode induced by addictive substances due to agonist administration and other pharmacological treatments occur [40].

HCQ is a multidimensional craving measurement tool, which includes 5 subscales, namely desire to use heroin, intentions and plans to use heroin, the anticipation of a positive outcome, relief from withdrawal or dysphoria and lack of control overuse. The presence of these sub-scales ensures the extension of the concept of craving beyond the limits of desire. The traditional notion of desire is correlated with similar ideas that conceptually share the same content as an intention to use heroin but which are considered particularly important for clinical assessment since they aim to specify the notion of craving [7].

As a tool for measuring psychometric dimensions, HCQ follows the principle that a measuring instrument cannot be used in a research study unless it has presented a satisfactory level of reliability and validity. The reliability and validity of measuring instruments are two key criteria for ensuring valid results [48]. The high reliability of a measurement tool is associated with the minimization of the random error associated with the consistency and stability it displays so that the variability of the results is small if the measurement is repeated under the same or almost similar conditions. Validity refers to whether a measuring instrument counts what it claims to count, reflects the meaning (i.e., dependent variable) it intends to illustrate. The construct validity is particularly useful for psychometric measuring instruments because it expresses the extent to which a measuring tool imprints the ideas of a theoretical framework [49]. It is necessary, however, to perform factor analysis and search for groups of conceptually and statistically related items in order to obtain a fully functional tool.

Conclusion

In conclusion, we report a successful validation of HCQ for former heroin users being in OMT under methadone and buprenorphine. Specifically, the questionnaire demonstrated internal consistency, reliability and predictive and construct validity reassuring that it can serve as a valuable tool for the assessment of opioid craving in research settings that require the use of repeated measures. At the same time, it can be useful for clinicians as a comprehensive method for the detection of craving symptoms, thus helping in better identifying former opioids using patients in an enhanced risk for relapse.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare no conflict of interest.

Acknowledgments

This article was supported by the Department of Nursing, University of Peloponnese, Greece.

References

1. Sadock B, Sadock V, Ruiz P (2015) Kaplan and Sadock’s synopsis of psychiatry: Behavioral Sciences/Clinical Psychiatry. (11th edtn), Wolters Kluwer LWW, Philadelphia.
2. Ahmadi J, Jahromi MS, Ghahremani D, London ED (2018) Single high-dose buprenorphine for opioid craving during withdrawal. Trials 19: 675.
3. Shiffman S (2002) Comments on craving. Addiction 95: 171-175.
4. Shi J, Zhao LY, Epstein DH, Zhang XL, Lu L (2007) Long-term methadone maintenance reduces protracted symptoms of heroin abstinence and cue-induced craving in Chinese heroin abusers. Pharmacol Biochemistry Behav 87: 141-145.
5. Drummond DC (2001) Theories of drug craving, ancient and modern. Addiction 96: 33-46.
6. Tiffany ST, Warthen MW, Goedeker KC (2009) The functional significance of craving in nicotine dependence. Nebr Symp Motiv 55: 171-97.
7. Tiffany S, Wray J (2012) The clinical significance of drug craving. Ann NY Acad Sci 1248: 1-17.
8. Anton RF (1999) What is craving? Models and implications for treatment. Alcohol Res Health 23: 165-173.
9. Kosten TR (1992) Can cocaine craving be a medication development outcome? Drug craving and relapse in opioid and cocaine dependence. Am J Addictions 1: 230-239.
10. Tiffany ST (1997) New perspectives on the measurement, manipulation, and meaning of drug craving. Human Psychopharmacol 12: S103-S113.
11. Hyman SM, Fox H, Hong KA, Doebrick C, Sinha R (2007) Stress and drug-cue- induced craving in opioid-dependent individuals in naltrexone treatment. Experimental Clin Psychopharmacol 15: 134-143.
12. Ren ZY, Shi J, Epstein DH, Wang J, Lu L (2009) Abnormal pain response in pain-sensitive opiate addicts after prolonged abstinence predicts increased drug craving. Psychopharmacol 204: 423-429.
13. De Jong AJ, Gongora CV, Engelhardt P, Breteler HM (2006) Effects of craving self-report measurement on desire for heroin in opioid dependent individuals. Substance Use  Misuse 41: 1695-1704.
14. Moore TM, Seavey A, Ritter K, McNulty JK, Gordon KC, et al. (2013) Ecological momentary assessment of the effects of craving and affect on risk for relapse during substance abuse treatment. Psychol Addict Behav 28: 619-624.
15. Weiss RD, Griffin ML, Mazurick C, Berkman B, Gastfriend DR, et al. (2003) The relationship between cocaine craving, psychosocial treatment, and subsequent cocaine use. Am J Psychiatry 160: 1320-1325.
16. Hartz DT, Frederick-Osborne SL, Galloway GP (2001). Craving predicts use during treatment for methamphetamine dependence: a prospective, repeated- measures, within-subject analysis. Drug Alcohol Depend 63: 269-276.
17. Shiffman S, Engberg JB, Paty JA, Perz WG, Gnys M, et al. (1997) A day at a time: predicting smoking lapse from daily urge. J Abnorm Psychol 106: 104-116.
18. Sayette MA, Shiffman S, Tiffany ST, Niaura RS, Martin CS, et al. (2000) The measurement of drug craving. Addiction 95: S189–S210.
19. Tiffany ST, Singleton E, Haertzen CA, Henningfield JE (1993) The development of a cocaine craving questionnaire. Drug and Alcohol Dependence 34: 19-28.
20. Ling W, Amass L, Shoptaw S, Annon JJ, Hillhouse M, et al. (2005). A multicenter randomized trial of buprenorphine-naloxone versus clonidine for opioid detoxification: Findings from the National Institute on Drug Abuse Clinical Trials Network. Addiction 100: 1090-1100.
21. Krupitsky E, Nunes EV, Ling W, Illeperuma A, Gastfriend DR, et al. (2011) Injectable extended-release naltrexone for opioid dependence: A doubleblind, placebo-controlled, multicentre randomised trial. Lancet 377: 1506-1513.
22. Dole VP (1994) What have we learned from three decades of methadone maintenance treatment? Drug and Alcohol Review 13:3-4.
23. McHugh RK, Fitzmaurice GM, Carroll KM, Griffin ML, Hill KP, et al. (2014) Assessing craving and its relationship to subsequent prescription opioid use among treatment-seeking prescription opioid-dependent patients. Drug Alcohol Depend 145: 121-126.
24. Tsui JI, Anderson BJ, Strong DR, Stein MD (2014) Craving predicts opioid use in opioid-dependent patients initiating buprenorphine treatment: alongitudinal study. Am J Drug Alcohol Abuse 40: 163-169.
25. Fareed A, Vayalapalli S, Casarella J, Amar R, Drexler K (2010) Heroin anticraving medications: a systematic review. Am J Drug Alcohol Abuse 36: 332-341.
26. Brown R, Kraus C, Fleming M, Reddy S (2004) Methadone: applied pharmacology and use as adjunctive treatment in chronic pain. Postgrad Med J 80: 654-659.
27. Saidak Z, Blake-Palmer K, Hay DL, Northup JK, Glass M (2006) Differential activation of G-proteins by mu-opioid receptor agonists. Br J Pharmacology 147: 671-680.
28. Dahan A, Yassen A, Bijl H, Romberg R, Sarton E, et al. (2005) Comparison of the respiratory effects of intravenous buprenorphine and fentanyl in humans and rats. Br J Anaesth 94: 825-834.
29. Leventelis C, Goutzourelas N, Kortsinidou A, Spanidis Y, Toulia G, et al. (2019) Buprenorphine and methadone as opioid maintenance treatments for heroin-addicted patients induce oxidative stress in blood. Oxid Med Cell Longev 9417048: 9.
30. Wang S (2019) Historical review: Opiate addiction and opioid receptors. Cell Transplantation 28: 233-238.
31. Strain EC, Bigelow GE, Liebson IA, Stitzer ML (1999) Moderate- vs high-dose methadone in the treatment of opioid dependence. JAMA 281: 1000-1005.
32. Saxon AJ, Hser YI, Woody G, Ling W (2013) Medication-assisted treatment for opioid addiction: methadone and buprenorphine. J Food Drug Analysis 21: S69-S72.
33. Mueller R (1999) Basic principles of structural equation modelling: an introduction to LISREL and EQS (2nd edtn) Springer Inc, New York.
34. Hu L, Bentler P (2009) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Model: A Multidisciplinary J 6: 1-55.
35. Heinz JA, Epstein HD, Schroeder RJ, Singleton GE, Heishman JS, et al. (2006) Heroin and cocaine craving and use during treatment: measurement validation and potential relationships. J Substance Abuse Treatment 31: 355-364.
36. Greenwald MK (2002) Heroin craving and drug use in opioid-maintained volunteers: effects of methadone dose variations. Experiment Clin Psychopharmacol 10: 39-46.
37. Greenwald MK (2005) Opioid craving and seeking behavior in physically dependent volunteers: effects of acute withdrawal and drug reinforcement opportunity. Experiment Clin Psychopharmacol 13: 3-14.
38. Greenwald MK, Johanson CE, Schuster CR (1999) Opioid reinforcement in heroin-dependent volunteers during outpatient buprenorphine maintenance. Drug Alcohol Depend 56: 191-203.
39. Greenwald MK, Roehrs TA (2005) Mu-opioid self-administration vs. passive administration in heroin abusers produces differential EEG activation. Neuropsychopharmacol 30: 212-221.
40. Falcato L, Beck T, Reimer J, Verthein U (2015) Self-reported cravings for heroin and cocaine during maintenance treatment with slow-release oral morphine compared with methadone: a randomized, crossover clinical trial. J Clin Psychopharmacol 35: 150-157.
41. Schmidt A, Borgwardt S, Gerber H, Wiesbeck GA, Schmid O, et al. (2014) Acute effects of heroin on negative emotional processing: relation of amygdala activity and stress-related responses. Biol Psychiatry 76: 289-296.
42. Dehghani-Arani F, Rostami R, Nadali H (2013) Neurofeedback training for opiate addiction: improvement of mental health and craving. Appl Psychophysiol Biofeedback 38:133-141.
43. Strasser J, Wiesbeck AG, Meier N, Stohler R, Dursteler-MacFarland MK (2010) Effects of a single 50% extra dose of methadone on heroin craving and mood in lower versus higher-dose methadone patients. J Clin Psychopharmacol 30: 450-454.
44. Schuster CR, Greenwald MK, Johanson CE, Heishman SJ (1995) Measurement of drug craving during naloxone precipitated withdrawal in methadone-maintained volunteers. Experiment Clin Psychopharmacol 3: 424-431.
45. Cox LS, Tiffany ST, Christen AG (2001) Evaluation of the brief questionnaire of smoking urges (QSU-Brief) in laboratory and clinical settings. Nicotine Tob Res 3: 7-16.
46. Anton RF, Moak DH, Latham P (1995) The obsessive-compulsive drinking scale: a self-rated instrument of the quantification of thoughts about alcohol and drinking behavior. Alcohol Clin Exp Res 19: 92-99.
47. Tiffany ST, Carter BL, Singleton EG (2000) Challenges in the manipulation, assessment, and interpretation of craving relevant variables. Addiction 95 (suppl 2): S177-187.
48. Polit DF, Beck CT (2008) Nursing research: Principles and methods. (8th edtn), Lippincott Williams and Wilkins, Philadelphia
49. Jacobson SF (1992) Evaluating instruments for use in clinical nursing research. In: Frank-Stromborg M (ed) Instruments for clinical research. Jones and Bartlett Publishers, Boston.