Open Journal of Social Sciences
Vol.06 No.03(2018), Article ID:83095,16 pages
10.4236/jss.2018.63009

The Development and Validation of a Questionnaire Measuring Barriers to Career Progression Faced by Women Dentists in Saudi Arabia

Mona Rajeh1,2, Belinda Nicolua2, Akram Qutob3, Pierre Pluye4, Shahrokh Esfandiari2

1Department of Preventive Dentistry, Faculty of Dentistry, Umm Al-Qura University, Makkah, Saudi Arabia

2Oral Health and Society Research Unit, Faculty of Dentistry, McGill University, Montreal, Canada

3Department of Dental Public Health, Faculty of Dentistry, King Abdulaziz University, Jeddah, Saudi Arabia

4Department of Family Medicine, Faculty of Medicine, McGill University, Montreal, Canada

Copyright © 2018 by authors and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY 4.0).

http://creativecommons.org/licenses/by/4.0/

Received: February 5, 2018; Accepted: March 16, 2018; Published: March 19, 2018

ABSTRACT

To develop and validate a questionnaire assessing barriers faced by Saudi Arabian female dentists in the progression of their career. We developed a three-part questionnaire based on a literature review, semi-structured interviews, and consultation with 3 panels of experts. The instrument was sent to a convenience sample of 150 female dentists who were faculty members at different university hospitals in Saudi Arabia. Cronbach’s alpha was used to test reliability. Exploratory factor analysis was used to evaluate construct validity. A total of 62 dentists returned the questionnaire (response rate 41.3%), 55 of which were useable for the pilot testing of the measure. The final instrument included 20 items divided into four subscales: Family Challenges, Environment Challenges, Interpersonal Challenges, and Sociocultural Challenges. Cronbach’s α for the total questionnaire was 0.899. Exploratory factor analysis on the questionnaire led to a set of subscales differing from those defined beforehand (KMO = 0.784 very good). A new valid and reliable questionnaire has been developed that measures barriers to female dentists’ career progression in Saudi Arabia.

Keywords:

Dentistry, Female, Middle East, Validation Study

1. Introduction

There is an upward trend in female enrollment in dentistry and female students now represent more than half of the student body worldwide. For example, 54% of students attending colleges in the United States are women, and females earned 57% of all bachelor’s degrees in 2000. This trend is also observed in the two oldest and largest dental schools in Saudi Arabia. In the years 2007 to 2014, the number of female graduates increased by 40.5% at King Saud University (KSU). Similarly, the number of females graduating each year from King Abdul-Aziz University (KAU) surpasses the number of male graduates with approximately 60% female. With this increasing proportion of females to males, the Saudi Arabian dental workforce is expected to be equally split between both genders by 2030 [1] .

Despite the growing number of female dentists who are competing to attain different professional degrees, they are discriminated against and underrepresented in dental organizations and faculty leadership positions. Evidence shows that Saudi Arabian female dentists in government institutions are granted lower work positions within the employment hierarchy, are less likely to hold additional degrees, and are paid less than male dentists [2] [3] [4] . Although Saudi women have broken national traditions by entering occupations previously dominated by men, such as dentistry, their professional arena remains limited by inequality in a society that considers men superior to women. Moreover, Saudi Arabian culture encourages exclusionary policies towards women, as it envisions women as homemakers and mothers rather than people who can contribute to the development of Saudi Arabia’s economy.

Despite emerging data, little is known about the challenges and barriers faced by female dentists in clinical practice, academia, and leadership. Currently, there is no valid national or international questionnaire to assess the barriers to career progression faced by female dentists in Saudi Arabia. Therefore, the aims of this study were to develop a valid and reliable questionnaire to identify and describe the perceived factors that prevent women dentists practicing in Saudi Arabia from advancing in their career.

2. Methods

Ethical approval for this study was obtained from the McGill University Institutional Review Board. The research was conducted in full accordance with the World Medical Association Declaration of Helsinki. Data were collected via a web-based questionnaire and then analyzed using SPSS software version 23. Participants who agreed to participate in the study signed the consent form and then were directed to the questionnaire items. A three-step approach was employed in the development and validation of the questionnaire: the selection of items, a pilot study, and the evaluation of reliability and validity. The following section describe these steps in detail. Finally, we revised the instrument. Figure 1 provides an overview of the steps involved.

2.1. Selection of Item

An item pool was generated from a thorough review of the existing literature,

Figure 1. Guidelines for scale development and analysis. Source: Hinkin, Tracey and Enz (1997).

consultation, and feedback from 3 experts in questionnaire development. Additional items were formulated based on a qualitative study that was conducted with 13 female dentists practicing in the Makkah region who were 1) general practitioners or specialists with 2) more or less experience, and 3) working either in government or the private sector [5] . We chose the Makkah region for three reasons. First, it includes all types of dental practices. Second, it was very familiar to the lead researcher, a Saudi woman dentist who used her intimate knowledge of the context to conduct rigorous, in-depth qualitative research with participants about sensitive issues. Third, this region was deemed the most cost-effective and convenient for this study. We excluded women dentists having less than two years post-graduate experience because they may not have gathered enough work experience to contribute to the study. Other dental professionals, such as dental students or dental assistants, were also excluded.

The glass ceiling theory guided the qualitative study. We used Bombuwela and Chamaru’s (2013) model as the conceptual framework for this study and the qualitative study [6] . Author MR, who is trained in qualitative research methodologies and methods, carried out the qualitative interviews at a location chosen by the participants. We used an interview guide that focused on participants’ professional perceptions, experiences, and challenges, and which consisted of the following core questions: 1) What are some of your experiences as a practicing dentist? 2) How does being a female dentist influence this experience? 3) What challenges have you faced throughout your career so far? 4) What barriers, if any, do you think could influence your practice? 5) How does being a woman influence these barriers? Full details of this work are reported elsewhere [5] .

Informed by the qualitative research described above and the literature search, instrument items were drafted. The initial questionnaire consisted of three sections. While the first section collected information on socio-demographic characteristics of the respondent, the second covered the professional practice characteristics of the dentist. Most questions had categorical response options. The last section was composed of questions related to external factors that may have influenced the respondent’s career path. To measure the relevance of the individual factors, the respondents rated the items on a five-point Likert scale, from “Strongly Agree” to “Strongly Disagree”. At the end of this section, two open-ended questions allowed the respondent to write about other possible challenges/facilitators that might influence their dental practice and possible practices that might be supportive of women dentists’ careers in Saudi Arabia. The initial instrument (section of the questionnaire on challenges) included 22 items divided into four subscales that assessed 1) family-related challenges, 2) sociocultural challenges, 3) workplace challenges, and 4) transportation issues.

Individual face-to-face cognitive interviews were conducted with two Saudi female graduate students at McGill University. Participants were encouraged to share their thoughts about the items with the researcher and provide feedback about the content and clarity of the draft questionnaire. Items were then revised in collaboration with the research team. We consulted three experts to ensure the face and content validity of the questionnaire. These experts evaluated the items of the questionnaire for their clarity, importance, and relevance. Based on the experts’ comments and feedback, minor changes were made in word choices and sentence structure to improve the clarity of the questionnaire.

2.2. Pilot Study

The pilot questionnaire was emailed to a convenience sample of 150 female dentists who were general practitioners or specialists working at different university hospitals in Saudi Arabia. We used the same inclusion/exclusion criteria described earlier for our qualitative study [5] . Participants were given four to six weeks to complete the survey. The questionnaire content was then modified and revised according to the participants’ responses.

2.3. Reliability and Validity Testing

2.3.1. Testing the Reliability

Internal consistency reliability was calculated using Cronbach’s alpha coefficient, and corrected item-total correlations were used to decide which low-contributing items should be removed from the scale. Cronbach’s alpha values of 0.70, 0.80, and 0.90 are commonly considered to be acceptable, good and excellent, respectively [7] .

2.3.2. Testing the Construct Validity

Exploratory factor analysis (EFA) was conducted using principal axis factoring (PA) as the extraction method and the Promax rotation in SPSS v23. Promax is one of a number of possible oblique rotations that are appropriate for correlated data, and is the most widely used [8] . The number of factors was determined by examination of eigenvalues > 1 and the screen plot [9] [10] .

3. Results

We received a total of 62 questionnaires (response rate: 41.3%). However, seven participants did not complete the survey and therefore were not included in the analysis. Thus, our final sample consisted of 55 questionnaires, including two from post-graduate students and one from a participant who was unemployed. The sociodemographic and relevant characteristics of the participants are shown in Table 1.

3.1. Item Selection

As described above, the 22 questionnaire items were originally formulated to be

Table 1. Sociodemographic and relevant characteristics of the participants.

representative of four constructs: family-related challenges (“family”), socio-cultural challenges (“social”), workplace challenges (“work”), and transportation issues (“transportation”). All 22 items were included in the construction of the correlation matrix (Table 2) and in the initial reliability and validity analyses. The initial exploratory factor analysis leads us to revise the constructs as shown in Table 3. The “work” construct was decomposed to reflect challenges that are more “interpersonal” (male dominance, gender discrimination, a male colleague, and patient attitudes) and “environmental” (lack of facilities, leadership, mentoring, and role models). Additionally, the “transportation” construct was added to the environmental construct. Finally, items 4 (financial) and 5 (income) were moved from “work” to “family” and item 15 (family friendly policies) was moved from “work” to “social”. The exploratory analysis also showed that results were

Table 2. Correlation matrix for challenges.

a Determinant = 4.033E-8.

Table 3. Item constructs for challenges.

improved by removing items 4 (financial) and 10 (education) from the models. Accordingly, we removed the two items; item 4 and 10. Thus, the revised questionnaire included 20 items reflecting 4 constructs: family, interpersonal, environmental, and social challenges. These decisions are explained in greater detail below.

3.2. Reliability

The internal consistency was very good when testing the 20 items related to the challenges questions of the survey that were retained. Cronbach’s alpha was found to be 0.899 (Table 4). The last column of the table indicates the alpha level when an item is deleted. In general, if the alpha is measurably higher than the overall alpha, the item can be removed [11] . In this case, items 2 (childcare) and

Item-Total Statistics Reliability Statistics

Table 4. Reliability Statistics.

a The items are: 1 through 12 (except 4 and 10) b The items are: 13 through 22.

3 (balance) each appeared to indicate an improvement by their removal; however, the change was extremely small, only 0.002 - 0.003 or about one-third of one percent.

As an additional step, split-half reliability was tested. This test breaks the items into two sets of equal size and computes Cronbach’s alpha separately on each set. In this instance, the items are simply split in order, with the first 10 falling in part 1 and the remaining 10 falling in part 2. For part 1, CA = 0.800, and for part 2, CA = 0.877. The part 1 value, while noticeably lower than the overall or part 2 results, are still within acceptable bounds. It is fair to say that the survey shows sufficient internal consistency.

3.3. Exploratory Factor Analysis

Examining the correlations between items, we noted that the correlations among the first three items were quite high, all above 0.80 (Table 2). In addition, the correlation between items 4 (financial) and 5 (income) was rho = 0.843. High correlations typically indicate that the items are either measuring the same construct or can be substituted for each other [12] . Moreover, heavily correlated items may later be problematic in analysis. Therefore, the decision was made to keep item 5 (income) and remove item 4 (financial) based on the exploratory analysis results described above. The first three items were all retained and did not cause any issues in subsequent analysis.

We also noted correlations of items 4 (financial) and 5 (income) with the three “family” items. This, along with the initial factor analysis results, lead to the decision to move item 5 (income) to the “family” factor. Most of the remaining correlations were neither too high nor too low [13] . Item 10 (education) was removed after the initial factor analysis. This decision was supported by the correlation table, as the item was very mildly correlated across factors and did not clearly belong to any one. The correlation table also supports the decision to refine the “work” construct into the more meaningful “interpersonal” and “environmental” constructs. The “transportation” items were mildly correlated with many of the other items, but most strongly with those grouped into “environmental”.

An examination of the screen plot was also used to determine the optimal number of factors. Figure 2 shows the screen plot for the first EFA that was

Figure 2. Screen plot for initial exploratory Factor analysis.

conducted. We looked for the “elbow” where the plot drops off. From this plot, one could choose anywhere from three to five factors, but the number of factors initially selected should be driven primarily by the theory [14] . It was determined at early stages that “work” should be separated into two dimensions; therefore, a five factor solution was briefly examined (results not shown). Ultimately, the four factors representing the “interpersonal”, “environmental”, “family” and “social” constructs were selected.

As indicated, the correlations informed the decision to remove item 4 (financial) from the questionnaire. Subsequently, an exploratory factor analysis was conducted with the remaining 21 items. The table of communalities from the EFA shows the relation between one item and all others. Extracted communalities can be interpreted as the percent of item variance explained by the factor. Items with communalities below 0.30 should generally be removed [15] [16] . Item 10 (education) had an extraction communality of 0.141 in this initial model and therefore was removed. Note that item 21 (location) had an extraction communality of 0.299, which was borderline; this item was retained and the communality improved to an acceptable 0.324.

Suitability of data for factor analysis was confirmed by KMO and Bartlett’s test of sphericity. The literature suggests that the KMO should be at least 0.60; thus, the obtained KMO value of 0.784 is very good [16] . The result of Bartlett’s test (p = 0.000) was below 0.05 and allowed us to reject the null hypothesis that the inter-correlation matrix is an identity matrix, that is, all ones on the diagonal and zeros otherwise. This result confirmed that factor analysis was appropriate.

As mentioned above, an initial exploratory factor analysis showed that it was worthwhile to reformulate the original constructs as shown in (Table 3). The final EFA specified four factors to be extracted. Table 5 and Table 6 show the resulting pattern, factor correlation and structure, matrices respectively. While the structure matrix gives the actual loadings, the pattern matrix shows more clearly which items load onto each factor. The loadings represent how much of an item is explained by the factor. Ideally one wants to see items load cleanly onto a single factor. However, in practice this will not always be the case; and indeed, theory can support this [8] . The pattern matrix shown in (Table 5) was used to finalize the revised construction of the items as discussed above.

Examination of the structure matrix showed the relationships between the items and the other factors (Table 6). Most notable was the lack of any cross-loadings on the “family” factor with the other factors. Further, the high values showed that the retention of the first three items was a good decision. The exception was items 5 (income), which cross-loaded on the “environmental” factor. In the pattern matrix, item 12 (leadership) was shown to be the weakest item on the “environmental” factor. We saw that it cross-loaded on both the “interpersonal” and “social” factors. In a similar fashion, item 15 (family friendly policies) was the weakest of the “social” construct items and cross-loaded with “environmental,” which is closer to where it was in the original construction.

Table 5. Final Exploratory Factor Analysis, pattern.

Extraction Method: Principle Axis Factoring. Rotation Method: Promax with Kaiser Normalization. a. Rotation converged in 5 iterations.

4. Discussion

We developed and validated an instrument to measure factors that may impact the careers of female dentists in Saudi Arabia. The questionnaire was designed to be short, practical, easy to analyze, and comprehensive. Shorter questionnaires are easier to complete and increase the quantity and quality of responses [17] . The process of developing the questionnaire involved a review of the literature, qualitative research, expert review, validity and reliability testing. The steps employed in constructing the scale closely follow the scale development guidelines proposed by DeVellis (2003) and Hinkin et al. (1997) [18] [19] .

Of the original 22 items, only two were removed: item 4 (financial) and item 10 (education). Item 4 (financial) clearly overlapped with item 5 (income), which was originally designed to measure two different aspects pertaining to money, reward and income. However, the results did not support this distinction. The education question (item 10) did not clearly align with any of the other

Table 6. Final exploratory factor analysis, structure.

items or with the constructs. In hindsight, this is consistent with the fact that the target audience was working dentists who face challenges in their practice.

Other items that were either weakly related or did not clearly load on a single factor included items 5 (income), 12 (leadership), 14 (role model), 15 (family friendly policies), 16 (stereotypes) and 21 (location). Looking at the raw data, we observed that in this sample 34.5% of respondents were neutral on item 12 (leadership), neither agreeing nor disagreeing. This might explain its weakness in the analysis. As discussed previously, item 5 (income) was deemed appropriate for the “family” construct while also relating to work through the “environmental” construct, taking the place of item 4 (financial). Item 14 (role model) loads on the “environmental” factor but also cross-loads on the “social” factor. This item correlated with the social construct items, especially items 17 (customs) and 19 (traditions). Item 15 (Family friendly policy) cross-loads with the “environmental” factor, where it was originally located, showing that the results are consistent with our conceptual model. Item 16 (stereotypes) cross-loads with the “interpersonal” factor. This too is theoretically consistent given that stereotypes encountered in the workplace are driven by the larger social environment. Finally, item 21 (location) cross-loads on both the “interpersonal” and “social” factors. It is not clear why this item loads on the “interpersonal” work component.

The final instrument consists of 20 items on four subscales (Appendix 1). We found that the subscales demonstrated good internal consistency reliability and construct validity when tested on a sample of female dentists working at different practice locations in Saudi Arabia. These results indicate that the questionnaire satisfies the criteria for an acceptable scale with regard to reliability and validity in the development of a new measure [19] .

The testing of the measure’s validity and reliability were our primary concerns. The internal structure of the questionnaire was tested by using Cronbach’s alpha [7] . The observed alphas for the total scale and subscales were higher than 0.70, which indicates an acceptable internal consistency [7] .

The validity of an instrument refers to how well the results of a test, as interpreted for a specific purpose, can be trusted. There are different types of validity, including face validity, content validity, construct validity, and criterion validity [15] . We applied various methods in this study. The basic principles for survey design recommended by de Vaus were applied to generate questions and identify domains at the first stage of the development [20] . Female dental practitioners and experts in questionnaire development and validation were involved in the item generation procedure to ensure that the items would be relevant to the purpose of the instrument, promoting content validity [15] [21] .

To test construct validity, factor analysis was employed. This analysis shows whether the individual items of the survey measure the constructs of interest. In other words, do the items align with the four challenges outlined through the qualitative study: family, social, work, and transportation challenges? [5] . In an EFA, the categorization of items into factors is driven by the procedure and not by theory [14] . Exploratory factor analysis produces factors in order of importance or impact. Thus, factor 1 corresponds to the “environmental” construct and orders the items as follows: mentoring, transport, driving, facilities, role model, location, and leadership. Factor 2 represents the “interpersonal” construct and loads the items as follows: gender, male, attitude, and patient. Next, factor 3 represents the “family” construct with the items ordered as follows: family, balance, child, and income. Finally, factor 4 represents the “social” construct items: societal, customs, tradition, stereotype, and family policy.

The correlations between factors reflect what was observed in the structure and pattern matrices as indicated in the original correlation table. The “family” construct (factor 3) did not correlate highly with the other three factors. “Environmental” and “interpersonal” factors were correlated with each other and each of these was correlated with “social”. These correlations make theoretical sense; the decomposed “work” construct exists in the context of the broader “social” construct. One should note that if the correlations were very high, it would have been an indication that the scale included too many factors. The revised constructs are better at capturing what the qualitative study identified as challenges or barriers in the workplace for female dentists in Saudi Arabia. Factors should, in general, contain a similar number of items and should not contain too few [12] .

There are limitations to this study; the main concern was the sample selection method. We pilot tested the questionnaire in a convenience sample with only female dentists who are faculty members at teaching universities in Saudi Arabia. This could have reduced the individual variability of the results, which could have lead to an underestimation of the internal consistency of some scales. Another limitation due to scarce resources was the small sample size, which may limit statistical power and introduce selection bias. Finally, we did not assess test-retest reliability in the current study, as the participants stated that they were too busy to complete the questionnaire a second time.

5. Conclusion

We have developed and validated the first questionnaire to measure factors that may affect the careers of female dentists, which was intended to be brief and user-friendly as it takes about five minutes to complete. The questionnaire has four subscales that enable the simultaneous measurement of different challenges associated with a female dentist’s career: Family Challenges, Environment Challenges, Interpersonal Challenges, and Sociocultural Challenges. Further research is needed to evaluate other aspects of reliability and validity of the questionnaire. For example, it would be informative to determine the test-retest reliability of the scale to evaluate whether the test results are consistent over time. However, based on our validation results, we are confident that the current questionnaire will be useful in its present form. Ultimately, it can be used to compare barriers to female dentists between different national and international dental institutions and help in planning relevant policy changes. The measure has utility for policy makers in identifying ways to create incentive systems and policies that maintain, develop and retain women in the dental workforce. A few suggestions arise out of this study: adopt family-friendly polices (e.g., part-time work options); promote equality concerning promotion and ensure equal representation of women in top management and leadership positions.

Acknowledgements

The authors would like to thank the practices and their staff for participating in the validation study. BN holds a Canada Research Chair Tier II in Life Course Oral Epidemiology. The present study did not receive any financial support.

Conflict of Interest

The authors declare that they have no conflict of interest.

Cite this paper

Rajeh, M., Nicolua, B., Qutob, A., Pluye, P. and Esfandiari, S. (2018) The Development and Validation of a Questionnaire Measuring Barriers to Career Progression Faced by Women Dentists in Saudi Arabia. Open Journal of Social Sciences, 6, 131-146. https://doi.org/10.4236/jss.2018.63009

References

  1. 1. Stewart, F.M. and Drummond, J.R. (2008) Women and the World of Dentistry. British Dental Journal, 188, 7-8.

  2. 2. Abdullah, S. and Saeed, T. (2013) Comparison of Role Stressors, Job Satisfaction and Turnover Intensions of Dentists Working in Public and Private Sectors. Pakistan Oral Dental Journal, 33, 412-417.

  3. 3. Ashri, N.Y., Al Ajaji, N., Al Mozainy, M. and Al Sourani, R. (2009) Career Profile of Dentists in Saudi Arabia. Saudi Dental Journal, 21, 28-36.

  4. 4. Pallavi, S.K. and Rajkumar, G.C. (2011) Professional Practice among Woman Dentist. Journal of International Society of Preventive Community Dentistry, 1, 14-19. https://doi.org/10.4103/2231-0762.86376

  5. 5. Rajeh, M., Nicolau, B., Pluye, P., Qutob, A. and Esfandiari, S. (2016) Are There Barriers for Professional Development of Women Dentists? A Qualitative Study in Saudi Arabia. JDR Clinical and Transational Research, 2, 119-131. https://doi.org/10.1177/2380084416685083

  6. 6. Bombuwela, P.M. and Chamaru, D.A.A. (2013) Effects of Glass Ceiling on Women Career Development in Private Sector Organizations—Case of Sri Lanka. Journal of Competitiveness, 5, 3-19. https://doi.org/10.7441/joc.2013.02.01

  7. 7. Kline, P.A. (1986) Handbook of Test Construction: Introduction to Psychometric Design. Methuen, London.

  8. 8. Mastsunaga, M. (2010) How to Factor-Analyze Your Data Right: Do’s, Don’ts, and How-To’s. International Journal of Psychological Research, 3, 97-110. https://doi.org/10.21500/20112084.854

  9. 9. Kaiser, H.F. (1960) The Application of Electronic-Computers to Factor-Analysis. Educational and Psychological Measurement, 20, 141-151. https://doi.org/10.1177/001316446002000116

  10. 10. Cattell, R.B. (1966) The Screen Test for the Number of Factors. Multivariate Behavioral Research, 1, 245-276. https://doi.org/10.1207/s15327906mbr0102_10

  11. 11. Gliem, J. and Gliem, R. (2003) Calculating, Interpreting, and Reporting Cronbach’s Alpha Reliability Coefficient for Likert-Type Scales. Midwest Research to Practice Conference in Adult, Continuing and Community Education. Columbus, OH.

  12. 12. Yong, A.G. and Pearce, S. (2013) A Beginner’s Guide to Factor Analysis: Focusing on Exploratory Factor Analysis. Tutorials in Quantitative Methods for Psychology, 9, 79-94. https://doi.org/10.20982/tqmp.09.2.p079

  13. 13. Osborne, J.W. and Costello, A.B. (2012) Best Practices in Exploratory Factor Analysis: Four Recommendations for Getting the Most from Your Analysis. Pan-Pacific Management Review, 12, 131-146.

  14. 14. Flora, D.B., La Brish, C. and Chalmers, R.P. (2012) Old and New Ideas for Data Screening and Assumption Testing for Exploratory and Confirmatory Factor Analysis. Frontiers in Psychology, 3, 1-21. https://doi.org/10.3389/fpsyg.2012.00055

  15. 15. Cook, D.A. and Beckman, T.J. (2006) Current Concepts in Validity and Reliability for Psychometric Instruments: Theory and Application. American Journal of Medicine, 119, 166.e7-166.e16. https://doi.org/10.1016/j.amjmed.2005.10.036

  16. 16. Beavers, A.S., Lounsbury, J.W., Richards, J.K., Huch, S.W., Skolits, G.J. and Esquivel, S.L. (2013) Practical Considerations for Using Exploratory Factor Analysis in Educational Research. Practical Assessment Research and Evaluation, 18, 1-13.

  17. 17. Rolstad, S. and Adler, J.A.R. (2011) Response Burden and Questionnaire Length: Is Shorter Better? A Review and Meta-Analysis. Value Health, 14, 1101-1108. https://doi.org/10.1016/j.jval.2011.06.003

  18. 18. DeVellis, R.F. (2003) Scale Development: Theory and Application. 2nd Edition, Sage, Thousand Oaks.

  19. 19. Hinkin, T.R., Tracey, J.B. and Enz, C.A. (1997) Scale Construction: Developing Reliable and Valid Measurement Instruments. Journal of Hospitality and Tourism Research, 21, 100-120. https://doi.org/10.1177/109634809702100108

  20. 20. De Vaus, D.A. (1991) Surveys in Social Research. 3rd Edition, UCL Press, London.

  21. 21. McGrath, C., Yeung, R.W.K., Comfort, M.B. and McMillan, A.S. (2005) Development and Evaluation of a Questionnaire to Evaluate Clinical Dental Teachers (ECDT). British Dental Journal, 198, 45-48. https://doi.org/10.1038/sj.bdj.4811957