Doctoral Degrees (Office of the Dean: Health Sciences)
Permanent URI for this collection
Browse
Browsing Doctoral Degrees (Office of the Dean: Health Sciences) by Subject "Assessment"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item Open Access Standard setting for specialist physician examinations in South Africa(University of the Free State, 2015-01) Schoeman, Frans Hendrik Scarpa; Nel, M. M.; Burch, V. C.English: Setting defensible and fair pass standards for high-stakes postgraduate specialist certification examinations is a critical quality assurance component of assessment. Doing so in a feasible and sustainable way, within a resource-constrained context such as South Africa, is challenging. Traditionally the 28 member Colleges of the Colleges of Medicine of South Africa (CMSA), the national specialist licensing examination body in South Africa, have used a fixed pass mark of 50%. This practice does not acknowledge the inherent variance in examination difficulty and so increases the risk of failing competent candidates (false negative outcome) and passing incompetent examinees (false positive outcome). In 2011, the College of Physicians (CoP), a large CMSA member College, addressed the matter by implementing a standard setting process for the written components of their specialist physician certification examinations. The aim of this study was twofold: i) To evaluate the knowledge, attitudes, views and perspectives of CoP examiners regarding standard setting, and ii) compare the performance and utility of the Cohen and Angoff methods to advise the CoP regarding an appropriate standard setting method in a resource-constrained setting. A literature review was done to conceptualise standard setting as it pertains to assessment in medical education. In addition, policies and regulatory systems relevant to specialist certification examinations in South Africa were reviewed to provide the context for this study. Two research components were concurrently conducted between 2012 - 2014: A prospective study evaluated the knowledge, attitudes, views and perspectives of CoP examiners regarding standard setting before and after training and 30 months of practical experience using both the Cohen and Angoff methods of standard setting. A comparative study evaluated the performance (pass marks and failure rates) and utility (according to a framework derived from the literature review) of the Cohen and Angoff methods using five cycles of examination data, including multiple choice questions (MCQ), short answer questions and short essay questions. The introduction of standard setting was successful and widely supported by the CoP examiners. The Cohen method performed well when used for test data with a reasonable number of test items (30 or more) in homogeneous exit-level cohorts of more than 50 candidates. Tests containing few test items (i.e. short essay questions) performed poorly. The performance of the Cohen method was variable for smaller cohorts (less than 100) of candidates drawn from heterogeneous populations, such as entry-level Part I MCQ test takers. The Angoff method yielded unacceptable outcomes regardless of test format. The utility comparison identified the Cohen method as the preferred standard setting method for the CoP. The findings of this study support the introduction and ongoing use of the Cohen method as a feasible and sustainable method of setting pass marks for the written components of the CoP certification examinations. Education and training in the use of standard setting methods, as part of a change management strategy, improved examiners’ understanding of the role, importance and basic methodology of standard setting and strengthened their support for the use of standard setting in certification examinations. More data are needed to evaluate the true impact of cohort size on the stability of the Cohen method for entry-level, heterogeneous cohorts of examinees. The purist Angoff strategy, used in this study due to resource limitations, performed poorly and was deemed ‘not fit for purpose’ by the CoP examiners. The usefulness of the novel standard setting utility framework developed in this study warrants further research in other examination settings such as performance–based examinations.Item Open Access A work-integrated learning education and training programme for radiography in South Africa(University of the Free State, 2015-06) Du plessis, Jeanetta Gesina Elizabeth; Bezuidenhout, J.English: An in-depth study was conducted to investigate current practices in the delivery of work-integrated learning (WIL) in Radiography training at selected universities in South Africa (SA) with the ultimate goal of developing an education and training programme for WIL in Radiography training. The study was prompted by a growing awareness that a gap existed in the delivery of WIL in Radiography programmes in SA. WIL is a relatively new jargon term that focuses attention on the integration of theoretical learning and learning in the workplace. Until recently the focus in many professional programmes has been on workplace learning (WPL) as a component of the training of students. Although the concept of placing students in the workplace to acquire work experience is not new, the rationale behind WIL goes beyond merely providing a physical workplace environment as a site for students to experience work or to learn from professional practice. However, to achieve success in the delivery of any WIL programme, it is important to structure the WIL component of any learning programme with regards to the diligent and constructive alignment of learning outcomes/objectives, the delivery of learning (i.e., facilitation methods), and assessment. For quality delivery of WIL, the coordination and monitoring of the learning processes, and hence of student progression, are also important. The research methods comprised a two-fold approach. First, a literature review was conducted in order to provide an appropriate background for the formulation of a conceptual framework and to contextualise the problem against related theory and research. Second, data were collected using questionnaires that generated both quantitative and qualitative data. The latter data were obtained by means of open-ended questions that allowed for comments that facilitated the gathering of information about the current status of the delivery of WIL in Radiography training. The compilation of an education and training programme for WIL for the Bachelor of Radiography in Diagnostics degree to enhance undergraduate radiography training in SA was achieved by merging the information from the literature review and the collected data. The findings on the current status of WIL, which emanated from the questionnaire survey, were compared with suggestions garnered from the literature for best practice in WIL in order to make recommendations to bridge the identified shortcomings in the delivery of WIL in Radiography training. By developing an education and training programme for WIL in Radiography and by providing recommendations towards improvements in the delivery of WIL, the study contributed significantly to the creation of new knowledge in the Radiology field and was thus successful in bridging the identified gap in the delivery of WIL. The implementation of the proposed programme for WIL can aid in the development of a curriculum for WIL that includes appropriate modules, learning units, assessment and monitoring strategies, and guidelines for the development of generic skills. The sound research approach and methodology that were employed ensured the quality, reliability and validity of the study. The completed research can form the basis for a further research undertaking.