English

Permanent URI for this community

http://hdl.handle.net/11660/122

Browse

Browsing English by Subject "Academic literacy"

Now showing 1 - 5 of 5
  • Thumbnail Image
    ItemOpen Access
    The design, refinement and reception of a test of academic literacy for postgraduate students
    (University of the Free State, 2012-01) Du Plessis, Colleen Lynne; Weideman, A. J.
    English: A number of studies have indicated that the literacy levels of students at tertiary institutions in South Africa are lower than required for academic success. Nonetheless, the possibility exists that some of these students may in fact manage to graduate with low academic literacy levels, and even be admitted to postgraduate study although they lack the ability to negotiate the advanced academic discourse required to succeed at that level. This study confirms the need for the development of a test of academic literacy for postgraduate students and shows how this can be accomplished by applying recognized design principles within the framework of applied linguistics. The constitutive and regulative conditions for the validation of language tests are dealt with as the necessary foundation for the design of a test of academic literacy for postgraduate students. Whereas language tests have conventionally been required to show validity and reliability, in terms of more contemporary thinking such tests must also possess what is referred to as consequential validity, a notion that refers to the impact of tests. Literacy tests should thus not only be consistent and theoretically justifiable, but should incorporate a multiplicity of evidence to back up their validation. The various phases involved in the design of a test are covered, including piloting and refinement. A theoretical articulation of academic literacy is provided, since this constitutes a crucial aspect of construct validity, with particular attention being given to the delineation of functional academic literacy. The identified ability is reflected in the blueprint for the test construct and is further specified in the task types selected for inclusion in the test. The definite move away from previous generations of ‘scientific’ and positivist thinking can be discerned in the kind of test tasks required of test takers and the way in which these emphasize the instrumental communicative function and mediating role of language as a social instrument within the material lingual sphere of academic discourse. Task types are evaluated in terms of their ability to be productive, based on a quantitative system of measurement and the application of appropriate statistical procedures such as point-biserial correlations. The research concludes with a reception study which assesses the extent to which the current version of the Test of Academic Literacy for Postgraduate Students (TALPS) is well received and whether it meets the regulative conditions of fairness, transparency, utility and accountability. In addition to confirming that the test has an acceptably high face validity, the survey section reveals that many postgraduate students have a restricted view of language and an erroneous perception of academic literacy. Much more needs to be done at undergraduate level to raise awareness about the important role played by language proficiency in all fields of study and how high academic literacy levels can contribute towards academic achievement and student success. The design of effective literacy interventions remains another area of concern. Research is also needed on the desirability and feasibility of designing subject-specific tests and ensuring equivalence where different test versions are in operation.
  • Thumbnail Image
    ItemOpen Access
    Investigating the efficacy of the "Skills for a Changing World" first-year literacy course
    (University of the Free State, 2011-11-30) Drennan, Laura Maria; Van Wyk, Arlys
    The academic performance of students entering higher education in South Africa has been high on the agenda of universities, organisations working in this sector, the Department of Education, and the media. The reason for this is that many students do not meet the admission requirements for higher education institutions. The low level of academic language proficiency of first-year students in particular is evidenced by entry-level proficiency testing. In response to this problem, new English literacy materials were generated at the University of the Free State to target such students and provide a potential access route to higher education institutions. The investigation of the efficacy of the Skills for a Changing World English literacy course employed a two-part study. The first, a pilot study, encompassed a non-equivalent quasi-experimental research approach which focussed on the performance of a non-equivalent control and experimental group in two different English literacy programmes. The results showed that the new English literacy course neither significantly improved the reading scores nor the academic performance of the students. Further qualitative research was required to investigate issues such as student motivation, students’ perceptions of learning, and facilitators’ perceptions of teaching. These are addressed in the current Master’s study, which adopts a mixed-method approach, where both qualitative and quantitative data was collected simultaneously in the form of pre- and post-test scores, facilitator journal entries, student focus groups, transcriptions of facilitator meetings, and a student questionnaire. The research methodology encompassed an ethnographic study, which involved working with students and facilitators who had been exposed to the Skills for a Changing World English literacy materials for one academic year. The goal of the Master’s study is to determine whether the course changed students’ performance on the National Benchmark Tests (NBT); what students’ perceptions were of their learning on the course; how facilitators experienced teaching the course materials; and whether students enjoyed the course content. The results unfortunately showed a drop in student performance on the NBT post-test, which could possibly be explained by lack of motivation to perform in a test that does not count for marks. Furthermore, the qualitative data seemed to indicate that some students failed to see the value of the course, and that some of the materials were irrelevant and uninteresting. It is postulated that this could have impacted on student motivation, and thus their performance on the course. The lack of facilitator training in English Second Language (ESL) composition also became apparent in the assessment of students’ work. Content-based instruction (CBI) is discussed as a potential solution to these issues, with a particular focus on formative assessment as an integral part thereof.
  • Thumbnail Image
    ItemOpen Access
    Konstitutiewe voorwaardes vir die ontwerp van 'n toets van akademiese geletterdheid
    (University of the Free State, 2010-11) Van Dyk, Tobias Johannes; Weideman, A. J.
    English: Throughput rates at South African universities are low and contribute, among others, to financial losses for a number of stakeholders: students, their parents/guardians, donors of scholarships, universities and the state. This compels institutions to investigate those factors influencing study success. Various international and local investigations indicate that academic language ability is one of several factors that has a significant influence on academic success. In order to make informed decisions about low stakes issues such as language support, universities need mechanisms to enable them to do so. One such mechanism is the use of measuring instruments, such as the Toets van Akademiese Geletterdheidsvlakke (TAG), to make a diagnosis of students‘ academic language ability, and to then place them on appropriate language support programmes. Tests, however, have the obvious power to touch the lives of people in some way. It is therefore necessary that an accountable and transparent approach is followed when designing and implementing tests such as TAG, and when interpreting test results. This is usually done by investigating the reliability and validity of a test; the latter by means of a process of validation. This study is an example of such an investigation where TAG is subjected to thorough and systematic scrutiny. A framework that considers applied linguistics as a discipline of design was followed, against which the construct (underlying theoretical framework), test specifications (the blueprint) and task types, as well as the reliability and validity of TAG were investigated. The framework suggests that a test is a technical design that can be used as solution to a (language) problem. This technical design is grounded upon certain constitutive conditions underlying it, and disclosed by regulative conditions. The former include aspects such as reliability and validity, and the latter utility, transparency, accountability and care. Although the focus of this study was on the constitutive conditions for designing and developing TAG, it repeatedly refers to the importance of the interplay between constitutive and regulative conditions, as well as the fact that a test such as TAG cannot be investigated from one perspective only. The argument of this study is thus that a technically accountable approach to the development and implementation of, and investigations into tests such as TAG is necessary in order to use them in a justifiable and responsible manner. The conclusion of this study is that TAG is a test of high quality and that it can without a doubt be used by universities for purposes of placement, because it was designed and implemented with care and its results are constantly being scrutinised.
  • Thumbnail Image
    ItemOpen Access
    A theoretical justification for the design and refinement of a Test of Advanced Language Ability (TALA)
    (University of the Free State, 2018-09) Steyn, Sanet; Weideman, A. J.; Du Plessis, C. L.
    The emphasis on political equality among the official languages of South Africa makes equivalence in the instruction and assessment of these languages at school level an important objective. The results of the National Senior Certificate (NSC) examination signal a possible inequality in the measurement of language abilities between the set of Home Languages (HLs) offered, as well as in the measurement of First Additional Languages (FALs). This necessitates action on the part of applied linguists to find a viable instrument for equivalent assessment. In order to do so, one must first find common ground among the various languages on the basis of which one can then derive a generic set of abilities that form part of an advanced language ability in any of these languages. As components of an overall ability, these will inform an idea of advanced language ability on which the further articulation of a construct for such a test should be based. This study explores the assumption that there are certain functions of language that all languages have in common, even though these different languages may not necessarily operate equally well in all material lingual spheres of discourse. Using as a theoretical basis the Curriculum and Assessment Policy Statement (CAPS), as well as current thinking about language teaching and assessment, this study not only provides a definition and further explication of advanced language ability but also describes the design of an assessment instrument to test this ability, the Test of Advanced Language Ability (TALA), that operationalizes the components of this construct. This test could potentially be the basis of a new, generic component of the NSC examination for Home Languages that might provide us with an instrument that can be demonstrated to be equivalent in terms of measurement, should it prove possible to develop similar tests across all the Home Languages. The study concludes with an evaluation of this instrument, a critical look at the limitations of the study and an overview of the potential utility of both the instrument and the findings of this investigation beyond its original aims.
  • Thumbnail Image
    ItemOpen Access
    Transparency, accessibility and accountability as regulative conditions for a postgraduate test of academic literacy
    (University of the Free State, 2012-01) Rambiritch, Avasha; Weideman, A. J.; Brokensha, S. I.
    English: This study is concerned with transparency, accessibility and accountability as regulative conditions for a postgraduate test of academic literacy. What it will propose to do is investigate how these can be incorporated into the design of one test, the Test of Academic Literacy for Postgraduate Students (TALPS), and theoretically accounted for in terms of a framework. A main focus is to show that the questions raised here about the social dimension of language testing cannot be adequately answered by experts in the field like Messick (1989b; 1996), Bachman and Palmer (1996), and Kunnan (2000; 2004). Instead these questions can be answered in a “third idea, other than validity and usefulness” (Weideman 2009a: 239), as outlined by Weideman, an idea that does not foreground one concept but rather identifies a number of fundamental considerations for language testing. The argument here is that construct and other empirically based forms of validity are not enough to validate a language test and that what is needed, in addition, is a detailed look at issues of transparency, accessibility and accountability. This study begins by contextualising the problem of poor academic literacy and outlining the need for academic literacy tests such as the Test of Academic Literacy Levels (TALL) and TALPS. This is followed by an in-depth study of previous work in the field of language testing. The literature on key concepts such as validity, reliability, accessibility, transparency and accountability is surveyed as well. An important part of this study is telling the story of TALPS from its initial conceptualisation to its final implementation. Included in this is a detailed study of the reliability and validity of the test, taking the form of a validation argument. Subsequent chapters (5, 6 and 7) focus specifically on issues of transparency, accessibility and accountability as they relate to TALPS. This study would not be complete without the voices of the test takers. A detailed summary of the data collected from a questionnaire administered to students who wrote TALPS is offered as well. The questionnaire has been designed to elicit information, comments, questions and reactions from the testees about the test. The final chapter in this study will attempt to provide a summary of the answers to the important questions that have been asked and answered in the course of this investigation. It will also consider the link between transparency, accessibility and accountability, and will focus briefly on other conditions in the framework that contribute to the design of fair and socially acceptable tests. This study hopes to make a contribution to the field of language testing by concentrating on an area of testing that has been largely ignored – the social dimension. One of the aims of this study is to show the complementarity among the empirical, social and ethical dimensions of TALPS. It therefore provides a framework that incorporates a concern for the empirical analyses of a test as well as a concern for the social dimensions of language testing. Test developers are challenged to consider important questions related to every aspect of the test, leading to the design of fair, accessible tests that are designed by test developers who are willing to be accountable for their designs.