Principle,_Theories_and_Their_Practical_Application_in_Assessment

Principle, theories and their practical application in assessment Introduction The purpose of this report is to detail and justify the approach taken with an assessment strategy for a non accredited Introduction to Computers course The approach has included identifying the characteristics of the individual learners (for example, age, sex, disabilities), the requirements of Wexford City council together with reference to, and analysis of, relevant learning styles, learning theories and assessment theories. The results guided the development of an assessment strategy (including method) that best suited the group profile, their learning styles, the expected objectives (outcomes) of the course whilst adhering to the principles guiding the undertaking of assessments. What is assessment' According to Corder (2002) assessment is about checking learner progress and achievements (what learning has taken place), and applies to all courses irrespective of whether it will lead to a qualification. Given the nature of the Cadishead course the purpose of the assessment was to provide information to the learner and the teacher on progress towards achieving course learning objectives and each learner’s personal objectives. The results allowed teaching and learning strategies to be adjusted as the course progressed to enable weaknesses (or gaps in skill levels) to be worked on and strengths to be further developed; therefore, it is what authors like Gravells (2007) refer to as assessment for learning. Assessment is therefore, not a one-off but should be integral to every session (formative) and would include a variety of methods, especially verbal questioning as well as regular written or online practical exercises. Conversely for an IT qualification like the European Computer Driving Licence (ECDL) there would also be a formal end of course (or summative) examination. Relevant group and individual characteristics According to Gravells (2007) it is important to understand the needs of individuals within the group to ensure that learners are treated individually and equally. This is particularly important when developing an assessment strategy because if the main purpose is to show progress then each leaner must be given the opportunity to perform to the best of their ability. The results of the group profile informed the assessment strategy. A detailed group profile, which also contains diary notes kept by the teacher, is included at appendix 2, however, in summary:      the group consisted of 8 learners, however, because of health issues 2 dropped out, the 6 remaining learners were over 60 years old with the eldest 83, there were 5 female and 1 male. 4 learners had little or no experience of computers and only one had been on any kind of course in the last 20 years. 1 learner with some kind of literacy problem which may be dyslexia but has never been tested. During the initial assessment for the course most of the group referred to bad experiences with close family members (e.g. a spouse or sibling) when trying to learn from them about computers, being subjected to a dismissive attitude. Overall, at the start of the course there was a lack of confidence in being able to learn new skills with age and previous experiences being the main influencing factors. From the initial assessment, observing the learners in the early weeks of the course, and through interaction during class, individual learning styles of the learners and the factors that would foster an effective learning environment became evident. All members of the group reacted positively to a teaching strategy that included a combination of learning styles that Fleming (1987) categories as visual, aural, read/write and kinaesthetic (VARK), however, there were individual preferences. Most of the group struggled with a recap exercise in week 2 that consisted of just words and handouts with mostly words were either not read or not understood. Both examples may be explained by a preference to a visual style or perhaps would prefer to be doing an activity. To improve the visual aspects images were added, as 1 appropriate, which had a positive impact on learning for all the learners. It was also noticeable that one learner in the group would instinctively start an activity without listening to (or reading) the instructions properly. It would perhaps be too simplistic to describe this as a kinaesthetic learner, however, it would probably be what Dunn and Dunn (1999) would categorise as impulsive or what Honey and Mumford (2000) would describe as an activist. It meant that an appropriate teaching strategy (including assessment) needed to include the teacher using appropriate verbal instructions and observing all learners. At least two learners displayed a very thoughtful approach to activities so taking time to read, listen and ask questions before taking action. Applying perhaps applying the Honey and Mumford (2000) learning style descriptions may conclude they were reflectors or global thinkers in terms of Dunn and Dunn (1999) as they also seemed to want to see the full picture before focusing on individual aspects. The teacher recognised, however, the potential influence of confidence so what may appear to be a particular learning style may have been a reflection of a lack of confidence. Perhaps individual learning styles change over time as confidence changes which could possibly explain why learners were prepared to “have a go” as the course progressed. Overall Dunn and Dunn (1999) identified a range of factors that could influence a person’s ability to concentrate on, process, use and retain new and difficult information, some of which included environmental factors such as noise and comfort. After a few sessions with the Cadishead group it was noticeable that responses from learners were far more positive when the library closed to the general public part way through the session. The environment was quieter and calmer. This was taken into consideration when planning each session with new material largely covered after the library closed. A strand of sociological factors identified how learners prefer to learn socially including or example, alone, part of a group or as a pair. One particular learner in the group tended to rely on a fellow learner while one kept distant throughout the course The teacher recognised, however, the potential influence of confidence so what may appear to be a particular learning style (or social leaning) may have been a reflection of a lack of confidence. Perhaps individual learning styles change over time as confidence changes which could possibly explain why learners were prepared to “have a go” as the course progressed. 2 Despite the low level of confidence there was still a significant positive influence within the group with a high level of motivation to learn and progress. Knowles (1970) and Lindeman (1926) in their contribution to andragogy both identified that adults want to learn as they experience needs and interests that learning will satisfy. The motivation for learning is therefore life centred rather than about subjects for example, with ICT wanting to keep up with the children or grandchildren, keeping in touch with friends and relatives, wanting to benefit from shopping online and finding find information about hobbies on the internet. An appreciation of the high level of motivation within the group was important in developing the assessment strategy, especially in the communication so individuals accepting the need to be aware of their progress in achieving learning objectives. The profile of the group made it even more important that reinforcement and recap of content occurred several times during a session otherwise there was little evidence of the practical application or understanding of the material being covered. It is perhaps what Briggs (1999) would refer to as surface as opposed to deep learning. The teaching strategy used regular recap together with the use of analogies that the learner would be able to relate new with familiar concepts (for example, firewall computer security being the equivalent of a garden fence) to encourage deeper learning. According to Skinner (1931) behaviour that is reinforced will tend to continue (known as Operant Conditioning). The re-inforcers could either be negative (i.e. punishing bad behaviour or positive. The use of negative reinforcement with the Cadishead group undoubtedly would have resulted in low learner retention, however, positive encouragement through regular praise is something, given the low level of confidence at the beginning of the course, was a fundamental element of the communication strategy adopted with the group. Appropriate motivation theories were also analysed in order to understand what conditions and factors would influence learner achievement. Maslow (1954) identified achievement (self actualization) as the ultimate goal but argued that the learner must already have had lower level needs satisfied (see figure 1). Failure to meet those lower level needs would therefore likely to lead to underachievement, however, a positive learning experience and achievement is more likely to result when certain basic personal needs have been met, 3 Appendix 3 contains a full analysis of the relevance of each level of need to the Cadishead group and as a result the factors that were considered when developing relevant learning and assessment strategies, however, the main elements included:      Comfort, food and the learning environment Health and Safety requirements and the correct use of the equipment Treating learners equally and fairly Encouraging learners to support each other Identifying achievements and giving genuine praise Figure 1 – Maslow’s Hierarchy of Needs Pyramid The Assessment Strategy Assessment Principles Although the course was not accredited (did not involve the formal assessment of the learner progress for the purpose of awarding a qualification) the relevant principles and theories of assessment were applied. Wallace (2004) identified that to be appropriate the assessment needs to be reliable, valid and sufficient. An assessment would be:  reliable if different teachers undertaking the same assessment would arrive at the same outcome, 4   sufficient if the results accurately reflects the actual competency level of the learner. valid if it accurately measures what it sets out to assess which in turn is likely to be determined by the method chosen. The assessment strategy was therefore designed to ensure adherence to the key criteria and this included ensuring that it met the needs of the organisation. The purpose of the course is for Wexford City Council is to encourage individuals to access services online rather than by telephone or face to face. The learning outcomes therefore centred on the key building blocks for example:    confident about being safe online, able to find information online including within the Wexford City council web site able to use email competently. The assessment therefore incorporated all those building blocks aspects. It did help that there is less scope for ambiguity with ICT compared with other subject areas which include an element of opinion or subjectivity, so it is easier to assess whether a task has been achieved. In this situation the course was very similar to many delivered throughout Wexford (in libraries and other community venues) with similar learning objectives, therefore, the same methods of assessment are applied in other locations and delivered by different teachers. The content and criteria of the assessment reflected the main learning objectives, for example, the ability to save a document, send an email with an attachment and find information through using a search engine or directly within a web site. Since the assessment of the learners was undertaken formatively i.e. throughout the duration of the course on a weekly basis, the teacher did not been rely on the results of one particular activity to make judgements on the skill levels being achieved. The assessment strategy followed was, therefore, likely to produce results that are reliable, sufficient and valid. The assessment method and delivery When developing the appropriate assessment method(s) consideration was given to the individual and group characteristics to ensure it was inclusive and appropriate for all the learners which included ensuring that the degree of difficulty challenged the more able learners. Communication (written and verbal) was key to the successful 5 development and implementation. Verbally to explain, clarify, reassure and praise while written with the content of the assessment. The teacher needed to ensure that the degree of difficulty of assessment reflected the capabilities of the learners and the needs of the course in the application of knowledge and skills. In terms of Bloom’s Taxonomy model (1956) this would equate to activity relating to the upper part of the pyramid (see figure 2 below), with learners required to do more than remember and understand but to apply the concepts covered and identify reasons when something doesn’t work. The assessment practical course so the application of learner that level of applied to a ultimately it is skills by the determined the achievement Figure 2 – Bloom’s Taxonomy After considering the issues arising from the individual and group characteristics it was decided to include, where possible, questions which allowed the learner to pick the answers from a jumbled list identified alphabetically. The assessment of the higher level skills (internet searching, saving images and email attachments) was undertaken through the practical application of those skills for the assessment method used). Evidencing the completion of all or parts of the task was particularly important, especially for saving an image and attachment. Full completion of the more complex activity required the learner to go through a series of tasks i.e. find an image, create a folder and save the image in the right folder on the pen drive. The marking system needed to reflect that sequence and complexity so a reference to the mark would easily indicate which aspect of the task (if any) couldn’t be completed. 6 The assessment was embedded in to the session as a recap of activities covered in previous sessions. The assessment was delivered in an informal way reflecting the type of course and the characteristics of the learners attending. A verbal explanation of requirements was given at regular intervals, especially for the harder tasks. When it became apparent that saving an image was causing problems to all but one of the learners the teacher did a recap before continuing with the assessment to prevent a loss in confidence. If this had been a formal accredited course like ECDL then the process would have included:     Learner registration Clearly identified and communicated assessment criteria Mock assessment with feedback to identify when the learner is ready for the formal examination Supervised under strict examination conditions (and access to reference materials (for example, course notes), no conferring between learners and a recorded fixed time for completion.   Consultation with the examining body relating to adjustments needed for learner support. Marking would be undertaken external to the organisation. Summary of results and feedback Teacher observation during the assessment was important in giving an indication of how individual learners coped with the exercise. An answer sheet was given out and each learner marked their own sheet for sections 1 to 3. The marking for saving an image and email attachment was undertaken by the teacher based on evidence including emails received from each learner. In this assessment since it was more important to identify issues for further recap it was the marks achieved within each section that was important rather the overall total marks. Feedback was undertaken as a group due to the nature of the course, because much of the feedback was common to all the learners and due to advanced development of the group. Tuckman (1965) identified four stages of team or group development ranging from forming to performing. The group were obviously at the performing stage evidenced by the level of interaction between group members and the support given by more able learners to the others. 7 . The feedback to learners was undertaken in an informal, non threatening way with learners prompted to form their own conclusions on areas requiring further recap. Taking into consideration Skinner’s positive re-inforcers, praise and constructive criticism was central to the feedback process while continually emphasising that areas where learners struggled would be the recap activity for future sessions. The marks became irrelevant as the group collectively knew which areas needed further recap. It was pointed out by the teacher what they had already achieved considering their starting point. Throughout the process there was compliance with the Data Protection Act (1998) in the keeping of learner records and confidentiality was maintained as appropriate. See appendix 5D for a summary of results. If this had been an accredited course like ECDL then the marking would have been undertaken external to the organisation and individual feedback, especially if the learner did not pass. Quality Assurance (QA) and Assessment Evaluation (see appendix 7 for the Evaluation Plan) According to Gravells (2007) all programmes should have a QA system to ensure that they are being delivered and assessed consistently and accurately. For the course covered by this report the progress of the various groups is discussed internally at team meetings were content and methods of consolidation can be analysed and improved. For a formal accredited course the process would have included formal verification consisting of internal and external resources to judge whether the key assessment criteria had been met, for example, other teachers involved in a sample of marking although for ECDL all marking is done externally. Learners would also have been informed of, and given details of, the appeals process should there be any dispute. In addition Gravells (2007) identifies feedback from learners and teacher self evaluation as important elements of the QA process. The feedback obtained from learners should allow for improved teacher performance, including the support given to learners together with helping to ensure the assessment processes used are effective, valid and reliable. Corder (2002) identifies that feedback obtained from learners is vital in enabling teacher self evaluation, therefore, for the group learner feedback (formal and informal) helped the teacher complete a self evaluation. Initially feedback was obtained informally through a group discussion which enabled 8 issues to be raised and any clarification provided. It helped that the teacher had already established a good rapport with the learners and that the group was already established. Therefore the issues raised during the discussion were mainly reflected in the formal feedback. Formal learner feedback was obtained through a questionnaire which was designed to be as simple as possible while covering the key issues. The learner feedback was very positive, not just regarding the assessment but also about the course generally with very few negatives identified. The main points relating to learner evaluations teacher evaluations include:  Included key skill areas.  Contained a variety of methods including a mixture of picking answers from a list and practical tasks e.g. saving an image, sending an attachment.  Easy to judge whether answers whether tasks completed successfully and with the practical elements teacher was able to assess the degree of completion.  This is a team resource so will be used at other locations.  Observation an important tool used in judging how learners are progressing.  Allowed (and required) the teacher to use minimum core skills Weaknesses: Areas for development It didn’t cover all learning objectives just the main ones Include regular similar assessment in other sessions and cover all the learning objectives Strengths The strong interpersonal skills of the teacher (including the plain written language used in the design and the verbal explanation given to learners) were the main strengths in enabling the assessment tool to be implemented effectively. The interpersonal skills and the strategy adopted enabled the assessment to be inclusive to all learners with the regular use of verbal questioning to test learner understanding. The feedback from learners shows that all felt included. The main development need for this assessment is in ensuring a similar approach is repeated 9 more frequently throughout the length of a course. However, a further self development need would be to manage formal assessment situations for accredited qualifications as the teacher recognises that experience has so far been in informal assessment situations. Conclusion The analysis has demonstrated to the teacher that assessment is an integral element of the teaching strategy and that a successful assessment strategy depends on key factors including:    Following established and agreed procedures from the course provider and awarding body for accredited courses) Teacher knowledge of the individual learners and the group Communication skills of the teacher in delivering the assessment. However, possibly the most crucial element is the openness and ability of the teacher to evaluate (including self) an assessment allowing successes to be built on. It is equally important to identify weaknesses and when it does not so well so that improvements can be identified, implemented and therefore avoid repeating those weaknesses . References and bibliography Biggs, J. (1999) Teaching for Quality Learning at University: What the Student Does. SHRE and Open University Press. Business Balls (2010. Tuckman forming, storming, norming and performing. [Online] Available at: http://www.businessballs.com/tuckmanformingstormingnormingperforming.htm Continuing Education (2009) The Theories of Learning in Educational Psychology. [online] Available at: ..http://www.lifecirclesinc.com/Learningtheories/behaviorism/Skinner.html Corder, N (2002) Learning to Teach Adults. Guildford: Biddles Ltd. Ecclesstone, R (2010) Transforming Formative Assessment in Lifelong Learning. Maidenhead: Open University Press. Gravells, A (2009) Principles and Practice of Assessment in the Lifelong Learning Sector. Exeter: Learning Matters Ltd. 10 Howard-Jones, (2008) Neuroscience and Education: Issues and Opportunities. London IoE/TLRP. [Online] Available at: http://www.tlrp.org/pub/documents/Neuroscience%20Commentary%20FINAL.pdf Brainbox (2011) Learning Styles – Honey and Mumford [online] Available at: http://www.brainboxx.co.uk/A2_LEARNSTYLES/pages/LSall4.htm Knowles, MS. Holton III, E. Swanson, R(2005) The Adult Learner. London: Butterworth-Heinemann. Learning and Skills Improvement (2010) Excellence Gateway Assessment [online] Available at: http://archive.excellencegateway.org.uk/page.aspx'o=114635 Learning Styles (2010) The Official Site of Dunn and Dunn Learning Styles [online] Available at: http://www.learningstyles.net/ilsn-a-research/documentdownloads'lang=en Skinner, B.F. (1971). Beyond Freedom and Dignity. New York: Knopf. Wallace, S (2004) Teaching & Supporting Learning in Further Education. Exeter: Learning Matters Ltd. 11