Item Level Data Item Level Data What is it? • A detailed breakdown of performance on each ‘item’ (usually questions or part questions within a script). • Item Level Data is available for most subjects (where marks are submitted via EMI or there is electronic script marking ESM). Item Level Data What is it for? For centres, it can be used to: • Evaluate their own performance against specification norms. • Identify strengths and weaknesses for a candidate cohort and suggest areas of improvement. • Assist individual candidates in improving their own performance. Item Level Data What is it NOT? • Information on grading and boundaries • A reflection of standards for a centre Access to Item Level Data First you will need to navigate to the WJEC secure website (www.wjecservices.co.uk or from a link on the public website), and then login. 12345 ······· Your examination centre number See your examinations officer for a password. The password is for staff use only and should not be shared with anyone. Access to Item Level Data A window similar to that below will open (some of the navigation buttons shown here on the left may not be available to you). Mouse click the ‘Walled Garden’ button (top left). Item Level Data A new window will open. There are two ‘buttons’ that give access to item level data: Choose a series from the ‘dropdown’ menu. Item Level Summary (Candidates) Centre Performance in Context Item Level Data A new window will open. There are two ‘buttons’ that give access to item level data: Choose a series from the ‘dropdown’ menu. Item Level Summary (Candidates) Choose the Item Level Summary (Candidates) to analyse data at the CANDIDATE LEVEL Item Level Summary (Candidates) For each unit a candidate has sat, the mark achieved on each item is given. Item Level Summary (Candidates) You can select a subject from the drop-down menu. Item Level Summary (Candidates) Click the candidate’s name to compare his/her performance with other candidates in the centre and overall as a report. Item Level Summary (Candidates) The report shows: • Mark achieved by candidate • Average mark achieved by centre • Average mark achieved for all candidates entered with WJEC (Click the bullets above to highlight the sections) Item Level Summary (Candidates) Click the ‘DOWNLOAD DATA’ icon for a complete set of candidate data that can be viewed and manipulated as a spread sheet. Centre Performance in Context To see how your centre performed, relative to other centres with WJEC, the Centre Performance in Context should be selected: Centre Performance in Context Centre Performance in Context This gives (for each item) the mean mean, facility factor and attempt % %, for all candidates as well as males and females separately. Centre Performance in Context - MEAN The mean mark (often called the average) is a measure of the overall performance by the candidates for a particular item. A simple comparison of the mean marks across all items will identify those items that have been well answered. Number of students mean 20 15 10 5 F 0 D C Test scores B A Centre Performance in Context - MEAN NOTE: The maximum mark may not be the same for each item and so a direct comparison of the mean score across items is not meaningful. For questions with different maximum marks, the facility factor should be used to compare performance. Number of students mean 20 15 10 5 F 0 D C Test scores B A Centre Performance in Context – Facility Factor The facility factor (FF) for an item describes the mean mark as a percentage of the maximum mark and is a measure of the accessibility of the item. Facility factor (FF) = mean mark × 100 max. mark For example, if the mean mark for a question with a maximum mark of ten was six marks, the facility factor would be 60 per cent. (6 divided by 10 multiplied by 100) Centre Performance in Context – Facility Factor The facility factor (FF) for an item describes the mean mark as a percentage of the maximum mark and is a measure of the accessibility of the item. Facility factor (FF) = mean mark × 100 max. mark A well answered question may have a facility factor of over 80 per cent. For example, a mean mark of 13 marks out of a maximum 15 marks would be a facility factor of 87 per cent. The facility factor allows you to quickly identify the items that candidates either struggled with or excelled at. Centre Performance in Context – Attempt % This is the percentage of candidates that have attempted a specific question and has two useful applications: • To assess if candidates are ‘leaving out’ those questions they find difficult. • Where candidates have a choice of question the statistics can evidence candidate preference or the teaching policy within centres. Using the Item Level Data • Take ownership – it’s YOUR data. • Understand what your data is telling you about your centre. • Make decisions or recommendations within your centres based on the data evidence. • Identify the accessible and challenging questions for your centre and address them.