TECHNICAL ANNEX - PHASE 1

Introduction

This Annex describes the methods that have been used to construct the value added indices for the institutions which volunteered to participate in the post-16 value added pilot.

Student coverage

The value added 'score' is calculated for those students who were:

• aged 17 on 31 August 1999;
• on roll at 20 January 2000 (for schools) or at the start of the Spring term 2000 (for colleges);
• entered for 2 or more GCE A levels or the AS/Advanced GNVQ equivalent in the 1998/1999 and 1999/2000 academic years (providing at least one of those entries was in 1999/2000); and
• entered for at least 1 GCSE/GNVQ by the end of their compulsory education.

Allocation of points to qualification grades

The 'input' and 'output' measures for the value added calculation are based on the points awarded to students' qualification grades. The following tables show how points are allocated to students' GCSE/GNVQ and GCE A/AS and Advanced GNVQ results.

GCSE results

GNVQ results

GCE A level and AS results

Advanced GNVQ results

Students who obtain a GCE A level and an AS in the same subject over the academic years 1998/1999 and 1999/2000 will have only the A level result included in the value added calculation, unless the student does not achieve a pass grade in the A level in which case the AS grade will be counted.

Students' Advanced GNVQ results are recorded at the end of two years of study. Those who continue to a third year of study are counted, but score no points. For these tables, the Department collected from the Awarding Bodies the results (Pass, Merit, Distinction or 'no award') of those 17 year olds who:

• were registered on an Advanced GNVQ course with an Awarding Body between 1 September and 31 December 1998; and
• had at least one unit certified in the 1999/2000 academic year.

The pilot schools and colleges were given the opportunity to remove any students who were not on roll in January 2000 or withdrew before the end of two years of study.

The input measure

The input measure for each 17 year old student is calculated as the total GCSE/GNVQ point score attained using the scores given previously. For example,

• for a student achieving 5 grade As, 2 grade Cs and an Intermediate GNVQ Part One (Merit), the total GCSE/GNVQ point score will be
  (5x7) + (2x5) + (1x12) = 57 points.

The output measure

The output measure for each 17 year old student is calculated as the total GCE A/AS and Advanced GNVQ point score attained using the scores given previously. For example,

• for a student achieving 2 grade B's at A level and a grade C at AS, the total point score will be (2x8) +3 =19 points. For a student achieving a grade B at A level and an Advanced GNVQ (Merit), the total point score will be 8 + 12=20 points.

Calculation of a student's value added 'score'

A student's value added 'score' is calculated as the difference, positive (+) or negative (-), between their total GCE A/AS and Advanced GNVQ point score and the median total GCE A/AS and Advanced GNVQ point score for all students nationally with the same[1] total GCSE/GNVQ point score. The median is the middle value - with half of the students having a GCE A/AS and Advanced GNVQ point score at or below the median, and half at or above the median.

The graph below gives the pattern of median GCE A/AS and Advanced GNVQ total point scores across the range of total GCSE/GNVQ point scores for all students nationally- joining the medians gives the 'national median line'. The 25th and 75th percentile lines provided on the graph illustrate the boundaries between which half of all students nationally fall. As the graph shows, students with a total GCSE/GNVQ point score of less than 40 or 80 and above have been grouped together at the highest and lowest extremes. This has been done to ensure there are adequate numbers of students from which to calculate a robust median as a result of the small numbers of students and the large variation in their performance.

Calculation and presentation of an Institution's value added

In the following example, Institution A has 11 students eligible for inclusion in the Value Added pilot with total GCSE/GNVQ and total GCE A/AS and Advanced GNVQ point scores shown below. The median total GCE A/AS and Advanced GNVQ point scores have been allocated according to the national median line shown above and a value added score calculated for each student.

An institution's value added score is a simple average (arithmetic mean) of the value added measures for all the students in the institution. Therefore, the value added score for institution A is -7 (sum of all the students' value added scores) divided by 11 (number of students in the institution) = -7/11 = -0.636.

The value added score for each institution has been published in terms of an index based around 100. Approximately half of the institutions taking part in the pilot will have an index number at or above 100, and half at or below 100. The value added index is calculated as 100+average value added score. In the example above the value added index for Institution A is 100 + (- 0.6) = 99.364. For publication, this is rounded to 1 decimal place and shown 99.4, using normal rounding conventions.

Interpretation of an Institution's value added index

The following graph shows the national distribution of all institutions' value added indices, not just those included in the pilot. It indicates the range of value added indices for all schools and colleges in England calculated according to the methods in the above paragraphs, but based on performance data unchecked with institutions.

The pilot institutions were chosen to be nationally representative of all schools and colleges with respect to student coverage of the pilot. The value added index of all students in the pilot institutions (based on data checked with them) was 100.5 - this is virtually identical to the value added index of students in all institutions.

The degree of significance that can be attached to any particular institution's value added index depends fundamentally on the number of students included in the value added estimate and the variation in its students' value added relative to such variation nationally. This is a calculation unique for each institution. In general, the fewer the number of students and/or the wider the variation in progress by students within that institution, the less confidence can be placed on the VA index being statistically different from average, or from that of other institutions.

The graph shows that, nationally, half of all institutions' value added indices fall between 97.6 and 102.3. As a rough guide, institutions with a value added index in this range and more than 30 students in the scope of the pilot, should not generally be considered to be different from the average institution.

When comparing the value added indices of two institutions, a difference of about 1.5 points, or less should not be considered to indicate a significant difference in aggregate performance.

In simple terms, an institution with a value added index of 102 means that, on average, every student in that institution gained the equivalent of two A level points (or one A level grade) more than the national median student with the same prior attainment.

The bars on the graph represent groups of value added indices. Each bar is one value added point in width, for example, the bar at the 100 value added index includes all the value added indices between 100 and 100.9.

1. Students who have gained Short Course GCSEs will have a total GCSE/GNVQ point score that includes half a point. Such students have been included with students who achieved the total whole number point score immediately below this half point. For example, a student with a total GCSE/GNVQ point score of 56.5 will have been included with students that have a total GCSE/GNVQ point score of 56.