TECHNICAL ANNEX - PHASE 2

Introduction

This Annex describes the methods that have been used to construct value added indices for the institutions who volunteered to participate in the Post-16 value added pilot.

Student coverage

The value added measure is calculated for those students who were:

• aged 17 on 31 August 1999;
• on roll at 20 January 2000 (for schools) or at the start of the Spring term 2000 (for colleges);
• entered for 2 or more GCE A levels or the AS/Advanced GNVQ equivalent in the 1998/1999 and 1999/2000 academic years (providing at least one of those entries was in 1999/2000); and
• entered for at least 1 GCSE/GNVQ by the end of their compulsory education.

Allocation of points

The tables below show how points are allocated to students' GCSE/GNVQ and GCE A/AS and Advanced GNVQ results.

GCSE results

Grade	GCSE points
A*	8
A	7
B	6
C	5
D	4
E	3
F	2
G	1

For GCSE (Short Course) grades, the points number of points is divided by 2.

GNVQ results

GCE A/AS level results

Advanced GNVQ results

GNVQs and the equivalent GCSE attempts

	Number of GCSE attempts
GNVQ Full
Intermediate	4
Foundation	4
GNVQ Part One
Intermediate	2
Foundation	2
GNVQ Language Unit
Intermediate	0.5
Foundation	0.5

5. Students who were registered on an Advanced GNVQ course with an Awarding Body between 1 September and 31 December 1998 (age 16 at 31 August 1998) and were:

• on roll at 20 January 2000 (for schools) or at the start of the Spring term 2000 (for colleges); and
• had at least one unit certified in the 1999/2000 academic year; but
• did not gain the qualification in the 1999/2000 academic year

The input measure

The input measure for each 17 year old student is their average GCSE/GNVQ point score. A student's average GCSE/GNVQ point score is calculated by dividing their total GCSE/GNVQ point score by the number of GCSE/GNVQ attempts using the tables given previously. For example,

• for a student achieving 5 grade As, 2 grade Cs and an Intermediate GNVQ Part One (Merit), the total GCSE/GNVQ point score will be
  (5x7) + (2x5) + (1x12) = 57 points.
• the total GCSE/GNVQ point score is then divided by the number of attempts. For the student above the average GCSE/GNVQ point score would be 57 / 9 = 6.3 points

The output measure

The output measure for each 17 year old student is calculated as the total A/AS and Advanced GNVQ point score attained using the scores given above. For example,

• for a student achieving 2 grade B's at A level and a grade C at AS level, the total point score will be (2x8) +3 =19 points. For a student achieving a grade B at A level and an Advanced GNVQ (Merit), the total point score will be 8 + 12=20 points.

Calculation of a student's value added measure

A student's value added measure is calculated as the difference, positive (+) or negative (-), between their total GCE A/AS level and Advanced GNVQ point score and the median total GCE A/AS level and Advanced GNVQ point score for all students nationally with the same average GCSE/GNVQ point score. The median is the middle value - with half of the students having a GCE A/AS level and Advanced GNVQ point score at or below the median, and half at or above the median.

The following graph gives the pattern of median GCE A/AS level and Advanced GNVQ total point scores across the range of average GCSE/GNVQ point scores for all students nationally- joining the medians gives the 'national median line'. As the graph shows, students with an average GCSE/GNVQ point score of less than 4 have been grouped together at the lowest extreme. This has been done to ensure there are adequate numbers of students from which to calculate a robust median as a result of the small numbers of students and the large variation in their performance.

Calculation and presentation of an Institution's value added

In the example below, Institution A has 11 students eligible for inclusion in the Value Added pilot with average GCSE/GNVQ point scores and total GCE A/AS level and Advanced GNVQ point scores shown below. The median total GCE A/AS level and Advanced GNVQ point scores have been allocated according to the national median line shown above and a value added score calculated for each student.

An institution's value added score is a simple average (arithmetic mean) of the value added measures for all the students in the institution. Therefore, the value added score for institution A is -6 (sum of all the students' value added scores) divided by 11 (number of students in the institution) = -6/11 = -0.545

The value added score for each institution has been published in terms of an index based around 100. Approximately half of the institutions taking part in the pilot will have an index number at or above 100, and half at or below 100. The value added index is calculated as 100+average value added score. In the example above the value added index for Institution A is 100 + (- 0.545) = 99.455. For publication, this is rounded to 1 decimal place and shown as 99.5, using normal rounding conventions.

Interpretation of an Institution's value added index

The graph below shows the national distribution of all institutions' value added indices, not just those included in the pilot. It indicates the range of value added indices for all schools and colleges in England calculated according to the methods in the above paragraphs, but based on performance data unchecked with institutions.

The pilot institutions were chosen to be nationally representative of all schools and colleges with respect to student coverage of the pilot. The value added index of all students in the pilot institutions (based on data checked with them) was 100.9 - this is virtually identical to the value added index of students in all institutions.

The degree of significance that can be attached to any particular institution's value added index depends fundamentally on the number of students included in the value added estimate and the variation in its students' value added relative to such variation nationally. This is a calculation unique for each institution. In general, the fewer the number of students and/or the wider the variation in progress by students within that institution, the less confidence can be placed on the VA index being statistically different from average, or from that of other institutions.

The graph shows that, nationally, half of all institutions' value added indices fall between 98.0 and 101.6. As a rough guide, institutions with a value added index in this range and more than 30 students in the scope of the pilot, should not generally be thought of as having a level of aggregate performance which is different from the average institution. If the difference between the value added index of any 2 institutions is about 1.5 points, or less, then the difference should also, generally, not be considered to indicate a significant difference in aggregate performance.

For example, an institution with a value added index of 102 means that, on average, every student in that institution gained the equivalent of 2 A level points more than the national median student with the same prior attainment.

The bars on the graph represent groups of value added indices. Each bar is one value added point in width, for example, the bar at the 100 value added index includes all the value added indices between 100 and 100.9.