TECHNICAL ANNEX

Introduction

This annex describes how the value added measures and associated figures have been calculated, including information about how the point scores allocated to GCSE and GNVQ examinations have been adjusted to allow all qualifications approved for use pre-16 to be included in performance indicators.

Topics covered include:

• allocation of point scores;
• discounting;
• calculation of the input and output measures used in the value added calculations;
• calculation of the value added scores;
• presentation of the value added measures; and
• length of time in current school.

Allocation of point scores

The 'input' and 'output' measures for the value added calculations are based on the points awarded to pupils' results. The following tables show how points are allocated to pupils' KS2, KS3, GCSE and GNVQ results.

KS2 Point Scores for all subjects

KS3 Point Scores

GCSE and GNVQ Point Scores for all subjects, as used in the main tables' and KS2-GCSE/GNVQ pilot tables.

Point Scores at age 15 - used to calculate the revised indicators

To enable all qualifications approved for use pre-16 to be included in the performance indicators, the following figures have been assigned to each qualification by the QCA:

• a qualification's contribution to the relevant NQF level threshold, based on 'size' only; and
• a new point score for each of the qualification's grades based on 'size' and 'challenge' (the latter determined by NQF level and grade).

More detailed information on how the figures are being calculated can be found in the QCA report Including all approved qualifications in school and college performance indicators: Phase one report. This is available online at

http://www.qca.org.uk/14-19/developments/ index_including-all-approved.htm

Point scores and contribution to level threshold figures for all qualifications which have been included in the pilot exercise can be found in Annex A.

Discounting

With the inclusion of a broader range of qualifications in the performance tables, the discounting rules needed to be refined to avoid double counting but to ensure that a pupil's achievement is recognised. Including all approved qualifications raises several issues for discounting and complicates the current situation.

Five basic principles have been applied:

1. Discounting is an issue only within the same subject area and within the same family of qualifications, for example GCSE and GCE, Applied GCSE and VCE but not GCSE and VCE nor GCSE and Key Skills.

2. Where a higher level qualification subsumes another, the lower level qualification is not counted, for example a GCSE and GCE AS or an NVQ level 1 and NVQ level 2. This will apply even if the lower qualification is larger. For example, if a candidate has both a Full GNVQ at Foundation level and a Part One GNVQ at Intermediate level in the same subject, the Part One is counted, because it is at the higher level.

3. Where a larger qualification incorporates another (eg has modules or units in common), the smaller is not counted, for example, a GCSE Short Course and GCSE Full Course. This will apply even if the qualifications have different classification codes.

4. Each qualification has a unique classification which identifies the subject area.

5. Where more than one qualification of the same size and level is offered, credit will be given to the one with the highest point score.

These principles are summarised in the following grid, in which qualifications lower down a column are not counted if there is also a valid qualification higher in the same column that has the same classification code or has a different classification code but uses the same modules or units.

Calculation of input and output measures

Four measures must be calculated for pupils in order that the various value added scores may be calculated:

KS2 average point score

The input measure for each pupil's value added score from KS2 to age 15, from KS2 to GCSE/GNVQ and historical value added score from KS2 to KS3 is calculated as the average (arithmetic mean) point score achieved in the English, maths and science tests at KS2, using the points allocated to each level.

Example :

KS2 APS = (27+21+33)/3 = 27

If any KS2 results for a pupil are disregarded, the input measure is calculated as the average of the remaining non-disregarded subjects.

KS3 average point score

The input measure for each pupil's KS3 to age 15 and KS3 to GCSE/GNVQ value added score is calculated as the average (arithmetic mean) point score achieved in the English, maths and science tests at KS3, using the points allocated to each level and applying the same method as that used to calculate the KS2 average point score.

The KS3 average point score is also used as the output measure in the historical KS2-KS3 value added calculation.

GCSE/GNVQ capped point score

The output measure for each pupil's KS2 to GCSE/GNVQ and KS3 to GCSE/GNVQ value added score is a capped point score achieved at GCSE/GNVQ. This is calculated as the best eight GCSE or GNVQ equivalent results, using the points allocated to each grade as used in the main GCSE/GNVQ performance tables, described earlier.

In order to calculate the capped GCSE/GNVQ point score for each pupil, each qualification has been divided into records to show its equivalence to the smallest qualification, which is a GCSE Short Course and has half the weight of a full GCSE as shown below:

In effect we are thus capping at the best 16 GCSE short course equivalents.

Each record is assigned a point score, which is calculated by dividing the point score for the qualification by the number of records for the qualification. The pupil's point score is then calculated by summing the point scores for each of the 16 highest scoring records.

Capped point score at age 15

The output measure for each pupil's KS2 to age 15 and KS3 to age 15 value added score is a capped point score achieved at age 15. This is calculated as the best eight GCSE or equivalent results, using the new point scores allocated to each grade as described earlier, and including all qualifications approved for use pre-16.

In order to calculate the capped point score at age 15 for each pupil, each qualification has been assigned a GCSE equivalent size. The size is directly related to the contribution to level 1 threshold figures such that 20% of the level 1 threshold is equivalent to a size of 1, or one GCSE. The qualification size can therefore be calculated by dividing the contribution to level 1 threshold figure by 20. As entry level qualifications do not contribute to any national qualifications framework level, size indicators have been assigned to them by QCA. These are shown in the point score tables.

Figures for the more widely used qualifications at age 15 are shown in the table below.

Contribution to the level 1 threshold figures which have been used to calculate size indicators for all other qualifications can be found in the point score tables.

To calculate the capped point score, the point score allocated to each result is divided by the size indicator to give a standardized point score. This is then used to rank the qualifications in order that the highest scoring qualifications can be included in the cap.

Finally, the point scores for qualifications are summed until a total size of 8 is reached or exceeded. If an individual qualification takes the total to more than 8 then a percentage of that qualification's point score will be used, according to the proportion of the size indicator required to reach the cap of 8. For example, only half the point score for a qualification would be included if it had a size of 2 and was being added to seven full GCSEs.

Example

A pupil who achieved two Bs and three Cs at GCSE, an A* for a GCSE Short Course, a full Intermediate GNVQ with distinction and a C at AS level would have their results broken down and their capped point score at age 15 calculated in the following way:

In this example a total size (= total number of GCSE equivalents) of 8 is both reached and exceeded by the fifth qualification. Therefore, we sum the point scores of the first four qualifications and then add on a proportion of the fifth qualification. As the first four qualifications have a combined size equivalent to 7.5 GCSEs, we need to add on the equivalent of 0.5 of a GCSE from the fifth qualification to take the total to 8. The fifth qualification has a total size equivalent to 1 GCSE and so we add on half of the point score for the qualification. The pupil's point score is therefore:

29 + 220 + 105 + 46 + (46 x 0.5) = 423

The average capped point score per pupil for a school is calculated by dividing the overall number of points (capped as described above) by the number of pupils.

Pupils included in the value added indicators

The value added measures are based on the results achieved by pupils:

• who were aged 15 on 31st August 2002; and
• for whom we have been able to match prior attainment in the relevant Key Stage tests.

All pupils for whom all results are disregarded at KS2 have been excluded from the historical KS2-KS3, KS2 to GCSE/GNVQ and KS2 to age 15 value added calculations, with one exception: if a pupil is disapplied in at least one subject and achieved at least one result at KS3, GCSE/GNVQ or age 15 respectively which was not disregarded, then he/she will be included in the calculation with an input score of zero. The same principle has been applied to pupils with all KS3 results disregarded for the KS3 to GCSE/GNVQ and KS3 to age 15 value added calculations.

Calculation of value added scores

Pupils' value added scores are calculated by comparing their performance at a particular stage of education with the median performance of other pupils in the same cohort with the same or similar prior attainment at a given stage of their education. For example, a pupil's value added score from KS2 to age 15 is based on comparing their performance at age 15 with the median performance of other pupils at age 15 with the same or similar prior attainment at KS2. The median value is the middle value - with an equal number of pupils having a total capped point score for the output measure at or below the median, and at or above.

Graphs and tables showing the KS2 to age 15 and KS3 to age 15 median lines are shown below. The graphs give the pattern of median point scores at age 15 across the range of KS2 point scores and across the range of KS3 point scores nationally - joining the medians gives the 'national median line'. There are two graphs for each of KS2 to age 15 value added and KS3 to age 15 value added; one graph shows median scores for pupils in mainstream schools and other shows median scores for pupils in special schools only.

KS2 to age 15 value added median line

Mainstream Schools

Special Schools

KS3 to age 15 value added median line

Mainstream schools

Special Schools

Similar graphs and tables for the KS2 to GCSE/GNVQ, KS3 to GCSE/GNVQ and historical KS2-KS3 median lines are available in the technical annex to the KS2 to GCSE/GNVQ pilot performance tables

Calculation of the school level value added measure

A school's value added score is a simple average (arithmetic mean) of the value added scores for all the pupils in the school eligible for inclusion in the value added measure.

In the example below, school A, a mainstream school, has five pupils eligible for inclusion in the KS2 to age 15 value added measure with average KS2 and capped point scores at age 15 shown below. Median capped point scores at age 15 have been allocated according to the national median line for mainstream schools shown above, and a value added score calculated for each pupil.

Therefore, the value added score for school A is:

Sum of pupils' value added scores / No. of pupils in value added measure = -131 / 5 = -26.2

Note: All figures will be rounded to one decimal place using normal rounding conventions.

Presentation of the value added measures

The KS2 to GCSE/GNVQ, KS3 to GCSE/GNVQ and historical KS2 to KS3 value added scores for each school have been published in terms of a measure centred around 100 - i.e. by adding 100 to a school's value added score. For a school with a value added score of 0.4, the value added measure would be:

0.4 + 100 = 100.4

Figures will be rounded to one decimal place using normal rounding conventions

The KS2 to age 15 and KS3 to age 15 value added scores have been presented in three different ways:

1.  centred around 1000 - i.e. by adding 1000 to a school's value added score;

2.  centred around the national average capped point score at age 15, which for 2003 has been calculated as 281.2; and

3.  as a ratio comparing the expected to the actual average capped point score for the school:-

Total of capped point scores at age 15 of pupils included in VA measure ÷ Total of median point scores at age 15 for all pupils in VA measure.

In the example above, the three different value added measures for school A would be:

1.  -26.2 + 1000 = 973.8

2.  -26.2 + 281.2 = 255

3.  (250 + 147 + 189 + 356 + 316) / (416 + 140 + 215 + 326 + 326) = 0.88*

* rounded to two decimal places

Average number of GCSE and equivalents taken by pupils in the value added calculation

For the KS2 to GCSE/GNVQ and KS2 to age 15 value added measures, the average number of GCSE and equivalents taken by each pupil included in the value added calculation is shown. These figures have been calculated by totalling the size indicators (as explained earlier) for all examinations taken by all pupils included in the value added calculation for each school and then dividing this by the number of pupils included in the calculation. The only difference between the two figures is that the average number of GCSE and equivalents shown for the KS2 to age 15 value added measure includes all qualifications approved for use pre-16 while the figure shown for the KS2 to GCSE/GNVQ value added measure includes GCSE and GNVQ exams only.

Example

A school has seven pupils who are included in the calculation. The examinations taken by each of the pupils are as follows:

The calculation in this example for the average number of GCSE equivalents taken by pupils would be:

(7.75 + 9 + 10 + 6 + 8 + 8.5 + 4.5) / 7 = 7.68

This would be rounded to 7.7 using normal rounding conventions.

Calculation of the coverage indicators

The coverage indicators show the percentage of pupils included in the school's number of 15 year olds on roll figure who were also included in each of the value added measures for that school.

For example if a school had 30 15 year olds on roll in 2003 but only 25 were included in a value added calculation, the value added coverage indicator would be:

(25/30) x 100 = 83.333%

This would be rounded to 83% using normal rounding conventions.

The stability indicator

One additional indicator is calculated for the KS3 to GCSE/GNVQ and KS3 to age 15 value added measures. This is called a stability indicator and shows the percentage of pupils included in the value added calculation who were in the same school at the end of KS3 and at age 15. For example, if a school has 10 pupils in the value added calculation and only seven were in the school at the end of KS3 and at age 15, then the stability indicator would be:

(7/10) x 100 = 70%

Length of time in current school

The final two columns of the tables give the percentage of 15 year old pupils on roll who have been in the school for the last two years and the percentage of 15 year old pupils on roll who have been in the school for all of their secondary education.

The indicators are calculated using the pupil date of entry included in schools' Pupil Level Annual School Census (PLASC) returns. As a result it has only been possible to calculate these indicators for schools who have completed PLASC returns.

The figure is calculated by first computing, for each pupil, the number of years that the pupil has spent in the school where they are currently on roll. If the pupil start date is prior to 1st October in any given school year, then the pupil is considered to have been in the school for the entire school year. Pupils starting school on or after 1st October are not considered to have been in the school for the whole year.

The indicators are then calculated by summing the number of pupils in the school for two years or more, or five years or more respectively, then dividing this by the number of 15 year old pupils on roll.

Example

If a school has 46 pupils and 35 of these have been in the school for five years or more, the five year PLASC mobility indicator for the school will be :

(35/46) x 100 = 76.09%

This will be rounded to the nearest whole number, using normal rounding conventions and so will be published as 76%.

[ Home Page | Previous page | Next page ]