<>Research Report Funded by PPTA. Completed in April 2103 and Launched 9 July, 2013. A short summary is at
http://www.eastonbh.ac.nz/2013/07/how-good-is-our-schooling/
This column was rejected by The Listener. It was published in Pundit on 8 September, 2014.
Keywords: Education; Maori
Executive Summary (Conclusion)
The average PISA scores on the three dimensions of reading, mathematics and science literacy of New Zealand fifteen year-olds are high among the OECD countries. There are differences by ethnicity and class (and to a lesser extent gender), which are explored in this report.
Ethnicity
Students were asked their ethnicity. It is a self-categorisation and is not an objective measure. It may even have a different meaning for boys and girls; it is possible that an individual’s ethnic choice is influenced by educational achievement. Even so the following conclusions may be reached – with caution.
The mean achievement scores for those who classify themselves as sole Pakeha are well above the OECD average, a situation which is often described as a ‘world class education’.
The same is true for those who classify themselves as sole Asian, although their scores are a little lower than the Pakeha ones. They too are in receipt of a ‘world class education’.
The heterogeneous Other group of those who classify themselves as sole ‘Other’ or of multiple ethnicities (other than Maori and Pakeha) score about the same as the OECD average.
The group of those who describe themselves as either sole Maori or Maori and Pakeha score lower than the OECD average. When their scores are adjusted for socioeconomic status they are very near the OECD average. They may be said to be in receipt of an ‘OECD average education’.
(Those who describe themselves as sole Maori are somewhat below the OECD, even after adjustment for SES, but they are offset by those who describe themselves as Maori and Pakeha who achieve more than the OECD average. It is possible that individual students may choose their ethnic classification in part – directly or indirectly – on the basis of their educational achievement. This is why the two groups have been combined.)
Those who describe themselves as Pasifika score markedly lower than the OECD average, even after adjustment for socioeconomic status.
Gender
There are large differences in scores in reading achievement by gender, with girls having a markedly higher achievement than boys. This is not peculiar to New Zealand but reflects OECD outcomes generally. As in the rest of the OECD, boys score a little higher in mathematics while the science literacy scores are about the same. (On the whole, the scores by the individual ethnic groups reflect these generalisations.)
Socioeconomic Status
Socioeconomic status seems to affect educational achievement, even after controlling for ethnicity and gender – students with higher SES tend to achieve better than those with low SES. Those in the top SES decile typically average over two years more in attainment than those in the bottom SES decile.
Conversely some of the difference among ethnic performance can be explained by the OECD SES variable. There are good reasons to believe that an SES variable (or variables) designed specifically for New Zealand would explain an even greater proportion.
Effectiveness of the New Zealand Education System
The OECD also finds not only that the New Zealand students perform well on average, but that their annual gain is higher than the OECD, suggesting that for those in the mid-teens the New Zealand (formal and informal) educational system is more successful on these achievement measures than that of the typical OECD country.
The superior effect of the New Zealand (formal and informal) education system is that New Zealanders are about a year ahead on the achievement measures compared to the OECD average.
Summary
New Zealand students up to the age of 15 experience a world class education system on these achievement measures. This applies especially to Pakeha and Asian students (on average).
Some minority ethnicities – including Maori and Maori-Pakeha and the heterogeneous Other groups – do not achieve as well; their level is comparable to the OECD average, including Britain and the United States, when socioeconomic differences are allowed for.
Only the Pasifika ethnic group scores markedly worse than the OECD average.
Introduction
Every three years the OECD Program for International Student Assessment, or PISA, assesses the educational achievement of 15 year old students, in a number of nations (including the entire OECD). The data base includes measures of student achievement in education, mathematics and science literacy, together with numerous measures of the student’s home and their parental social characteristics.
While the OECD does not collect ethnicity data, the New Zealand survey does. This report is a preliminary exploration of the relationship between ethnicity and educational attainment together with the interactions with gender and socioeconomic status.[1]
Defining Ethnicity
The students were asked to identify their ethnicity/ethnicities. The precise question is recorded in the appendix. Their responses are consolidated into Pakeha, Maori, Pasifika, Asian and Other.
A student could choose more than one category. This presents a problem of analysis. Rather than go through the complexity of arguments and approaches, this study simply reports that it uses the following categories:
– Pakeha (sole);
– Maori (sole);
– Maori and Pakeha;
– Pasifika (all);
– Asian (sole);
– Other (including multiple ethnicities but excluding Maori-Pakeha and all Pasifika).
This approach avoids double counting, while maintaining an adequate size of each grouping to give some statistical confidence in its use. (The ‘Other’ is a small heterogeneous category, included for completeness.)
(For reasons explained below, the report sometimes combines sole Maori and Maori-Pakeha into a ‘Both’ group. A similar approach, for similar reasons, could have been done for Pasifika, but because the numbers were smaller, a single all Pasifika category was used.)
Table 1 summarises the ethnicities of the students.
Table 1. Reported Ethnicities
Categories Used in Analysis | All Responses | |||
Number | Share of Total | Number | Share of Category in All | |
Pakeha |
2725* |
58.7% |
3283 |
83% |
Maori |
369* |
7.9% |
833 |
44% |
Maori-Pakeha |
334** |
7.2% |
** |
|
Pasifika |
465*** |
10.0% |
465 |
56% |
Asian |
528* |
11.3% |
647 |
82% |
Other |
222**** |
4.8% |
107 |
|
TOTAL |
4643 |
100.0% |
5334 |
* Only one ethnicity chosen.
** Chose Maori and Pakeha only (40% who chose Maori said they were also Pakeha but no other ethnicity)
*** All who chose Pasifika including where there was other choices
**** Residual (includes Maori and Pakeha with Asian or Other).
Source: Ministry of Education PISA data base.
Ethnicity is not an objective categorisation like descent. Not only may an individual choose their ethnicity based on their social context, but they may vary their choice for different circumstances. An even greater complexity arises for the purposes of this study if one of the social variables which influence the student’s ethnic choice is educational achievement. It seems possible that a student is more likely to choose one ethnic classification if they are of high educational achievement, another if theirs is low. (Data presented below may be interpreted this way.) Where this occurs one has to be cautious about making inferences about ethnicity and educational achievement because a reverse or two-way causality may be occurring.
An insight into the complexity is evident in Table 2 which shows the gender breakdown.
Table 2: Gender by Ethnicity
Boys as % of ethnic group | |
Pakeha |
50.8 |
Maori |
57.2 |
Maori-Pakeha |
48.8 |
Pasifika |
52.7 |
Asian |
53.4 |
Other |
49.5 |
TOTAL |
51.6 |
Source: Ministry of Education PISA data base.
There is no reason to believe that the proportions should be markedly (and statistically) different for Pakeha, Maori and Maori-Pakeha if they were objective.[2] Yet boys are more likely to describe themselves as Maori than girls. This suggests that there is a social element in the choice of ethnicity. If it can be observed for gender it is likely to apply for other social characteristics.
Suppose educational status affects ethnicity choice; for instance those from Maori backgrounds with poor educational achievement might tend to choose sole Maori ethnicity, those with higher achievement might add another ethnicity. In order to dampen – but regrettably not to eliminate – the possibility that ethnic choice is influenced by educational achievement, much of the analysis uses a category which pools both Maori and Maori-Pakeha. The name of the category is abbreviated to ‘Both’.[3]
As much as possible, the studies treat the ethnicity-gender categories as different, avoiding automatically assuming that a boy and a girl who state they are Maori are meaning the same thing. We cannot treat ethnicity as an objective fact independent of social circumstances.
Measuring Socioeconomic Status
PISA assesses socioeconomic background with an index of social, cultural and economic status (SES), which is based on information provided by students about their parents’ education and occupations and their home possessions, such as a desk to use for studying and the number of books in the home.
On this index, one ‘unit’ is equivalent to one standard deviation across all OECD students. So across all OECD countries, about two-thirds of students are from a socioeconomic background that is between one unit above and one unit below the average.
(It seems likely that, were there the resources available, a better New Zealand index of socioeconomic status could be constructed. It might well sharpen up the SES impact on educational achievement.)
Gender
Despite also being a self-categorisation, the gender variable does not present the difficulties that ethnicity of socioeconomic status does.
Ethnicity and Educational Achievement
PISA assesses its fifteen year old students on three main dimensions, reading, mathematics and science literacy. Although there are various subcategories the focus here is on a single measure for each dimension which summarises the assessment.
The scale is open ended but the following may be a useful way of interpreting it. PISA 2009 says that ‘[f]or the 32 OECD countries in which a sizeable number of 15-year-olds in the PISA samples were enrolled in at least two different grade levels, the difference between students in two grades implies that one school year corresponds to an average of 39 score points on the PISA reading scale.’ The figure from reading is 44 score points for New Zealand.[4]
Unfortunately no equivalent figure was published for maths and science literacy in PISA 2009. But the 2003 international report gives the equivalent for mathematics of 41 score points as the OECD average and 50 score points for New Zealand.[5] The 2006 international report gives an average of 28 score points for OECD countries in science literacy and 43 score points for New Zealand.[6]
In each case the New Zealand annual increment exceeds the OECD average. This is consistent with the data about to be reported which shows New Zealand achievement scores are above the OECD average (as one might expect from accumulating annual increments).
It also might suggest that the New Zealand educational system is better at increasing achievement on the given measures since an extra year adds more. (So it is not a matter of New Zealand students starting a lot higher when they are younger and the relativity being maintained. It is being increased.) However any success is from the entire system including the informal sector of parents, the media and out-of-school activities, as well as the formal sector of schooling.
The averages for the ethnic groups are summarised in Tables 3R (for reading), 3M (for mathematics) and 3S (for science literacy). To assist interpretation, the score is bolded where it exceeds the OECD average.[7]
Table 3R: Mean Reading Scores by Ethnicity
Boys | Girls | Total* | |
Pakeha |
528 |
566 |
547 |
Maori |
440 |
474 |
457 |
Maori-Pakeha |
488 |
538 |
513 |
Both |
461 |
507 |
484 |
Pasifika |
426 |
482 |
454 |
Asian |
507 |
540 |
524 |
Other |
480 |
537 |
508 |
TOTAL NZ |
503 |
543 |
524 |
OECD |
474 |
513 |
493 |
* The total assumes equal numbers of boys and girls
Source: Ministry of Education PISA data base.
Table 3M: Mean Mathematics Scores by Ethnicity
Boys | Girls | Total* | |
Pakeha |
550 |
537 |
544 |
Maori |
462 |
450 |
455 |
Maori-Pakeha |
511 |
505 |
508 |
Both |
483 |
478 |
481 |
Pasifika |
450 |
451 |
451 |
Asian |
547 |
520 |
533 |
Other |
501 |
505 |
503 |
TOTAL NZ |
528 |
516 |
521 |
OECD |
501 |
490 |
496 |
* The total assumes equal numbers of boys and girls
Source: Ministry of Education PISA data base.
Table 3S: Mean Science Literacy Scores by Ethnicity
Boys | Girls | Total* | |
Pakeha |
563 |
560 |
562 |
Maori |
466 |
465 |
466 |
Maori-Pakeha |
514 |
529 |
522 |
Both |
487 |
498 |
493 |
Pasifika |
443 |
464 |
452 |
Asian |
538 |
527 |
532 |
Other |
503 |
523 |
513 |
TOTAL NZ |
534 |
535 |
535 |
OECD |
501 |
499 |
501 |
Source: Ministry of Education PISA data base.
* The total assumes equal numbers of boys and girls
On all dimensions Pakeha and Asians are well above the OECD averages. So are the national averages, by the equivalent of at least a year of schooling. It is these scores which are sometimes used to say that the New Zealand educational system is ‘world class’ (for those up to 15 years of age – there are not comparable measures for older students).
On the other hand the scores for sole Maori and Pasifika are somewhat below the OECD average. However the group which describes itself as both Maori and Pakeha are comfortably above the OECD average, although not as spectacularly as for sole Pakeha or sole Asian. When the sole Maori with the Maori-Pakeha groups are pooled, their averages are below the OECD averages, although much closer than in the case for sole Maori.
(The Other group has scores about or just below the OECD average. Because it is such a heterogeneous group there seems little point in discussing this result, especially as the sample size is small.)
The Socioeconomic Gradient: Educational Achievement and Socioeconomic Status
It is well established that educational achievement is related to socioeconomic status (SES). Moreover there is a correlation between SES and ethnicity. This section investigates the SES effect on the educational achievement scores.
The measure of SES used here is the standard one provided by the OECD in the international data base. It is probably not the best measure for New Zealand, but it has the merit that the results are comparable with the OECD’s international research. A New Zealand constructed index might well show a greater effect on educational achievement.
The analysis is based on ranking the students by SES dividing the surveyed population into ten categories (deciles). The deciles are numbered from 1 which has the lowest SES scores to 10, which has the highest. (Theses SES deciles are not the same as the deciles which are used to rank schools.)
Table 4 illustrates the general method showing the reading achievement scores of sole Pakeha boys and Maori and Maori-Pakeha boys by decile.[8]
Table 4: Reading Achievement by SES Deciles
Boys | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
Pakeha | 458 | 486 | 505 | 512 | 519 | 524 | 524 | 548 | 564 | 583 |
Both | 412 | 456 | 465 | 456 | 459 | 471 | 482 | 498 | 512 | 517 |
Source: Ministry of Education PISA data base; some calculations.
To simplify the interpretation of the table the results are shown visually in Figure 1.
Figure 1: Reading Achievement by SES Deciles
Source: Table 4.
Both show a distinct rising – and, as it happens, statistically significant – trend with higher SES. However the Maori and Maori-Pakeha scores are below the Pakeha ones in each decile. (Even so, those of the combined Maori groups in the top third of the socioeconomic ranking are higher than Pakeha at the bottom third of the ranking.)
This trend or gradient can be estimated econometrically. The gradients are summarised in Table 5. The measure shown is the estimate of the difference between the average score at the bottom decile and the top gradient.
Tables 5: Difference in Educational Achievement between Top and Bottom SES Decile.
Average Score |
Reading |
Mathematics |
Science Literacy |
|||
Boys |
Girls |
Boys |
Girls |
Boys |
Girls |
|
Pakeha |
105 |
102 |
92 |
112 |
100 |
97 |
Maori |
45 |
92 |
46 |
78 |
53 |
109 |
Maori-Pakeha |
117 |
106 |
109 |
143 |
124 |
123 |
Both |
87 |
112 |
84 |
132 |
95 |
128 |
Pasifika |
143 |
99 |
141 |
129 |
158 |
110 |
Asian |
110 |
96 |
112 |
135 |
113 |
103 |
Other |
139 |
126 |
134 |
132 |
140 |
151 |
1 Year* |
44 |
50 |
43 |
* OECD estimate of the effect of one school year.
Source: Ministry of Education PISA data base; some calculations.
With the exception of sole Maori (especially boys) the gradient magnitudes are much the same. So will be the score difference between the top and bottom SES decile. The implications of the low sole Maori gradients are discussed in the next section.
The gradients are large. Typically students from the highest SES families average more than two years in their achievement relative to those in the bottom SES families. It would appear that given a choice between being in a top SES family or being in a bottom SES family and having two extra years of schooling, the first option would give a higher educational achievement on average. (In practice the student in the low SES family is likely to get fewer rather than more years of schooling.)
Of course the outcome may not be only the pure effect of the socioeconomic status of the student’s family, since other effects may be subsumed in it. For instance, those from high SES families may go to better-resourced schools.
Where gradients exceed the Pakeha ones – which is generally true – there is a convergence with rising socioeconomic status. That means that the gap between these ethnicity achievement scores is proportionally greater at the low SES level than at the high level.
Maori and Maori-Pakeha
In many circumstances it does not matter for purposes such as this study that the ethnicity is not an objective characteristic but reflects social circumstances (including the context when the self-report is made). However here it would matter if one of the determinants of an individual’s choice of ethnicity was their educational achievement.
For instance suppose there was a tendency by students of Maori descent with low educational achievement to categorise themselves as ‘sole Maori’ but those with higher achievement to categorise themselves as both Maori and Pakeha. Given that educational achievement is positively correlated with socioeconomic status, we would expect to see a higher proportion of Maori-Pakeha in upper SES deciles than in lower ones, and a stronger gradient for Maori-Pakeha, than sole Maori. This is exactly what happens.
This fact does not prove the conjecture that the choice of ethnicity is affected by educational achievement, but in case further investigation found some greater credibility for the conjecture, the analysis has combined Maori and Maori-Pakeha into a single ‘Both’ category which is reported in the tabulations.[9]
The result from combining the two ethnicities is a gradient for each achievement dimension more like those for the other ethnic categories. Even the slightly lower gradients could be explained if some higher achievers of Maori descent chose only a ‘Pakeha’ categorisation.
If the conjecture were to apply for Maori, it might also apply for Pasifika (only 56% of all Pasifika describe themselves as sole Pasifika, as distinct for an equivalent statistic of 44 % of those who describe themselves as Maori). The gradients (not reported here) of the sole Pasifika were low like the sole Maori. (Numbers were small; hence the use of an ‘all Pasifika’ category.)
The purpose of this section is not to argue the conjecture is correct, but to point out the complications if a factor in the choice of ethnicity is educational achievement. Using the Both (sole Maori and Maori-Pakeha) and the (all) Pasifika categories reduces any such effect.
Educational Achievement adjusted for Socioeconomic Status
The various ethnic groups are not spread evenly through the SES ranking so that proportionally more Pakeha are clustered in the top of the ranking and proportionally more sole Maori are in the bottom (Maori-Pakeha more middling). Given the SES gradient the average scores are affected by the distribution of socioeconomic characteristics.
This effect can be eliminated by deriving an average as if the two groups had exactly the same socioeconomic structure. The outcome for groups shown in table 4 (the reading scores of boys from two ethnic groups) is shown in Table 6 where it is assumed that each has 10 percent of the population in each decile, but the achievement score in each decile is exactly the same as in the original survey. (The adjusted scores are thus the averages of the deciles shown in Table 4.)
Table 6: Unadjusted and Adjusted Reading Scores (Boys)
Boys |
Unadjusted |
Adjusted |
Difference |
Pakeha |
528 |
524 |
-4 |
Maori & Maori-Pakeha |
461 |
470 |
9 |
Source: Ministry of Education PISA data base; some calculations.
Pakeha tend to be slightly more preponderant in the higher SES groups. If they were spread evenly though the deciles (with the same average score in each decile) their reading score would average 4 points lower at 524 rather than 528. Conversely the Maori & Maori-Pakeha are more likely to be in the lower SES deciles. If they were spread evenly their reading score would be 470 instead of 461, some 9 points higher. Thus the difference between the two ethnicities of 67 points is reduced to 54 points when the results are controlled for SES. There is a sense in which some of the difference between the two groups’ score can be explained by differences in SES.
Since there is almost certainly a better measure of SES for New Zealand than the OECD one, the estimate that class explains only a quarter of differences is probably too low. Calculating the actual figure would involve more resources than are available for this project. What this report demonstrates is that ethnicity cannot be the sole explanation for differences in achievement scores between the two groups.
The adjusted and unadjusted scores for all the ethnic groups used in this report are shown in Tables 7R, 7M and 7S. Again bolding indicates that the score is above the OECD average.
Table 7R: Mean Reading Scores by Ethnicity
Boys |
Girls |
|||
Unadjusted |
Adjusted |
Unadjusted |
Adjusted |
|
Pakeha |
528 |
524 |
566 |
557 |
Maori |
440 |
450 |
474 |
493 |
Maori-Pakeha |
488 |
500 |
538 |
546 |
Both |
461 |
470 |
507 |
523 |
Pasifika |
427 |
453 |
483 |
502 |
Asian |
507 |
503 |
540 |
539 |
Other |
480 |
494 |
537 |
533 |
TOTAL NZ |
503 |
503 |
543 |
543 |
OECD |
474 |
513 |
Source: Ministry of Education PISA data base; some calculations.
Table 7M: Mean Mathematics Scores by Ethnicity
Boys |
Girls |
|||
Unadjusted |
Adjusted |
Unadjusted |
Adjusted |
|
Pakeha |
550 |
540 |
537 |
523 |
Maori |
462 |
473 |
449 |
466 |
Maori-Pakeha |
511 |
523 |
505 |
518 |
Both |
483 |
492 |
478 |
497 |
Pasifika |
450 |
476 |
450 |
474 |
Asian |
547 |
543 |
520 |
519 |
Other |
501 |
494 |
505 |
501 |
TOTAL NZ |
528 |
528 |
516 |
516 |
OECD |
501 |
490 |
Source: Ministry of Education PISA data base; some calculations.
Table 7S: Mean Science Literacy Scores by Ethnicity
Boys |
Girls |
|||
Unadjusted |
Adjusted |
Unadjusted |
Adjusted |
|
Pakeha |
563 |
559 |
560 |
555 |
Maori |
466 |
478 |
465 |
480 |
Maori-Pakeha |
514 |
528 |
528 |
539 |
Both |
487 |
497 |
498 |
518 |
Pasifika |
443 |
471 |
464 |
485 |
Asian |
538 |
534 |
527 |
526 |
Other |
503 |
537 |
523 |
518 |
TOTAL NZ |
534 |
534 |
535 |
535 |
OECD |
501 |
499 |
Source: Ministry of Education PISA data base; some calculations.
While the scores do not move much they result in one major change to the earlier conclusions. While the educational achievement scores for sole Pakeha and sole Asians were among the best in the world, it now appears that the scores for the Both group (i.e. combined sole Maori and Maori-Pakeha group) are close to the OECD average when there is an adjustment made for their tendency to have lower socioeconomic status.
The implication is that Maori may not be getting a ‘world class education’, but what they do get is typical of the OECD. Probably they are getting an education – on these measures – similar to that they would get in Britain or the United States if they were in a similar socioeconomic situation.
Table 8: Some International Comparisons
Boys & Girls |
Reading |
Mathematics |
Science Literacy |
Both (adjusted) |
497 |
495 |
507 |
Pasifika (adjusted) |
478 |
475 |
478 |
OECD Average |
493 |
496 |
501 |
Britain |
494 |
492 |
514 |
United States |
500 |
487 |
502 |
Source: Ministry of Education PISA data base; some calculations.
However the all Pasifika group remains below – but closer – to the OECD average.
Conclusion: Ethnicity and Educational Achievement
The conclusions are set out in the Executive Summary. It should be emphasised that this is a preliminary exploration, and the data base is such that a more refined statistical analysis is possible and would almost certainly be worthwhile.
APPENDIX: THE ETHNICITY QUESTION IN THE PISA SURVEY
Q4: Which ethnic group(s) do you belong?
You may tick more than one box
New Zealand Māori
New Zealand Pākehā/European
Other European
(Please say which)
Samoan
Cook Island Māori
Tongan
Other Pacific Island
(Please say which)
Chinese
Indian
Other Asian
(Please say which)
Other Group
(Please say which)
Note that in the 2006 New Zealand census of those aged 10 to 14 (the midpoint of the group would have been 15 at the time of the 2009 PISA survey), 55% were reported as New Zealand European, 19% as Maori, 9%, as Pacific Islander, 8% as ‘other’ and 8% were reported as ‘New Zealanders’ (an option not offered in the PISA survey). These figures include multiple choices. They do not match well with the PISA responses. Differences may reflect the options given for answering the question, that many of the fifteen year olds may have had their census ethnicity chosen for them by their parents when they were twelve, and that individuals change their ethnicity over time or for particular circumstances.
Acknowledgements
I am grateful to Judie Alison, Warwick Elly, Liz Gordon, John Hattie and Geraldine McDonald for comments on early drafts and for assistance from Jit Cheung and Steve May from the Ministry of Education. All the remaining errors are mine, as are any interpretations of the findings.
The research is funded by a grant from the Post Primary Teachers Association.
Endnotes
1. The ethnicity data of the students was kindly supplied by the Ministry of Education, providing that the individual responses of the students were not disclosed. The rest of the PISA data is in the international domain. The report uses the term gender rather than sex because, like ethnicity, it is a self-categorisation.
2. There may be differential gender migration for the other categories.
3. Note that in their PISA studies the Ministry of Education estimates of Maori includes all those who describe themselves as Maori, including those with multiple ethnicities.
4. Volume II (2009), p.27 and Table A1.2.
5. Volume II (2003), p.60 and Table A1.2.
6. Volume II (2006), p.55 and Table A1.2.
7. The results weight each student equally. There are sampling weightings, but they have not been used.
8. That the Maori and Maori-Pakeha data is less regular than the Pakeha data probably reflects the smaller sample.
9. The Both group is similar to the prioritisation method, in which those who categorise themselves as Maori are allocated to that group and no other, and the double-count method in which a person with multiple ethnicities is allocated to all categories. However the Both group excludes the sixth who chose at least one of their ethnicities as Maori but also mentioned another other than Pakeha – such as Pasifika or Asian.