December 14, 2010
12/14/10—At this busy time of year, some may have missed the startling findings of the latest international comparison of 15-year-olds’ examination results that appeared on the front page of Tuesday’s New York Times. The results released by the OECD’s PISA (Program for International Student Achievement) showed that 15-year-olds in Shanghai, which was taking part in PISA for the first time, finished first relative to their peers in 65 countries in every subject – reading, science, and math. The US was respectively 17th, 23rd and 31st (see Table below).
This may not seem like news. We have become used to the articles after each new round of PISA results that bemoan the weaknesses of the US education system and call for school reform. But to someone like myself, who has spent the last quarter century comparing different countries’ education and training systems and how the skills they produce are related to economic performance, the results were a real shock. The shock wasn’t that Shanghai finished ahead of the US, which again was about average in all categories. Nor was it that Shanghai surpassed the perennially top-scoring nations, such as Singapore, Finland, and South Korea, though this was a bit of a surprise. The real eye-opener was the magnitude of the Shanghai advantage. On a test where a few points’ difference in average results can move a country up 10 places in the rankings, Shanghai students finished 38 points ahead of second-ranked Singapore in the math exam, the same gap that separated Macao in 12th place from the US in distant 31st. The Shanghai lead in science and reading scores was a bit narrower, though still very impressive.
These dramatic results in China’s first time in the study conjure images of Bob Beamon’s shattering of the long jump record at the Mexico City Olympics or Flo Jo’s (Florence Griffith Joyner’s) memorable Olympics double 20 years later in the 100- and 200-meter dashes where her winning margin longer than her fingernails. Yet the Shanghai students’ results were achieved without the aid of altitude or the suspicion of steroid use.
As with any achievement that seems to fall well outside the norm, they will no doubt attract a lot of skepticism and scrutiny. The Chinese chose to take part with 3 major cities–Hong Kong and Macao, along with Shanghai–since the task of drawing a random sample of 15-year-olds from a nation of 1.4 billion people seemed too daunting. But the Shanghai sample alone contained 5,100 students, the same as all of the US. Some may question whether there was cheating or coaching on the exam, which has been a problem in the past in China with high-stakes standardized tests. Others like Mark Schneider, a commissioner of the Department of Education’s research arm in the George W. Bush Administration, may contend that the Shanghai students were more motivated to do well because they were told the test was important for China’s global image. “Can you imagine the reaction if we told the students of Chicago that the PISA was an important international test and that America’s reputation depended on them performing well?” Mr. Schneider told the New York Times. I am not sure of his tone when he said this, but my hunch is that such admonitions would not make a whole lot of difference.
Perhaps the most serious challenge regarding the PISA results is whether these 5,100 students are really representative of Shanghai’s population, since omitting even a small number of low-performing students can have a big impact on the average results. For example, the children of migrant workers are often unable to enroll in public schools in the large cities because their parents do not have official residency status. Thus, as with the failure of PISA to reach early drop-outs from US inner city high-schools, it may be that the Shanghai results reflect the school population, but not the full population of 15 year-olds.
My guess is that, with the possible exception of the migrant worker issue, the results will stand up well to scrutiny. The PISA exams were administered in 2009; the international research team responsible for the study would have been well aware of how much attention these results would draw and no doubt spent much of the last year checking all aspects of the data from Shanghai in painstaking detail.
When I asked my colleague Prof. Mingwei Liu, who is one of the top experts on China labor issues what he thought of the results, his response was very telling:
“I'm not surprised at all. Chinese students learn far more academic stuff in primary and secondary education than students in most countries. In addition, Chinese students are good at getting high scores in all kinds of exams. They are good not only at memorizing, but also how to resolve test problems, answer questions, and various skills on taking exams per se (e.g. getting the right answers without even understanding the questions). The huge amount of homework and extra tutoring pay off in exams.”
Raising additional alarm bells for those concerned about US competitiveness, he noted:
“Students in Beijing and Shanghai are often not the ones who get the highest scores, because generally they don't work as hard as many in other provinces, since they have more opportunities and exams there are easier.”
He predicted that “if students in some provinces such as Jiangsu, Shangdong, Hubei, Zhejiang, and Fujian take the PISA test in the future, they will outperform Shanghai students.” He added a very important qualifier on the practical implications of these results, pointing to the difference between exam results and real-world skills. “Some students with high test scores are also capable of resolving practical problems, but others not. In China, people, especially employers, often describe these students as "high scores, low capability". Indeed, a core issue in our current skills project comparing the skill development systems of India and China is China’s efforts to transform its education system from one that excels at rote learning to one that develops a new generation of innovators.
Even with this caveat, these international comparisons are likely to intensify calls for education reform in the US. President Obama already has drawn an analogy to Sputnik in the late 1950s, when fears of losing the space race to the other superpower, the Soviet Union, spurred a focus on upgrading US science and math education. Today’s debates also echo concerns from the 1980s, when an influential report, A Nation at Risk, argued that the US was losing ground to its main economic rivals–Japan and Germany–by committing an act of “unthinking, unilateral educational disarmament.”
The US emerged stronger from each of these eras, and retained its place as the world’s dominant global economy. Can we do the same in this latest phase of global competition, where success depends more than ever on the skills and knowledge of our workforce? To do so we must summon the political will across parties to address the problem, not a very likely prospect at the moment, and then undertake significant reforms of all stages of our education and training system.
The key focus for these reforms will be our major cities, where our population and our greatest educational challenges are concentrated. In contrast, based on the PISA results, China appears to have solved the riddle of urban education, at least in its national context. This will be a vital advantage for it going forward, as Shanghai and Beijing continue to expand beyond 25 million people each, and China’s second and third-tier cities of 5-15 million people continue to grow in the decades ahead.