RStyles team published an article on PISA methodology in OECD Working Papers Series.
The abstract of the paper is:
The OECD’s PISA (Programme for International Student Assessment) is expanding on new educational systems from cycle to cycle. Most of the new participants differ significantly and negatively from the core participants in the level of educational proficiency. The study has investigated a potential expansion of the new participants’ proportion from 15% of low performing countries to 50% of low-performing countries in the PISA sample. A simulation study was performed that aimed to check whether an increasing share of low performing countries among PISA participants can affect: a) key country parameters (means and within-country standard deviations), b) item parameters (difficulty and discrimination), and c) sensitivity and specificity of differential item functioning procedures. The results of the study point out that the PISA procedures are fit and robust to the increasing proportion of low-performing countries resulting in highly reliable inter country score differences and estimates of country means.
More on this topic and the paper in open access on OECD webpages.