Murray and Irit Aitkin Statistical Modeling of the National Assessment of Educational Progress Springer 2011.
This book describes our joint contract work for the US National Center for Educational Statistics, on an integrated psychometric and statistical model for the large-scale NAEP surveys of the US Department of Education. The model incorporates the clustered survey design through multiple model levels, and allows for stratification and oversampling of minority ethnic groups. We allow for guessing through a two-class latent class model with different item parameters in the classes. We are extending this work with a current Institute of Education Sciences grant to examine Bayesian model comparisons for the competing possible models for the survey data, and to obtain the full information from records with incomplete convariate information.
Statistical Inference: an Integrated Bayesian/Likelihood Approach , Chapman and Hall/CRC Press 2010
This book sets out an integrated Bayesian and likelihood approach to statistical inference, using the likelihood as the primary measure of evidence for statistical model parameters, and for the statistical models themselves.
To assess the strength of evidence from the data for competing parameter values or competing models, it uses likelihood ratios between the parameter values or the models. To interpret the likelihood ratios it uses a Bayesian approach which requires in general only the non-informative priors that are widely used in posterior inference about model parameters, though it can accommodate informative priors.
Comparison of different statistical models requires a treatment of the unknown model parameters. This problem is usually treated by Bayes factors; the book gives a different approach which uses the full posterior distribution of the likelihood for each model. This quite small change to standard Bayesian analysis allows a very general and unified approach to a wide range of apparently different inference problems.
A further contribution of the book is to develop a general Bayesian approach to finite population inference. This approach, using the multinomial distribution and a non-informative Dirichlet prior, can also be adapted to provide a general Bayesian "non-parametric" analysis.
The book is intended to provide both an exposition of an alternative to standard Bayesian inference, and the foundation for a course sequence in modern Bayesian theory at the graduate or advanced undergraduate level.
I have been working since 2003 with Irit Aitkin, in a series of research contracts with the US National Center for Education Statistics and several research grants from the Institute of Education Sciences, on the development and implementation of fully model-based approaches to the analysis of the large-scale US National Assessment of Educational Progress (NAEP) surveys of primary and high-school students' achievement. We have developed multi-level model-based approaches for the clustered and stratified survey designs, examined alternative distributions for the latent ability, and formulated alternative latent class models for guessing, or "engagement" on the test items. For each of these developments, we have compared the results of the model-based approach with the current methods of analysis using both simulated and actual NAEP data.
A detailed description of this approach, its principles and a summary of each project can be found here.