Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • apa.csl
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Sample Size and Chi-Squared Test of Fit: A comparison Between a Random Sample Approach and a Chi-Square Value Adjustment Method Using Swedish Adolescent Data
Karlstad University, Faculty of Arts and Social Sciences (starting 2013), Centre for Research on Child and Adolescent Mental Health (from 2013).ORCID iD: 0000-0001-9541-3673
2015 (English)In: Pacific Rim Objective Measurement Symposium (PROMS) 2014 Conference Proceedings: Rasch and the Future / [ed] Zhang, Quan., & Yang, Hong. (Eds.), Springer, 2015, p. 197-211Conference paper, Published paper (Refereed)
Abstract [en]

Background: Significance tests are commonly sensitive to sample size, and Chi-Squared statistics is not an exception. Nevertheless, Chi-Squared statistics are commonly used for test of fit of measurement models. Thus, for analysts working with very large (or very small) sample sizes this may require particular attention. However, several different approaches to handle a large sample size in test of fit analysis have been developed. Thus, one strategy may be to adjust the fit statistic to correspond to an equivalent sample of different size. This strategy has been implemented in the RUMM2030 software. Another strategy may be to adopt a random sample approach.

Aims: The RUMM2030 Chi-Square value adjustment facility has been available for a long time, but still there are few studies describing the empirical consequences of adjusting the sample to correspond to a smaller effective sample size in the statistical analysis of fit. Alternatively a random sample approach could be adopted in order to handle the large sample size problem. The purpose of this study was to analyze and compare these two strategies as test of fit approximations, using Swedish adolescent data. 

Sample:The analysis is based on the survey Young in Värmland which is a paper-and-pencil based survey conducted recurrently since 1988 targeting all adolescent in school year 9 residing the county of Värmland, Sweden. So far, more than 20,000 individuals have participated in the survey. In the analysis presented here, seven items based on the adolescents, experiences of the school environment were subjected to analysis, in total 21,088 individuals.

Methods: For the purposes of this study, the original sample size was adjusted to several different effective samples using the RUMM2030 adjustment function, in the test of fit analysis. In addition, 10 random samples for each sample size were drawn from the original sample, and averaged Chi-Square values calculated. The Chi-Square values obtained using the two strategies were compared.

Results: Given the original sample of 21,000, adjusting to samples 5,000 or larger, the RUMM2030 adjustment facility work as well as a random sample approach. In contrast, when adjusting to lower samples the adjustment function is less effective in approximating the Chi-Square value for an actual random sample of the relevant size. Hence, fit is exaggerated and misfit under estimated using the adjustment function, in particular is that true for fitting but not misfitting items.  

Conclusion: Even though the inferences based on p-values may be the same despite big Chi-Square value differences between the two approaches, the danger of using fit statistics mechanically cannot be enough stressed. Neither the adjustment function, nor the random sample approach is sufficient in evaluating model fit, instead several complementing methods should be used.

 

Place, publisher, year, edition, pages
Springer, 2015. p. 197-211
Keywords [en]
Polytomousrasch model, Rasch model for ordered response categories, Fit analysis, Chi-Squared statistics, Random samples, Adjusted samples
National Category
Social Sciences Probability Theory and Statistics
Research subject
Public Health Science
Identifiers
URN: urn:nbn:se:kau:diva-37414DOI: 10.1007/978-3-662-47490-7_15ISBN: 978-3-662-47490-7 (print)OAI: oai:DiVA.org:kau-37414DiVA, id: diva2:845088
Conference
PROMS 2014 (Pacific Rim Objective Measurement Symposium). 2-6 August 2014, Guangzhou, China.
Available from: 2015-08-10 Created: 2015-08-10 Last updated: 2020-12-22Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Authority records

Bergh, Daniel

Search in DiVA

By author/editor
Bergh, Daniel
By organisation
Centre for Research on Child and Adolescent Mental Health (from 2013)
Social SciencesProbability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 3381 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • apa.csl
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf