Change search
Link to record
Permanent link

Direct link
BETA
Publications (10 of 36) Show all publications
Tengberg, M., Roe, A. & Skar, G. B. (2018). Interrater reliability of constructed response items in standardized tests of reading. Nordic Studies in Education, 38(2), 118-137
Open this publication in new window or tab >>Interrater reliability of constructed response items in standardized tests of reading
2018 (English)In: Nordic Studies in Education, ISSN 1891-5914, E-ISSN 1891-5949, Vol. 38, no 2, p. 118-137Article in journal (Refereed) Published
Abstract [en]

This article reports from a study of interrater reliability of constructed response items in standardized tests of reading. Two panels of raters (lower secondary teachers and test developers) were asked to rate student responses on 11 different items taken from the Norwegian national reading test in eighth grade. Consensus estimates and measurement estimates were combined with a qualitative analysis of difficult-to-score student responses. Based on findings about rater agreement, distribution of severity, and troublesome response characteristics, the article provides knowledge about both actual and possible levels of interrater reliability and discusses the use and development of open-ended reading test items.

Place, publisher, year, edition, pages
Universitetsforlaget AS, 2018
Keywords
Assessment, Constructed response, Interrater reliability, National tests, Reading
National Category
Pedagogical Work
Research subject
Educational Work
Identifiers
urn:nbn:se:kau:diva-69005 (URN)10.18261/issn.1891-5949-2018-02-03 (DOI)2-s2.0-85048856410 (Scopus ID)
Available from: 2018-09-05 Created: 2018-09-05 Last updated: 2019-02-28Bibliographically approved
Klette, K., Sahlström, F., Blikstad-Balas, M., Luoto, J., Tanner, M., Tengberg, M., . . . Slotte, A. (2018). Justice through participation: Student engagement in Nordic classrooms. Education Inquiry, 9(1), 57-77
Open this publication in new window or tab >>Justice through participation: Student engagement in Nordic classrooms
Show others...
2018 (English)In: Education Inquiry, ISSN 2000-4508, E-ISSN 2000-4508, Vol. 9, no 1, p. 57-77Article in journal (Refereed) Published
Abstract [en]

In this article, we approach large questions regarding justice and equality in the Nordic classrooms. A substantial body of previous research emphasises the importance of student engagement in teaching and learning. Drawing on video data from Norway, Sweden and Finland, we focus on whole-class teaching, i.e. situations in which the teacher addresses the class from the front of the classroom, to investigate justice trough participation. We have approached our topic through two concerns: student participation in classroom discourse and student engagement as providing access to content. Our findings seem to pose some serious challenges for the Nordic welfare society vision of classrooms as core societal hubs for justice and equality. While whole-class teaching is one of the primary tools available for attempting to achieve justice and equality for all, this interaction format seems to contain inherent constraints that do not support equitable student engagement. Further, the way the Nordic classrooms have responded so far to the massive digitisation in their societies seems to pose serious questions rather than provide comforting answers.

Place, publisher, year, edition, pages
Taylor & Francis, 2018
National Category
Educational Sciences
Research subject
Education
Identifiers
urn:nbn:se:kau:diva-66615 (URN)10.1080/20004508.2018.1428036 (DOI)
Available from: 2018-03-08 Created: 2018-03-08 Last updated: 2018-08-31Bibliographically approved
Tengberg, M. (2018). Validation of sub-constructs in reading comprehension tests using teachers’ classification of cognitive targets. Language Assessment Quarterly, 15(2), 169-182
Open this publication in new window or tab >>Validation of sub-constructs in reading comprehension tests using teachers’ classification of cognitive targets
2018 (English)In: Language Assessment Quarterly, ISSN 1543-4303, Vol. 15, no 2, p. 169-182Article in journal (Refereed) Published
Abstract [en]

Reading comprehension is often treated as a multidimensional construct. In many reading tests, items are distributed over reading process categories to represent the subskills expected to constitute comprehension. This study explores (a) the extent to which specified subskills of reading comprehension tests are conceptually conceivable to teachers, who score and use national reading test results and (b) the extent to which teachers agree on how to locate and define item difficulty in terms of expected text comprehension. Eleven teachers of Swedish were asked to classify items from a national reading test in Sweden by process categories similar to the categories used in the PIRLS reading test. They were also asked to describe the type of comprehension necessary for solving the items. Findings of the study suggest that the reliability of item classification is limited and that teachers’ perception of item difficulty is diverse. Although the data set in the study is limited, the findings indicate, in line with recent validity theory, that the division of reading comprehension into subskills by cognitive process level will require further validity evidence and should be treated with caution. Implications for the interpretation of test scores and for test development are discussed.

Place, publisher, year, edition, pages
Routledge, 2018
National Category
Educational Sciences
Research subject
Education
Identifiers
urn:nbn:se:kau:diva-68392 (URN)10.1080/15434303.2018.1448820 (DOI)000435123300004 ()2-s2.0-85048700313 (Scopus ID)
Available from: 2018-07-04 Created: 2018-07-04 Last updated: 2018-07-05Bibliographically approved
Tengberg, M. & Skar, G. B. (2017). Hur tillförlitligt är det nationella provet i läsning i åk 9?. Utbildning och Demokrati, 26(2), 113-137
Open this publication in new window or tab >>Hur tillförlitligt är det nationella provet i läsning i åk 9?
2017 (Swedish)In: Utbildning och Demokrati, ISSN 1102-6472, E-ISSN 2001-7316, Vol. 26, no 2, p. 113-137Article in journal (Refereed) Published
Abstract [en]

This article reports from a study of reliability in the Swedish national reading test in ninth grade. Based on a representative sample of 500 students, the study investigates internal consistency and power to discriminate between different levels of reading ability. Data was analyzed using both classical test theory and Rasch modelling. Results from the study suggest that neither the subscales nor the whole scale of items included in the test reach desirable levels of internal consistency. Three out of four subscales include items with strikingly low degree of internal correlation (.23–.46). Furthermore, results from the Rasch analysis suggest that the test is capable of reliable separation between two groups of test-takers only: lower performing students and higher performing students. The empirical support for categorizing student performances into six separate grade levels is therefore questionable. Implications for future development of the national test of reading are discussed.

Place, publisher, year, edition, pages
Örebro: Örebro universitet, 2017
Keywords
construct validity, national test, Rasch analysis, reading, reliability
National Category
Educational Sciences Didactics General Language Studies and Linguistics
Research subject
Education
Identifiers
urn:nbn:se:kau:diva-65464 (URN)
Available from: 2017-12-22 Created: 2017-12-22 Last updated: 2018-01-13Bibliographically approved
Tengberg, M. (2017). National reading tests in Denmark, Norway, and Sweden: A comparison of construct definitions, cognitive targets, and response formats. Language Testing, 34(1), 83-100
Open this publication in new window or tab >>National reading tests in Denmark, Norway, and Sweden: A comparison of construct definitions, cognitive targets, and response formats
2017 (English)In: Language Testing, ISSN 0265-5322, E-ISSN 1477-0946, Vol. 34, no 1, p. 83-100Article in journal (Refereed) Published
Abstract [en]

Reading comprehension tests are often assumed to measure the same, or at least similar, constructs. Yet, reading is not a single but a multidimensional form of processing, which means that variations in terms of reading material and item design may emphasize one aspect of the construct at the cost of another. The educational systems in Denmark, Norway, and Sweden share a number of traits, and in the recent decade, the development of national test instruments, especially for reading, has been highly influenced by international surveys of student achievement. In this study, national tests of L1 reading comprehension in secondary school in the three Scandinavian countries are compared in order to reveal the present range of diversity/commonality within the three test domains. The analysis employs both qualitative and quantitative aspects of data, including frameworks, text samples, task samples, and scoring guidelines from 2011 to 2014. Findings indicate that the three tests differ substantially from each other, not only in terms of the intentional and operative constructs of reading to be measured, but also in terms of testing methods and stability over time. Implications for the future development of reading comprehension assessment are discussed.

Place, publisher, year, edition, pages
Sage Publications, 2017
Keywords
Assessment; cognitive target; national tests; reading comprehension; response format
National Category
Didactics
Research subject
Education
Identifiers
urn:nbn:se:kau:diva-43484 (URN)10.1177/0265532215609392 (DOI)000392606400004 ()
Projects
National assessment of reading comprehension
Available from: 2016-06-21 Created: 2016-06-21 Last updated: 2017-09-21Bibliographically approved
Tengberg, M. (2016). Rätt mätt?. Paper presented at Andra nationella konferensen i Pedagogiskt arbete 2015. Kapet (avslutad tryckt version), 12(1), 21-37
Open this publication in new window or tab >>Rätt mätt?
2016 (Swedish)In: Kapet (avslutad tryckt version), ISSN 1653-4743, Vol. 12, no 1, p. 21-37Article in journal (Refereed) Published
Abstract [sv]

Under det senaste decenniet har kunskapstest av olika slag fått en ökad betydelse i skolan och testresultat inverkar i allt högre grad på såväl pedagogiskt som ekonomiskt och politiskt beslutsfattande. Men tester, vars resultat får stora konsekvenser för elever och skolor, måste underställas höga krav på reliabilitet och validitet och bli föremål för kritiskt prövning både av vetenskapssamhället och av dem som använder testet. I den här artikeln diskuteras aspekter av reliabilitet och validitet i det svenska nationella läsprovet i årskurs nio. Till grund för diskussionen ligger analyser av provinnehåll, elevresultat och experimentella data där lärare fått klassificera provuppgifter utifrån innehållsliga kategorier. En slutsats från dessa analyser är att det nationella läsprovet dras med flera kvalitetsproblem som omfattar aspekter av både mätsäkerhet och innehållsvaliditet. Några förslag till utveckling av provsystemet diskuteras i slutet av artikeln.

Place, publisher, year, edition, pages
Karlstad: Karlstads universitet, 2016
National Category
Pedagogical Work Pedagogy
Research subject
Educational Work
Identifiers
urn:nbn:se:kau:diva-43708 (URN)
Conference
Andra nationella konferensen i Pedagogiskt arbete 2015
Available from: 2016-06-28 Created: 2016-06-28 Last updated: 2018-11-27Bibliographically approved
Tengberg, M. & Skar, G. B. (2016). Samstämmighet i lärares bedömning av nationella prov i läsförståelse. Nordic Journal of Literacy Research, 2, 1-18
Open this publication in new window or tab >>Samstämmighet i lärares bedömning av nationella prov i läsförståelse
2016 (Swedish)In: Nordic Journal of Literacy Research, ISSN 2464-1596, Vol. 2, p. 1-18Article in journal (Refereed) Published
Abstract [sv]

Tillförlitlighet i bedömning är en avgörande komponent i varje testprogram där testtagares resultat bygger på bedömares tolkningar utifrån en bedömningsskala eller en bedömningsguide. Utförliga svar på öppna uppgifter bedöms exempelvis sällan som antingen ”rätt” eller ”fel”. Istället tillämpas skalan eller bedömningsguiden för att fastställa i vilken utsträckning svaret uppvisar den efterfrågade kompetensen. I den här artikeln redovisas resultat från en studie av bedömarreliabilitet på öppna uppgifter i det nationella provets svenska läsförståelsedel i årskurs nio.

För att undersöka i vilken mån provsystemet skapar förutsättningar för god bedömarreliabilitet har sex lärare fått bedöma tre elevers lösningar av 14 öppna uppgifter, totalt 252 bedömningar. Analyserna innefattar konsensusestimat (procentuell samstämmighet och Cohens kappa) och konsistensestimat (ICC). Dessutom har kvalitativa analyser genomförts på uppgiftsnivå för att visa på aspekter i uppgiftskonstruktionen som kan ligga till grund för låg bedömarreliabilitet.

Resultaten från studien visar på moderata nivåer av bedömarreliabilitet, både ifråga om kappavärden (.73) och ICC (.82), vilket motsvarar en variation mellan bedömningarna som får stora konsekvenser för elevernas slutgiltiga provresultat. I artikeln diskuterar vi resultatens implikationer för rättvis bedömning av elevers läsförmåga i Sverige. Vi för också ett resonemang om olika sätt att stärka bedömarreliabiliteten det nationella provet i läsförståelse.

Keywords
bedömning; interbedömarreliabilitet; läsning; nationella prov; reliabilitet
National Category
Didactics
Research subject
Education
Identifiers
urn:nbn:se:kau:diva-43483 (URN)10.17585/njlr.v2.230 (DOI)
Projects
National assessment of reading comprehension
Available from: 2016-06-21 Created: 2016-06-21 Last updated: 2017-03-29Bibliographically approved
Tengberg, M. (2015). Att bedöma läsförmåga: teoretiska utgångspunkter. In: Gustaf Skar & Michael Tengberg (Ed.), Bedömning i svenskämnet: Årskurs 7-9 (pp. 87-108). Stockholm: Natur och kultur
Open this publication in new window or tab >>Att bedöma läsförmåga: teoretiska utgångspunkter
2015 (Swedish)In: Bedömning i svenskämnet: Årskurs 7-9 / [ed] Gustaf Skar & Michael Tengberg, Stockholm: Natur och kultur, 2015, p. 87-108Chapter in book (Other academic)
Place, publisher, year, edition, pages
Stockholm: Natur och kultur, 2015
Series
Svensklärarserien ; 237
National Category
Didactics
Identifiers
urn:nbn:se:kau:diva-35971 (URN)978-91-27-13844-5 (ISBN)
Available from: 2015-05-07 Created: 2015-05-07 Last updated: 2015-07-10Bibliographically approved
Skar, G. & Tengberg, M. (Eds.). (2015). Bedömning i svenskämnet: Årskurs 7-9. Stockholm: Natur och kultur
Open this publication in new window or tab >>Bedömning i svenskämnet: Årskurs 7-9
2015 (Swedish)Collection (editor) (Other academic)
Place, publisher, year, edition, pages
Stockholm: Natur och kultur, 2015. p. 242
Series
SvensklärarserieN ; 237
National Category
Didactics
Research subject
Education
Identifiers
urn:nbn:se:kau:diva-35972 (URN)978-91-27-13844-5 (ISBN)
Available from: 2015-05-07 Created: 2015-05-07 Last updated: 2016-06-08Bibliographically approved
Tengberg, M., Olin-Scheller, C. & Lindholm, A. (2015). Improving Students’ Narrative Comprehension through a Multiple Strategy Approach: Effects of Dialogic Strategy Instruction in Secondary School. L1-Educational Studies in Language and Literature, 15, Article ID 1.
Open this publication in new window or tab >>Improving Students’ Narrative Comprehension through a Multiple Strategy Approach: Effects of Dialogic Strategy Instruction in Secondary School
2015 (English)In: L1-Educational Studies in Language and Literature, ISSN 1567-6617, E-ISSN 1573-1731, Vol. 15, article id 1Article in journal (Refereed) Published
Abstract [en]

This article reports on an intervention study using a multiple strategy approach – called Dialogic Strategy Instruction (DSI) – to improve seventh grade students’ narrative comprehension in terms of students’ ability to generate inferences and develop interpretations of characters and events. DSI combines strat-egy instruction with dialogical principles for classroom discussion and response writing. After a 13-week intervention period, students in the intervention condition demonstrated significant improvement in generating inferences and interpretations compared to pre-test results. However, the average effect was small and the improvement was not significantly different relative to controls. An analysis of stu-dents with low pre-test scores showed that the intervention group had made significant and strong improvement relative to controls. Results for middle and high achievers were non-significant in this respect. Therefore, although DSI appears valuable for low achievers, results indicate that the instruc-tional principles implemented may not offer instructional support for all children. Possible reasons for the absence of a general effect are discussed and directions for future research are suggested.

Keywords
dialogue, intervention study, narrative comprehension, reading, strategy instruction
National Category
Didactics
Research subject
Educational Work
Identifiers
urn:nbn:se:kau:diva-35973 (URN)000364642500001 ()
Funder
Swedish Research Council, 2011-5540
Available from: 2015-05-07 Created: 2015-05-07 Last updated: 2018-06-14Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-9375-9512

Search in DiVA

Show all publications