kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Adaptive Comparative Judgement in Practice: A case study of the formative use of ACJ in a collaborative open-ended design project between Swedish upper primary schools under authentic classroom conditions
KTH, School of Industrial Engineering and Management (ITM), Learning.
KTH, School of Industrial Engineering and Management (ITM), Learning.
2023 (English)Independent thesis Advanced level (professional degree), 20 credits / 30 HE creditsStudent thesisAlternative title
Adaptiv komparativ bedömning i praktiken (Swedish)
Abstract [en]

Digital assessment has the potential to increase equity in the assessment by facilitating a smother collaboration between teachers. It also allows peer assessment between students from different classes or even schools, which allows the students to view a potentially larger and more varied set of examples of the same assignment. In this study, we have examined the potential of using Adaptive Comparative Judgement (ACJ) as a collaborative method of formative assessment of student portfolios created in the open-ended design project: Public Spaces. In the project, the students were given the task of creating an improvement suggestion for a public space from an Agenda 2030 perspective. ACJ has been used with positive results in several similar studies, but many of these have been conducted in more controlled environments with fewer variables. This study, therefore, focuses on examining ACJ from a more practice-based perspective with younger students. The students in the study were taught by their regular teachers who were instructed to plan their lessons according to the teaching material created for Public Spaces within this master thesis, which can be found in appendixes A, B and C. This study was conducted through a collaboration between four upper primary school classes from different municipalities in Sweden. The students and teachers participating in this study completed two ACJ assessment sessions, one in the middle of the project and one at the end. During the assessments, they were also instructed to provide written feedback. When analysing the content of the feedback it became clear that although the majority of the comments could be classified as specific, meaning they clearly specified which area of the assignment the feedback referred to, the feedback was often rather shallow and potentially not very helpful. The students and teachers rarely motivate their feedback, and the students rarely provided clear improvement suggestions. When analysing the reliability of the portfolio rankings from the ACJ assessments we found that they were notably lower than in many of the previous studies. Furthermore, we noticed that several of the teachers seemed to have struggled to find the time to read the teaching material. This problem could potentially have been alleviated by communicating the material more clearly at the start of the project.

Abstract [sv]

Digital bedömning har potential att öka likvärdigheten i bedömningen genom att främja ett smidigare samarbete mellan lärare. Den öppnar även upp för kamratbedömning mellan elever från olika klasser och skolor, vilket kan bidra till att eleverna får tillgång till fler och potentiellt mer varierade elevexempel. I denna studie har vi undersökt möjligheten att använda adaptiv komparativ bedömning (ACJ) i formativt syfte vid sambedömning av elevportfolier skapade i det öppna designprojektet: Offentliga Rum. Projektet gick ut på att låta eleverna ta fram ett förbättringsförslag för ett offentligt rum utifrån ett Agenda 2030 perspektiv. ACJ har använts med positiva resultat i flera liknande studier, men många av dessa har utförts i mer kontrollerade miljöer med färre variabler. Denna studie fokuserar därför på att undersöka ACJ ur ett mer praktiknära perspektiv och med yngre elever. Eleverna i denna studie undervisades av sina ordinarie lärare som var instruerade att planera undervisningen enligt ett undervisningsmaterial skapat för Offentliga Rum inom detta mastersarbete, vilket finns tillgängligt i appendix A, B och C. Denna studie utfördes genom ett samarbete mellan fyra mellanstadieskolor från olika kommuner i Sverige. De deltagande lärarna och eleverna genomförde två ACJ bedömningar, en i mitten av projektet och en vid slutet. De var också instruerade att ge skriftlig återkoppling under bedömningarna. Vid analysen av innehållet iåter kopplingen blev det tydligt att trots att majoriteten av kommentarerna kunde klassificeras som specifika, dvs. att de tydligt specificerade vilket område av arbetet som återkopplingen refererade till, var återkopplingen ofta ganska ytlig och potentiellt inte särskilt hjälpsam. Eleverna och lärarna motiverade även sällan sin återkoppling och eleverna gav sällan ett klart förbättringsförslag. Vid analys av reliabiliteten hos portfoliorankingarna från ACJ-bedömningarna fann vi att de var märkbart lägre än i många av de tidigare studierna. Dessutom uppmärksammade vi att flera av lärarna verkar haft det svårt att få tid att läsa undervisningsmaterialet. Dessa problem skulle eventuellt kunna mildras genom tydligare kommunikation av materialet vid projektets start. 

Place, publisher, year, edition, pages
2023. , p. 198
Series
TRITA-ITM-EX ; 2023:22
Keywords [en]
Digital Assessment, Adaptive Comparative Judgement, RM Compare, Feedback, Peer Assessment, Formative Assessment, Teaching Material, STEM
Keywords [sv]
Digital bedömning, Adaptiv Komparativ Bedömning, RM Compare, Återkoppling, Kamratbedömning, Formativ bedömning, Undervisningsmaterial, STEM
National Category
Other Engineering and Technologies Educational Sciences
Identifiers
URN: urn:nbn:se:kth:diva-323974OAI: oai:DiVA.org:kth-323974DiVA, id: diva2:1737577
Subject / course
Technology and Learning
Educational program
Master of Science in Engineering - Engineering and of Education
Presentation
2021-08-13, 00:00
Supervisors
Examiners
Available from: 2023-02-17 Created: 2023-02-17 Last updated: 2025-02-18Bibliographically approved

Open Access in DiVA

fulltext(70172 kB)488 downloads
File information
File name FULLTEXT01.pdfFile size 70172 kBChecksum SHA-512
d9f09dd21c1cb33909f35ba46c1be162cea8cebff923b934d37b0d7c1f55f541caa2288f24d2ee4a0fa0051aa5f28a541b5725e7fb63d615c1a46c93fb13a13a
Type fulltextMimetype application/pdf

By organisation
Learning
Other Engineering and TechnologiesEducational Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 488 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 585 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf