Contribution: This study evaluates the generalizability of previously identified perceptions among engineering students of assessments in introductory programming (CS1). The students’ perceptions of their instructors’ and teaching assistants’ (TAs) roles in these assessments are also studied, and differences based on prior programming experience, gender, and course explored.Background: Basic programming skills are desirable also for students who are not majoring in computer science (CS). Students’ experience of assessments has not been fully explored.

Research Questions: 1) How do engineering students experience the assessment (lab assignments, midterm exam, and project) in their CS1 courses? 2) What are the students’ perceptions of the TAs and instructors in relation to these assessment situations? 3) What significant differences can be found based on students’ prior experience in programming, gender, and course?

Methodology: Previously identified themes from an interview study worked as a framework for the formulation of 25 statements used in a survey among 137 students in six CS1 courses (second part of a mixed-method study). Descriptive statistics, Mann–Whitney U Test, and Kruskal–Wallis tests with Bonferroni corrections, were used to analyze the data.

Findings: Laboratory assignments were experienced as an opportunity to learn while the exams were viewed as predictable. The projects were perceived as authentic, although varying in difficulty, and as a huge leap from the lab assignments. Students perceived the instructors to put their touch on the course but viewed their TAs as carrying out the assessments, and experienced variations between TAs. Female students experienced these variations to a larger extent and perceived received feedback as less useful.

Assessment plays an important role in education, and can both guide and motivate learning. Assessment can, however, be carried out with different aims; providing the students with feedback that supports the learning (formative assessment) and judging to which degree the students have fulfilled the intended learning outcomes (summative assessment). In this study, we explore the instructors’ perspective on assessment within the context of introductory programming courses offered to non-computer science majors at a public tuition-free state-funded university in a Nordic country. These courses are given to a large number of students and also employ several teaching assistants (TAs). We used constructivism as a basis for our study and investigated how instructors implement formative and summative assessments, how they view their role, and what expectations they have of their TAs in these assessments. We interviewed seven course coordinators (main instructors for introductory programming courses with additional administrative duties but without formal responsibility of the grading) and analyzed 205 course artifacts, such as syllabi, lab assignment instructions and course material from the cross-department TA training course. The results showed that course coordinators use formative and summative assessments both separately and within the same activity. They view themselves as responsible for the assessments, as the planners and material developers, as the organizers and administrators, as well as monitors of the assessments. However, the results also showed that these course coordinators delegate much of the responsibility for the assessments to their TAs, and expect TAs to both grade the students and provide them with feedback and guidance. In addition, the TAs are also expected to act as informants about their students’ performance. The course coordinators’ role entails many different aspects, where communicating through instructions to both students and TAs are essential. We see that this implementation of assessment, with lots of responsibility distributed to the TAs, could be difficult to manage for a single faculty member who is not necessarily responsible for the grading. Based on the results, we outline some recommendations, such as offering TA training.

Assessments of students' performances, including providing the students with feedback, are central parts of course design in higher education. These assessments can be carried out in different ways and with different purposes. The introductory programming courses (CS1) are often given to a large group of students, many of whom are non-computer science majors. To conduct the assessments and provide the students with individual feedback, teaching assistants (TAs, students with prior experience of the subject) are often employed to assist the course coordinators (main instructors) in these courses.

The practice-based research aim of this thesis is to give recommendations to course coordinators and TAs on how to integrate assessment situations in CS1 courses, aiming to be experienced as fair and valid examinations and opportunities to provide the students with useful feedback. To do so, the experiences and use of assessments within CS1 courses given to non-computer science majors, are explored from the perspective of students, TAs, and course coordinators. The three stakeholders' experiences are also compared, and to further understand the experiences, the TAs' and course coordinators' perceptions of their roles in relation to the assessments are explored. The studied assessment types include lab assignments, midterm exams, and individual final projects. 

By using a mixed-method approach with a qualitative starting point, each of the stakeholders' perspectives has been studied in detail, mostly within a Swedish context. The research uncovers the complex role of the TAs and the other stakeholders' strong dependency on them. The results suggest that each of the three studied assessment types has weaknesses and strengths that often are experienced differently by the stakeholders. This includes pitfalls with how assessment situations, designed by course coordinators to be both learning activities and graded, are challenging for the TAs to conduct. The assessments then risk being experienced by the students as unfair or dependent on the TA. Further, the results suggest that TAs face challenges related to the student-TA relationship, specifically how to handle being friends with the students they teach. The large group of students limits the course coordinators' role in the assessments, and they have a monitoring role not always visible to the students.

Recommendations for TAs and course coordinators, grounded in the research results, are presented in the thesis. Further implications from the research, in the form of TA training initiatives, are also described and evaluated as part of this thesis work.

Bedömningar av studenters prestationer, inklusive att ge studenterna återkoppling, är centrala delar av kursutformningen inom högre utbildning. Bedömningar kan dock göras på olika sätt och med olika syften. Inledande programmeringskurser (CS1) ges ofta till en stor grupp studenter där många valt ett annat huvudämne än datavetenskap. För att kunna genomföra bedömningarna och ge studenterna individuell återkoppling, anlitas ofta lärarassistenter (studenter med tidigare erfarenhet av ämnet) som hjälper kursansvariga (huvudlärarna).

Det praktiknära forskningssyftet med denna avhandling är att ge rekommendationer till kursansvariga och assistenter om hur bedömningssituationer kan integreras i CS1-kurser, med en strävan att upplevas som både rättvisa och relevanta examinationer samt utgöra möjligheter att ge studenterna användbar återkoppling. För att kunna göra det utforskas erfarenheter och användning av bedömning i CS1-kurser, som ges till studenter som inte har valt datavetenskap som sitt huvudämne, ur studenters, assistenters och kursansvarigas perspektiv. De tre intressenternas erfarenheter jämförs också med varandra och för att ytterligare förstå erfarenheterna undersöks även assistenters och kursansvarigas syn på sina roller i relation till bedömningarna. De studerade bedömningstyperna inkluderar labbuppgifter, mittkursprov och individuella slutprojekt. 

Genom att kombinera kvalitativa och kvantitativa metoder, med en utgångspunkt i det kvalitativa, har var och en av dessa intressenters perspektiv studerats i detalj, främst i en svensk kontext. Forskningsresultaten belyser assistenternas komplexa roll och hur beroende de andra intressenterna är av dem. Resultaten tyder på att var och en av de tre studerade bedömningstyperna har svagheter och styrkor som ofta upplevs olika av intressenterna. Detta inkluderar fallgropar där bedömningssituationer, som är utformade av kursansvariga för att vara både lärandeaktiviteter och betygsätts, är utmanande för assistenterna att genomföra. Bedömningarna riskerar då att upplevas av studenterna som orättvisa eller beroende av vilken assistent som genomför dem. Vidare tyder resultaten på att assistenterna ställs inför utmaningar som är relaterade till relationen mellan student och assistent, särskilt hur assistenterna hanterar situationer där de också ser sig som vänner till studenterna. Den stora gruppen studenter begränsar kursansvarigas roll i bedömningarna och de har en mer övervakande roll som inte alltid är synlig för studenterna. 

Rekommendationer för assistenter och kursansvariga, grundade i forskningsresultaten, presenteras i avhandlingen. Ytterligare implikationer av forskningen, i form av assistentutbildningsinsatser, beskrivs och utvärderas också.

Teaching assistants (TAs) are often used in computer science (CS) courses to conduct tutorials in smaller groups and to be able to provide students with one-to-one help sessions. TAs can also assist with the assessment, which can be time-consuming in large classes. Previous research does, however, indicate that TAs can be poorly prepared for their work tasks. In this experience report, we present a TA training course that addresses this issue and prepares new TAs for their responsibilities. This includes conducting tutorials, tutoring and providing students with useful feedback in lab settings, and conducting assessments. In addition, we also aimed at creating a safe space for our TAs to discuss challenges and fears that they foresee or have already experienced as TAs. To do this, we developed an introductory course consisting of five modules, that are based on previous research on TAs in CS and in the Technological Pedagogical Content Knowledge (TPACK) framework. The modules are: HR questions, classroom teaching, helping and supervising in lab sessions, assessment, and reflection and discussion. The course is given in a blended learning format, where the first three modules are student-paced online modules, while the last two are synchronous meetings. To evaluate the training, we present course evaluation results from 53 TAs who enrolled in this 6-hours course during the 2020/2021 academic year. We conclude that this TA training course has been well appreciated by almost all the participants, discuss lessons learned and future plans.

Teaching assistants (TAs), students who assist the faculty, are widely used in computer science (CS) courses. Previous studies have, however, shown that TAs could be poorly prepared and need training. Particularly, an interview study has shown that one of the areas where the TAs experience uncertainty is when assessing students’ oral presentations of their lab assignments. Based on that result and by interviewing course coordinators, we have developed and offered training workshops about assessment in CS. We invited our TAs in the introductory CS courses to participate on a voluntary basis. By distributing pre-workshop surveys at the beginning of each semester, and post-workshop surveys at the end of the semesters, to both workshop attendees (50) and non-attendees (44), we studied how the TAs conducted the assessments and what impact the training workshop had on their self-reported practice.  Both surveys had 11 identical statements that the TAs were asked to rate on a 7-point Likert scale. We also conducted interviews with four workshop attendees and three non-attendees. The results showed a significant difference between the two groups in the post-workshop survey: workshop attendees disagreed more with the statement “I try to assess students' understanding rather than the program”, which is more in line with the instructions given. In addition, when comparing pre- and post-workshop answers, the workshop attendees stated that they were less inclined to ask for help, experienced that the lab instructions were not detailed enough, and were more inclined to ask questions that convinced them that the students had written the program themselves. In the control group, no significant differences between pre- and post-tests were found.

Teaching assistants (TAs) are heavily used in computer science courses as a way to handle high enrollment and still being able to offer students individual tutoring and detailed assessments. TAs are themselves students who take on this additional role in parallel with their own studies at the same institution. Previous research has shown that being a TA can be challenging but has mainly been conducted on TAs from a single institution or within a single course. This paper offers a multi-institutional, multi-national perspective of challenges that TAs in computer science face. This has been done by conducting a thematic analysis of 180 reflective essays written by TAs from three institutions across Europe. The thematic analysis resulted in five main challenges: becoming a professional TA, student focused challenges, assessment, defining and using best practice, and threats to best practice. In addition, these challenges were all identified within the essays from all three institutions, indicating that the identified challenges are not particularly context-dependent. Based on these findings, we also outline implications for educators involved in TA training and coordinators of computer science courses with TAs.

The COVID-19 pandemic caused many universities to switch from on-campus education to distance education, in just a couple of days. In this study, we investigate how computer science (CS) majors experienced the emergency remote education during the first months of the pandemic. We sent out a survey to bachelor’s and master’s students, which yielded 794 responses. In the survey, we asked the students to compare the online education to their on-campus education on a number of aspects, such as motivation, experienced stress and the possibility to collaborate with other students. We also asked the students to compare the quality of learning activities, such as lectures and tutorials. The results show that students preferred the on-campus setting in all aspects, but on open-ended questions about advantages, the students stated the new education format made their schedule more flexible, increased accessibility of learning material, and reduced commuting time. We also identified some differences between the students’ year of study, implying that different courses did not adapt to the new conditions equally well.

When the COVID-19 pandemic broke out, universities all over the world had to close down their campuses and move all education online in just a matter of days [1]. At KTH, where a vast majority of the education typically takes place on campus, this caused a very rapid change. In this study, we wanted to evaluate how the students had experienced their education and surveyed students enrolled in year 1-3 of the 5 year computer science and engineering programme, and in both years of the master of science program in computer science.

The surveys were distributed in May 2020, as part of a mandatory assignment in the program integrating courses in the respective programmes [2]. It was mandatory for the students to fill out the survey, but the students were not graded on their responses. We got responses from almost all active students, altogether 794 respondents (139, 173, 148, 166, 168, for each grade respectively). We asked the students to compare the typical campus education before the pandemic, with the spring semester's digital education, on a number of aspects, such as stress, procrastination, motivation, and their experienced possibility to fulfill the learning outcomes. We also specifically asked the students to compare the quality of different course activities, such as lectures, tutorials, and computer labs. All of these questions were asked on a 7-point Likert scale ranging from 1 = Distance education is considerably better, to 7 = Campus education is considerably better, and with 4 being a neutral value. In addition, we also asked the students to name the main advantages and disadvantages of this new education format, in two open-ended questions.

In general, the results show that the majority of the students preferred the on-campus setting in all aspects and regarding all course activities. This was shown to be statistically significant as one-sample Wilcoxon signed ranked tests, testing the null hypnosis that the median =4 (that is that distance education and on-campus were experienced as equivalent), all had p-values< 0.05, favoring the on-campus setting. The students were slightly more positive about the online lectures, and many students stated, in the open question, that having recorded lectures makes it easy to pause and rewatch if needed. The students also appreciated the flexibility that comes with remote teaching and that their commuting time disappeared. There were also a couple of differences between different grades, where for instance first-year Master students were more positive about online lectures compared to 2nd and 3rd-year bachelor students. The main disadvantages that students mentioned were lack of motivation, study discipline, and lack of social interactions with other students as well as teachers. During the conference, a more detailed presentation of the collected data will be given. It is, however, also important to remember that the results show the students’ perceptions of online education given during the first months of the pandemic. It does not imply that distance education, in general, is experienced as less sufficient, it rather shows that last-minute changes from a familiar on-campus setting to a remote one are very challenging. When evaluating the results during the pandemic it is important to note that distance education is not the same as emergency online education [3].

Remote emergency teaching is challenging, and this is well reflected in the students’ perception of their education during the spring semester of 2020. We should, however, learn from the experience and for instance, recorded lectures seem to be something the students would appreciate also in a post-covid-19 world.

Teaching assistants (TAs), students that are employed to assist the faculty, play an important role in many courses, by for instance conducting tutorials, helping and guiding students, and assessing assignments [1]. Especially in the courses that are given to a large number of students, TAs are widely used as a way to enable one-to-one tutoring and individual feedback [2]. However, previous studies have shown that TAs are not always offered sufficient training [3]. In 2019, the EECS school decided to initiate a mandatory training course for all computer science TAs [4], and this is a brief report on the experiences of this initiative. The purpose of this initiative has been to make our TAs more comfortable and prepared in their TA role. This includes multiple aspects, for instance, to empower the TAs to ask for well-defined lab grading criteria, if that is missing.

The first offering of the TA training course was given in January 2020. The course consists of five modules (expected time for each module in parentheses): 1. HR questions (30 min) 2. Classroom teaching (60 min) 3. Helping students and supervising lab sessions (60 min) 4. Workshop regarding assessment (120 min) 5. Discussion and reflection seminar about modules 1-3 (90 min)Modules 1-3 are online modules in Canvas [5], each containing several quizzes. Modules 4-5 are scheduled sessions that first took place on campus but since March 2020 have been conducted through Zoom. During the spring semester, the course was given in Swedish. In the summer of 2020, the course material was translated to English, and modules 4 and 5 are since then offered in both English and Swedish. The TAs are paid the same hourly salary for taking part in the TA training as they earn for teaching. TAs that are employed as amanuensis (sv. amanuens) are also required to enroll in the course and for them it counts as part of their competency development. All TAs that have participated in the TA training course have been given the opportunity to anonymously fill out a customized course evaluation, and informed consent to use the answers for research purposes has also been collected.

Most of the TAs enrolled in the training course have given it positive reviews. A commonly mentioned strength of the course has been the interactive sessions, meeting and discussing with other TAs. Many TAs have also experienced the course as valuable for their TA development. For example, a TA wrote: “Great initiative, the course gives you tools that will make your role as a TA easier, and some grounds to develop within!”

As instructors in the course, we have also learned from the processes and continued to improve and develop the course throughout the year, based on the feedback from the TAs. During the conference, we will be able to share more examples from the course content as well as results from the course evaluations.

To offer an introductory TA training course has been a great experience for us involved in the course and, in general, resulted in positive evaluations from the TAs who participated in the training. A portion of what we teach is specific to computer science, while other elements of the training are more general and focus on pedagogical aspects and the TA role. We hope this initiative of TA training can spread across KTH and are more than happy to share our course material and experiences.

Third-cycle education is an important part of higher education at KTH, but has been limited discussed during previous KTHSoTL conferences. Third-cycle education can, however, face a different set of challenges compared to first and second-cycle, and there have for instance been alarming reports concerning the well-being of doctoral students [1]. The COVID-19 pandemic, has also come with an additional set of challenges for doctoral students [2], and balancing the different roles within a doctoral education program has been reported as difficult [3]. The purpose of this workshop is to highlight, reflect on, discuss and find possible solutions to some of the key issues which were identified in the doctoral student survey, that was conducted by the PhD Chapter at KTH in December 2019 [4].

The workshop will be structured around the questions:

• How should third-cycle education at KTH look like in ten years, 2031?'

• What do we need to change to reach that vision?

In December 2019, the PhD Chapter (THS) sent out a survey to all doctoral students that had a registered study activity and email address in Ladok [4]. KTH has also recently sent out a survey to all doctoral students that were admitted to their doctoral studies between the years 2012-2016 [5]. In addition, in UKÄ’s recent review of KTH’s quality assurance system, they highlighted that the doctoral students at KTH were not given as good opportunities to influence their education as first-and second-cycle students, but at the same time recognizing that this is something KTH works with [6]. A short summary of the survey results from the PhD Chapter’s report on “Consequences of COVID-19” gave some insights into how the current pandemic influenced KTH’s doctoral students at the beginning of the pandemic [7].Results/observations/lessons learnedThird-cycle education differentiates from first- and second-cycle, and comes with a set of own challenges. Based on the four sources regarding KTH’s doctoral students [4-7], we like to focus on some key topics that have been identified to have room for improvements:

• Supervision [4-5] - what characterizes good supervision and how do we achieve that?

• Doctoral student influence and evaluations of study programs [4, 6] - how can this be strengthened and unified across KTH?

• Well-being and balance between private and working life [4] - what can be done to support the doctoral students to achieve this? For instance, how can the work culture support a good balance?

• Study and work environment [4] - what is working well and what can be improved?

• Career opportunities and guidance for doctoral students [4-6] - what can be done to strengthen this?

• COVID-19 pandemic consequences [7] - what can we learn from this experience and how can we minimize potential damage to doctoral education on short and long term?