A tool to assess computational thinking skills

A fundamental skill

"Computational thinking (CT)” is widely recognized as a fundamental skill of computer science since it consists not of thinking like a computer but of thinking in a computational way, i.e. that is, to be able to translate an idea into a language that a computer can understand and execute.

Computational thinking thus refers to a thinking process in which students learn to break down a complex problem or system into smaller, manageable parts, find and identify similarities between these different components. They learn to sort and select only the relevant information and to formulate a solution as a series of steps or "rules" that a computer will then be able to follow and execute. This set of rules is what we call "algorithm".

Despite the fact that it is an essential skill for digital education, few tools exist to assess the level of students.

Tools to assess computational thinking

To meet this need, a partnership bringing together EPFL, the SUPSI-DFA, the USI and the FHNW was set up around the project entitled “CT-Skills”. As part of the National Research Programme (NRP77) dedicated to "Digital Transformation" and supported by the Swiss National Science Foundation (SNSF), this project aims to assess students' computational thinking skills in an automated way, for example through a tutoring system that students interact with during the problem-solving process.

The cross-institutional collaboration has led to the development of a first evaluation tool for computational thinking skills named the “Cross Array Task (CAT)”. This evaluation tool effectively takes the form of an unplugged activity --an activity which is done without a computer-- and which can be carried out at all levels of compulsory education.

Developing the Cross Array Task

Objectives of the unplugged activity

The Cross Array Task aims to assess students' ability to recognize patterns in a cross-shaped template made up of colored dots and to describe this pattern to another person using different means of communication (voice, gestures, drawings, etc.) in order for their partner to reproduce the pattern on a blank template.

This activity is designed to assess basic computer skills such as the ability to break down a problem into parts (decomposition) or pattern recognition and the ability to describe a procedure (algorithm).

Increasingly complex diagrams

At the start of the task, students are paired up and separated by a partition which prevents them from seeing each other. To describe the diagram to student B, student A is first asked to use only words; if that fails, a blank sheet of paper and a black pen is available and can help. If that still does not work, a blank diagram is provided. Finally, if that isn’t successful student A is given colored markers. If necessary, student A can also inquire at any time how and what student B is coloring.

The task consists of deciphering and synthetically describing a sequence of 12 increasingly complex diagrams. In the best case scenario, student A is able to correctly describe the 12 diagrams to student B through strictly only verbal cues, without seeing what student B is doing and within an allotted time of approximately 5 minutes.

From research to practice: testing the Cross Array Task in schools

Between March and April 2021, the Cross Array Task was tested in an experimental study carried out in 8 classes of 3 public schools in the canton of Ticino.

In order to cover all school levels, the research team selected a sample including a kindergarten school (3 to 6 years old), three primary classes - a 1st, a 3rd and a 5th (6 to 11 years old) , and four lower secondary classes - one 2nd, one 3rd and two 4th (12 to 16 years old).

A total of 109 pupils (51 girls and 58 boys) from all levels of primary school, ie aged 3 to 16, took part in the experiment.

Results

The main results of the study reveal that algorithmic skills increase with age, in particular with regard to autonomy and the ability to use symbolic artifacts to describe their algorithms. The largest increase is seen between students in lower and upper primary education.

The study also shows that the very young, i.e. kindergarten-level children and first cycle primary school students, are already able to describe complex algorithms (one-dimensional and two-dimensional algorithms).

No significant difference related to gender was observed.

These results are a first step in being able to identify computational thinking in an automated way. The publication with all the detailed results of this study is available here.

The Cross Array Task along with its description and worksheets is available on the roteco website.