3 Knowledge tests

Determining the level: simple or complex

Below you can read the most important characteristics of simple and complex knowledge tests. It is important to make this consideration first, before choosing an alternative test format. After all, the alternative must be of a comparable level. Tools: test matrix, possible answer models or assessment criteria.

Simple knowledge tests

Tests with multiple choice or closed questions (multiple choice, fill in the blank tests), but also simple open questions and most short case studies fall under the category of simple knowledge tests. These tests require the reproduction (based on memory) of knowledge and its simple application. These are lower taxonomy levels. For simple knowledge tests, you usually use an answer model.

Please note: Does the test include closed questions, but are the students required to think through a number of steps to find the right answer? Did you indicate the higher taxonomy levels in your test matrix? In that case, please refer to the guidelines for complex tests below.

Complex knowledge tests

Tests with open questions, larger case studies or essay tests fall under complex knowledge tests. These tests require insight and analysis. These are higher taxonomy levels. For complex knowledge tests, you usually use lists of assessment criteria, rubrics or extensive answer models for the assessment.

Please note: Does the test include open questions, but is there really only one correct answer that requires only a few thinking steps of the student? Did you indicate lower taxonomy levels in your test matrix? In that case, please refer to the guidelines for simple tests below.

Test security

Simple knowledge tests

In these types of tests, the test answers are not unique to each student, because there is really only one correct answer. In addition, the test does not require any insight from the student. As a result, simple knowledge tests have a high risk of fraud. Rotterdam University of Applied Sciences has decided to not use any possibilities for online proctoring (online surveillance). This means that the fraud risk must be weighed for each study programme and mitigated as much as possible.

Complex knowledge tests

The answers to complex tests are (mostly) unique to each student. In addition, these tests require the student to justify or substantiate their own insight. This reduces the risk of fraud.

Customised or alternative test formats

Option 1: Administer the test online

The knowledge test remains summative and is taken online with a test programme such as Remindo, Learnsharp or Möbius (only for exact science subjects). You can also use an online environment such as Mastering Physics | Pearson. To limit fraud, this option requires multiple versions of the same test. In addition, you can use a time lock so students have to complete and submit the test within a specific time frame. The testing programmes described above support this option.

• Correcting open-ended questions online is not done automatically in these testing programmes.
• In Remindo, students cannot make drawings, perform calculations or write formulas in the test.

HINT will soon offer a comprehensive page with more information. Until then, you can visit the Remindo website. The information is available in both Dutch and English.

Why does this work?

Although you will have to make some adjustments to your current test (see the tips below), your summative knowledge test can be administered in the way it was conceived. Please take into account that this option is highly vulnerable to fraud.

Comments

Because of fraud vulnerability, online testing programmes are not suitable for administering crucial tests or tests that will impact major decisions. Tests are defined as crucial when:

• irresponsible social risks arise if the student does not master the learning objectives;
• the learning objectives are not tested anywhere else in the curriculum;
• the learning objectives are important for the core of the profession and are not tested elsewhere in the curriculum.

Option 2: Convert to an assignment or oral exam

Simple knowledge tests

Especially if the test consists of simple application questions (lower taxonomy levels), you can convert it to a simple hand-in assignment with a time lock or an open book. A short online oral test is also possible. The main advantage of a hand-in assignment or oral test compared to an online knowledge test (option 1) is that they require the student to provide more unique information, which reduces the risk of fraud. The disadvantage may be that an assignment requires more insight than intended by the learning objectives. You can check this with the test matrix and taxonomy level. An oral test can be a lot of work.

Complex knowledge tests

Does a test consist of questions that require the student to figure something out, make calculations and/or a drawing or write an argument? In that case you can convert the test into a hand-in assignment. Think, for example, of a simple professional product or an assignment in which students have to write their own questions and answers. Tests in which the student gives an argument or reasoning can also be converted into oral tests. It is also possible to combine both options, in which the student first creates and submits an assignment, followed by an oral validation. Complex knowledge tests can also be converted in a ‘take-home’/open book case test that students have to complete within a set time frame.

Please note: Make sure that when you convert a test into a hand-in assignment or oral test, you don’t make it too difficult. If students are prepared for open knowledge questions, however complex, writing an essay will demand more from them. In that case, provide an outline of an essay, for example, so that they have some guidance. The same applies if they have to create a complete professional product instead of answering application questions.

Option 3: Test learning objectives in another test

If similar learning objectives are tested elsewhere in the curriculum, you should consider integrating the test into another, second test. The advantage is that this will reduce testing pressure in the future. There are two options for this. In both cases the student obtains the credits for the original test if they pass the second test.

1. The learning objectives of the test will be tested at a higher level in a second test later in the curriculum. This is the case if the student cannot pass the second test without mastering the learning objectives of the original test. As soon as this is possible again, offer the student exercises and formative testing so that the step to the second test does not become too big.
2. The learning objectives of the course are partly tested later in the curriculum, or fit well into another, second test, so this test can easily be expanded or adapted. In that case, the test is integrated into the second test. This will also require practice opportunities and formative testing to properly prepare the student for the second test. The idea is not to administer two separate tests at once at a later time. If integration is only possible to a very limited extent, it is better to postpone the test (option 3).

Why does this work?

When students can show on a test with a higher processing level that they are able to apply the basic knowledge, this confirms that the students have also mastered the basic knowledge. This not only ensures less fraud-prone testing, but also reduces the pressure of testing for both students and lecturers.

Comment

A pitfall here is to not limit yourself to moving the core learning objectives to another test, which results in a test that is much too big. Failure to maintain basic knowledge with opportunities for practice and formative testing can result in insufficient basic knowledge and students failing the second test.

Option 4: Postpone the test

Before you decide to postpone a test, you need to assess how crucial a test is. Crucial tests must be postponed. Tests are defined as crucial when: irresponsible social risks arise if the student does not master the learning objectives;

1. the learning objectives are not tested anywhere else in the curriculum;
2. the learning objectives are important for the core of the profession and are not tested elsewhere in the curriculum.

If the knowledge test meets these conditions, then consult at least with the exam board, the curriculum committee and the programme manager about postponing the test until further notice. If the knowledge test does not meet the conditions, it is not recommended to postpone the test. In that case, check again whether the learning objectives can at least partially be tested in a different test or if it can still be administered in a modified format.