The value of this tool is evident : automatic scoring and feedback of learners' answers means not only major time-savings for the teacher, they also allow the learner to self-test repeatedly (without feeling self-conscious) -for example when needing to learn language vocabulary, or master difficult mathematical algorithms. They can enjoy immediate automated feedback without depending on the presence or availability of the teacher. Test scores can be automatically integrated in each student's grade-book, and data such as dates, number of attempts etc. forwarded to the reporting tool to assist the teacher in tracking and monitoring students' progress.

A well designed test can prove an invaluable motivational tool for the learner. The teacher can build in extensive and meaningful feedback and suggestions and design stimulating questions and feedback which can involve not only text but graphics, audio video, etc.

The “Tests” tool could equally be termed the “Exercises” tool, inasmuch as its use is not restricted to simple “summative” quizzes at the end of a chapter or module. For example, some teachers use the test tool as the main vehicle for presenting new material (rather than simply displaying a series of documents) in order to present materials in a more interactive way (e.g. requiring the learner to complete a question or two at the end of each page.) Using the test tool in this way to create interactive exercises (involving formative rather than summative assessment approaches) keeps learners alert and enriches their learning experience. It is also enjoyable and rewarding for the teacher, challenging him/her to develop ever richer and more stimulating course content.

Of course, there are times when the teacher simply wants to set a formal, timed “Exam” for students, withholding feedback and scores for sharing at a later date. The tests tool can be configured in this way, too.

Because the tests tool is very fully featured, it takes a little time to learn how to use. The following sections explain in detail how to make full use of the Tests tool in Chamilo.

.

Illustration: Exercises – Create a new test

The test will be displayed on the main page of the test tool, listed along with other tests created in the course:

Illustration: Exercises - List

Because there are so many different ways the test tool can be used, there is a good chance the default options are not entirely suitable your needs. Be careful, therefore to consider the extensive Advanced settings section available in the create/edit test page:

Illustration: Exercises – Creating an exercise – Advanced options

The settings are fairly self-evident:

• feedback can be provided either at the end of the test or withheld, e.g. for an exam

• questions can be presented either on a single page (e.g. for a brief text quiz) or on separate pages (e.g. when the question and answers are lengthy or incorporate media)

• scores and correct answers can be shown (e.g. for an interactive exercise) or hidden (e.g. for an exam).

• Questions and answers can follow a pre-determined sequence or be randomized and shuffled

• The relevant questions category can be displayed or hidden

• The number of attempts allowed can be limited to a given number - “infinite” attempts might suit a “practice” exercise

• Start and end times for the test can be set and the time allowed to complete the test can be limited; )a real-time counter will appear to the user)

• Negative scores can be “zeroed” or carried forward to the next question

• Students can review their answers before submitting the test

• A pass percentage can be set to indicate success or failure to the student

• A closing message to be viewed on completion can be composed.

Note : The platform administrator may enable an additional option in the “Feedback” section of the advanced settings:__Self-evaluation (immediate feedback). This option, if checked, enables the “scenario” question types which allows teachers to devise quizzes based on multiple-choice or image mapping questions which, depending on the answer given, lead the learner to alternative subsequent questions. This is a powerful tool – however, be aware that once self-evaluation is checked, these advanced settings cannot be changed: you will instead have to create a new test not based on the Self-evaluation (immediate feedback) option.

This section describes some of the key educational concepts which have steered the design of the Tests feature in Chamilo. For teachers new to on-line teaching and learning, it is useful to consider some of the terminology Chamilo uses in relation to these concepts prior to developing the tests themselves.

Automated feedback

A major advantage in on-line learning is that it can offer the learner enormous flexibility in terms of where and when he/she chooses to study. A student can undertake a fully automated test if they wish anywhere and at any time and receive their score immediately. However, this freedom also means that there is no teacher to hand to explain why they have got a question wrong in a test or exercise, as there might be in a traditional classroom. In automated on-line tests, therefore, the teacher needs to pay particular attention to the provision of automated on-line feedback.

This means that as well as receiving a score, the student can receive a brief (pre-written) comment relating to their answer, whether it is right or wrong. Typically, if he gives the right answer, you might congratulate him: “Well done, I can see you have a good grasp of this concept!”. If he answers incorrectly, you can offer immediate feedback to encourage him to look beyond that demoralizing red “X”, and help make the error part of the learning process. This is the best time for him to learn where he went wrong and how to avoid the error next time. Use hints e.g : “Sorry, wrong answer! Have you read the Open Standards chapter deeply enough? A quick revision of Chapter 3 should help you out! Remember the example of OASIS and how it was designed?”. (This feedback could also contain a direct link to the “Chapter 3” example.)

Such feedback can “humanize” the test and is often really appreciated by students. It means a bit more work for the teacher when creating the test, but provides an important way of encouraging and correcting the learner during his auto-evaluation attempts.

Scoring tests

Assigning a score represents a “raw”, even crude way of evaluating a student's skills. Nevertheless, scores matter a lot to students even though they are of limited use in terms of formative assessment. You have a lot of flexibility in setting up scores, pass marks etc. in Chamilo, including assigning overall weightings to tests and the option to assign negative scores to individual answers. You should use these features with care, and avoid careless errors especially (a wrong score for a correct answer is incredibly frustrating for a student). If you give a automatically score to a learner, it can prove difficult to adjust it afterwards. Updating a score may be necessary under specific circumstances (e.g. where the test is complex), and this will require a manual revision by the teacher.

Displaying correct answers

The option to display correct answers is a simple and direct way of providing feedback by demonstrating to the learner what he should have answered. Sometimes displaying the answers in this way is desirable, but given that the learner may wish to re-take the test without being given the answer, or that you may well want to re-use the test on a future occasion, it is recommended you don't display the any answers, but instead use feedback to let the learner know how (rather than what) he should have answered and set him on the right course to get it right next time through his own efforts.

You will have gathered by now that the term Tests used for this tool is a generic term used to describe a range of on-line learning activities which, along with assignments and forums, can be used to generate evaluative information which can guide students' further learning. A key aspect of the Tests tool is that it can be used to provide instant automated responses to student input without the need to wait for the teacher to respond. How such automated responses are delivered and used however can vary widely in the course of the student's journey through the course. Below are some suggested broad categories of evaluative activity which can be effectively set up using the Tests tool.

• Exams refers to (generally time-controlled) exercises for which no immediate feedback, nor any answer, is given to the learner : he will obtain his results when the teacher considers it right. This type of summative assessment is often undertaken at the end of a course and may carry a high weighting in the overall assessment of the student. Because of its significance scoring and feedback are generally withheld until the teacher has had the chance to go through at students' results with a “fine-tooth comb”.

• Exercises allow the teacher to assess the learner throughout the course e.g. at the end of a lesson, unit or module. Less formal than an exam, such tests might allow students three or more attempts and represent a more formative approach to assessment, involving plenty supportive feedback. Some elements of these tests might not be fully automated, requiring some level of direct assessment by the teacher. Most scoring would be immediately supplied, but feedback would tend to fall short of displaying the correct answer directly, challenging students to apply themselves to study revision and reflection to work out the correct answer for themselves.

• Automated tests can generally be available to take an unlimited number of times. Their key purpose would be to allow the learner to “drill” himself, e.g. to learn vocabulary in a language course, rehearse a particular mathematical algorithm or memorize a safety procedure. Such tests would be free of any time constraints and would be fully automated displaying scoring, feedback and even correct answers to encourage improvements through practice.

• Scenarios represent a more advanced form of interactive exercise where the teacher can pre-determine alternative next steps in the test predicated on student's response to a question, leading the student through a particular path of questioning. This type of activity can be achieved using Chamilo's Tests tool and can provide students with a very effective interactive on-line learning experience, but of course is by its very nature quite complex and demanding to set up.

The above descriptions demonstrate how the Tests tool can function depending on how it is configured by the teacher. The tool is designed to be as flexible as possible while remaining relatively easy to use, so variations on all the above “models” can be achieved to meet requirements.

On creating a test, click Proceed to questions you will be taken to the questions page:

Illustration: First view of the questions list

In addition to the toolbar the page displays an array of different types of question. (The same page can be accessed at a later time using the edit icon for any given test listed in the Tests page.) It offers a range of question types from which to choose, which are described in this section. Each test can contain a combination of question types – this can prove handy for ensuring tests don't become monotonous.

Fill banks/form

multiple choice

Open question

image zones

Unique answer + unknown

Combination T/F/ ?

multiple answer

matching

Oral expression

exact selection

Multiple answer T/F/ ?

Global multiple answer

Illustration: Exercises – Question types

Questions are created using an on-line form which differs according to the question type. The following subsections describe each form in detail. Completion of the forms is quite intuitive. The following general points regarding the completion of the on-line forms apply to all or most types of question:

1. The “Comment” column will only be shown if you selected the show feedback_option to in the _advanced settings when you created the test (if you don't want to show any feedback, then there is little point in showing this column on the page.)

2. Any fields marked with a red asterisk * must be completed.

3. Every question has a title and an “enriched” description. We generally recommend putting a short title in the title box, and putting the real question in the enrich question box. This will allow you to present a much fuller, more meaning question, use images, and/or include audio or video.

4. All answer and comment boxes have editing tools which are hidden by default but can be viewed by clicking the small black arrow on the top-left corner of the text box. This will allow you also add images, links and audio to your answers or feedback comments.

5. Every question type involves the allocation of scores. In the case of questions requiring scoring later by a teacher, this will simply be a maximum score against which the teacher will mark the test. In the case of objective tests involving learners selecting an answer, the score for each right or wrong answer chosen can be set, and can include negative values. Teachers should give some thought, particularly in tests involving different question types, to what weighting within the overall test each question should carry.

6. Teachers should also consider the number of options they wish to offer in the case of multiple choice type questions – this can be determined using the red and green remove/add answer option buttons at the foot of the form.

8. You can also assign a difficulty level for each question in the advanced options. This can be handy for later fine-tuning the evaluation of students' work, or reviewing the test overall. It can also be very useful when it comes to recycling questions, allowing you filter questions by difficulty.

Type 1 : Multiple choice – Single answer

The classic multiple choice question is one in which only one answer can be considered valid for scoring purposes. It is possible to make the test a little bit more challenging by assigning a negative score to a wrong answer. When this is highlighted (e.g. in the test introduction or the question itself), this encourages the learner to think more carefully before selecting ananswer at random, and incurring a penalty.

Illustration: Tests – Multiple choice, single answer

Type 2 : Multiple choice – multiple answer

This question type is a variation of the previous simple multiple choice question. It allows you to assign more than one valid answer. Again, positive and negative scores are possible. This type of question can prove more challenging to students (you may wish to indicate in the question that there are two answers – so as not to “trick” them!

Illustration: Tests – Multiple answer question

Type 3 : Fill the blanks

This question type can be used to produce the classic “cloze passage” exercise favoured by language teachers, in which certain words in a written passage can be selected to be displayed as a blank field for students to fill in. First, the teacher simply needs to type the passage (or list of sentences) and then add square brackets around the words to be “blanked”. (These words will be displayed to the learner as empty text fields, which will be marked as correct if the student enters the correct answer matching the word typed by the teacher.) As soon as the square brackets are added, the form will display a “score” field for each word to be corrected. To make the test a little easier, the teacher may choose to list the words to be chose in the “enrich question” box. It's a handy test for checking key terminology or vocabulary.

Illustration: Tests - Fill the blanks

The question will appear like this to the learner, who simply needs to type the correct word in to the blank fields:

Type 4 : Matching

This type of question is designed to have the learner link together text elements from one list with text elements in another. An element might be a word, a sentence or part-sentence. It can also be used to ask a user to sort elements in a certain order.

Illustration: Tests – Matching text elements

The learner will view the matching question like this:

or, from 1.9.6, a draggable circle will make it a little more appealing to students.

Type 5 : Open question

The open question type allows the learner to write whatever they wish as an answer (a kind of “mini-assignment). As such an activity has no single give answer, it will not be automatically evaluated at the end of the test, but will have to be evaluated manually by the teacher before the final test result can be scored. As with assignment and marked forum posts, the teacher is able to do this via the Recording tool which will flag up open questions needing marked.

Illustration: Tests - Open questions

The learner will be presented with a text field in which to write. (He can use the small arrow on the left of the top border of the field to make use of a range of fields if he wishes to enrich his presentation. e.g. with heading or highlighted text.)

Type 6: Audio answer

This new question type, available with Chamilo 1.9, will only appear only if your administrator has enabled the Nanogong audio recording feature. The question can be compared to “open question” type described above, requiring manual marking by the teacher, but instead of asking the learner to write a text passage, you ask him to record an audio answer. This has obvious uses for language courses, of for learners with particular access needs, but can provide a useful alternative in many study fields – encouraging learners to develop their oral presentation skills.

(note: This question type requires the learner to have Java applets supported in his browser. If this is not the case, he will need to to record the audio separately and then upload it as an MP3 file to the server.)

Because of the additional requirements for this question type, you should make sure your learners are aware that the test will contain an audio question and that they have a more than one chance to try the test to check whether their browser supports it or not. (Bear in mind that taking a test is generally a stressful experience. You don't want to give your learners a nervous breakdown because they can't manage to record an audio answer!)

Illustration: Tests – Oral expression

The learner simply needs to press the Record answer when he is ready to talk, and follow the instructions for the on-screen Nanogong recorder, which will automatically send his recording to the teacher for marking and scoring.

Type 7 : Hot spot, or image area

In this type of question, the learner will be asked to identify areas within an image by clicking on them. These areas have been previously defined by the teacher. Building one of these questions require slightly more work than other types.

Illustration: Exercises - Hot spot question creation

Having uploaded your image and named your question, you can add or remove hotspots, and define the area using an ellipse, a rectangle or a polygon shape. As with other question types, you can add comments for feedback purposes and determine a score for each correctly identified area.

The learner needs to click on the picture to identify the areas listed – the numbers will appear on top of the areas on which he clicks:

Illustration: Exercises - Hot spot learner view

Type 8 : Exact combination

This is a variation of the multiple answer type test looked at above. The only difference is that no partially-correct combination will be accepted. This means that if, between options A-B-C-D, only B and D are correct, you will have to mark both B and D in order to be given the points assigned to this question. There is therefore only one overall score field shown on the form.

Illustration: Tests - Exact combination

Type 9 : Unique answer with unknown

Sometimes a teacher wants the student to indicate that they don't know the answer to a question, rather than just guess an answer. This question type is a single-answer multiple choice question, but reserves the last answer as a “Don't know” option, with an automatic score of zero. Students can be discouraged from guessing by penalizing a wrong answer with a negative score.

Illustration: Tests – unique answer with unknown

Type 10 : Multiple answer True/False/Don't know

This is a another combination format which includes a zero-scored “don't know” option as with the above question option, but using true/false answers rather than multiple choice. Again, negative scores can be assigned to discourage guessing, and admit “honest” ignorance!

Illustration: Tests – Multiple answer true/false/don't know

Type 11 : Combination True/false/don't know

This option offers a single score covering one or more questions. The form looks similar to the “exact answer” form, but the test is displayed to the student differently, as illustrated below.

Illustration: Tests – Combination true/false/don't know

The learner will view the question(s) and use radio buttons like this:

Note: a “don't know” will generate a zero score, so this format might most fairly contain one or two questions – requiring prudent use from the teacher.

Type 12 : Global multiple answer

This question format combines the multiple answer format with a single score , providing an option for ensuring that no negative score is accrued

Illustration: Tests – Global multiple answer

Any question created, independently of the test or course to which it has been attached, feeds the questions database. This base can be very useful for teachers wanting to re-use questions that are very general or, needing questions to serve as “fillers” when you're short of a few questions in a new test.

During test creation, the database shows questions from other tests, as well as orphan questions (i.e. those questions which have belonged to a test which has been removed).

When the question database shows, it presents a list of all existing questions inside the course. This is where assigning questions to categories and adding a difficulty to tests can prove very useful for identifying individual questions: use the Filter and Difficulty drop-down lists to filter the questions accordingly.

Illustration: Tests – Questions database

As well as allowing teachers to create their own tests as described above in sections 12.2 to 12.6, Chamilo enables teachers to easily upload and include tests created in external tools, in particular tests created using Hot Potatoes, Qti2 and Microsoft Excel. These are popular authoring tools which can produce tests in a standard format suitable for use in many, LMSes, including Chamilo. Details of how to use these software packages are obviously outwit h the scope of this guide, but experienced online educators may well be familiar with one or all of them, and may have produced tests with them which they wish to upload to Chamilo.

To do so, the teacher need simply click on the relevant icon in the toolbar on the main Tests page and follow the dialogue box to locate and import the relevant file.

Illustration: Tests – import external tests from Tests page

Illustration: Tests - Import HotPotatoes, Qti2 and Excel tests

Illustration: Exercises - List

This will display a page listing all the results for the relevant test.

Illustration: Exercises – Results list

Here, the teacher can get an overview of learners' times and scores, and can score or edit open questions and offer feedback. He can also delete a test attempt to allow the learner to do the test again (if there is a valid reason to do so e.g. there was an error in the test).