Evaluation+and+Assessment

 =Evaluation and Assessment=

**Benefits: **
The benefits of on-line constructivist environments for formative and summative assessment can be significant. Based on research in this area we have outlined several below. **We encourage you to add others to this collaborative section.**

· Formative assessment opportunities are more frequent because the students are producing work the teacher can easily access. (Ross & Ross, 2005) · The ability to divide and monitor a student's participation in more traditional “group work”. (e.g, can assist in making judgments around ATL marks) · The ability to provide timely feedback, while the work is still relevant to the student. · Provides an venue for regular peer assessment. · Provides greater opportunities for assessment FOR learning (instead of assessment OF learning) such as teacher or peer feedback and links to additional resources that might enhance learning. · A reduction in the “halo effect” through the potential use of anonymous posts. · <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Recent studies have shown that active participation of generally less motivated students increases. <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Computer tracking tools allow for quick teacher diagnostic about which students are not contributing or managing expectations. Can also allow for some data to be used in ATL assessments. <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Access to electronic testing and quizzes, which allow for instant feedback; <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Potential for a reduced focus on marks (the “what did you get effect?”) and thus competition and an increased focus on more beneficial approach of personal and group learning. <span style="color: gray; font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;"> <span style="font-family: 'Times New Roman','serif';">

**<span style="font-family: 'Times New Roman','serif';">Challenges: **
<span style="font-family: 'Times New Roman','serif';">Assessment of on-line learning environments can also be challenging. The tasks are more open-ended, more process oriented (see designing environments) and less defined.

Here are some noted challenges researchers found in assessing work in these environments. You may have discovered these in your own practice and may have more to add in our collaborative environment: <span style="color: red; font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;"> <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Ross & Ross (2005), note many challenges in assessing on-line learning environments. They argue that <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Allowing students to participate in multiple ways can make the teacher’s job more laborious (Hsu, Marquez, Hamza & Alhalabi, 1999; Levin, Levin & Waddoupts, 1999; MacDonald, 2001). <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">The amount of reading generated by on-line student discussions may encourage students to participate in “surface processing” rather than the deep learning that a single source may allow for. <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Students with weaker writing skills and/or cultural differences may create divergent quality of participation that may be difficult to assess fairly. <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Written products, which are those usually assessed in these contexts, can limit the ability for students to demonstrate real learning and, more importantly, to participate in meaningful and divergent applications of new learning.

Other challenges, based on the research of Chan & Aalst (2003), and Brooks and Joeng (2006), Hew & Cheung (2004) and others include: <span style="font-family: 'Times New Roman','serif';"> <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif';">Online discussions often produce discussion that lacks coherence and depth, with students often not responding to what other students have said. <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif';">It can be challenging to properly assess the content learning of passive learners. <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif';">What is expressed in these contexts is not always reflected in written or created responses. <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Assessing social development, which is an important aspect of interactions and contributions in these collaborative learning environments, can be difficult to assess, especially under ministry and IB criteria. <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Although the differentiation required in all classrooms may be easier in an on-line environment, it is also potentially more public and this can cause social and emotional challenges for students of differing abilities and/or prior knowledge. <span style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol; mso-fareast-language: EN-CA; msobidifontfamily: Symbol; msofareastfontfamily: Symbol; msofareastlanguage: EN-CA; msolist: Ignore;">· <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Fluid participation patterns and uneven participation or understanding can make it difficult to assess according to timelines and detailed criteria.

<span style="font-family: 'Times New Roman','serif';">

**<span style="font-family: 'Times New Roman','serif';">Putting It Into Practice: **
<span style="font-family: 'Times New Roman','serif';">Setting up assessments for these on-line environments can be challenging. Our intent here is to outline some suggestions, taken from research in this area, which may help teachers do this.

<span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;"> E.g., ask students to find and validate new resources to support and build on new ideas. They can create visual representations of their understanding, using tools such as [|mind maps] (and others found in our Tools section).
 * //<span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Construct activities that require more than written responses: //**

<span style="font-family: 'Times New Roman','serif';">Not surprisingly, many researchers encourage teachers to have a very clear and detailed understanding of the desired learning outcomes (e.g., Wiggins and McTighe) before designing the task and the corresponding rubric and then <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">design assessments to measure learning objectives, not to merely increase the frequency of student interaction. <span style="font-family: 'Times New Roman','serif';">
 * //<span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Learning Objectives Drive Activities: //**

<span style="font-family: 'Times New Roman','serif';">As in any classroom, give varied assessment tasks. Don’t just assess posted responses to on-line discussions in the same way. Try using tools such as a reflective summary, an online survey, peer or self assessment tasks, peer assessment using a peer rating system. (Vonderwell, Liang, and Alderman (2007). These methods can increase student engagement and independence.
 * //<span style="font-family: 'Times New Roman','serif';">Use a variety of assessment types: //**

<span style="font-family: 'Times New Roman','serif';">Researchers encourage teachers to assess not only the understanding and knowledge application demonstrated in productions created in on-line environments, but to assess the method of interacting with peers (i.e. Referring directly to specific work of their peers, using appropriate and discussion-building language etc.)
 * //<span style="font-family: 'Times New Roman','serif';">Include assessment of application of productive collaborative techniques: //**

<span style="font-family: 'Times New Roman','serif';">One of the great challenges of on-line contributions is that teachers must find time to build in meaningful and timely feedback on student contributions. Teachers must build this into to planning time. Feedback must be acted upon in order to provide learning opportunities for the students.
 * //<span style="font-family: 'Times New Roman','serif';">Provide Timely and Detailed Feedback for Additional Learning //**

<span style="font-family: 'Times New Roman','serif';">The quality of postings or contributions will always be valued over the quantity in good on-line assessment. Questions Hew and Cheung (2003) recommend teachers asking are: //<span style="font-family: 'Times New Roman','serif';">What are the types of thinking skills exhibited by the online learners during the online discussion? What are the levels of information processing found in the online learners' messages? //
 * //<span style="font-family: 'Times New Roman','serif';">Require Demonstrations of Higher Level Thinking //**

**<span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">The following table from indicates how scaffolding might be put in place and assessment tools would focus upon a Level 5 in the active construction of knowledge ****<span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;"> (Hew and Chueng, 2003): ** (after Kanuka & Anderson, 1998)
 * <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Table 1: **<span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;"> Five phases in the active construction of knowledge


 * <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Phase I ||  || <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Sharing and comparing of information. For example: //Statements of agreement or corroborating examples from one or more other participant.// ||


 * <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Phase II ||  || <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Discovery and exploration of dissonance or inconsistency among the ideas, or statements advanced by different participants. For example: //Identifying and stating areas of disagreement or asking and answering questions to clarify the source and extent of the disagreements.// ||


 * <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Phase III ||  || <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Negotiation of meaning. For example: //Negotiation of the meaning of terms or identification of areas of agreement or overlap among conflicting concepts.// ||


 * <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Phase IV ||  || <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Testing and modification of proposed synthesis or co-construction. For example: //Testing the proposed synthesis against formal data collected or against contradictory information from the literature.// ||


 * <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Phase V ||  || <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Statement or application of newly constructed knowledge. For example: //Summarising of agreements or students' self reflective statements that illustrate their knowledge or ways of thinking have changed as a result of the online interaction.// ||

<span style="font-family: 'Times New Roman','serif';"> The following judgment examples from Hew and Cheung (2003) might also help teachers consider the assessment of higher level thinking skills in on-line contributions. <span style="font-family: 'Times New Roman',Times,serif; font-size: 11pt;"> __Critical thinking - surface level:__ <span style="font-family: 'Times New Roman',Times,serif; font-size: 11pt;">//"I find that there are too many empty (white) spaces on the presentation slides."// (This was classified as critical thinking - surface level of information processing since the author made his conclusion without giving any justification as to why it was not good to have too many empty spaces on a presentation slide) __Critical thinking - in depth level:__ <span style="font-family: 'Times New Roman',Times,serif; font-size: 11pt;">//"I feel that the choice of your illustrations are quite well chosen, except for the birds. I feel that the birds are distracting because of their movements and they don't blend well with the other illustrations."// (This was coded as critical thinking - in-depth level of information processing because the author expressed a judgment and provided a plausible argument as to why his judgment was valid.)

<span style="font-family: 'Times New Roman','serif';">Scaffolds work both around content and on-line processes and ensure that these supports change as the course and the student learning progresses (see above).
 * //<span style="font-family: 'Times New Roman',Times,serif;">Build in Scaffolding: //**

<span style="font-family: 'Times New Roman','serif';">Build in regular and required elements of peer and self assessment according to strict criteria that require written reflections by the students involved. <span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">
 * //<span style="font-family: 'Times New Roman','serif';">Build in Peer and Self Assessment //**

**<span style="font-family: 'Times New Roman','serif'; mso-fareast-font-family: 'Times New Roman'; mso-fareast-language: EN-CA;">Additional Resources **
<span style="font-family: 'Times New Roman','serif';">For additional reading and viewing on Assessments in On-line environments, you may wish to visit the following links: Dugas on evaluating on-line environments: [] An article which gived rubrics for writing: [] For an audio Powerpoint on On-line assessment [] An article on preventing plagiarism in on-line environments: [] An overview of the benefits of the influence of ICT and on-line discussions (don’t be thrown off by the silly beginning. Some valuable content here)

media type="youtube" key="EwAo1Cc8FAo" width="296" height="245" align="center"