SPARKPlus: moderating group work assessment - UTSOnline Help Search Generic filters Hidden label Exact matches only Hidden label Hidden label Hidden label Home Information for all users Information for all users Blackboard Mobile and Tablet applications Collaboration Tools Collaboration Tools Campus Pack For students For students Avoiding Plagiarism for students Downloading a Subject Outline in UTSOnline How to successfully submit a UTSOnline assignment SparkPlus for students Turnitin for students Kaltura Kaltura Download your media from Kaltura Kaltura Closed Caption Kaltura MediaSpace: Use Channels to Group and Control Access to Your Media Restrict access to your media with Scheduling Supervisors Access to MyMedia of Kaltura Why Publishing Medias? Logging into UTSOnline Navigating UTSOnline Navigating UTSOnline The My Subjects module Portfolios Privacy Statement for UTSOnline Supported Operating Systems and Browsers UTSOnline Support Information for Staff Information for Staff 1. Grades Submission Automation - Getting Started 1. Grades Submission Automation - Getting Started 2. Grades Submission Automation - Setting Up Your Grade Centre for Automation at the Start of Session 3. Grades Submission Automation - Finalizing and Submitting Your Grades at the End of Session 4. Grade Submission Automation Processes for Faculty Admin & Support Adding Your Subject Outline to UTSOnline Automated UTSOnline Subject Creation and Enrolment Automated UTSOnline Subject Creation and Enrolment Forms for Subject Coordinators Session key dates in UTSOnline Building Assessment Tasks Building Assessment Tasks Changes to inline document markup in the Blackboard Assignment tool How to access Review Portfolio Assignments for Staff Quiz recommendations UTS Avoiding Plagiarism Building Content Building Content Adobe Captivate Content Layout Tool How to make a video or screencast - examples IML Sound Booth Lecture capture with Echo360 ALP LinkedIn Learning Managing the availability of Echo360 lecture recordings Practical Ideas - Video Reading Lists Communicating and Collaborating Communicating and Collaborating A.nnotate Instructions for Academics Zoom in UTSOnline Construct Learning Pathways and Self-paced Learning Construct Learning Pathways and Self-paced Learning Adaptive Release How to create sets of questions and direct students based on their results Learning Modules Lesson Plan Differences between UTSOnline Subjects, Forums, and Communities Echo Universal Capture Frequently Asked Questions How Do I Download and Upload Grade Centre Data? Kaltura in UTSOnline Kaltura in UTSOnline Kaltura video assignment Managing Your Sites Managing Your Sites Add/Remove Users Available Sessions in UTSOnline Copyright and Reading Lists How to archive a UTSOnline subject Managing subject site availability Obtaining the UTSOnline site ID Renaming Your UTSOnline Site Staff Accounts Student Preview Mode Temporary Accounts UTSOnline site roles Marking and Grading Marking and Grading How to Upload Marks to UTSOnline Marking with Rubrics REVIEW FAQ The Gradecentre Turnitin and Feedback Studio Multi enrollment UTSOnline Forum Sites Qwickly Tools Suite Qwickly Tools Suite Qwickly Attendance Tool Qwickly Productivity Tools Using Qwickly Attendance REVIEW Staff User Guide Setting up the look and feel of your site SPARKPlus: moderating group work assessment Turnitin Turnitin Detecting similarity matches for consecutive student submissions in the same subject Detecting similarity matches to earlier submissions How to mark Turnitin assignments in UTSOnline Using H5P on UTSOnline (Blackboard) UTSOnline integration request UTSOnline Support For Staff UTSOnline Support For Staff IML Learning Technologists Working with Groups Working with Groups The Import Groups tool The UTS Batch Group Enrolment tool Known Issues Sitemap UTSOnline Support Home » Information for Staff » SPARKPlus: moderating group work assessment SPARKPlus: moderating group work assessment The SPARKPlus Group Contribution mode is the primary focus of this guide. About SPARKPlus Who is this guide for? What is SPARKPlus? Group Contribution mode Can I use SPARK for marks or grades? Set up a SPARKPlus rating session Request access or a new SPARK user account Duplicating a previous SPARK session Add a staff member to your SPARK subject Creating a new Group Contribution Mode setup When to run a SPARK rating session Rating categories and criteria Help students make effective comments Student group setup Adding students to a SPARK subject from Canvas or UTSOnline Bulk upload student groups into SPARK Student is in the wrong group Common questions answered What does SPARK capture? Is there a commonly accepted threshold where deviation from ‘average’ or equal contribution prompts grade moderation? Encouraging students to complete SPARK ratings Explaining SPARK to students Ratings sliders: why can’t the sliders be pushed all the way to the ends? Are raters identities hidden? Will students see individual comments? Should I publish ratings (and optionally, comments)? Going deeper Learn more about the most popular SPARK modes Other modes and features of SPARK Learn more about group work in your teaching Who is this guide for? The information on this page is aimed at UTS teaching staff wishing to conduct self and peer assessment activities with their students using SPARKPlus in Group Contribution mode. Talk to a learning designer or academic developer for your faculty if you wish to determine whether SPARKPlus can facilitate other types of activities in your subjects, or you simply require guidance on implementing group work effectively. The LX.lab and the learning and teaching Resources are a great place to find help. SPARKPlus is not just for teachers and students. Professional staff and researchers at UTS who have a legitimate need for the unique capabilities of SPARKPlus should make an IT Service request. The tips and recommendations on this page outline some of the ways you can use SPARK Group Contribution Mode effectively in your teaching. General functionality can be explored in the SPARKPlus user guide (ver 3.12). What is SPARKPlus? SPARKPlus is an online, multi-modal, feedback and assessment tool that combines self and peer assessment with sophisticated reporting. SPARK is an acronym for “Self and Peer Assessment Resource Kit”. There are several modes of operation in SPARKPlus hence the ‘plus’ in the name, but we will just simply call it SPARK. Key features: The ability to provide insights into collaborative and team based activities A way to capture self and peer feedback as the basis of additional reflexive activities or to support individual grades moderation in group work contribution (fairness) A blend of quantitative and qualitative measures (factorised group member comparisons along with supporting anecdotal reports) A variety of feedback options that can be published (and incorporated into reflection activities) Targeted and individualised reminders to non-participants Customisable (‘made to order’ setups) A Benchmarking mode where students’ assessments are compared to a benchmark assessment to help align standards. A Multiple Assessor mode Similar to benchmarking, but which allows rating across many participants allocated to all, or specific, assessable items. SPARK is best used as a low-stakes or formative assessment tool and for supporting activities that engage students in reflecting and in making effective judgements on themselves and others, or on objects, media, performances, events – anything. It can be used to create and capture flipped learning activities which form part of broader teaching approaches and strategies. For students, SPARK’s Group Contribution Mode can help address common complaints in group work: Equal marks for unequal contributions Free-riders known also as social loafers and passengers not getting penalised Better students inadequately rewarded and demotivated because they feel their efforts are ‘carrying’ the team. SPARK can play a role in group work and assessment to support the pedagogical approaches to developing student capability for evaluative judgement (Boud et al. 2018) and to help develop skills for reflexivity and self-regulation. “We do not learn from experience… we learn from reflecting on experience” ― John Dewey Back to top of page ↑ Group Contribution mode In Group Contribution mode, students assess both their own and each other’s contribution to the group work task. The students rate everyone’s performance against criteria and write comments (or feedback) to explain and justify their ratings. Group Contribution Mode focusses on the equity and the distribution of group work and can play a role in learning to be an effective collaborator. The students gauge everyone’s relative contribution to the process of delivering the group task. The ‘ideal’ is when all group members have been judged by each other to have equally contributed. Moving toward equal contribution in group work (fairness) Comments can be withheld from publication to students to encourage frank and honest reporting. The feedback and the rating scores (qualitative and quantitative information) that SPARK produces in group contribution mode can be used by the teacher to help inform the moderation of grades in group work activities. Benefits of using SPARK in group work: Helps students to make judgments, reflect and understand effective group work and feedback Promotes more equal and thoughtful participation (fairness) Can help identify dysfunctional behaviour to inform the moderation of individual student grades. Provides insight into the challenges of group-based learning activities Back to top of page ↑ Can I use SPARK for marks or grades? SPARK is very useful in terms of moderating marks given for a group project in Contribution Mode, but does not provide a reliable mark or grade that you can use directly. Instead the results in SPARK should only be used to help inform the marking process alongside other observations and evidence. SPARK is particularly helpful in locating those students who have been free-riding or disruptive, as well as those who have gone above and beyond the call. It does however require some considered interpretation of results by the teacher to spot situations where students have tried to game the system or where students simply picking on someone they don’t like. Keep in mind: Marks awarded for simply completing a SPARK activity serve no value in terms of learning. If necessary, you might link a small proportion of marks to the level of student engagement, the depth and quality of comments and generosity of feedback. Effective group work and team-based skills should never be assumed and may need to be scaffolded for the students. For more marks, a separate individually assessed self-reflection responding to feedback received through SPARK can demonstrate a student’s understanding of what they are doing well and what they can improve upon (ie. what they have learned). Back to top of page ↑ Request access or a new SPARK user account Subject coordinators can request a new SPARK user account through ServiceConnect. The Chief Instructor in SPARK (usually the subject coordinator) is responsible for managing basic instructor accounts in their own subject. Instructors (usually tutors) should request access to a SPARK subject directly from the Chief Instructor. Authorisation from the previous Chief Instructor or higher authority is necessary if you want a clone of a previous SPARK session where you were not the Chief Instructor. The authorisation for access needs to be provided in a separate request directly to ITD (Learning and Teaching Systems – LaTS) through ServiceConnect as an ‘IT Service request’. There are two roles (access permissions) for teaching staff: Chief Instructor and Instructor. Chief Instructor role (CI). This is the subject owner, usually the subject coordinator. Create new subjects Create Instructor accounts Add/Remove Instructors from a SPARK subject Modify subjects Connect/Enrol from equivalent UTSOnline subject Clone subjects (where user already exists as CI) Manage, upload students and create groups View activity logs (search log) Create new, modify, clone and delete task Delete Individual and Group Ratings Publish Results Instructor role (I) Similar to tutors, this role is set by the Chief Instructor. Manage, upload students and create groups Create new, modify, clone and delete tasks Back to top of page ↑ Duplicating a previous SPARK session Only chief instructor’s in SPARK can make a copy (a clone) of a previous session. Locate yourself within the old SPARK subject you want to clone: With the old subject selected where you are the Chief Instructor Click the ‘clone’ button Update the details for the new session (ie. session and year) Set the Chief Instructor if it is not you (careful, you wont be able to easily switch back) Choose the task options for copying (leave as it is if you want an exact copy) Submit. Then, to complete your SPARK setup for the new subject, you need to adjust the following as required: Set new dates and times for the rating period to take place within Create or modifying rating categories and criteria for tasks if necessary. Refer to the section ‘Setting rating categories and criteria’. Define student groupings for tasks. Refer to the section ‘Student group setup’. Add other instructors to this subject Finally ensure ‘Student access to task’ is set to ‘Active’. Back to top of page ↑ Add a staff member to your SPARK subject Subject coordinators (the Chief Instructor in SPARK) can add additional staff to their subject in the role of Instructor. Additional staff members must already have a SPARK user account. Select the Show button next to the Subject Instructors heading (under the Active Subjects listing). This reveals the list of staff currently in your SPARK subject. Select add to Subject to open the Add Instructors to Subject pop-up window. Select Search for instructors and search for the staff member by name or staff ID. Select the staff member from the results listing (repeat for any additional staff). Select the Add button to complete the process. Back to top of page ↑ Creating a new Group Contribution Mode setup It is the responsibility of the subject coordinator to set up and configure a SPARK session for a subject. If your subject has used SPARK previously and you were the coordinator, you can clone and update the previous version yourself. Subject coordinators who are new to SPARK must first ensure they have an account before any assistance with setup can take place. A consultation with the Learning Technologies Support team is the most effective way to get started for the first time in SPARK. When seeking assistance, be prepared to supply the following information: Subject name and number Name for the group task (this would usually be the same as published in the subject outline). Start and Finish dates/times for the SPARK session. Rating categories and criteria. A spreadsheet listing details of the student ID’s and group membership. To create student groups in large classes, you will want to ‘Bulk upload student groups’. Back to top of page ↑ When to run a SPARK rating session The rating period for a Group Contribution mode task is usually set to take place at the conclusion of a group work project, after task submission but before marking commences. The rating session is typically open for one week or more to allow all students time to submit a rating. Consider running an interim rating session roughly half way through a group work task. This gives students an opportunity to get familiar with the SPARK rating process. It can also assist in two important ways: Provides an opportunity for the students to engage with the SPARK criteria and to be reminded about what is important in their collaboration. Serves as part of a ‘health check’ for student groups to calibrate their group work if necessary and an opportunity for the teacher to monitor activity and to intervene if necessary. Read more about monitoring groups in Enhancing Experiences of Group Work. In that resource, a ‘Team Review Exercise’ for student groups can also be downloaded from the section on monitoring group dynamics. Back to top of page ↑ Rating categories and criteria In SPARK the Chief Instructor sets up a scale for criteria-based ratings. If you are familiar with REVIEW, these scales are similar to the marking sliders in REVIEW. There are default rating scales, but you can easily modify these or create your own depending on the nature of your SPARK session. An appropriate rating scale and criteria are necessary for the students to be able to make effective judgements about themselves and their peers. The criteria set by the Chief Instructor should be aligned with the learning objectives of the group work activity being assessed and the objectives of the subject as a whole. In other words, it should be clear to the students what and why they are rating. Criteria for self and peer assessment are grouped within overall categories, usually focused on a process or product. These will vary across disciplines and subjects. Categories might include, but are not limited to: contribution, teamwork, collaboration, quality of work, professionalism, leadership, inclusiveness etc. Avoid extremely large groups and/or too many separate categories and criteria if you want the students to fully engage, complete the ratings and write effective comments. Most SPARK instructors find that a group size of 4-6 students and 3-5 rating criteria are adequate. Some examples of categories and criteria are listed below in no particular order or relationship. You are strongly encouraged to develop your own that specifically relate to your own subject discipline and learning activity. Avoid ambiguity and stick to realistic, measurable things that describe the kind of group work behaviours valued in the assessment task. SAMPLE CATEGORIES Efficient functioning of the group Inclusiveness, diversity and interpersonal relations Leadership and responsibility Communication and coordination Collaboration and cohesion Professionalism Innovation and creativity SAMPLE CRITERIA Displayed an openness in engaging with other group member’s ideas and suggestions Demonstrated enthusiasm & participation (in-person and/or online) Actively participated in group meetings, on time and came prepared (attended in-person and/or online) Motivated others to participate, built trust (e.g. by listening and responding, encouraging others etc.) Helped resolve conflicts sensitively through negotiation and compromise Proactively and productively contributed to the group’s objectives (e.g. offering ideas, asking questions) Performed their allocated tasks on time and at the agreed level of quality Offered valuable ideas (e.g. about content, processes, relationship building) Followed through on agreed tasks and responsibilities, effectively and reliably Assisted others voluntarily, sharing information openly for everyone’s benefit Generated new ideas and offered alternatives in a constructive and sensitive manner Offered practical and actionable (task-oriented) feedback These examples are in the past tense for a final reflective rating session at the completion of a group assessment task. For interim SPARK rating sessions run while students are still engaged in their group work activities (perhaps during a group work ‘health check’), write the criteria in the present tense, eg. demonstrates, assists, helps, participates etc. Setting the categories and criteria can be very easily configured manually, but you can also upload a spreadsheet for a single bulk operation. Download this template and modify the contents in Excel to suit your particular task. Always save listings in tab-delimited text only format (.txt) for uploading to SPARK. The template has the following sample categories and underlying criteria (remember to change these for your own purpose): EFFICIENT FUNCTIONING OF GROUP Actively participated in group meetings, on time and came prepared Displayed enthusiasm and participation Performed their allocated tasks on time and at the agreed level of quality LEADERSHIP Motivated others to participate, built trust Offered practical and actionable (task-oriented) feedback Helped resolve conflicts sensitively through negotiation and compromise Back to top of page ↑ Help students make effective comments It is important that your students understand the value of providing and receiving constructive and generous feedback in group work prepares them for professional workplace settings. Opportunity to write comments exists in most of the SPARK modes, but the purpose differs. In Group Contribution Mode, encourage students to engage thoughtfully in explaining their ratings of peers, especially when they have rated a peer particularly high or low. Comments on a peer’s contribution are entered on the same page below the rating criteria sliders. Commentary on contribution ratings can help the instructor understand the context and reasons for ratings (or perhaps reveal sources of group conflict). Comments explaining ratings can be very important when they are made available (published) for the students to read. Publishing comments can also form part of a learning activity that fosters skills in both providing and acting upon feedback effectively. In this scenario students may need instruction on what effective feedback should be, that is: constructive, specific, kind (but honest), justified and relevant. Be mindful of how and when you choose to make student comments about each other available. The purpose of your SPARK activity will determine whether comments need to be revealed. Ratings and comments are always anonymous, up to a point. It is helpful to remind students about what they should focus on when adding comments to their ratings. Here are two example prompts that could be included in the task setup criteria. When group contribution comments are not published: COMMENT ON [peername] Comment here about [peername]’s overall contribution to your group, particularly if you rated them very low or high on criteria. Only teaching staff can see these comments. When group contribution comments are published (shown to students): Make constructive comments here about [peername]’s overall contribution to your group. Explain what they did well and what they can improve. [peername] will see what you have written for them, but they will not see who wrote it. These are Group Contribution Mode example prompts only. The [peername] placeholder text is automatically replaced by the name of peers in a group as long as you maintain the square brackets exactly as shown. You are encouraged to write a comment prompt appropriate to your assessment criteria and learning objectives. Back to top of page ↑ Student group setup Enrolments in your subject may change quite a bit in the first few weeks of semester so delay group formation if possible until enrolment is stable – usually by around week four or census date. SPARK works best with groups of 4-6 students (anonymity cannot be assured in small group). Group membership can be changed easily in SPARK up until the point at which ratings start, but thereafter group membership changes present challenges. Setting groups up correctly requires a specific two-column spreadsheet format. For small student cohorts, you can manage groups and student membership manually within SPARK. An accurate groups listing is essential to avoid delays in setup. To avoid changes to group membership, wait a few weeks (until after census date if possible), or right up until you run you run a SPARK rating session to create student groups in SPARK. The latest class lists from MSA+ or your UTSOnline subject are accurate sources of student enrolment data. Groups created in UTSOnline can be exported and adapted for use in SPARK. Back to top of page ↑ Adding students to a SPARK subject from Canvas or UTSOnline Adding students to a SPARK site from Canvas Staff teaching in Canvas who wish to use SPARK will need to add their students to their SPARK site via file upload. Adding students via file upload will create SPARK accounts for those students who do not already have them. It is not currently possible to export the class/enrolment data required for SPARKPlus from Canvas. The following headers and data are required to add students to a SPARK site via file upload (save a copy of this template and populate the fields in Excel). IMPORTANT: Save the file from Excel in tab-delimited plain text (.txt). IMPORTANT: If a student ID contains one or more leading zeros (e.g. 0xxxxxxx) then: In Excel, format the cell containing the student ID as text Check that the leading zero(s) of the student ID are visible in Excel. If not, add them manually Export the file from Excel and upload it to SPARKPlus as normal If the leading zeros are not present when the student’s SPARKPlus account is created then they will not be able to log in and enter ratings. FirstName LastName AccountID Email John Citizen 12345678 john.citizen@student.uts.edu.au This data can be obtained by running a custom report in My Timetable (a.k.a. Allocate+ or MSA+) and selecting the following fields: STUDENT.FIRST_NAME STUDENT.LAST_NAME STUDENT.STUDENT_CODE STUDENT.EMAIL_ADDRESS We recommend that you perform this task after the census date for the teaching session. Doing so will make it more likely that the enrolment data is more accurate, as students cannot enrol or withdraw without penalty from a subject after the census date. To import the student data: In SPARK, ensure the correct subject (not the task) is selected under the Active subjects. Click the Upload button in the Enrolment section. Choose the tab-delimited plain text (.txt) file containing the student data and click submit. SPARK will confirm that the file was uploaded successfully and display the following information: Existing accounts New accounts Existing registrations New registrations If you are updating existing enrolments then you will be given the ability to dis-enroll any students not listed in the new file. To do this, deselect the checkbox for any students you wish to dis-enroll and click Delete. Otherwise, click Close to complete the upload process. Adding students to a SPARK site from UTSOnline Staff can add students to a SPARK site from UTSOnline. Follow these steps from within SPARK to add students to a SPARK site: Ensure the correct subject (not the task) is selected under the Active subjects. Click the Settings button in the Enrolment section. This opens a UTSOnline Settings pop-up window that may briefly display the message Connecting to UTSOnline (UTSOnline is where SPARK draws enrolment information). Be sure to select the correct subject/session from the displayed listing (it should be displayed in a format similar to: ‘654321-2019-SPRING-CITY’) Make a selection as instructed below the subject listing. The default is usually OK – Remove students who are un-enrolled when enrolments are next updated. Click the ‘Save’ button, then the subsequent ‘Close’ link that pops up. This returns you to the main window. Select ‘Update Enrolment’. If you get an error message because there are already students in this subject: First delete all current students via the ‘View’button. In the pop-up window select all then click delete. Select ‘Update Enrolment’ again. Use the ‘View’ button to examine your student list. Sometimes UTSOnline Student_View accounts are also imported. You can delete these, or simply ignore them. As long as these types of users are not placed into a student group in a task, they won’t be able to do anything. Back to top of page ↑ Bulk upload student groups into SPARK To bulk upload group listings for a task into SPARK you will first need to create a two-column spreadsheet with the following headers: ‘AccountID’ and ‘GroupName’ (AccountID’ is same as student number and ‘GroupName’ can be anything with letters and numbers only). Create a spreadsheet in Excel (or download this simple template) and enter the details under the headers described above. Save this spreadsheet listing out as a tab-delimited plain text file (.txt). This is now ready for upload it directly into SPARK: Access SparkPlus via https://spark.uts.edu.au/ Select on your subject Select your assessment task In the ‘STUDENTS AND GROUPS” section, navigate to the ‘File of students in groups’ row Select ‘Upload’ to create your groups in the current task Follow the prompts to upload the tab-delimited .txt file of the groups. Back to top of page ↑ Student is in the wrong group If a student is in the wrong group, a change to group membership is only possible if rating has not already begun. Once students start the rating process, groups are locked and the only option is to conduct a workaround for the affected students/groups. To make changes to group allocation BEFORE the rating process has started: Select Manage Groups while viewing the affected task to open the Group Editor pop-up window. Select the affected group(s) in the drop-down menu Add or Remove students using the buttons in the middle of the two columns. Close the floating Group Editor window (there is no save button). Workarounds if students have already started rating each other are to: (A) Cancel the SPARK session, make the group changes and start the session again. Because this negates existing ratings, consider this option ONLY if none or just a couple of students have rated so far. Those same students will have to be asked to conduct their ratings all over again and that is not feasible with many students and/or lengthy question sets. (B) Allow the SPARK session to run through to completion. Only the groups with errors in membership will be compromised. Ignore (and preferably switch off) the ratings for the affected individuals in the wrong groups. Have the affected groups provide either paper-based responses, or have them send individual emails to you with the ratings and comments. You wont have any numerical results in SPARK for students in the wrong group. (C) Delete all the ratings for the group(s) the student is being moved out of. Students in that group who’ve already rated will then need to re-enter their ratings. You cant re-enter the comments/ratings on behalf of those students who have already rated, but you can download them before deleting the results. The members of the group that the student is being moved into will also need to go back in and add ratings/comments for the new student. To delete ratings and re-start affected groups: Select the task you want to delete ratings from in the ‘Setup’ tab, then select ‘Change to Inactive’ (under the ‘Student Access To Task’ section) Go to the ‘Results’ tab and select the ‘Delete Group Ratings’ option (under the ‘Students And Groups’ section) Select the group you want to delete ratings for and select ‘Delete Group Ratings’. Back in the ‘Setup’ tab, select ‘Manage Groups’. Move the student(s) to the correct group(s) using the ‘Group Editor’ pop-up window. Select ‘Change to Active’ (under the ‘Student Access To Task’ section) Back to top of page ↑ What does SPARK capture? SPARK produces two quantitative indicators from the student self and peer ratings representing contribution and feedback: RPF and SA/PA. These are shorthand for Relative Performance Factor and Self Assessment relative to Peer Assessment. The sentiments captured in the comments that students make about their peers to justify the ratings they allocated are very important. Often these comments help explain the numbers. The factors when considered alongside the student comments help you get an insight into the group’s dynamic. This anecdotal evidence can help inform the moderation of individual grades in group work assessments. Relative Performance Factor (RPF) is the student’s rating relative to the group average rating. The RPF factor is usually interpreted as the student’s contribution compared to the average contribution for the group. If the performance factor is less than 1.0 then the student performance, as rated by peers, is lower than the average performance for the group. The feedback factor (SA/PA) is The Self Assessment relative to the Peer Assessment, or the student’s self-rating relative to the average of this student’s rating by peers. If the performance factor is greater than 1 then the student’s self-rating of contribution is greater than his/her rating of contribution as given by peers. Interpretation of these factors requires care and thought if they are to influence the weighting of individual marks for assessment of a group activity. The term ‘average’ in SPARK doesn’t mean a student performed ordinarily (or was mediocre), it actually means relative to the other group members they were equal contributors in effort. TIP: When configuring a SPARK GCM task, rename the default “AV” (Average) rating label to “EQ” and the text to “Equal to others” to limit confusion around the meaning of the word ‘average’. SPARK can help reveal that a problem exists within a group, but it doesn’t necessarily pinpoint where a problem lies or what the exact causes are. Early intervention with student(s) is recommended in order to address group work issues. The RPF reflects how a student’s performance compares to others in the group (across all criteria). A value greater than 1.0 suggests they are perceived to be performing above the group average. If it is less than 1.0, the student may be performing below the group average. SA/PA tells how a student’s self ratings match up with their peers’ ratings. If it is greater than 1.0, it means their self appraisal is higher than how their peers’ have viewed their performance. If it is less than 1.0, the student’s peers view their performance more positively than they do of themselves. You may want to seek guidance from the LX.lab Support team when interpreting final SPARK ratings. Back to top of page ↑ Is there a commonly accepted threshold where deviation from ‘average’ or equal contribution prompts grade moderation? The nature of your SPARK configuration, your students group dynamic and their approach to the SPARK rating process might influence your decision on what constitutes equal contribution in a group task – the norm. In large classes anyone below .90 and above 1.10 on the RPF scale might be considered for moderation. You can disregard ratings within this range (between 0.90–1.10), and assume all team members are roughly contributing equally to the group task. This generous threshold helps limit the number of students you need to look at closely. In a small cohort of students you can afford to examine every students ratings and comments. In this case you might narrow the range for what you consider is an average (or equal) contribution to between .95 and 1.05. A high SA/PA (feedback factor) can reveal a student trying to ‘game’ or ‘sabotage’ the ratings by overrating themselves. This factor helps reveal the low contributing student who is trying to inflate their own efforts. This behaviour skews the overall result for the other group members. In this scenario you can exclude that student’s rating from the overall calculation and the relative performance within the group will return to a more balanced representation. In all cases where the ratings are outside the band (or norms) you have set, it is necessary to look closely at the student comments for evidence supporting the high or low scores. When used effectively, SPARK can help inform the grade moderation process in group work. The decision to moderate a student mark always rests with your judgement as the subject coordinator. Back to top of page ↑ Encouraging students to complete SPARK ratings Meaningful ratings in Group Contribution Mode can only be produced if all—or most—students in a group complete their ratings. Explain this to students as the best way to achieve fairness in group work. You might consider a small penalty for non-raters. If the intention is to publish ratings and comments (de-identified), explain to the students that anonymity is more likely as the number of participants increases (beyond three). This may nudge students in larger groups into completing the SPARK rating. You will need to regularly monitor progress during a rating session and use the “email non-submitters” button on occasion to chase up those who have not yet rated. Explain to those students the value in participating and the potential consequences if they don’t (note: ratings cannot be entered on behalf of a student). Explaining to students the practical value of developing skills in evaluative judgement can help justify the purpose and value of full participation in a SPARK rating activity. Think carefully about giving ‘marks for participation’ simply to motivate students. This is like giving students marks for simply handing in an assignment – there is no learning involved and nothing to really assess. Back to top of page ↑ Explaining SPARK to students How and when will you be explaining SPARK to your students? Ideally you should allow time for you and your students to learn what you need to know about SPARK before using it — and before the students begin their group activity. Support and assistance with SPARK is not always available at the last minute so allow plenty of time to prepare. Students should know before they start on their group task how and why SPARK is being used – especially if it leads to a moderated grade. When students know that they are being rated by their peers they might realise that free-riding and other dysfunctional behaviours are likely to incur a cost. If you are intending to use SPARK to help inform any moderation of students’ marks you should communicate this in class and in your subject outline. This can help students see that a lack of participation and contribution *may* have an impact on their final grade. Explicit categories and rating criteria in SPARK help students understand what is expected of them in the group work process – explain these up front. A dedicated discussion forum is the recommended way to manage questions from students about SPARK. Download this basic guide to rating to share with your students. It outlines the basic steps necessary for students to complete a standard SPARK rating session in Group Contribution mode. Place this somewhere appropriate in your Canvas subject and point this out to the students (the sample email below addresses this in paragraph three). Below is an example email announcement to students. Replace the parts in curly brackets as required. SPARK for {Subject/Task} is now open – participation is mandatory. It is time to start assessing your group member’s contribution to {task name here}. A SPARK session is now available at ‘spark.uts.edu.au‘. It must be completed no later than {date here}. There are instructions to help you access and complete the group contribution survey. These instructions can be accessed in the {explain here where in your UTSOnline subject site the instructional guide can be found}. Your SPARK contributions can have an impact on final grades. Constructive feedback you provide about your teammates is {anonymous}{anonymous and will be published}. Be fair, realistic and respectful in your assessments of yourself and others. Credit teammates who are generously contributing and respectfully note (with context) any teammate who is not doing their fair share. Regards, {coordinator’s name here} Back to top of page ↑ Ratings sliders: why can’t the sliders be pushed all the way to the ends? A common source of confusion for students is when forced norms is ‘on’ and they are expecting to be able to drag the rating sliders to any position. This misunderstanding of the SPARK slider behaviour is usually when students think that they are marking each other. In GCM each student is making judgements on their peers’ contribution to the group work process and allocating what they believe to be an appropriate ’share’ of the overall effort with respect to ‘average’ or equal contribution. The forced norms setting encourages the students to consider the effect of their rating in the context and spirit of collaborative work where equal and fair group work contribution is valued. When ‘on’ the forced norms setting helps reveal students from simply rating themselves high or rating someone else low – thats is, gaming or sabotaging the results, or perhaps not taking the task seriously. Photo by Annie Spratt on Unsplash Think of GCM in SPARK like slicing up a cake, where everyone helps decide how the ‘group work cake’ is divided. If you allocate equal slices for everyone this implies all members performed equally well (and the rating slider ‘averages’ toward the middle). If you give one student a bigger slice than everyone else, then logically, there is less cake left to be split up with the rest of the group (and the slider adjusts to reflect that unequal distribution). It becomes very obvious when a student has attempted to take the biggest slice for themselves! Turning forced norms off can be an appropriate option depending on the nature of the task and the capacity of the student cohort to engage in a GCM rating session with honesty, integrity and a realistic appreciation their own and their peer’s contribution. Back to top of page ↑ Are raters identities hidden? Will students see individual comments? Ratings and comments are always anonymous. Students will only see comments if you select that option in the task setup and then take the extra step to publish the final results. Although student names are not published alongside the comments made, it is very important to realise that it can be very easy for students to work out who made particular comments, especially in small groups. Some students may have been unguarded and ruthless in their comments on each other. If you choose to publish comments alongside ratings, you will need to moderate the comments before publishing – which can be time consuming in a large subject. Back to top of page ↑ Should I publish ratings (and optionally, comments)? There is no need to publish results if your purpose is to simply explore potential cases where you need to moderate student grades. This may either due to a lack of contribution, or less frequently, a widely acknowledged generosity of contribution in a group assessment task. Since we are seeking a measure of equality in the relative contribution of students in group work, avoid rewarding students who through dominating, overbearing or sabotaging behaviours, appear to have done more work than their peers. This behaviour is also undesirable in developing effective and professional teamwork skills. Don’t publish comments unless it presents a learning opportunity where you have adequately scaffolded students with the skills to write and reflect on feedback in a professional manner. You might intentionally run a SPARK session ‘in the open’ as a peer assessment exercise, where students are required to provide constructive feedback to each other. They may then take that further by individually reflecting on that feedback, in which case the students need to see each other’s comments in order to frame their own response. Apart from an exercise in developing effective teamwork skills, this approach might be for class presentations, role plays, pitches, 360° reviews, interview and negotiation skills and so on. Seek advice from a learning designer on the best approach and SPARK mode for your activity in these cases. Back to top of page ↑ Learn more about using SPARK If want to explore how SPARK can be used in your subject, or simply need a refresher in managing SPARK for yourself you can ask for help at the LX.lab. Occasionally there are workshops on using SPARK with a focus on Group Contribution Mode. General functionality is explained in SPARKPlus user guide (ver 3.12 July 2019) Back to top of page ↑ Other modes and features of SPARKPlus The modes support publishing options that allow students to compare their judgement, answers and reasoning with instructors and/or peers. –SPARKPlus developers Student Benchmarking Individual student assessments are compared to a benchmark assessment and peers. Benchmarking provides an opportunity to refine judgements on anything against assessment criteria (a report, design, performance, film etc.). As a formative task, students learn how to assess work and, in turn, become aware of the strengths and weaknesses of their own work. When modelled on a forthcoming summative assessment task, students are better prepared to do well. Multiple Assessor Similar to benchmarking, but allows rating across many participants allocated to all, or specific, assessable items. All the assessments are placed on a scale in a distributed comparison indicating the minimum, maximum, average and standard deviation of participants’ ratings, making it easy to identify the criteria where there is general agreement and those where participants have quite different opinions. Supports identified or anonymous reporting. Used for benchmarking, standards construction and collaborative assessments for example of submissions, applications or even tenders. Survey/Formative Review Same as multiple choice except not graded and participants can be asked to provide consent (useful for research projects). Multiple Choice Facilitates multiple choice questions, with or without explanation of reason for chosen answer and free response questions. Multiple Choice – Multi Attempt Multiple choice questions where you specify the number of attempts allowed. Immediate feedback is provided. Flexibility to determine mark for the attempt they chose the correct answer. Clicker Student use mobile phones, tablets or PCs to answer questions in class, results displayed via histogram, comments categorised for different answer responses. NOTES: Multiple choice and Clicker modes are not often used at UTS given that type of functionality is largely addressed by other tools. SPARK quizzes (like all the other modes) are best used in formative, flipped and classroom engagement activities. A unique feature of SPARK’s survey and quiz modes is the ability to summarise results visually in a histogram to show the distribution of answers. This provides an opportunity for the teacher to gauge understanding and for lively classroom discussion. Request a consultation with an academic developer or learning designer to explore whether you have a learning and teaching, or research activity, that SPARK can help you with, or whether there are better suited tools for your purpose. Back to top of page ↑ Learn more about group work in your teaching To learn more about group work in your teaching, consult this online resource to help lecturers and tutors manage and motivate student groups, or read an overview of assessing group work. Get help through the LX.lab and explore the learning and teaching Resources on collaboration, student engagement and active learning. Find articles tagged with ‘group work‘ or ‘groups‘. This post on Learning about group work is a compilation of links and further reading. The UTS HELPS website has material you can share with students to help them make the most of group work activities. Pages 5. Grade Submission Automation – Q&A & Troubleshooting Are they checking? Calendar & date management – UTSOnline Capturing Webcam video in Windows Movie Maker Change to online.uts.edu.au landing page Changing the ‘look & feel’ of your subject in UTSOnline Chemistry notation support Clickers for Engaging Large Classes Configuring REVIEW for Grades Submission Consent Matters Content Layout Tool New Features Copy & paste in the new UTSOnline Embedding a YouTube video in UTSOnline FEIT – Safety and Wellbeing Essentials Forumeng32 G Suite: Google Drive plus more for UTS staff/students Getting Ready in UTSOnline Getting your video onto YouTube and embedding it into UTSOnline Grade centre improvements – UTSOnline How do I make my video in YouTube unlisted? How to create “Camera facing” video Improved Mobile Experience Inline marking & rubrics – UTSOnline Intermittent issue with students uploading Turnitin assignments in Blackboard when using Safari ITSC Reference Known Issues Cannot hide the Quiz attempt score from students Captivate issues with embedded YouTube video Chief Instructors in SPARK cannot select the correct UTSOnline subject as an enrolment source Comments in the inline document viewer are partially hidden Content Usage Statistics Report issue Course Activity Overview and Student Overview for Single Course reports shows no data Deleting groups linked to group assignment results in inaccessible assignment submission Document does not open in the inline document viewer in the Blackboard Assignment tool Emails received from UTSOnline shows additional email address Error when accessing assignment submissions in Bb Grader Group journals are visible to all enrolled students in a UTSOnline site Incorrect Rubric display in graded Portfolio Artefacts Last Access Date for a student is incorrect or is not recorded LaTex Content is not rendered automatically Missing scroll bars in Grade Centre on Mac Opening the Last Graded view in the MyGrades tool generates an error Opening the Test Options page generates an error Original draft version is displayed in a Blackboard Assignment rather than the latest submission Portfolio instructions do not appear when editing your Portfolio Statistics Reporting for a content item contains incorrect data Subject Coordinator role not supported in Blackboard Instructor app Subject Coordinator unable to use the UTS Batch Group enrolment tool Submitting Group Assignment Attempt Results in Error when Group Contains a Disabled Enrollment Turnitin submission error – An unusual number of excessively long or short words Turnitin submission error – Gradebook was not modified Unable to attach files in Internet Explorer 11 Unable to open context menu in Google Chrome Unable to submit content using Safari 11.1 Visual bug on Test Presentation when help text is off Zoom cloud recordings are not showing in ‘Gallery View’ Audio auto-play in Chrome browser Files attached to Blackboard Blogs & Journals not opening due to filename format Even if scroll bar is displayed user unable to scroll Filter Function does not work in the Announcements module Line break issue Internet Explorer 11 font issue Missing equation content when marking assignment Survey results downloaded file issue Turnitin bulk download issue Learning and Teaching System access for Faculty Administrators Making short videos to present content: The process Making Video – General Guidelines for Best Practice Marine science flipped Pencast video Placemat Preparing and optimising your video for teaching REVIEW and SPARK similarities and differences REVIEW new features REVIEW November 2017 Upgrade Summary REVIEW was upgraded to version 3.6.11 on February 27 2019 Search Results Setting up a YouTube account to upload videos to Shooting and uploading video to YouTube using a Smartphone or Tablet Similarity Scores not updated after Assignment Due Date Site design and student engagement Sitemap Smartviews and groups in UTSOnline SPARKplus upgrade from v3.05 to v3.12 Student Portfolio Overview Subject Coordinator unable to create and configure Campus Pack content Think – Pair – Share Tips on creating formative quizzes Tips on preparing and assigning reading work Tips on using media in your online space Using Audacity for your Soundtracks UTSOnline and Qwickly Tools UTSOnline Calendar UTSOnline Exam Invigilators Guide UTSOnline is temporarily unavailable SPARKPlus is temporarily unavailable UTSOnline Regular Maintenance Window UTSOnline Tools Value lines Voice Narrated PowerPoint using Office 365 for PC Zoom Meeting Guide Information for all users For students Avoiding Plagiarism for students Downloading a Subject Outline in UTSOnline How to successfully submit a UTSOnline assignment SparkPlus for students Turnitin for students Kaltura Why Publishing Medias? Restrict access to your media with Scheduling Download your media from Kaltura Kaltura MediaSpace: Use Channels to Group and Control Access to Your Media Supervisors Access to MyMedia of Kaltura Kaltura Closed Caption Privacy Statement for UTSOnline Navigating UTSOnline The My Subjects module Logging into UTSOnline Collaboration Tools Campus Pack Portfolios UTSOnline Support Blackboard Mobile and Tablet applications Supported Operating Systems and Browsers Information for Staff 1. Grades Submission Automation – Getting Started 2. Grades Submission Automation – Setting Up Your Grade Centre for Automation at the Start of Session 3. Grades Submission Automation – Finalizing and Submitting Your Grades at the End of Session 4. Grade Submission Automation Processes for Faculty Admin & Support Automated UTSOnline Subject Creation and Enrolment Forms for Subject Coordinators Session key dates in UTSOnline Construct Learning Pathways and Self-paced Learning Adaptive Release How to create sets of questions and direct students based on their results Learning Modules Lesson Plan Echo Universal Capture Frequently Asked Questions How Do I Download and Upload Grade Centre Data? Kaltura in UTSOnline Kaltura video assignment Multi enrollment UTSOnline Forum Sites Qwickly Tools Suite Qwickly Attendance Tool Qwickly Productivity Tools Using Qwickly Attendance REVIEW Staff User Guide SPARKPlus: moderating group work assessment Turnitin Detecting similarity matches for consecutive student submissions in the same subject Detecting similarity matches to earlier submissions How to mark Turnitin assignments in UTSOnline UTSOnline integration request Adding Your Subject Outline to UTSOnline Using H5P on UTSOnline (Blackboard) Differences between UTSOnline Subjects, Forums, and Communities Managing Your Sites Available Sessions in UTSOnline How to archive a UTSOnline subject Managing subject site availability Obtaining the UTSOnline site ID Renaming Your UTSOnline Site UTSOnline site roles Staff Accounts Temporary Accounts Removal of temporary UTSOnline accounts Copyright and Reading Lists Student Preview Mode Add/Remove Users Setting up the look and feel of your site Working with Groups The Import Groups tool The UTS Batch Group Enrolment tool Building Content Adobe Captivate Content Layout Tool Basic Tiles Enhanced Tiles Flip Cards Home Page Welcome Tiles Lesson Structure and Layout Elements IML Sound Booth LinkedIn Learning Managing the availability of Echo360 lecture recordings Reading Lists How to make a video or screencast – examples Practical Ideas – Video Lecture capture with Echo360 ALP Communicating and Collaborating A.nnotate Instructions for Academics Zoom in UTSOnline Building Assessment Tasks Changes to inline document markup in the Blackboard Assignment tool How to access Review Portfolio Assignments for Staff Quiz recommendations UTS Avoiding Plagiarism Marking and Grading How to Upload Marks to UTSOnline The Gradecentre Marking with Rubrics REVIEW FAQ Turnitin and Feedback Studio UTSOnline Support For Staff IML Learning Technologists UTSOnline Help Level Of Use Archives December 2017 March 2016 February 2016 January 2016 October 2015 September 2015 August 2015 July 2015 April 2015 February 2015 November 2014 Categories AC2016 (2) Creating video (8) For Students (1) How-to videos (14) Learning design (6) Learning.futures (8) Practical examples (6) Stories (1) Tools (14) What's new? – UTSOnline (1) WordPress Log in WordPress Subscribe Entries (RSS) Comments (RSS) Links UTS homepage UTS library Current students Staff Connect © Copyright UTS - CRICOS Provider No: 00099F - 2 March 2017 9:32am. This page is authorised by the Director, Institute for Interactive Media and Learning.
