FLO Upgrade Path 2012 Requirements Analysis May 2010 Table of Contents Introduction................................................................................................................................................... 1 Summary of requirements ............................................................................................................................ 2 Description of requirements ......................................................................................................................... 3 FLO should: align with Flinders University’s strategic objectives.............................................................. 3 FLO should: align with university structure and governance .................................................................... 3 FLO should: run on multiple client‐side platforms.................................................................................... 4 FLO should: provide flexibility in assessment options .............................................................................. 5 FLO should: provide flexibility in reporting options .................................................................................. 7 FLO should: provide an engaging and user‐friendly interface .................................................................. 8 FLO should: provide an open and extensible platform ........................................................................... 10 FLO should: support collaboration and group work................................................................................ 12 FLO should: allow for content sharing and re‐use .................................................................................. 13 FLO should: support the University’s policy on accessibility................................................................... 14 FLO should: support the implementation of University‐wide initiatives ................................................ 15 Introduction This document details the requirements for a new implementation of FLO, to be launched at the start of 2012. The requirements have been sourced from a number of areas. Firstly, an extensive brainstorming session was conducted with FLO faculty support staff, to determine what they saw as the key features and functionality required. Secondly, all staff and students who currently use FLO were invited to complete a survey about their opinions of the current version, and their ideas for improvement. A targeted number of staff and students also participated in focus groups, where they elaborated upon these ideas in more detail. A number of academic staff members sent in email correspondence with further ideas and comments. Qualitative data analysis was undertaken to identify “themes” or common ideas in the feedback received. These themes form the basis of this requirements specification. In addition to feedback from stakeholders, institutional policies and projects were taken into account. These include the need to support policies on areas such as copyright and accessibility, and various university‐wide initiatives including the Web Presence in Every Topic (WebPET), ePortfolios, and text‐ matching software. It is to be noted that the WebPET guidelines are currently under review, with a view to expanding the minimum standard for every topic. A discussion paper on what these new standards might include was used as a basis for several identified requirements. The requirements listed in this document are intentionally broad in nature, and should be used as a guideline for discussion of potential implementations of FLO. Note: Themes were identified that are not pertinent to requirements for the product underpinning FLO, rather they identify key areas where we may improve the administration of FLO, and enhance the provision of support and other services. All such feedback has been noted and will be forwarded to the relevant parties for consideration. 1 Summary of requirements The following section identifies requirements for the new implementation of FLO. It is to be noted that “FLO” does not necessarily refer to a single product (a learning management system). Rather, these requirements may be fulfilled by a number of integrated systems, including a traditional learning management system along with third‐party commercial or open‐source software, and/or tools developed in‐house. FLO should… 1. align with Flinders University’s strategic objectives 2. align with university structure and governance 3. run on multiple client‐side platforms 4. provide flexibility in assessment options 5. provide flexibility in reporting options 6. provide an engaging and user‐friendly interface 7. provide an open and extensible platform 8. support collaboration and group work 9. allow for content sharing and re‐use 10. support the University’s policy on accessibility 11. support the implementation of University‐wide initiatives 2 Description of requirements FLO should: align with Flinders University’s strategic objectives Key stakeholders: CEdICT, DVC(A) For example: • increase accessibility for all students to an enhanced student‐learning experience • support the university increasing the number and nature of topics, courses and programs that are offered through various flexible delivery modes • support the university disseminating and embedding good teaching practices across the university • represent the minimum web presence in every topic (WebPET) Supporting data: • Appendix I: Strategic Plan 2010‐2014 FLO should: align with university structure and governance Key stakeholders: FLOAGs, staff members For example: • allow support staff (eg FLOAGs, student help desk) to be responsible for maintaining and able to directly support their own areas of the system • map FLO roles/permissions with Student Two roles/permissions • provide flexibility for topic coordinators to control or devolve responsibility over maintaining their site • use Flinders‐specific terminology • maintain an internal representation of Flinders’ organisational structure • support Flinders branding • align with practices of granting users access to sites • provide acceptable levels of data security • aid compliance with copyright policies, academic integrity Supporting data: • Features requested by FLOAGs (Appendix A): o sites: DAGs can create / remove, Library own areas o use own role names (topic coordinator, tutor, etc), create and describe our own roles, map custom roles to SIS roles o define own terminology 3 o map to university’s structure with flexibility (eg accommodate SoM departments), accommodate non‐university uses (eg IELI), organise into teaching & non‐teaching o ability to theme / brand • Staff focus groups (Appendix E): o support: copyright management – something that makes it easier for users of FLO to adhere to copyright requirements – perhaps make it more prominent (without policing nit). Maybe an icon as an addition to WebPET. Risk management. • Copyright policy (Appendix J) FLO should: run on multiple client‐side platforms Key stakeholders: FLOAGs, staff members, students For example: • run on most (if not all) browsers and operating systems • access via mobile devices • should not require the installation of additional software (eg Java, Flash) to perform standard tasks • provide an unobtrusive method of testing system requirements • provide an equally satisfying and useful user experience on different platforms/devices • adapt to new platforms as they are released • able to be accessed from behind firewalls Supporting data: • Features requested by FLOAGs (Appendix A): o less reliance on java / browser requirements, multiple browsers / OSs (inc Linux), platform agnostic o no java applets o quicker log in, less reliance on accepting certificates • Student survey (Appendix B): suggested improvements o compatible with more browsers o less reliance on java o no popup browser checker o access from mobile devices • Student focus group (Appendix C): feedback o FLO works great at uni but when at home not so good, due to plugins (Java, correct browsers, pop‐ups blocked) missing • Staff survey (Appendix D): suggested improvements o less technical requirements o compatible with Safari, Firefox 4 o less reliance on Java • Staff focus groups (Appendix E): o from support/admin group: suggestion to improve – FLO working differently on different computers • Horizon Report (Appendix M) o identifies mobile computing as a key trend FLO should: provide flexibility in assessment options Key stakeholders: FLOAGs, staff members, students For example: • wide range of assessment types (eg quizzes, assignment submission formats) • wide range of question types • flexible grading forms • flexibility in requesting and granting extensions (to individuals or groups) • security options (IP blocking, proctor password) • flexible marking and feedback options (including the capability for online marking) • flexible peer assessment functionality • assignment dropbox, including clear notification of successful submission • text matching software, incorporating an expansive database • provide flexibility in file types submitted • provide good quiz randomisation and branching options, with the possibility of narrative Supporting data: • Features requested by FLOAGs (Appendix A): o grading options: more and more flexible, grading form that doesn’t have marks – to apply with a grading rubric, consistent view for grading o customisable start page to quizzes with html / surveys, self‐tests o keep / recording scores for self‐tests o settings customisation / options for quizzes, self‐tests o question database – local design, flexibility – separation of quiz database from survey database; ability to create question type, iterative question types, ability to import questions (having formatted them elsewhere) o being able to create media rich quizzes – video clips, etc, multimedia integration with gradebook (eg flash) o file attachment ability o security settings, eg proctor password, ip masking 5 o feedback to students – easy to use: hints, different modes of feedback (eg coaching), rich media feedback (eg audio comments) o apply extension to individual or group of students (through assignments and/or assessments) o request an extension (with controls) – quiz / assignment o quizzes: branching questions, parameterised questions o individual and group assignments o types of assignments: text & google docs, assessable discussions (better interface when viewing one’s contribution to discussions for grading), grading of a variety of assignment/assessment types (eg wikis) o ways of assessing different types of assignment, particularly marking essays online and providing feedback online o assignment dropbox – easier resubmission • Student focus group feedback (Appendix C): o quizzes: heavily used in science – quite painful to use – issues with passwords and expiry dates and sometimes lecturers can’t reopen them. if you spell a one word answer wrong it will mark you wrong • Staff survey: suggested improvements (Appendix D): o assignments: better sorting required, ability to take back file if posted to wrong student, easier to use, allow more submission types (eg video files), include cover sheet, use in conjunction with marking forms) o gradebook: make it more flexible o quiz: more flexible quiz options, easier to change for individual students (reset, time limits), make it easier to use, misc • Staff focus groups (Appendix E): o Power users: online exams – security issues. Students need to be blocked from going to outside world or talking to each other. Locking down to particular hour, lab, etc. and partition off from internet and each other. o Power users: peer assessment – really cumbersome in FLO. Quiz per student and others do the quiz to assess that student – good cause it goes to gradebook – student doesn’t see feedback and need to setup a lot of quizzes. Or students posts self‐appraisal and then students can grade this – but it doesn’t send grades to gradebook. o Power users: quizzes – progressive scheme. o Power users: quizzes that open at different times for different students is cumbersome. Managing groups and quizzes is hard. o Assignment submission – if you allow them to take it back it will only let them edit – whereas the most common problem is that they submitted the entirely wrong file and editing doesn’t help them. • Correspondence from academic staff (Appendix F): o I use FLO to administer an online test to students. My ability to set up a test so that it randomises questions is difficult and it is too easy for students to copy the test and therefore cheat. So I wonder if it is possible to make it easier to 1. randomise questions and answers and 2. to make it impossible to 'screen print' or copy the questions. At them moment, it will randomise answers, but not the questions (at least not easily). 6 o The FLO options also do not appear to allow me to put an overall time limit on the test‐ it is either unlimited within a certain time frame (for example, the test can be 'open' between certain dates, but students can enter/exit multiple times) or it is a set time limit (for example, 1 entry and complete within a set time, e.g, 1 hour), but nothing in between. I'd like to be able to let students have multiple entries, but have the option to set the number of entries, say for example, so I could say you can enter the test 3 times over the 5 days that it is open, and you have a maximum of 3 hours to complete the test. At the moment, it doesn't let me do that. o Finally, at the moment, if I am looking at students assignments, etc. it alphabetises according to the FIRST name, rather than the surname. Obviously that is ridiculous. We also do not have student ID numbers in FLO which is critical when students have very similar or identical names. o I would like to see peer evaluation systems to be built into FLO • Educational Technology Policies, Guidelines and Projects (Appendix J) o text matching software • Expanded WebPET draft guidelines (Appendix K) o assignment dropbox o online marking FLO should: provide flexibility in reporting options Key stakeholders: FLOAGs, staff members For example: • inform regarding changes in enrolment • identify students at risk, based on specified criteria • provide data on user activity • provide data on copyright compliance • provide student‐centred reporting options (including access and results), as well as topic‐centred • ensure reporting is backwards‐compatible (i.e. comparison across years is meaningful) Supporting data: • Features requested by FLOAGs (Appendix A): o alerts: there are x new students, Y unenrolled (withdrawn) students since you last logged in o ability to identify students at risk: missed assignments, haven’t logged in enough / at all / by a certain date, activity stops o tracking copyright, eg record FAN of dodgy content • Educational Technology Policies, Guidelines and Projects (Appendix J) o identifying students at risk o annual statistics on use of FLO • Copyright policy (Appendix J) 7 FLO should: provide an engaging and user‐friendly interface Key stakeholders: FLOAGs, staff members, students For example: • support common internet conventions (eg use of back button, ability to open links in new window) • provide a modern, aesthetically pleasing and engaging interface • allow for clear, flexible and intuitive organisation of topic content • provide fast access, even over dialup speeds • be easy to use, even for those with poor computer skills • allow the organisation of the site to reflect how the topic is taught • highlight new / modified content and new activity • be internally consistent in terms of interface and usability • customisable for students • provide basic options for beginner users, and advanced options for advanced users • able to represent content in different languages (eg for NESB students, language topics) Supporting data: • Features requested by FLOAGs (Appendix A): o usability: intuitive to use, consistency across tools, support common conventions (eg right‐ click for context menu), less steps to achieve something, clear visual indication of what’s happening, easy to navigate o feed of new files to site, indication of changed files • Student survey: suggested improvements (Appendix B): o interface: better organisation, easier to navigate, less cluttered, less cumbersome, better colour scheme, better usability, more interesting / engaging / modern, allow multiple windows, improve aesthetics, allow back button, allow personalisation / customisation o notifications: fix incorrect notifications, put announcements in home page, deadline reminders, notifications for more things (eg content) o improve user interface for: discussion, email, calendar o remove irrelevant / unused tools o access: faster access speed • Student focus group (Appendix C): o bit hard to get around at first. • Staff survey: suggested improvements (Appendix D): o gradebook: easier to use, make gradebook less “clunky” o notifications: fix incorrect notifications, more obvious when new content o file management: make it easier to upload files, including media files o discussions: make it less clumsy, improve organisation, organise new messages into threads 8 o interface: less “clunky”, less ugly, course list more manageable, stop the LHS menu from auto‐resizing, more elegant / engaging / Web 2.0, better organised, allow multiple windows, better interface for uploading files, less confusing terminology, easier to switch between roles, easier to navigate, more intuitive / user‐friendly, less steps to do things, conform to internet conventions (eg back button), structure the site as a course outline (weeks), more web‐like (links vs folder) o speed: respond more quickly, easier to use via dialup o learnability: improve learnability, memorability • Staff focus groups (Appendix E): o CDE: clunky interface – nothing intuitive. Many students are on Facebook, Twitter, etc – really high end things that have loads of ease of use, beauty and customisability and we give them a cardboard box with a texta. o CDE: Feedback from students is that they find it difficult to navigate their way around. The cohort is not terribly computer literate and we spend a large part of sem 1 teaching students how to navigate FLO. Need to make the interface easier to use. And make the help site easier to read and navigate – for staff and students. o CDE: Discussion board is hopeless – very difficult to navigate. Good discussion tool – allow students to upload photos or automatically upload student photos from student cards. Can choose to go into a particular thread and can read in order they were posted and can jump to who was this person responding to and then you can reply to the person that just spoke or to any indiv post or the thread as a whole. If I have a reply then the system notifies me and I can go directly to the new posts. Discussions ‐ all new messages are listed – need to work the way through to determine the thread. Apparently no logic to the way the messages are listed. o CDE: FLO should have the option of being a website, not just a file repository. o Power Users: FLO is really limited in that it is not well organised for structuring a course that Moodle may offer – sequencing rather than dumping into folder structure. o Power Users: no notification that new activity has happened on FLO o Power Users: Needs to have a gradebook. Current version is cumbersome – configurability is limited. Lots of clicks and often have to go back and review data and go back and add comments etc. Takes a long time to do for 400+ students. Topic with streams – lots of manual data manipulation. Clumsy interface. Have to select student, click in right spot, enter data, remember to click save, wait for it to update and then go back in and add a comment. Downloading, editing in Excel and uploading csv is usually quicker. Can’t set size of columns. Lock who gradebook or certain sections. No quick way of seeing when it was updated and by who. o Power Users: Look and feel – needs some flair. Why do all topics look the same? o Power Users: A good interface is absolutely essential. o Power Users: Should be able to go to week 12 and access everything – separate on basis of content, not media – or have the choice. o Distance: FLO Looks a bit busy – standardisation has benefit. Too many icons on the first page. Students returning to study after a time or first time study can struggle with FLO – especially if the path to get to certain information is somewhat convoluted. o Distance: Reduce options. Make it simpler for them to find information and to post information. 9 o Distance: Will platform allow learning materials to be put on so it’s friendly and attractive not just documents posted? Week by week layout – embedded links and podcasts. Flows nicely. Looks more professional and is more intuitive for the user to use. o Distance: Perhaps we need to provide options to that people can present information sequentially or in folders according to their preference. o Support: Assignment submission – students find this difficult in FLO – the process is the same as attaching a file to an email but the use of terminology in FLO confuses students (esp international students) o Support: Students want fast access to information – they don’t want to waste time learning how to use something. No‐one does a course on learning to use Facebook so maybe this is about motivation / incentive, not usability of FLO. But FLO does have to have more complexity by nature. o Support: customisability – maybe add some flexibility to FLO – let them hide things they don’t use and let them add things they want (like iGoogle). Might help students engage with the interface. • Correspondence from academic staff (Appendix F): o The problem with FLO is not what it can do (I would expect any current system to offer a similar set of capabilities) but rather with its interface. I've learned how to work around its frustrations, crashes, and idiosyncrasies ‐‐ but others less motivated have no doubt given up, particularly since the final result isn't very rewarding. And from the student's point of view, I suspect the image they get is of an institution that's still using yesterday's technologies. I haven't looked at any particular alternate systems (although now I will do so), so I don't have any specific suggestions. But if we're serious about engaging students (and staff!) in today's GenY world, we need a system that is a pleasure to use (dare I say "fun"), both for staff and students. o I feel that FLO's structure does not allow us develop an "immersed" experience. Ideally, I would like students to enter my fictional organisation's intranet site seamlessly when the log into FLO so from day one they believe that they are working for the fictional organisation. FLO should: provide an open and extensible platform Key stakeholders: FLOAGs, staff members, students For example: • integrate with existing and potential Flinders University systems (including single sign‐on and two‐ way data transfer) • provide flexible methods of pushing or pulling data out of the system (email, RSS, SMS, iCal, web services, clickers) • give users control over what data they receive • allow embedding of content • the ability to transfer data between gradebooks in different sites 10 Supporting data: • Features requested by FLOAGs (Appendix A): o integrate with other systems: : text‐matching, internet accounting (JET), content management system, FLO Live, media delivery tools, other boutique systems (eg MedLink), eReadings, library, SETs, support, topic information, ePortfolios, Flinders email, other Web 2.0 (Facebook, Second Life, Twitter, LinkedIn, other social networks), room bookings, appointment bookings (exchange), consultation times, events calendar, tutorial software (Captivate), open (develop own plugins) o feed of new files to site o calendar: integrate with staff/student calendar systems and event management calendar; web service – download/upload from own personal calendar system (meeting maker / exchange) o phone integration: messages out (link with email / announcements / discussion), ability to send data out (announcement / email / discussion) • Student survey: suggested improvements (Appendix B): o notifications: email, SMS, RSS • Staff survey: suggested improvements (Appendix D): o integrate with other systems: SETs, more open in general o notifications: email notification when new discussions, SMS notifications • Staff focus groups (Appendix E): o CDE: option to link discussion tool to your email – FLO would send any new discussion posts to email o CDE: duplication of tasks – eg posting new material on FLO and then going to email and emailing students that there is new material. Or posting an announcement and then copy and paste into email. Need a link between announcements and email. o Power Users: ePortfolios – have an option in FLO – or integrate ePortfolio into FLO. FLO needs to talk to whatever is selected. o Power Users: link discussion tool with email – but give control to users to opt‐in or opt‐out. Give students power to pull stuff from FLO, rather than pushing to them. Give students ability to pull the calendar to their mobile phone. o Power Users: Trying to present different info to diff groups of students is cumbersome. GoogleDocs does this well – but if you use a third party system you need to seamlessly integrate, not get students to register or double login. o Distance: Assignment dropbox – message comes up in yellow saying “this has been done” – students are always asking if it has been submitted successfully – so students tend to miss this message. Automatic email notification from the system saying that their assignment has been successfully submitted. o Support: worst thing: not getting notification when you get private mail in FLO (need to check this everyday and not everyone does). Mail tool in FLO (no notification, so need to check every topic – which is an issue if a person has a lot of topics to check). o Support: mail tool (notification, rss out?) – this confuses students o Support: Time to recognise that FLO isn’t going to do everything but whatever we add needs to be integrated with FLO so FLO needs to integrate new tools and we need to push things in and out of FLO more easily 11 • Correspondence from academic staff (Appendix F): o Blackboard and similar products might be old fashioned. Would like our requirements identification to include a critical exploration of the move towards mLearning, eg the ability to push stuff onto Facebook (one per topic), podcasts, and mobile phones. • Horizon Report (Appendix M) o identifies open content as a key trend FLO should: support collaboration and group work Key stakeholders: FLOAGs, staff members, students For example: • provide tools and functionality to support students’ group work • provide synchronous (eg virtual classroom, audio/video collaboration) and asynchronous (eg document sharing) capabilities • provide teacher‐driven and student‐driven tools for collaboration • completeness, coherence and consistency in how staff members manage groups (eg integration with gradebook so tutors can only see their students, can allocate staff responsibility for different groups) Supporting data: • Features requested by FLOAGs (Appendix A): o Collaborative document editing (GoogleDocs) o Whiteboard tool (access to tool from communications tool; synchronous?) o People create their own groups • Student survey: suggested improvements (Appendix B): o need a msn/facebook chat feature, more collaborative tools, need a space for groups to discuss • Staff survey: suggested improvements (Appendix D): o more up‐to‐date collaboration tools, document sharing (eg Google Docs), social bookmarking (eg Delicious), wiki, project management tools • Correspondence from academic staff (Appendix F): o I would like to see the wiki, document editing options (like Google Docs), and FLO Live to be integrated into FLO. At the moment I am forced to provide multiple logins to students who have to leave the FLO site in order to use some of the more innovative online tools. 12 FLO should: allow for content sharing and re‐use Key stakeholders: FLOAGs, staff members For example: • support the dynamic use of templates • provide a repository for sharing and re‐using teaching content between sites and across years • share administrative content (eg grading rubrics, policy wordings, cover sheets) Supporting data: • Features requested by FLOAGs (Appendix A): o templates: ability to template sites, adjust template as you go, add a tool/content into templates and have it flow through to all linked sites o sharing content across sites: push into other sites, repository (external?), subscribe to content o content in one place, use it in multiple topics (without duplication), more than public/private, give permissions to roles/individuals (eg “give” content to a specific TC / course) o templates / proformas for creating content: SAMs, weekly schedule, home page, auto‐ generate content o create pdfs automatically o mimic local intranet o allow staff to collaborate o google docs‐type thing o whole document management system o collaboration, change tracking o types of permissions to documents (read‐only, write, etc) o file is linked to from within topic o could have changes flow through to site, or stick with a particular version • Staff survey: suggested improvements (Appendix D): o calendar: calendars shared across multiple topics / users o more flexibility in sharing content across topics, more flexibility in connecting topics • Staff focus groups (Appendix E): o CDE: same topic with internal and external students – need the ability to load one file for more than one topic site. Need to be able to make the designation at the point that you upload the file. Multiple sites share same content – but keep multiple sites. o Power Users: Create material that can be accessed by different topics. One copy – access from more than one place. • Horizon Report (Appendix M) o identifies open content as a key trend 13 FLO should: support the University’s policy on accessibility Key stakeholders: FLOAGs, staff members, students For example: • content must be accessible to those with assistive technologies • students with slow off‐campus access to FLO should be able to perform the same tasks • support offline building and delivery of content • assist files to be posted in an accessible format • must be accessible to those with learning disabilities • accommodate a range of student learning styles • comply with W3C accessibility standards Supporting data: • Features requested by FLOAGs (Appendix A): o accessibility: conform to standards and guidelines (W3C), less rant‐inducing (easy to use with screen reading software), accessible to remote users with dialup • Student focus group (Appendix C): o people posting word documents not pdf and some file sizes are large. Not everyone has access to Word. Conversion to pdf in uploading system would get around issue of lecturers not having access to pdf writer or not knowing how to. o Worst thing: accessibility (dial‐up users, no internet at home) o inconsistent file formats, not accessible, too large in download size o FLO assumes everyone has access to the same technology – they don’t – access and equity issues. Need to also provide lecture notes in hard copy for students. All sorts of issues with the format of the electronic copies and create problems with printing. Lecturers do everything in different formats. Accessibility, legibility, we are not all equal. Access – people with no internet or slow connections feel greatly disadvantaged. People are sometimes ashamed of this. • Accessibility policy (Appendix J): o learning management systems and online learning resources are fully accessible to students with a disability 14 FLO should: support the implementation of University‐wide initiatives Key stakeholders: FLOAGs, EMAG members For example: • Academic Integrity Management Strategy (provision of site to all users, text matching software) • Library Assignment site (provision of site to all students) • student feedback project (providing feedback to topic cohorts) • expanded WebPET (monitoring and enforcing compliance with WebPET guidelines) • identifying students at risk of withdrawal • ePortfolios • initiatives to address the Strategic Plan • work‐integrated learning • graduate qualities • Second Life Supporting data: • Features requested by FLOAGs (Appendix A): o customise plagiarism check for submission of assignments • AUQA review (Appendix G): o commendation from AUQA on development of Academic Integrity Management Strategy o AUQA recommendation that Flinders formalise a systematic approach to communication of student feedback outcomes, noting that processes should cover students across all locations • Strategic Plan (Appendix I): o implement a transparent, responsive and accountable mechanism for responding to student evaluation of teaching and topic delivery • Educational Technology Policies, Guidelines and Projects (Appendix J) o Library Assignment (online quiz offered to all students via FLO) o Academic Integrity site ‐ all staff and students have access to a website “Academic Integrity at Flinders”, containing a quiz. Teaching staff may choose to make this a part of the curriculum o text matching software may be used in topics and is available to students for their own educational purposes o identifying students at risk of withdrawal • Expanded WebPET draft guidelines (Appendix K) o monitoring and enforcing compliance 15