JISC e-Learning Webinars: Making Assessment Count

Friday 3rd February 2012 1-2pm

Online via Blackboard Collaborate

Presenters: Professor Peter Chatterton (Daedalus e-World Ltd) and Professor Gunter Saunders (University of Westminster)

The objective of Making Assessment Count is primarily to help students engage more closely with the assessment process, either at the stage where they are addressing an assignment or at the stage when they receive feedback on a completed assignment. In addition an underlying theme of MAC is to use technology to help connect student reflections on their assessment with their tutors. To facilitate the reflection aspect of MAC a web based tool called e-Reflect is often used. This tool enables the authoring of self-review questionnaires by tutors for students. On completion of an e-Reflect questionnaire a report is generated for the student containing responses that are linked to the options the student selected on the questionnaire.

The session will provide an overview of MAC and highlight some of the variant MAC processes that are being developed by six different universities, as well as drawing out strengths and weaknesses of MAC. There will be a demonstration of how the e-Reflect tool works but the presenters will also show how MAC can work without that tool.  Participants will be engaged by seeking their views on the affordances offered by MAC as well as their input into identifying barriers and enablers in applying MAC in their own institutional and subject contexts.

The webinar is free to attend.

The Webinar

Joined about 20 minutes into the event after teaching ‘Social Media for Job Hunting‘. Thanks to @sarahknight for sending me the login details which hadn’t arrived! 

MACE: https://sites.google.com/a/staff.westminster.ac.uk/mace/home

Immediate reaction from staff is that the workload is likely to be high re learning journals, but find that a few comments actually doesn’t take that long, especially in comparison to the improvement demonstrated from students.

Offers the opportunity for small, but detailed, positive feedback suggesting actions… rather than “So..?” as a typical written comment on an assignment.

What should the balance be between e & f2f feedback:

My comment:

I’m interested to see how coaching practice, etc. is impacting upon how things work. With manipulating-media.co.uk we give them ‘consultancy sessions’ as a group as feedforward, before they submit their assignments, and they write reflective blog post every week. They tend to use FB to connect with each other. Agree that we should look at the e, but ultimately it’s about ensuring that it meets the needs of the course.

Some chat comments:

  • The record of what everyone says is increasingly important, a big advantage of ‘e’.
  • Is dialogue about scaffolding or about generating cognitive conflict? different processes and different models of dialogue; is all dialogue equally productive?
  • Awarding micro grades for demonstrating action on feedback

Further links:

Audio Feedback (@jisc_rsc_se)

Live notes from the session:

Information is on the Moodle site.
Research that has been done is wholly positive about the effects of audio feedback.

Portsmouth University. Find it personable, absorb information in different ways (skim read otherwise), listen to it on mobile devices on the move, shareable, can go through feedback in tutorials/agree targets.

Negatives seem to be all from the tutors side – about getting on board. Ultimately should be saving you time in the long run… Where it has been embedded evidence is that it saves time.

Negatives from students – some felt tutor was ‘cheating,, not a replacement, no visuals, can be too personal for comfort, can’t answer back/engage in dialogue. Some thought the novelty helped with success.

Range of tools that can be used for audio feedback. MP3 recorders (£8-20), record to iPhone (add http://www.ipadio.com/… A bit like audioboo). Particularly good for formative assessment. Screenr and Jing also useful. Example of staff member who does feedback into the device in class and then bluetooths it to student phones in class.

No eportfolio system? Try Posterous? [issues with privacy?]

Also podcasts useful, so e.g. Mike into an iPod. Recorded – placed into iTunes – so then becomes public data. Look at small microphones (£2ish – usually adequate).

Material will be on: http://www.jiscrsc.ac.uk/southeast/courses-and-resources/moodle-courses.aspx?

Tutors often prefer to record to PC as they have control over the file names. Most of these other ones don’t have meaningful names.

Microsoft sound recorder or audacity – can be provided portable style to be plugged into laptops.

Impressive. Tweeted about ipadio – had response from @ipadio with a link to http://ipadio.com/page.asp?section=99&sectionTitle=ipadio+in+Education.

Return assignments via Moodle. Most are uploaded individually via advanced assignment. Others put in folders online but are privacy issues there.

Conversation – run it in a ‘test’ assignment. Get students involved. Describe it like the Director’s Commentary … Talk as you read… Finding people’s way in. Efficient way to save time, and distributing it … Everything takes so long on Moodle. People more comfortable with audio than video (even a level/BTEC level – particularly self conscious).

Don’t get too hung up on scripting, etc — they are used to our voices, more engaging if not scripted. Expect effort in first stages, find what works for you – e.g. A list of repeated errors?

Care with quality of editing – too hi, the files will be too large. Compress audio, e.g. Lossless = like zip for audio. Much higher compression = lossy compression, most use MP3 – but it’s a licensed format hence why often a cost involved. Firefox doesn’t support MP3 as not an open source format. 96kbitspersecond about lowest can go to be usable – fine for text. 10 minutes about 1MB. Most 2-3 minutes, although exam paper can be 10-15 minutes – quicker than doing smaller bits – the file saving more of a complex process.

Cloud storage can be quite expensive. 365live – can give a lot of access control.. Could use something like Dragon to produce a text file – but would then have to proofread it. Could possibly ask students to bring in USB sticks that they keep just for feedback // need to take a backup copy .. Not ideal but an option.

Look also at – http://www.jiscdigitalmedia.ac.uk/audio/.

Care with recording straight to apple devices – uses AIF files by default – have to go into recording format settings to MP3 – most widely used format.

http://moodle-rsc.ukc.ac.uk/course/view.php?id=154

Nanogong… http://gong.ust.hk/nanogong/ – can speed up & slow down, embeddable in Moodle. Have choice via (grade book) to give individual or general. Doesn’t yet work for Moodle 2. Can preset text to go into the text box… Then add file. If already pre-recorded quick to upload, but hoping for a neater way of doing it.

Asked students to write in the notes to give feedback and got v. Good feedback. As always, has to be one method out of several for feedback.

Office 2007 – can embed the audio directly into the Word doc http://www.howtogeek.com/80016/insert-audio-into-word-2007-documents-2. Office 2010 – not so good… More clunky? Not seen add-ins that make it better… powerPoint works well.
Oo
http://www.knovio.com/ … Uses JPEGs & PPTs only. Can’t use live, but prerecord – audio feedback with video. Student has control over whether text or tutor appears large on their screen. Useful for diagrams, art, etc – can put multiple pics in so can talk through several images. Link can be emailed to any number of students – if not appropriate – copy link & share. Embed code & social media buttons. Querying if a plugin allowing material to be downloaded.

What about http://keepvid.com/ to retain files? What about using the annotate features in YouTube – allows students to deconstruct a video. Staff could also write video comments as feedback. Can only annotate ones you’ve uploaded – but can give rights. Look for edit/annotate. Could set up e.g. A programme channel – give passwords so all can upload. Cn turn the annotations off.

Plagiarism detection – Turnitin. Farnham – made it part of HE policy that all assignments submitted. Students really like it – linked into Grademark so works well… Now picks up on Google translate re-translations – which students were using as a getaround. Avoids collusion. Leave up to professional judgement as to whether deal as ‘catch out’ or ‘teaching’. More effective if student uploadsn self. Need a policy to deal with it afterwards.

Other options a cut & paste suspect sections into Google.

Screen capture: Jing. screenr (no software required – also 5 min limit). Camtasia. http://www.yawcam.com/.

e-assessment showcase (eastern region 30 – March).

Adam etc happy to come for a 1-2 hour kick start session if can get small group (e.g. 10) together. Useful to collect case studies applicable to particular programmes… Are linked from the Moodle site.

Clickers in the Classroom

Really interesting video giving lots of examples of how this is used:

Plagiarism Software

My colleague, Nicole McNab, has looked at plagiarism quite extensively over the past couple of years, so this has gained my interest … especially as Nicole tends towards the idea that TurnItIn, etc. should not be used for detection, but for training students to understand!

Students who are aware that their work will be checked by plagiarism-detection software are just as likely to cheat as those who are not, a study suggests.

Turnitin software, which is used by thousands of universities worldwide, extracts text from submitted essays and checks it against other sources, such as online documents.

However, the study conducted by a researcher at California State University suggests that such measures should not be regarded as a “silver bullet” in the battle against “deviant” academic practices.

Robert J. Youmans, a cognitive psychologist, says in the paper published in the journal Studies in Higher Education that he expected to find that fewer students would cheat if they were warned that their work would be scanned.

This proved not to be the case.

Read full story.

Publishing in a Digital Age

http://www.sxc.hu/photo/1360030

As I work towards publishing my PhD, it’s interesting to see what the publishing market is doing. I have already been told that if I can deal with the ‘image issue’ I am likely to have a fight on my hands from the publishers – as my PhD is not an obscure topic – but one that many have an interest in (judging by the 1000s that visit this blog) …so it’s just finding time to do it!

Open-access publishing models will end once and for all the outdated and expensive production of little-read monographs that “exist for the purposes of existing”.

The prediction was made in a debate titled Open Access: The New Future of Academic Publishing? at the British Academy, where William St Clair, senior research fellow at Trinity College, Cambridge, said that the print-only monograph was now little more than “a union card to apply for jobs in academia – if there are any”.

Mr St Clair, who is also co-founder and chairman of Open Book Publishers, said the typical university press monograph “is printed in an edition of 500 copies, much the same number as in the 18th century. They are priced at about £60 to £100.

Read full article.

Disruption with Online Education

http://www.sxc.hu/photo/987822

A really interesting article in this weeks Times Higher Education which considers how the US system (and probably those of us who end up echoing their systems) is about to face turbulent times with higher and higher fees causing ‘disruption’ as students look for other ways to engage:

This disruption, they say, will force down costs, lure prospective students away from traditional “core” universities, transform the way academics work, and spell the end for the traditional scholarly calendar based around face-to-face teaching

….

Online education will bring a shift by opening higher education to a new middle group in the concentric circles, they argue.

“We use the word ‘disruptive’ not because it is a breakthrough improvement for that middle group, but because it transforms the product or service into something that is so much more affordable and simple that a whole new population can afford it and find that it is accessible to them,” Christensen says.

The book asserts that until now, unlike other industries, higher education has not had a “disruptive innovation” that has forced the sector to drive down costs. The result, says Christensen, has been “sustained and difficult price increases”

….

“Almost invariably, the [established] leaders find it impossible to lead the industry in disruption,” he says. “It’s not technology per se that keeps them in the middle, but the very fact that it is affordable and accessible makes it almost impossible for the [sector's traditional] leaders to address.”

Christensen suggests that instead it will be new institutions and providers that will lead the way in online learning innovations. “What you will see is that online learning will take root in this larger population: people who, either because of the nature of their life or their situation, can’t go to a campus but can do it online.

This is a challenging piece, and one worth a read, as I’m sure the questions will be asked in the UK shortly.

Copyright: Reclaiming Fair Use

I get asked a lot of questions about copyright, and as a non-lawyer, I have to say that it’s not my strongest area of expertise. I went to a day event at the Institute for Historical Research on copyright whilst undertaking my PhD, and every other sentence was ‘this is guidance, check with the lawyer’ … The following text looks interesting, although largely for US audiences:

This lively book, Reclaiming Fair Use: How to Put Balance Back in Copyright, is designed to liberate people from the “Mind Forg’d Manacles” of copyright law. The authors – film and media scholar Patricia Aufderheide and professor of law and stalwart defender of the public interest Peter Jaszi – hope to help readers “understand how to think about and use copyright, and especially your right to use copyrighted material without permission or payment when you make a work – whether a blog entry, a song, a mashup, a poem, a documentary, a magazine article, a lesson plan, a scholarly archive, a slide show, a technical manual, a scrapbook, a collage, or a brochure”.

The broad and flexible defence of fair use was codified in the US copyright act in 1976. The defence provides that the use of copyright material for “purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright”. This defence has also been applied in a wide range of cultural and technological contexts.

Read the full article, or buy the book.

Unregulated KIS’s?

http://chrislorensson.com/design/hefce-kis/

I’ve always been keen to see data that engages more with ‘how did we develop this student from where they started from?’ rather than final grades, etc. so the new KIS are of concern – read more about them here:

I recently spent an enthralling Sunday morning renewing my car insurance via a price-comparison website. In the past, I’d always performed the insurance-renewal ritual via a series of telephone calls in which I’d asked patient and blameless call-centre workers whether the companies employing them were having a laugh. While this involved some cheery conversations and usually resulted in a decent outcome, it did take rather a long time.

The website I used allowed me to be precise in my search. But the process took as long as ever. I found myself having to compare seemingly similar products that were actually quite different. This was because, in the key information provided, critical data were missing. For instance, the website identified whether a product included legal cover and at what cost, but not the level of cover provided. In most cases the absent details could be obtained only by making a phone call …

Nonetheless, car insurance is fairly straightforward; and although we all wince at its cost, policies are far cheaper, simpler and easier to compare than the complexity of UK university courses. As we know, the idea that prospective undergraduates should be able to make informed comparisons between programmes and institutions is central to the government’s vision of market-orientated higher education. But its plans for the provision of vital data, via Key Information Sets, are inadequate and likely to be misleading and counterproductive.

Read full story, and read more on the HEFCE site. I also thought this story about ‘adding social value‘ was of related interest…  see, e.g.

Universities have long measured their financial value, for example the spending power of their staff or their total turnover, she explained. But the report recommends finding an economic price for all university “outputs”, including those not captured by financial analysis….

A “social weight” could then be applied to this economic value to reflect social priorities, for example by counting an activity as more valuable if it delivers to the poor rather than the rich.

Students demanding feedback…

http://www.sxc.hu/photo/866529

Students used to ‘quick feedback’, are becoming more and more demanding.

Students currently going through the higher education system are part of a “net generation” who expect instant feedback because of their heavy reliance on mobile phones, social media and video games, a conference has heard.

Arlene J. Nicholas, an assistant professor in the department of business studies and economics at Salve Regina University in New England, spoke at the Society for Research into Higher Education conference, held in Wales last week, giving findings from her research on learning methods among 100 students at a small private university.

She told delegates that the current generation of US students – defined as those born between 1981 and 2000 – were the most diverse, with a third defined as non-white or Latino. But they are also the most demanding, Dr Nicholas claimed. “This multimedia generation seems to expect multiple methods to learn,” she said.

She added that the net generation could be easily upset by negative feedback but nevertheless wanted more feedback than any other generation.

Read full story, and read more about FASTECH, a project at the University of Winchester considering how technology can enhance feedback.

RSA Animate – Changing Education Paradigms (Ken Robinson)

A really interesting video (using the beautiful RSA Animate style), which takes Sir Ken Robinson, arguing that the education system is no longer suitable for the modern day:
The video was suggested by Clare Killen at the #jiscel11 conference.