Skip to main content

Course Evaluations via AI... "I'm sorry Dave. I'm afraid that's what I think"

So the shape of things to come, I suppose. We just finished up with March Break and we send a Report Card (marks & comments) home with the students as they go on holiday.  As they were coming back for the last term, I thought I would do a temperature check with them when they got back (if I get to write an evaluation of them, it only seems fair!)


This time, rather than using Microsoft Forms (like here), I used a chat bot (an automated conversation) where the student would talk with the chat bot and then sentiment analysis would be performed on the data.  Not only would the student's written content be considered but also how they wrote it and what word or phrasings they used.
I had all three sections interact with the chat bot (the link was posted in their OneNote) and I stepped out of the room to give them a chance for some privacy. I did mention to them that this was experimental, that they didn't have to do it and that it was anonymous (modulo the usual "nothing is really anonymous on the internet"). They had no issue getting onto the chatbot and no one had any issues with how it interacted with them, or what to do once they clicked on the link.

I used https://hubert.ai which is presently in beta -- but given the rate at which Microsoft is improving their chatbot framework, I can see school districts developing their own tools in the future.  PowerBI, after all, does do sentiment and text analysis.
Once the students were done, I logged into the results to see how I did. I only blur out the overall because our school could use it as part of my own evaluation and I don't think I'm yet comfortable with that.

You can download a CSV filled with the actual conversations the students had, so it's like doing a Microsoft Form (or, umm, paper) course evaluation, or do your own PowerBI analysis of the text (I haven't had time to do that yet.)
One caveat to be aware of was the jokes that popped up! Now, I had tested the chatbot myself and didn't discover this, but trust a student to push the envelope.  And they are terrible Dad jokes -- and also aimed at a post-secondary crowd rather than early high school (alcohol was mentioned in one joke).
And out of the 47 students, I did have two students loudly vocalize how they found it "creepy" to be doing this.
What I do look forward to is having the AI dig deeper once it notices a pattern, or if it reads avoidance or suggestive messages.  And I'd love to see us be able to have it focus on particular topics (I use #VNPS and would like to ask questions specifically on that, for example).
Do I think they were any more honest than using Microsoft Forms or paper? Scanning through the Excel sheet of actual conversations, I don't think so.  But, I likely got MORE content than I would off of a Form or paper-- and the added layer of sentiment & textual analysis helped avoid focusing so much on an individual written comment (that we always obsess over). And it definitely SAVED TIME -- the students took maybe ten minutes to do the chat and then click--> I had the results.

I will point out that there are two sessions on AI during the LearnTeams conference in a few days! 
https://www.learnteamsconference.com/ (yes, I'm doing one related to Education).
Labels:

Comments

Post a Comment

Popular posts from this blog

Desmos, OneNote & Replay

So using Desmos activities are a great way to encourage exploration and discussion in math class -- if you haven't tried them, I encourage it.  They're collected at  https://teacher.desmos.com/  But ... Desmos doesn't give you quite enough.  It doesn't have a way of capturing the work that the student does within their space, and it doesn't allow for annotation of class contributions as we come together to discuss.  Well, not surprisingly, OneNote comes to the rescue.  Using the Windows shortcut Windows-Shift-S it is really quick to snag the Desmos screen and pop it into a waiting OneNote page.  From there, we can grab our pen and (using wireless projection) talk about what all the different responses mean and where to go from there. (An aside : one of the nice features of Desmos activities are the way you can hit PAUSE and it will pause all the screens of the students working.  I always give them a heads up "10 seconds to pause..." and it's...
21 comments
Read more

So you want to hack your OneNote Class Notebook

Taking a brief break from my "Getting Started with OneNote Class Notebook" series (you can start that one here )... This is a little advanced so if you're not comfortable setting permissions inside of Office365 you may want to avoid this.  Or set up a Class Notebook to play with so that it doesn't affect any existing Class Notebooks.  Yeah, the latter is a good option. One of the great powers of OneNote is that you can do some really neat permissioning of the Section Tabs. When the Notebook is created, of course, it gives you an "open permissions" on the Collaboration Space and student-read-only on the Content Library.  And then each student space is wide open to each individual student. But we've found that occasionally you want to mix up the permissions a little.  For example, you could create a space in a student section for your private notes that the student couldn't see, or maybe you want a tab in the Collaboration Space that students cou...
40 comments
Read more

Making your own font

Slid in amongst all the announcements for Ignite, Microsoft's big conference in September, as a tool that I thought was quite cool.  Not original, since similar things have existed elsewhere & when, but a nice option nevertheless. Microsoft's Font Maker allows you to create your own font using digital ink.  You get all 26 characters, numbers and punctuation (for English languages) on which you draw your font for each character. (For me, it's the first 128 printable characters out of the ASCII table!)  Using your #digitalink pen, you draw out what you want each character to look like. I just quickly wrote out the alphabet as you can see below: You don't have to do it all at once and you can keep working on your Font as you go; it saves as a JSON Project File which means you can send these between collaborators. Once you have your font done, you can adjust the spacing between characters & words to make it look good (it uses a scene from Hamlet -- I...
Post a Comment
Read more