Skip to main content

Course Evaluations via AI... "I'm sorry Dave. I'm afraid that's what I think"

So the shape of things to come, I suppose. We just finished up with March Break and we send a Report Card (marks & comments) home with the students as they go on holiday.  As they were coming back for the last term, I thought I would do a temperature check with them when they got back (if I get to write an evaluation of them, it only seems fair!)

This time, rather than using Microsoft Forms (like here), I used a chat bot (an automated conversation) where the student would talk with the chat bot and then sentiment analysis would be performed on the data.  Not only would the student's written content be considered but also how they wrote it and what word or phrasings they used.
I had all three sections interact with the chat bot (the link was posted in their OneNote) and I stepped out of the room to give them a chance for some privacy. I did mention to them that this was experimental, that they didn't have to do it and that it was anonymous (modulo the usual "nothing is really anonymous on the internet"). They had no issue getting onto the chatbot and no one had any issues with how it interacted with them, or what to do once they clicked on the link.

I used which is presently in beta -- but given the rate at which Microsoft is improving their chatbot framework, I can see school districts developing their own tools in the future.  PowerBI, after all, does do sentiment and text analysis.
Once the students were done, I logged into the results to see how I did. I only blur out the overall because our school could use it as part of my own evaluation and I don't think I'm yet comfortable with that.

You can download a CSV filled with the actual conversations the students had, so it's like doing a Microsoft Form (or, umm, paper) course evaluation, or do your own PowerBI analysis of the text (I haven't had time to do that yet.)
One caveat to be aware of was the jokes that popped up! Now, I had tested the chatbot myself and didn't discover this, but trust a student to push the envelope.  And they are terrible Dad jokes -- and also aimed at a post-secondary crowd rather than early high school (alcohol was mentioned in one joke).
And out of the 47 students, I did have two students loudly vocalize how they found it "creepy" to be doing this.
What I do look forward to is having the AI dig deeper once it notices a pattern, or if it reads avoidance or suggestive messages.  And I'd love to see us be able to have it focus on particular topics (I use #VNPS and would like to ask questions specifically on that, for example).
Do I think they were any more honest than using Microsoft Forms or paper? Scanning through the Excel sheet of actual conversations, I don't think so.  But, I likely got MORE content than I would off of a Form or paper-- and the added layer of sentiment & textual analysis helped avoid focusing so much on an individual written comment (that we always obsess over). And it definitely SAVED TIME -- the students took maybe ten minutes to do the chat and then click--> I had the results.

I will point out that there are two sessions on AI during the LearnTeams conference in a few days! (yes, I'm doing one related to Education).


Popular posts from this blog

Teacher Professional Development and Microsoft OneNote

During the first three weeks of July, I have the amazing opportunity to work at the Park City Mathematics Institute.  It is, without exaggeration, the best professional development opportunity for teachers of mathematics.  Participants spend three weeks thinking deeply about mathematics and mathematics education.

There are three main aspects of PCMI:

learning mathematicsreflection on practice (RoP)becoming a resource to others.I'm part of the team for RoP and in charge of the third aspect, in which participants consider a gap in professional development back at their home districts and work in small groups to help fill that hole by developing a rich PD seminar on that topic.

It is not easy to develop professional development.  Teachers who haven't written PD have to patiently learn how to write (essentially) lesson plans for someone else.

This year, I used Microsoft OneNote to facilitate the process.  We have a central OneNote Notebook through which I lay out the daily schedule…

Escape Room / BreakOut in OneNote

[[Part 2 of this article is here: Link]]

So when I was visiting Anna in Edinburgh during March Break, she showed me how she used Password-Protected OneNote sections within the OneNote ClassNotebook to help students check their work -- she set the password to the correct answer, so they knew they had it right when the Section opened up.

I figured I could use this for Math Review, so I set aside a couple of hours (turned out to be 3 hours but a fair chunk of that was solution-time) the other night to put an Exam Review together for my Grade 10 Mathematics course.  I pulled together as many multiple choice questions and short answer questions on the topics as I could Google and tried to balance each Section with a mix of topics and then threw in a couple of pop-culture questions, too.  The students worked on the problems in each section and used the answers as passwords to unlock the next section until they got to the Prize section.
Result?  Near total continual engagement for the 60 min…

You need a Team to do that

So yesterday I mentioned about using Teams to do Screencasts - you can see that blog post by clicking here.  Here's a brief run-down of the process.
When you're in the Teams application, either as a reply to a message, or a brand new conversation, click on the MEET NOW button.  It looks like a little video camera.
You'll get Teams Meeting popping up... you'll be doing a call, but of course, there's no one on the other end.  Type in a Subject -- I'd choose the Course, Textbook, page and question number if it's applicable, otherwise some combination of curriculum standard.

Click on MEET NOW to being the meeting. 
We want to start recording in order to get the video part of the screencast.  Click on the 3-dot menu button and choose START RECORDING.  (It says "Preview" because recording is still in pretty active development -- see below).  The recording has now started and you'll see the red icon in the upper right of the screen.

Now, you're g…