The Rise, Fall and Rise Again of AI
Written by Ben Lawless, Researcher, Instructor, Author, Consultant at Aitken College, The University of Melbourne, Matilda Education, Lawless Learning
When I first got access to ChatGPT I lost my mind with excitement.
I couldn’t believe what I witnessing. It felt like the calculator moment for text. Like all labour saving devices though, we didn’t use all the time it saved us frolicking in the fields playing guitar to each other, but doing even more work, being even more productive. In those early days we were told that every aspect of modern life would be impacted by this generational technology that had our unemployment written all over it.
I can now take on projects that would have taken me five hours to do and smash them out in ten minutes. So I take on more projects. Honestly I would say my total productivity has tripled since I became a ‘heavy user’. I am able to say yes to more things. I can do my normal job way quicker. I can do entirely new things I was never able to do before.
However, there are quite a few exciting, promising things that I thought I was going to be able to do that I have now realised I can’t do. ChatGPT has read the entire internet via a disturbing sounding process called a ‘webscrape’. That is the strength and also a huge weakness of the whole large language model framework. It is great that it has read everything on the internet… Unfortunately a large part of what is on the internet is nonsense, irrelevant or just plain false. So I had my junior high school students writing open-ended responses to a question (“How has human life changed since the Stone Ages?”). I then trained a thread of Lady Chatterley (my pet name for ChatGPT) to give them five pieces of feedback:
Spelling
Punctuation
Sentence structure
Essay structure
Advice about slightly more content they could add
I even said make sure you show the example from their text with the error, teach the rule and then show the improved sentence. I thought I was such a champion! So I got the students to do this 3-4 times, thinking my little experiment would vastly improve student writing ability and showcase my total legendariness… But no. It made very little if any difference to their writing, and used up about 2 hours of their valuable time instead.
What this episode taught me is that there’s isn’t some mysterious developmental progression of writing ability out there in the world that AI will slowly walk people through. Especially in the field of education, there are endless fads and incorrect information and teaching suggestions. Letting AI run wild with all of that stuff on your students is downright dangerous.
Other mistakes I made in those early days include:
Making 100s of multiple choice questions, printing them as playing cards, laminating them, cutting them out separately and then realising the answer to every single question was “A”.
Asking for information about a new book a colleague had wrote, only to be told a completely different person had written it – a hallucination of the very personal type
Lazily using it to read academic journal articles and summarise for me, only to actually read one of the papers and realise its summarising skill wasn’t that great. Yes, it made the text shorter but did not have the expertise to realise what were the most salient points for my purposes.
I have learnt my lesson, and the lessons are:
The human user needs to have the required skill to complete the task without AI support, otherwise, they won't know if what the AI generates is high quality or not
Every single output produced from AI needs to be quality-checked by a skilled human
AI is more useful as an assistant (especially for ideation and writing)
AI-generated material is often better as a very loose first draft, like ‘sentence starters’ in literacy teaching
AI is useful for generating options, but requires a person skilled in the area to choose from those options
AI is good for producing a first draft of a rubric or progression.
It can often just blindly apply a learning taxonomy (for example, Bloom, 1956), specific prompting to avoid this is sometimes required
Break down tasks into small chunks. Asking AI to complete a major task in one step is not as successful as determining the smaller steps required to complete the bigger task and then getting AI to complete those tasks. Not only will the quality be better, but mid-process errors can be addressed as they come up
Never assume AI is actually intelligent. As Leon Furze warns us, “Chatbots don’t make so sense, they make words.”
If there is a specific expert activity you want AI to complete, considering uploading a PDF of instructions first, and then get it to perform the task
The more you can think like the AI, the better the results will be. The more expertise you have, the better you will be able to determine if what the AI pits out is any good.
Happy prompting!