GPTs are taking the world by storm, and while ChatGPT has been amazing to play with, I wanted to go further and see what it could look like embedded in an actual workflow and context.
Recently, I soft-launched a beta report card writing solution for ManageBac. It will read your report card grades, pseudonymize the prompts, allow teachers to personalize them, and then fire off and write reports for students. You can see it in action in the video below (starts at 0:44 seconds, so you can see how quick and easy it is).
The Reponse
People were initially impressed with the potentials, but there was also a growing chorus of concern – was this fair to parents and students? By automating this, what does this do to the parent/teacher and student/teacher dynamic that currently exists?
Report cards are important records of learning and achievement, and there is definitely validity to the criticism. Interestingly, no one’s first reaction was to question the quality of the reports generated (which weren’t always that great) – our awe of this tech being what it is now – one of my hopes is that by embedding the technology in context, we can get a better sense for just how capable (or not) it is.
A Trail to Clarity
As technology improves, we are going to be ever more confronted with similar questions. By adopting this new tool, what do we lose?
With all else being equal, we only need to consider two Ps in order to begin answering this question the process and the product.
Much of these tools help us to lower the burden (reduce or simplify the process), in order to attain a similar final result (the product).
The Process
If what we value is the journey and the experience, then automation makes little sense. Driving through a tunnel to the other side of the mountain vs. hiking over it serve very different purposes. Generally, if our goal is to learn and develop our own selves, we should be wary of shortcuts.
Studies have shown that spell-checkers help ensure our work is free of errors, but they don’t actually teach us to spell. We will continue to spell words incorrectly and rely on the tool to fix it. This kind of augmented intelligence, extending our capacities through our use of technology, is valid only if we consider the skill something inessential. I may not be overly bothered if my doctor can’t spell obsequious, but if they need to refer to a Google image search to make a diagnosis (true story), I may have my reservations.
The sweat and toil of the hard way builds up our capacity, but if technology is ubiquitous, is that skillset necessary?
By riding the automation super-highway, we elevate the skills of the novice to competent, while also collapsing the expert to the same. This equalizing force offers competency to all, but will not help us to develop skills. By displacing experts, we are also furthering our reliance on technology (that may be commercial or concentrated in the hands of a few) – what happens if the technology goes down? Do we have the capacity to suffice without it?
The Product
If I can snap my fingers and make report cards appear through the magic of GPT-3, what does that mean? What is being lost by not having the teacher hand-write the reports? Does it lack personal insight into the student? What if offering a personal insight is a requirement prior to submitting a prompt? Does that ameliorate the concern?
Assuming (with all the dangers that come with that term) that GPT can produce reports of as high a quality as a teacher, what is lost through the use of it? Does the reflection it takes to write report cards offer value that can’t be discovered elsewhere (during parent teacher conferences, for example) – and if not, what is the consequence?
Automation has taken over in the fields of construction, assembly and data management – mechanical problems with fixed qualities that signify success are attainable, and for a lower cost, what could dissuade management from going this route? However, if what we are looking for is not just the final product, but a connection to another human, the question becomes simple. A stamped autograph is very different from one that has been signed by hand.
The Muddled Answer
Especially for those of us who are decision makers, and must decide whether we allow these new super powers into our organization, the answer is never going to be easy. There is always going to be loss associated with the gains that are made. However, by breaking down the decision into two points – that of the process and that of the product, we can better contextualize the gains and losses inherent in this transaction.
GPT Reports:
Process:
Grades are entered into the grade book.
Customized prompts are crafted based on grades, rubrics and content covered.
Reports are generated.
Teachers self-edit, and then reports are peer-edited/reviewed by administration
Relevant assessment? Yes
Individualized? Yes
Time spent? 1-5 minutes per class
Editing and review? Yes
Product:
Individualized reports for each student based on the material covered in class, their grades, attainment levels on rubrics and course material covered.
Relevancy to a specific student is dependent upon the teacher writing student specific prompts.
Individualized? Automatically by grades and rubrics, individually by teacher input. Write-ups will be different per student.
Relevant? If the data (input) is correct, and the content is edited, then it will be relevant.
Hand-written Reports
Process:
Grades are entered into the grade book.
The teacher reviews each student, and writes a comment based on their performance in class and their knowledge of the student’s progress.
The teacher writes a report for each student, or builds one from a template/comment bank.
Teachers self-edit, and then reports are peer-edited/reviewed by administration
Relevant assessment? Yes
Individualized? Dependent on teacher
Time spent? 1-5 minutes per student
Editing and review? Yes
Product:
Reports are submitted for each student.
Dependent on teacher, they can be mostly formulaic/copy and pasted, or completely individualized.
Relevancy to students is dependent upon the effort put into report generation by the teacher.
Individualized? Dependent upon the teacher
Relevant? Dependent upon the teacher.
We can see one main distinction in the process – that reports are generated automatically (and likely much more quickly) rather than laboriously written or assembled by teachers.
Exceptional hand-written reports require a lot of composition time. GPT reports are able to greatly reduce the time necessary to compose the reports, but do not remove the opportunity for individualized comments and reflections.
There is what it is, and there is what is seen
From a school level, I don’t think there are large consequences to GPT derived report comments, especially if a comprehensive review process is in place. However, the school, teachers and administrators are not the only ones involved in this process. How do students and parents feel to be receiving computer generated content?
We need to be mindful of how this impacts all stakeholders in our organization. I think that if schools are forthright about the tools and technologies they use, and teachers are available for follow-up conversations and guidance, then there likely won’t be a major conflict brewing as a result. But, if organizations take a clandestine approach to their technology usage, then that type of duplicitness is going to breed mistrust.
As with anything, if you do decide to adopt this technology, its best to be honest. It may start a good deal of conversations about what else can be automated, but hopefully with the two Ps you have a structure from which you can begin evaluating.
Trying it out
If you are considering trying GPT generated reports, feel free to join our closed beta and try it out for yourself.
