I spent some time today playing with ChatGPT between meetings. I figured that I should see what it could come up with when asked about job application documents. It’s idea of what a resume looks like is rooted in the resumes of yesteryear, but it’s not so bad that I’d throw it out, just guide the writer in how to explain their impact in the prior roles.
However, what I really wanted to see was what it could do for a cover letter. I started out simple by seeing if it knew the formatting, and the result was… well I guess that counts?Read more: AI Hasn’t Cracked Persuasion Yet
Dear [Hiring Manager],
I am writing to apply for the [position] at [Company]. With my extensive experience in [field], I am confident that I can quickly become an integral part of your team.
In my current role as [position title], I have been responsible for [responsibility 1], [responsibility 2], and [responsibility 3]. These experiences have given me an understanding of [industry] and a valuable set of skills. I also have experience with [software/platform], which could be beneficial to your team.
I am very excited about the opportunity to join [Company] and believe I have the dedication, knowledge, and enthusiasm to excel in this position. I am a quick learner and would appreciate the chance to bring my skills to the team.
Thank you for taking the time to consider my application. I look forward to hearing from you and discussing my qualifications in more detail.
It kinda looks like a narrative version of a resume though and that’s not the point of the cover letter. The broad strokes of the formatting are correct but it’s as if the system doesn’t know the point of the format. As you’ll see, that was the theme of my tests.
Next up I gave it a job posting (I’m redacting certain info because it was an internal posting)
Dear Hiring Manager,
I am writing to apply for the Financial Analyst Co-op position at XYZ. With my educational background in finance, my enthusiasm for working in the capital markets and my strong analytical skills, I am confident that I am a perfect fit for this role.
I have obtained a Bachelor of Science degree in finance, and I am currently pursuing a Master of Science degree in finance. My studies have provided me with the knowledge and experience to understand the complexities of the capital markets. I have also developed strong analytical skills, including the ability to analyze datasets, develop insightful commentary, and create analytical tools and initiatives.
I have excellent verbal and written communication skills, and I can work independently and under pressure in a high-pressure, high-reward environment. I am also motivated to gain my CPA or CFA and am eager to take on increasingly greater accountabilities.
XYZ is an organization that I truly admire and I am excited at the prospect of joining your team. I am confident that my enthusiasm and experience will be an asset to your organization. I look forward to discussing further how I can contribute to your team.
It was able to pull some things in from the job posting, but it didn’t understand the required education and also wasn’t quite sure how things connected together. Instead what I got was a listing of skills in the job posting but with the words “I have excellent” ahead of it. It’s exactly what I expect to see of a generic cover letter from someone who doesn’t actually know what they have that’s valuable to the employer. There’s no real content in the cover letter it’s just reposting what the employer asked for. Also, Masters of Science in Finance? Really? That wasn’t asked for in any way, and the job is for someone with two years of education.
After some tweaking I got a bit better of a cover letter
Dear Hiring Manager,
I am writing to express my strong interest in the Financial Analyst (Co-op), Capital Markets Performance Management position at XYZ. With my background in accounting/finance and reporting, my exposure to Capital Markets products and businesses, and my experience with analytical tools and managing large datasets, I believe I would be a great fit for this role.
My background includes a B.Sc. in Accounting and Finance, in addition to experience in PowerPoint presentation, verbal and written communication, and strong analytical skills. I am a motivated individual who is comfortable working under pressure and can prioritize activities to ensure accuracy and timeliness of reporting.
I am excited to join a team that is dedicated to providing trusted advice, helping clients thrive, and achieving success that is mutual. With the opportunity to do challenging work, take on greater accountabilities, and build close relationships with counterparts across the company, this position is a perfect fit for me.
I am eager to learn more about this position and discuss how I can contribute to your team. I am available to discuss this opportunity further and can be reached at (000) 000-0000 or at [email protected]
Thank you for your time and consideration.
At least now it looks like a cover letter that I wouldn’t immediately throw out. But it still doesn’t understand that the point is to persuade the reader that they’re a good choice, not just restate what’s in the job posting with some platitudes.
I’d share what happened with another job posting, but it was more of the same. Lots of platitudes and restating the job posting.
I figured that it didn’t understand how to connect to the things in the posting because it didn’t have a resume, so I fed an example resume (we make fake resumes based off fictitious students to use as examples) and job posting in. The resume was one that would be just slightly indirectly connected to the job posting so that it was a bit harder. But it would still be something that I could easily walk a student through on how to make those connections. The AI on the other hand… well…
Dear [Hiring Manager],
As a 4th year Bachelor of Arts majoring in English and minoring in Psychology with an intention to pursue a career in the not-for-profit sector, I believe I am the perfect candidate for the position. My comprehensive knowledge of customer relations, developed through 5 years of high-caliber service in the food and beverage industry, combined with my strong team player and proficient interpersonal communication skills fostered by 3 years of experience as a community volunteer in event planning make me an ideal candidate.
I am also highly organized and have excellent time management skills; maintaining a full course load with a cumulative 3.95 GPA while working a part-time job and volunteering. I am proficient in all MS Office Programs (Outlook, PowerPoint, Word, Excel). My experience volunteering at Anytown Shelter Thrift Store, assisting clients with community resources, as well as helping with social media posts and event management will be beneficial to the role.
I have also presented a poster on sleep deprivation during student exams at the Psycology Conference in Anytown and have been awarded the Coop Student of the Year Award for my excellence and hard work as an Anytown University Coop Student. I am confident that my skills and experience make me an ideal candidate for this position.
Thank you for your time and consideration and I look forward to hearing from you soon.
My first thought was “YES, they got the idea” but then I kept reading. There were four connection points between the resume and cover letter, and the AI was able to figure out one of them. Even worse, it literally just dropped in wording directly from the resume without elaborating on it. And finally, there was something mentioned multiple times in different sections of the job posting (something a human would read as a flag that you need to mention it) but the AI didn’t bother.
I ran that same resume through with a job posting for a psyc student co-op job with a child care company and a huge red flag went up for me. As soon as it had both a feminine name and childcare connected job the AI decided to stop overselling and be a little more humble about itself. It’s writing about how perfect it is for the job, over and over again, until it thinks it’s a woman applying for a job working with children then suddenly it’s talking about how it’s only a “good fit” for the role instead of “perfect fit” and now instead of talking about itself in an expansive way it talks about how it can provide “valuable support to the team”. Remember everyone, AI isn’t unbiased, it just lacks the ability to critically examine its bias.
After all of that… well it’s a fun toy, and it’s better than nothing. If you had a resume with one of these cover letters and a resume without a cover letter I’d pick the one with the ChatGPT one. But it only understood structure and copying content, it didn’t understand what the job postings meant, or where the information in them was important or how to frame things in a way that was convincing.
It seems like it may be a wile before AI has figured out what makes human communication convincing. But since that’s something a lot of humans haven’t figured out yet perhaps it’s not doing as bad as I think. Basically it’s the same issue that AI has with essay writing. The writing fits the structure it’s imitating, and mentions the things in the prompt, but it doesn’t have the ability to write in a way that makes connections between the material and the audience in a meaningful way, or to connect things between the material in a way that is convincing and not simply “hey these two phrases probably go beside each other”.
I intend to use the output from this in my teaching on cover letters this year as it shows the difference between a good cover letter (which these aren’t) and a bad cover letter (which these aren’t) and a boring/formulaic/lazy cover letter (which these most assuredly are).