Noah D. Arney

Category: Technology

  • AI Hasn’t Cracked Persuasion Yet

    I spent some time today playing with ChatGPT between meetings. I figured that I should see what it could come up with when asked about job application documents. It’s idea of what a resume looks like is rooted in the resumes of yesteryear, but it’s not so bad that I’d throw it out, just guide the writer in how to explain their impact in the prior roles.

    However, what I really wanted to see was what it could do for a cover letter. I started out simple by seeing if it knew the formatting, and the result was… well I guess that counts?

    Read more: AI Hasn’t Cracked Persuasion Yet

    Dear [Hiring Manager],

    I am writing to apply for the [position] at [Company]. With my extensive experience in [field], I am confident that I can quickly become an integral part of your team.

    In my current role as [position title], I have been responsible for [responsibility 1], [responsibility 2], and [responsibility 3]. These experiences have given me an understanding of [industry] and a valuable set of skills. I also have experience with [software/platform], which could be beneficial to your team.

    I am very excited about the opportunity to join [Company] and believe I have the dedication, knowledge, and enthusiasm to excel in this position. I am a quick learner and would appreciate the chance to bring my skills to the team.

    Thank you for taking the time to consider my application. I look forward to hearing from you and discussing my qualifications in more detail.

    Sincerely,

    [Your Name]

    It kinda looks like a narrative version of a resume though and that’s not the point of the cover letter. The broad strokes of the formatting are correct but it’s as if the system doesn’t know the point of the format. As you’ll see, that was the theme of my tests.

    Next up I gave it a job posting (I’m redacting certain info because it was an internal posting)

    Dear Hiring Manager,

    I am writing to apply for the Financial Analyst Co-op position at XYZ. With my educational background in finance, my enthusiasm for working in the capital markets and my strong analytical skills, I am confident that I am a perfect fit for this role.

    I have obtained a Bachelor of Science degree in finance, and I am currently pursuing a Master of Science degree in finance. My studies have provided me with the knowledge and experience to understand the complexities of the capital markets. I have also developed strong analytical skills, including the ability to analyze datasets, develop insightful commentary, and create analytical tools and initiatives.

    I have excellent verbal and written communication skills, and I can work independently and under pressure in a high-pressure, high-reward environment. I am also motivated to gain my CPA or CFA and am eager to take on increasingly greater accountabilities.

    XYZ is an organization that I truly admire and I am excited at the prospect of joining your team. I am confident that my enthusiasm and experience will be an asset to your organization. I look forward to discussing further how I can contribute to your team.

    Sincerely,

    [Your Name]

    It was able to pull some things in from the job posting, but it didn’t understand the required education and also wasn’t quite sure how things connected together. Instead what I got was a listing of skills in the job posting but with the words “I have excellent” ahead of it. It’s exactly what I expect to see of a generic cover letter from someone who doesn’t actually know what they have that’s valuable to the employer. There’s no real content in the cover letter it’s just reposting what the employer asked for. Also, Masters of Science in Finance? Really? That wasn’t asked for in any way, and the job is for someone with two years of education.

    After some tweaking I got a bit better of a cover letter

    Dear Hiring Manager,

    I am writing to express my strong interest in the Financial Analyst (Co-op), Capital Markets Performance Management position at XYZ. With my background in accounting/finance and reporting, my exposure to Capital Markets products and businesses, and my experience with analytical tools and managing large datasets, I believe I would be a great fit for this role.

    My background includes a B.Sc. in Accounting and Finance, in addition to experience in PowerPoint presentation, verbal and written communication, and strong analytical skills. I am a motivated individual who is comfortable working under pressure and can prioritize activities to ensure accuracy and timeliness of reporting.

    I am excited to join a team that is dedicated to providing trusted advice, helping clients thrive, and achieving success that is mutual. With the opportunity to do challenging work, take on greater accountabilities, and build close relationships with counterparts across the company, this position is a perfect fit for me.

    I am eager to learn more about this position and discuss how I can contribute to your team. I am available to discuss this opportunity further and can be reached at (000) 000-0000 or at [email protected]

    Thank you for your time and consideration.

    Sincerely,

    Your Name

    At least now it looks like a cover letter that I wouldn’t immediately throw out. But it still doesn’t understand that the point is to persuade the reader that they’re a good choice, not just restate what’s in the job posting with some platitudes.

    I’d share what happened with another job posting, but it was more of the same. Lots of platitudes and restating the job posting.

    I figured that it didn’t understand how to connect to the things in the posting because it didn’t have a resume, so I fed an example resume (we make fake resumes based off fictitious students to use as examples) and job posting in. The resume was one that would be just slightly indirectly connected to the job posting so that it was a bit harder. But it would still be something that I could easily walk a student through on how to make those connections. The AI on the other hand… well…

    Dear [Hiring Manager],

    As a 4th year Bachelor of Arts majoring in English and minoring in Psychology with an intention to pursue a career in the not-for-profit sector, I believe I am the perfect candidate for the position. My comprehensive knowledge of customer relations, developed through 5 years of high-caliber service in the food and beverage industry, combined with my strong team player and proficient interpersonal communication skills fostered by 3 years of experience as a community volunteer in event planning make me an ideal candidate.

    I am also highly organized and have excellent time management skills; maintaining a full course load with a cumulative 3.95 GPA while working a part-time job and volunteering. I am proficient in all MS Office Programs (Outlook, PowerPoint, Word, Excel). My experience volunteering at Anytown Shelter Thrift Store, assisting clients with community resources, as well as helping with social media posts and event management will be beneficial to the role.

    I have also presented a poster on sleep deprivation during student exams at the Psycology Conference in Anytown and have been awarded the Coop Student of the Year Award for my excellence and hard work as an Anytown University Coop Student. I am confident that my skills and experience make me an ideal candidate for this position.

    Thank you for your time and consideration and I look forward to hearing from you soon.

    Sincerely,

    Ella Pink

    My first thought was “YES, they got the idea” but then I kept reading. There were four connection points between the resume and cover letter, and the AI was able to figure out one of them. Even worse, it literally just dropped in wording directly from the resume without elaborating on it. And finally, there was something mentioned multiple times in different sections of the job posting (something a human would read as a flag that you need to mention it) but the AI didn’t bother.

    I ran that same resume through with a job posting for a psyc student co-op job with a child care company and a huge red flag went up for me. As soon as it had both a feminine name and childcare connected job the AI decided to stop overselling and be a little more humble about itself. It’s writing about how perfect it is for the job, over and over again, until it thinks it’s a woman applying for a job working with children then suddenly it’s talking about how it’s only a “good fit” for the role instead of “perfect fit” and now instead of talking about itself in an expansive way it talks about how it can provide “valuable support to the team”. Remember everyone, AI isn’t unbiased, it just lacks the ability to critically examine its bias.

    After all of that… well it’s a fun toy, and it’s better than nothing. If you had a resume with one of these cover letters and a resume without a cover letter I’d pick the one with the ChatGPT one. But it only understood structure and copying content, it didn’t understand what the job postings meant, or where the information in them was important or how to frame things in a way that was convincing.

    It seems like it may be a wile before AI has figured out what makes human communication convincing. But since that’s something a lot of humans haven’t figured out yet perhaps it’s not doing as bad as I think. Basically it’s the same issue that AI has with essay writing. The writing fits the structure it’s imitating, and mentions the things in the prompt, but it doesn’t have the ability to write in a way that makes connections between the material and the audience in a meaningful way, or to connect things between the material in a way that is convincing and not simply “hey these two phrases probably go beside each other”.

    I intend to use the output from this in my teaching on cover letters this year as it shows the difference between a good cover letter (which these aren’t) and a bad cover letter (which these aren’t) and a boring/formulaic/lazy cover letter (which these most assuredly are).

  • AI and Hiring

    A former twitter thread

    In response to this tweet:

    NEW: the insurance startup Lemonade claimed it was analyzing “non-verbal cues” like eye movements and speech patterns to reject insurance claims.

    then the company deleted a bunch of tweets, and now it’s saying “we def do not do phrenology”

    https://twitter.com/janusrose/status/1397602847064215554

    This is part of a trend of problems with companies who use AI to make decisions about people. Now, I’m not involved in AI ethics, for that you should follow Timnit Gebru, but my work in career development involves understanding how people assess other people.

    A problem in hiring is the role bias plays. If you ask 100 people if bias plays a role in their decisions about job candidates you’ll probably have 90 saying no. That’s actually one of the problems. Our brains do this wonderful thing where they say “I’ve seen that behaviour before and it meant they were lying”. But the problem is that the experiences your brain is assessing against are based in the culture you are within. The things it assesses are what people within your culture do when they’re lying. That may be different for those raised in other cultures. But your brain tells you that your experience is universal. And that’s not just about lying. It does the same for what means attentive, friendly, pleasant, combative, and dedicated. That means that when we’re interviewing someone for a job and we think about “fit” it’s very very easy to favour those raised in your own culture.

    So let’s talk AI. The cues that AI is told mean certain things are also culturally conditioned. Usually from how the system was trained. The problem is that AI can’t critically assess itself and say “wait, is that true or just what my upbringing says?”

    Now there are many employers who don’t critically assess their biases, and that’s a problem. But transferring those biases to AI and then claiming it’s unbiased because it’s AI is much much worse. So that’s where we are. AI remains subject to the garbage in garbage out problem, so pretending that it’s unbiased is untrue. What AI does is apply the same biases to everyone. That’s wildly different from being unbiased.

  • Automation and Career Development

    This was originally a twitter thread

    I’m seeing a lot of people talking about how people should go into HS only jobs or trades instead of university. Lets put aside that the unemployment rate for trades is often worse than jobs that require a university degree, instead I’ll tell you a story about the economy.

    I grew up in BC. And the alternative to university that was pushed when I was in high school was either the family farm (I lived in a farming community) or the lumber industry. FYI, this is a #CareerDevelopment story.

    It was the 90s and the lumber industry was strong. If you weren’t from a farming family the non-university jobs talked about were forestry/lumber, construction, plumber/electrician, and first responders. In the mill towns it was pretty much just forestry/lumber.

    The forestry and lumber industry was very people intensive. People to cut trees, people to plant trees, people to move logs, people to run the mills, people to support all of those industries, people to work in secondary industries (wood product manufacturing).

    So it’s the 90s and there’s about 100,000 jobs in the industry. They’re good jobs, well paid jobs. Most of them require no post-secondary or maybe a certificate.

    When I moved to Calgary five years ago the way people talked about the oil sands was exactly the way people talked about the lumber industry in BC when I was a kid.

    Now, I say that there were good jobs, and there were, but the number of jobs wasn’t really going up. And this doesn’t get noticed in the short term, but what it means is that the industry isn’t growing, which means the future won’t be bright for people trying to get in.

    Oh, productivity kept going up, the money the industry brought into the province kept going up, but employment was stagnant. That was never mentioned to teens looking to what their future could be though.

    So, what happened to that industry? Well, the 2000s happened. And at the end of it the industry had shrunk 50%. The 2000s were filled with talk about how we needed to “retrain” forestry and lumber workers.

    Magic bullet after magic bullet was proposed. The government started talking up trades, while ignoring the increasing trades unemployment rate. The jobs that had lower unemployment? Work that required a bachelors degree.

    FYI, here’s the Forestry & Lumber industry over 20 years. Yeah, it was bad.

    I talked in depth about the so-called Trades shortages about five years ago. TL:DR the only trades that have lower unemployment than bachelors degree requiring jobs are the ones that required two years of post-secondary apprenticeship program.

    That’s an important point a lot of people forget. Trades school in Canada is run through the same post-secondary system as Bachelors. The programs are generally 1/2 the length, but that’s it. So when I talk about post-secondary I mean Certs, Diplomas, Trades, and Degrees.

    What’s the point of this story?

    1. jobs that don’t require post-secondary are being automated
    2. once a resource extraction industry automates they never bring the jobs back
    3. people with post-secondary have an easier time changing industries when jobs disappear

    So, if you want to tell someone not to go to get a Bachelors degree, you’re still probably going to be telling them to go to post-secondary. That’s the way of the world now.

    As I look back on the people who talked up forestry when I was a kid I notice something. Most of them were let go when the mill automated or they changed industries in their late 40s. Some of them went to post-secondary then to retrain/reskill, and that’s a good thing.

    But here’s where it comes to Alberta. The same automation warning signs are there for the oil & gas industry. I had a student who I worked with a few years ago. He’d spent 15 years in the oil sands and decided to change jobs. Why? Because he saw the signs. He knew that his job was going to be automated in the next five years, so he decided to train now for the IT job that was going to replace 10 people who were doing what he was doing before.

    And that’s where we get back to #CareerDevelopment. Students need to learn not what the past industries were, but what industries are growing and flourishing. That is going to require post-secondary, of some kind.

  • My Inbox Zero Method

    Good Morning! I said I’d write a twitter thread about inbox zero, but it was kinda too long for one, so instead it’s a blog post. Here we go, I wrote this mostly as I worked on getting back to inbox zero yesterday.

    I just got back from 4 days of vacation, and I had let everyone know I would be on vacation, so I didn’t get as many emails as normal (and outlook tells other people in your org that you’re on vacation before they send you an email). Thankfully I only had around 200 emails to go through.

    I did hit inbox zero before I left for vacation, that lasted about 5 minutes. But I didn’t want to be checking my email constantly. I did checked it twice to make sure nothing urgent came in, and I responded to the one super urgent email I had.

    So, here’s how I got back to inbox zero.

    Ground Rules

    1. the inbox is a to-do list, but not the only to-do list
    2. you don’t need to be on as many listservs as you think, and if there’s a daily, or better weekly, digest for it you should be using that
    3. if you’ve dealt with an email it gets filed
    4. dealt with means that you have no current actions for it
    5. if you’re waiting on a response via email it’s dealt with
    6. if you added it to your formal to-do list it’s dealt with
    7. nested folders are your friend
    8. only have folders you use a lot
    (more…)

  • Surface Phone?

    I love my Windows Phone. Unfortunately it’s time to upgrade and I can get a very good Android phone for $200 cheaper than the new Lumia 950. So I’m going to have to say goodbye to the Windows Phone for a few years. That being said Microsoft has done a good thing getting their ecosystem into Android, so although I’ll have an Android it will be running the same ecosystem apps I’m used to (oh Office, you make my life so much easier).

    But I’m not here to lament the cost of the new Lumia, but rather to look ahead at what might be. A Surface Phone.

    (more…)

  • I Am Not a Wallet

    I had this thought last night when trying out Sims Free Play on my phone. Everything takes forever to complete. While most of these pay to speed up games take about one to two minutes to do a task early in the game, just long enough to figure out what else you can do, Sims takes 10-120 minutes. So you set one task, close the game and come back to it later. It’s rather boring. But the game is fun when you’re playing it. Which is all designed to get you to pay for premium currency to make the game play faster. (more…)

  • Time for Inside Higher Ed to End Anonymous Comments

    There have been many cases against anonymous comments.

    Popular science removed their comments entirely for a good reason, as referenced in a New Yorker article:

    “The editors argued that Internet comments, particularly anonymous ones, undermine the integrity of science and lead to a culture of aggression and mockery that hinders substantive discourse.”

    Huffington post brought up To Kill a Mockingbird:

    “Lee’s basic claim is this: We are capable of doing far worse things to one another when we do not have to own up to the things we do.”

    The cases for anonymous comments tend to focus on the lack of effect restricting it will have as people are still willing to say terrible things in public under their own names.

    (more…)

  • On Technological Change

    We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten.
    -Bill Gates