When a Phishing Email is Too Good?

If one of the methods to spot a phishing email is to look for poor grammar, spelling errors, word choice, etc., then it is clear to see the potential for AI to limit these red flags.

A new Apple commercial displaying the company's new "professional" writing feature clearly shows the potential.

In the commercial, "Warren" a seemingly underperforming employee drafts an unprofessional message to his boss about a project. With the click of a button, the draft is updated to a very well written and thoughtful email. So much so, Warren's boss is in disbelief that Warren authored the email.

Security research firms claim 40-50% of phishing emails targeting corporate accounts are now written using AI, and that usage has led to an overall increase in phishing attempts of 50-60% since 2023.

I was curious to understand how an email can be identified as being AI generated. Basically, you as a reader or a tool used for detection, would analyze the:

"writing style, which often exhibits characteristics like overly formal tone, repetitive phrasing, lack of personalization, inconsistent context, unusual sense of urgency, and a lack of subtle nuances that human writing typically has; essentially, an email that sounds too perfect or robotic can be a sign of AI generation."(2024, Google AI)

In the past, red flags were poor grammar, now a red flag is excellent grammar.