What happens when AI speechwriting goes wrong?

So, there's this recent incident in the Canadian legislative assembly that's got people talking about the potential pitfalls of relying on AI for speechwriting. Bill Oliver, a member of the Progressive Conservative Party, gave a speech that included a line that seemed to be an LLM prompt instruction - yeah, it was pretty awkward. The speech had a section that started with "here's a more natural, flowing version of that section," which sounds a lot like an LLM suggesting an alternative style option in response to a prompt. It's got a lot of people wondering if Oliver just blindly relied on AI to craft his words.

The whole thing has sparked a discussion about how much we're using AI in professional settings, and whether we're relying on it too heavily. I mean, AI can be a huge help with speechwriting, but it's no substitute for human judgment and critical thinking. The fact that Oliver read out the apparent LLM response without even realizing it wasn't part of the actual speech is a pretty big red flag - it shows that professionals need to be more mindful of how they're using AI. As one expert noted, it's crucial to find a balance between leveraging AI's capabilities and maintaining human oversight.

This isn't an isolated incident, by the way - lots of professionals have faced embarrassment when their AI use has been revealed. Lawyers, authors, journalists, academics... they've all had problems when their LLM use has come to light, often because of obvious errors in their work. A study from Duke University found that workers tend to hide their AI use because their colleagues see it as "lazy" or "replaceable" when they use LLMs. So, as AI becomes more prevalent in professional settings, it's pretty clear that we need to establish some guidelines and best practices for using it - that way, we can avoid incidents like this in the future.

Source: Ars Technica