Asking the right questions
Making sense of AI governance, one complication at a time
A friend of mine told me over dinner the other night that when she uses her favorite chatbot, she can’t help but get sucked into a vortex of curiosity — each question leading to the next, something illuminating sure to be revealed if she sent off just one more inquisitive prompt. I recognized the feeling. It’s intoxicating to get answers to enormously complicated questions in under a minute.
But asking questions isn’t free, whether our questions are about AI, about policy, or about society. The questions we ask shape — and constrain — the answers that can be revealed. How can we prevent AI-driven harms to young people will yield a dramatically different answer than how can we verify whether an AI system has the capability of deception? Both questions can be valuable, depending on the circumstances. But only if you’ve chosen your question with an eye toward what kind of answer will actually change someone’s thinking, or better, their behavior. As Einstein is said to have put it, given an hour to solve a life-or-death problem, he’d spend 55 minutes figuring out the right question to ask.
Something you should know about me: I’m a bit of a word nerd. I like when I find a word that means exactly what I need it to mean (indeed, I find AI a useful tool to find such words, as a sort of super-powered thesaurus), and I like when I can spot two people using a word they think reflects a shared understanding of a concept but actually means something entirely distinct to each person (see e.g. “bias”). I’ve competed in pun competitions (I haven’t won, but neither have I been disqualified for coming up with an idiom rather than a pun).
I like the word “elicitation” for exactly these reasons. It means a method to draw out latent or unrevealed information of some importance. It also, conveniently, refers to the practice of researchers probing AI systems for hidden capabilities and knowledge — and, on the other side of the coin, to AI agents gathering information from users that might be necessary to achieve an objective accurately and safely. Elicitation means everything I want it to mean for what I’m doing here: asking interesting questions about the state of AI, the efficacy of safety and governance interventions, and the potential impact of emerging research and policy proposals on the responsible development of artificial intelligence.
I plan to use this space to explore those questions, and to feature friends and colleagues whose analysis has made me reconsider my own understanding of a topic. In a domain that often thrives on conflict and division over terms, priorities, ideologies, and frames, what I most value in is cultivating our ability to disagree accurately — to ask curious questions, and hard ones, that get to the marrow of an issue. To elicit our way to solutions that actually make a difference, rather than ones that just sound good on paper.
In that spirit, my first question is this: how will ad-driven business models reshape interactive AI systems?
(A version of this article ran in the Globe and Mail earlier this month)
Earlier this year, when I learned of OpenAI’s plans to add advertisements to ChatGPT, my heart sank. But I can’t say I was surprised.
Before taking my current role at the Center for Democracy & Technology, a 30-year-old nonprofit focused on civil rights and civil liberties, I worked at Meta as the company was investing in increasingly sophisticated ways to deliver ads to its billions of users. The news of ads’ arrival in AI systems felt eerily familiar.
Like most frontier AI companies (and like social media companies before them), OpenAI faces mounting pressure to justify immense investments in infrastructure, energy, and other operating costs, with subscription revenue covering only a fraction. Tech company employees have lots of different opinions about how their products work, but at the end of the day it’s economic incentives that shape the tools. And I’m pretty sure I’ve seen this movie before — the sequels, too, and the remakes.
From what I can see, nothing is stopping any of these AI labs from following the playbook of platforms that came before it. Once the seal is broken, the lucrative revenue spigot will be hard for others to resist. Meta has already announced plans to use conversations with its AI chatbot to power ad targeting across Facebook and Instagram. Google now displays ads in its AI-generated search summaries. Looking at the business models frontier AI companies are pursuing, a clear pattern is emerging: advertising is coming to AI.
So what does this move mean for AI, and for all of us?
In advertising, the name of the game is intent: whether people are poised to take an advertiser’s desired action. The stronger the intent, the higher the prices ads command. Searching the web for a hotel in Barcelona signals much stronger intent than casually flipping through a travel magazine. Conversations with AI assistants are uniquely revealing: after a sustained exchange with a chatbot to map out a vacation to Spain, the chatbot knows not just that you’re planning a trip, but your budget, travel companions, and which specific experiences appeal to you. And as AI tools integrate memory, these insights accumulate rapidly.
These deep signals mean that — as long as they don’t completely destroy user trust — ads are likely to be highly effective, attracting more advertisers and making the business model appear to kick into gear. The ads will seem relatively benign at first: paid promotions set apart from organic responses, with commitments that ads will be helpful to people. But if these first tentative moves avoid significant backlash, things likely won’t stop there.
As more advertisers vie for space, the next challenge is figuring out which ads to surface to whom. At Meta, this happened by predicting an ad’s relevance — how likely it is to result in the advertiser’s desired action. Since it’s tough to know for sure what people find useful, ad platforms rely on proxies: clicks, cart additions, purchases. Once some people act on some ads, developers can build AI models to predict who else might do the same, based on whether they share characteristics like demographics, interests, or behavior.
As revenue from basic ads fills the coffers, developers will start getting creative. OpenAI’s current CEO of Applications pioneered creative ad formats in her previous job at Instacart, from sponsored products to last-minute checkout suggestions. The company has assured us that ads will be clearly marked and won’t change the organic output. But chatbots are naturally interactive, and it won’t be long before the pull to make monetization feel organic overcomes existing hesitation. I’ve watched this particular assurance erode before.
Generative AI leaders have already floated the idea of affiliate marketing, where outputs remain organic but the company gets a kickback if the user makes a purchase based on ChatGPT’s recommendations. This theoretically maintains AI companies’ editorial discretion, but incentives could nudge platforms toward more frequently recommending the more lucrative options — or letting profit tip the scale between equally useful responses. As AI tools are increasingly marketed as companions or friends, they risk taking on the dynamics of online influencers (or worse), raising questions about what they should disclose about financially motivated recommendations.
Affiliate marketing can also easily bleed into lead generation. OpenAI stated in its announcement that “we never sell your data to advertisers,” but lead generation is different: platforms prompt users to voluntarily submit information to advertisers seeking prospective customers for big ticket items like loans or business software. Lead generators offer lucrative commissions and are common in healthcare, legal services, staffing, and higher education, but are also known for peddling harmful products like payday loans and for-profit universities.
Consider what happens when a user asks a chatbot for advice on how to afford college. It would be simple for a chatbot to check whether they qualify for student loans, then offer to pass their information to solicit bids — including to lead generators for lenders offering high interest rates. And it would be easy to justify that the AI was simply being helpful, and that nothing it recommended was influenced by ads.
To be sure, advertising isn’t the only problematic business model for AI. Hyper-personalized subscriptions can lead to addictive products and discriminatory outcomes. Government contracts create dependencies that could compromise companies’ independence, if the risk of losing the contract would hit the bottom line hard enough (or lead the government to make trouble for a company’s other business lines). Commerce features that blur assistance and sales raise conflicts of interest. Each introduces its own perverse incentives.
But OpenAI’s tentative embrace of advertising is clarifying. How they roll out these offerings, and whether they stay true to their current commitments, will speak volumes about the pressures on companies to prioritize revenue over the interests of the people they claim to serve. We’ve seen with social media what happens if that pressure wins out. We know where it can lead.
The AI industry keeps insisting this time will be different, that they’ll build responsibly, that they’ve learned from other companies’ mistakes. I worked inside one of those companies. I saw how good intentions are vulnerable to crumbling under revenue pressure. I argued against systems I knew would cause harm, and watched them ship anyway. General purpose AI products are still in their relative infancy, and there’s still time to change course.
So one question I have is how these business models will play out. They exert a force that individuals — even CEOs — find nearly impossible to resist. And they’ll tell us a lot about whose interests these tools may ultimately serve.