There’s a case to be made that most popular chatbots will reduce to another attention economy — e.g. running advertisements, like they already had begun experimenting with on BingChat. Attention is the most popular measurement in the digital economy to date because of its universality. I’m hoping that AI enables a new bunch of internet mega-corporations that are more based on services and generating value rather than extracting value from users. Until this is proven, attention is a useful domain for analysis. In the case when performance is based on services provided, these companies will need new methods of measurement
Product-focused organizations likely have better tools for measuring user behavior, as we’ve seen with Meta’s versus Snap’s response to the ATT shakeup of advertising. Following the gap between ML-first organizations like OpenAI and Anthropic versus those with a history of delivering user benefits will be fascinating in the chatbot space. It’s likely the biggest reason to bet on Meta’s Llama moat strategy. I’ve heard from big model labs that they have evaluation problems — they don’t compare to other big models enough. If this is the case, they can be left behind quickly if one organization actually figures out how to measure and improve based on user engagement.
Meta’s Llama open-source strategy implies a bet that the model layer is not where user engagement will be captured. By releasing the model, Meta concedes that value accrues at the product layer — the organization that figures out how to measure and optimize user engagement on top of an open model may win, regardless of who trained it.The general hypothesis reduces to the following fact: how people use technology depends heavily on the context, and people will be emotionally available in a much deeper way in a direct messaging app. News feeds are for passive engagement, messages are intentional
We are going to increase the scale of the sociological experiment we’re running: transitioning from the era of making friends in person and transitioning them online, to the era of making human friends online, to the final frontier of making artificial friends online.
[removed: short reaction]While I’m mostly concerned about this feedback loop because I care deeply about how AI interfaces with society, there are also clear modalities where letting that outer feedback loop on user behavior evolve without constraint can be seen as beneficial, like entertainment. This same process of integrating the behavior of the user can let scenarios like entertainment become deeply personal.
[removed: personal reaction]They can run more experiments, and see how the RL policies are changing the context of the domain they’re operating in, but so long as RL systems are designed with a specification that the problem is static, the long-time impacts will be hard to change. RL technologies were developed primarily in toy domains where the robots are always the same, but now they’re going to slightly nudge humans, resulting in a long-term dance where the models and users co-evolve together.
We saw this with both Stable Diffusion for image generation and ChatGPT to text manipulation. As time goes on, the number of untapped modalities of digital information will continue to shrink, so opportunities for this may whither
Heidegger’s concept of “standing reserve” — treating all resources as raw material for extraction — applies to AI consuming digital modalities. But the critique assumes technology serves economic value. If technology is employed to make stories more sacred or to create artistic rather than economic value, the standing-reserve framing may not hold.Though, these companies need to signpost what they’re doing and capture reputation, so most new domains will be shipped when it is still a little too slow to feel right. ChatGPT woke everyone up to try products, which can only happen once.
[removed: paraphrase without original observation]We’re not used to working in domains where information can come and go in tons of different modalities. We’re also not used to the words we type immediately becoming tangible in the real world. The litmus test for these is that we know they’re coming, but it is incredibly hard to imagine life with them
When new modalities emerge, the hardest question is where success will be measured. The pressure to serve market adoption forces developing products toward user-friendliness as the default metric, even when the real value may lie elsewhere.
