The session you cannot take with you
earendil.com
July 31, 2026
14 min read
🔥🔥🔥🔥🔥
67/100
Summary
Inference APIs allow users to send input and receive output, facilitating conversation archiving and replaying. However, challenges such as prompt cache storage on external GPUs, differing tokenization between models, and intentional sampling unreproducibility complicate session portability.
Key Takeaways
- Inference APIs are increasingly returning non-portable session data, which limits user ownership and access to their AI interactions.
- Features like encrypted blobs, hidden instructions, and provider-bound state prevent users from fully understanding or continuing their sessions with different models.
- A truly portable session should allow users to export, inspect, replay, audit, and delete their session data independently of the provider.
- The marketing of encryption features often misleads users, as the data is typically only accessible to the provider, not the user.
Community Sentiment
Positives
- This article gives a very good overview of a problem that most users of AI rarely evaluate, highlighting important issues with provider lock-in and user experience.
- I'm optimistic that dark patterns will eventually be replaced by more respectful practices in AI, as the market matures and user awareness grows.
- Many commenters are finding innovative ways to maintain context across sessions, suggesting a creative push towards better user experience in AI interactions.
Concerns
- If switching models is painful, providers can create vendor lock-in, enshittify the user experience, and drive up costs — this is a huge concern for future competition.
- The coupling of powerful extensions with inference APIs creates unnecessary barriers, limiting user freedom and innovation.
- Some models are overly verbose, resulting in poor signal-to-noise ratios that frustrate users and hinder productivity.
![[PSA] Anthropic's Method to Losing Goodwill in a Few Easy Steps](https://pub-90f0ac00d93c47daac3e1d2cdd28d496.r2.dev/articles/bc0d5e0d91ce2945c6d54237cac8fe90.webp)
