Inference APIs allow users to send input and receive output, facilitating conversation archiving and replaying. However, challenges such as prompt cache storage on external GPUs, differing tokenization between models, and intentional sampling unreproducibility complicate session portability.

Tinfoil verifies the identity of models served through inference APIs, ensuring users receive the exact model weights as published. It addresses concerns about potential modifications, such as quantization or altered context windows, in open-source models.