A single function Jev-like wrapper for LLMs, including vision models

Overview

I was intrigued by Jev and the self-hostable projects appearing around it, such as OpenJev and SemIf. Reading about them introduced me to a neat trick: reading an LLM's token probabilities. Apparently this is an old trick for some people. See e.g. OpenAI's logprobs cookbook. But it was new to me. I believe the basic idea is to write a prompt like this: State: My order arrived broken and I want a r...

Summary from allanrbo.blogspot.com.