I was going to say that’s wild, but that’s the whole point of the model isn’t it.

I don’t remember how it all works, but I imagine it’s something like:

1. in = encode(prompt)
2. result = applyModel(in)
3. saveState(prompt, result)
4. out = decode(result)

I think these would all be model aware steps. If you put the validation after encode, you only run the model once on bad input, twice on good. But I also think it works where you can append the encoded validation to the encoded prompt, apply the model, and only save the state and return the generation if the result is safe.

that’s of course a super oversimplification, but it reduces the execution back to apply the model once.