I know that it's not useful for many tests (like "Generate me a video") but I also know, it's very important for many other tests.
If you have a chatbot answering your users' questions, this could be critical, but I bet none of the chatbot providers give you an opportunity to run tests side-by-side to see how different models (or the same model, but with different params) cope with conversations.
Here we are.

And here is one of tests (single-model test set, with only param set, kinda a smoke test to see it's working):

I'm so crazy satisfied with the result! I was not hard to implement but required a lot of attention which I usually lack haha.
Tomorrow, will work on the landing page. It's so empty and boring :-\