SIGNAL BROKER - AI GAMEPLAY COMMENTATOR SCORECARD ================================================= A method, not a score. Watch ten minutes of one recording or one session of your own game, and fill in a row each time the commentator speaks. The full method is at https://thesignalbroker.com/blog/how-to-evaluate-an-ai-gameplay-commentator/ Date: ____________ Game: ______________________ Session length: ________ Build (Help/About): ______________ Role: SPECTATOR / PLAYER Talkativeness: ___ Source: recording / my own session OBS recorded: yes / no HOW TO FILL A ROW Relevant? hit = names or reacts to something visible or just finished miss = wrong object or wrong outcome, or filler that fits any game trivial = relevant but not worth a line (not a miss) On time? immediate = within a beat of the moment a beat late = the moment is still alive when the line lands stale = the scene has already changed (menus, respawn, next play) Repeat? yes = the same point, phrase or joke again with no awareness of it (referring back to an earlier moment is memory, not a repeat) Keep? your taste: would you want that line in the session, yes or no ----+------------------------+----------------------+---------+-----------+------+----- m:ss| What was on screen | What he said (short) | Relev. | On time? | Rep. | Keep ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- | | | | | | ----+------------------------+----------------------+---------+-----------+------+----- SILENCES (each quiet stretch longer than a minute) from-to | what was on screen | right / wrong / voice recovering ----------+-------------------------------+--------------------------------- | | | | | | Right silence: a menu, a loading screen, a cutscene, nothing in particular. Wrong silence: a clear moment passed unremarked while the Broker Bar read LISTENING (not PAUSED). VOICE RECOVERING is a service state, not a judgement. In a recording you cannot see the Bar, so judge silence by the screen. TOTALS Lines heard ____ Hits ____ Misses ____ Trivial ____ Immediate ____ A beat late ____ Stale ____ Repeats ____ Wrong silences ____ Voice recovering ____ Verdict in one sentence: _________________________________________________ NOTES * Signal Broker's words and voice are generated by AI; a line can be wrong, repeated or badly timed. This sheet records your own judgement of one session; it is not a benchmark and the maker publishes no scores. * Change one thing at a time between runs (role, Talkativeness, game). * If you publish a sheet, say which build, role and Talkativeness you used.