if in a timeline the players simply decide to not play the game,does that count as a doomed timeline or is the game eventually use time mumbo jumbo to convince them to play it?

Yes, there was a timeline where Gamzee killed everyone before they could play and that counts as a doomed timeline