What decides sprites' color?

For the trolls it was their text color.

For the kids except Rose, it was their post/pre-scratch guardian’s text color