would a prince of time be able to destroy the memories of something that happened in the past, or is that more of a mind thing?

That sounds more of a Mind thing, if anything the Prince of Time would destroy the effects of an event but not the memory of it.