Headcanon Hannibal and Will had soul contracts so that explains everything.