I like to think Hannibal is okay with Will touching his furniture, books, and the like, freely because Will is good with them. He is mindful that these are things Hannibal takes care of and it shows in how he handles them with decency. Versus Franklyn who can’t read the room to save his life (literally) and throws away used tissues on Hannibal’s table.