Qifrey minds them, oh boy he so does, since he has a major grudge against the people who did it to him and is trying everything in his power to stop them. But, yeah, he really doesn't want people knowing his past. He's never even told his students about it so having random people find out will be very annoying to him.