Earning wrath by being veg

I left meat eating and egg eating from my diet some 8 odd years ago. From then on, every time someone offers me a biscuit or cake, I politely refuse to have them. They get curious to know why. I tell them I avoid eggs and meat from my diet. They scorn at me, stupefied by my refusal to have those biscuits or cakes. What is wrong? My resolution to not have them or their anger at my refusal to have something that they offer out of courtesy or affection? I don’t know.