respect/violated

I am presently working in a man environment, we are only 5 ladies out of 35-40 mens working there. What is nice about it, is there is less hormones spears between ladies but there is a lot of testosterone which I had to deal with, this week. These mens are usually very nice and friendly in a good way, always willing to help and never forget to say hi when we cross each others path compare to ladies that it is very hard to approach and by chance they look at you, you are really not sure if they are pleased to see you or else... There is that gentleman he is so nice to me and very talkative he enjoyed joking around but this week he kind of make me really unconfortable and I felt like being violated by what he said he didn't mean to hurt me, he was just joking but that is not the type of jokes that I can tolerate. I honestly had to tell him that I can't take those type of jokes but how can you do it without being rude?