Have you been in a position where you just couldn’t catch a break? You were damned if you did and damned if you didn’t? Jesus spent His whole life like that. The religious establishment had condemned John the Baptist and said he was crazy for the lifestyle he lived (locusts, honey, camelhair, etc.). Then Jesus came and did not eschew comforts and hung out with people they didn’t like and they railed against Him as a libertine.
In Luke 7:33, just before our focal passage, Jesus says,
For John the Baptist came neither eating bread nor drinking wine, and you say, ‘He has a demon.’ The Son of Man came eating and drinking, and you say, ‘Here is a glutton and a drunkard, a friend of tax collectors and sinners.’
So, having been called a drunkard, glutton and friend of tax collectors (that was really the worst) by the religious establishment, He accepts an invitation to dine from one of those worthies. One might logically expect that, having now agreed to dine with the “right” crowd, he would be invited with open arms.