Back in high school a million years ago I had a summer job making cold calls for a "market research" company. We'd actually show up to work in the morning, rip the next page out of a big telephone directory - and get dialling in our cubicles.
It was crumby work, but at the end of the day you get pretty hardened up to rejection. To be honest, you're not going to hurt anyone's feelings by just hanging up.
Being actively rude isn't cool though - the person's just doing his or her (low-paid) job.