Hard to say really, I guess there are loads of places that are rough and plenty of nice places which has its rough parts too!
For me, having grown up on the Lightbowne estate in Moston (classy) I wouldnt really be scared of walking through any part of Manchester!
However, it is always good practice to be sensible and I wouldn't advise any of you Sarf Manchester lot to be walking through the following areas, in particular at night!
Broughton (lower and higher)
Cheatham hill (especially near the red light area!)
Collyhurst/Miles platting (either side of oldham rd heading towards town centre)