So I actually wrote my doctoral dissertation on foreign accent in Mandarin and how native speakers distinguish between native and non-native voices (with and without other cues).
There are basically three things to sounding native-like: the sounds, the tones, and the overall prosody, like not letting English sentence-level intonation creep into your Chinese sentences. Well, and fluency, but that’s another matter. If you’re halting, they’ll probably perceive an accent no matter how accurate your speech is (assuming again that they can see you and/or know you are a foreigner in the first place; perceptions also change when native speakers are mistakenly identified as foreigners, though that doesn’t seem to happen too often.) I didn’t test on visual identification for my dissertation experiments, so I don’t know what would happen if you had an Asian-appearing person who was not a native speaker, like an ABC/CBC/etc, but logically it should have some effect as it “confuses” the brain that’s expecting the person to belong to a certain group and speak a certain way.
To the OP’s question: a couple of things help a foreigner’s accent to sound more native-like (which native variety is another question as has been said above). Look hard at vowels and consonants and get someone who knows segmental phonetics to tell you exactly what part(s) of your mouth/tongue need to be in exactly what position and what has to happen to make the “native” sound. Better yet get someone who can compare how the “same” sound is produced in English and Mandarin. Most Chinese teachers really don’t know how to do this as they are mostly out of literature backgrounds, not linguistics.
Narrow your tone space. We tend to be taught tones in an exaggerated way (which IMO is needful), and that sticks with us. When I was in interpreting school, my Chinese-A classmates suggested doing reducing the size of my tone space and it helped a lot in evaluations. You will still pronounce tones, but over a narrower band of pitches within your particular voice space. (If you also teach Chinese you’re SOL on this one as that’s what’s needed for beginners, really. lol)
Anyway just some thoughts. The best evaluation I ever got of my over-the-phone accent was one time years ago when a client said, “Oh, you’re American? So that’s why you sound like a drunken Cantonese speaker.” I decided to take that as a compliment based on where my Mandarin really was/is.