The Secret Service group assigned to the White House, both uniformed and non-uniformed, are assigned to the “Office of the President,” not to the individual POTUS. They protect members of the Presidential staff and family as well as members of Congress and Senate if needed.
Point being - The Office holders may change but the Secret Service members quite often remain on that particular assignment. The most common reason for a Secret Service member assigned to White House detail to leave the team is job ‘burn-out.’ Most are married and the demands of the assigment are not conducive to a family routine.
Being assigned as a member of a protective team for the highest profile world leader is not a task for those lacking in dedication. They are not in it for the money; its a Civil Service job - starting at, I believe GS-14 level - which ain’t all that much living in D.C. Perks are part of the deal - travel on a moments notice, shifts of 18 - 20 hours routinely, free Air Force 1 ash trays and glasses, Diplo-Red passports, being under scrutiny every minute of your day - whoopee…but for the people who chose to be put in rotation for the teams it can be the highlight of their career. And it looks very good on your resume’ (c.v.) after you leave the Treasury Department. (You did know that the U.S. Secret Service was a division of the Treasury Department?)
There is an old maxim in the personal security business - “We have to do it right everytime, they only have to do it right once.”