Criminal defendants commonly have to surrender their passports as part of bail or other types of agreements, but Americans who owe the IRS more than $55,000 in back taxes can also have passport problems.