Do I have to pay capital gains tax when I downsize?

Paying capital gains tax when you downsize depends on your basis in the home you are selling and how much it has appreciated since you bought it, not on the fact that you are moving to something smaller. Long ownership in a fast-appreciating home can create a larger gain than a shorter hold, and your filing status and use of the home as a primary residence both affect the calculation. Brian Beatty can supply a valuation and sale history to hand your CPA, since only they can run the actual number for your return.