CPT® 99350 Medicare denial

We are getting denied from Medicare for CPT® 99350 (billed for a patient home visit) due to missing/incomplete/invalid service facility address and when we talked with Medicare representative they said that we need to bill service location address in Box 32. We are billing patient home visits without service facility address and some of our claims are getting patient and others denied (mostly denied claims are those where Medicare is secondary payer). Please help me what I do in this scenario. Thank you!