Concerns over accommodation for asylum seekers have brought political division into the spotlight in parts of Wales