He says he drew on a Kerry brogue using a dialect coach’s guidance after viewers deemed his performance distracting