After killing all the adds, I ended up luring him down near the entrance gates to give myself enough time to make it over to the exit gates (the ones that open up after he's defeated).
I tucked myself in the southmost corner at the exit gates to see if he'd be dumb enough to get stuck near the ledge on his way over to me. He did. I took advantage of his stupidity and meteored him to death.