Drummond was plagued with ill health late in his career. After brain surgery in 1969, he was working his way back to the majors when he died while umpiring a minor league game in 1970. He was a member of the umpiring crew that called a balk on Stu Miller during the 1961 All-Star Game at Candlestick Park when the little pitcher was blown off the mound by the wind.