He attended George Watson's Hospital, Edinburgh, before entering the employment of Captain Brown, an ornithologist, as draughtsman and colourist.
He did not, however, remain long in that position, and found more congenial work in the teaching of drawing before becoming a student at the School of Design, under William Allan.
He was eighteen years of age when he first exhibited in the Royal Scottish Academy. He became an associate of the academy in 1846, and was elected an academician in 1852.
He had a prominent role teaching drawing and in 1857 was chosen as librarian of the academy. On the death of W. B. Johnstone in 1868, Drummond was appointed curator of the National Gallery of Scotland. He was elected a fellow of the Royal Scottish Society of Antiquaries in 1848 and curator of the National Museum of the Antiquaries of Scotland between 1851 and 1854 and from 1861 to his death.
