Kelly has published more than fifty books of poetry and prose, including Red Actions: Selected Poems 1960-1993 (1995) and a collection of short fictions, A Transparent Tree (1985). Many were published by the Black Sparrow Press. He also edited the anthology A Controversy of Poets (1965).Kelly was of great help to the Hungryalist group of poets of India during the trial of Malay Roychoudhury,with whom he had correspondence,now archived at Kolkata.

Kelly received the Los Angeles Times First Annual Book Award (1980) for Kill the Messenger Who Brings Bad News and the American Book Award, Before Columbus Foundation (1991) for In Time.