During the 1960s Sharma supported Indira Gandhi's quest for leadership of the Congress Party. He was elected as the President, AICC in 1972 and presided over the AICC session in Calcutta. From 1974, he served in the union cabinet as the minister for Communication from 1974–77. In 1971 and 1980 he won a Lok Sabha seat from Bhopal. Later on, he was given a variety of ceremonial posts. In 1984 he began serving as a governor of Indian states, first in Andhra Pradesh. During this time, his daughter Geetanjali Maken and son-in-law Lalit Maken, a young member of parliament and a promising political leader, were killed by Sikh militants. In 1985, he left Andhra Pradesh and became governor of Punjab during a time of violence between the Indian government and Sikh militants, many of whom lived in Punjab. He left Punjab in 1986 and took up his final governorship in Maharashtra. He remained governor of Maharashtra until 1987 when he was elected for a 5-year term as the eighth Vice-President of India and chairman of the Rajya Sabha.
Sharma was known to be a stickler for parliamentary norms. He is known to have broken down in the Rajya Sabha while witnessing the members of the house create a din on a political issue. His grief brought back some order into the proceedings of the house.
