Modi's Bharatiya Janata Party (BJP) thrashed Congress in the last two general elections in 2014 and 2019. Having ruled the country for much of its independent history, Congress is now in power in only three of India's 31 major states and federal territories.
It lost power in Punjab state last week to a decade-old party, while the BJP retained control of four other states where elections were held in the past month.
A Congress spokesperson said its president Sonia Gandhi had asked state party heads of Uttar Pradesh, Uttarakhand, Punjab, Goa and Manipur to resign "in order to facilitate reorganisation" of their local units.
The party's top leadership met on Sunday to analyse the losses, during which Gandhi and her two children who are in senior positions had offered to resign.
Senior Congress members, however, said they would continue to back the Nehru-Gandhi family, which has controlled the party for decades since the time of India's first prime minister, Jawaharlal Nehru. Modi has repeatedly denounced Congress's dynastic politics.
Congress said in a statement on Sunday it "unanimously reaffirms its faith in the leadership of Sonia Gandhi and requests the Congress president to lead from the front, address the organisational weaknesses, effect necessary and comprehensive organisational changes in order to take on the political challenges".