The men were allegedly led by suspended lawmaker Ovie Omo-Agege who seized the symbol of authority from the upper legislative chambers sitting in Nigeria's capital Abuja, the Senate said.
The incident was branded "an act of treason" by Aliyu Abdullahi, a Senate spokesman who said the lawmaker was trying to overthrow a branch of the Federal Government of Nigeria.
"This action is an affront on the legislature," he said in a statement.
"Security agencies must stand on the side of due process and immediately mobilize their personnel to retrieve the mace and apprehend the perpetrators of this act," Abdullahi said.
Laws cannot be passed by Nigeria's Senate in the absence of the mace, a symbol of authority binding decisions taken on the Senate floor.
Despite the incident, sessions continued Wednesday in the 109-member assembly with a spare mace, Shehu Sani, a senator representing Kaduna Central said.
"I and others insisted that the Senate must not adjourn. I removed my waist belt and lay to serve as a mace for us to continue."
"My colleagues seconded. Then a spare mace brought in and the session continued," Sani added.
The lawmaker who condemned the attack on Facebook called the incident an attempted coup that could fester if not addressed.
"Those who sanctioned,organized and supported this will someday do that to the Presidency or the judiciary," he added.
Nigeria's ruling party, the All Progressives Congress, APC, has called Wednesday's incident an attack on the country's democracy and called for arrests.
"The APC hereby condemns this action and views it as an attack on our democracy and a desecration of the hallowed institution of the National Assembly, " Bolaji Abdullahi, a spokesman for the party, said in a statement.
"We therefore call on security agencies to take all necessary actions to recover the stolen mace and ensure that the perpetrators and their sponsors are brought to justice," he added.
Violent outbursts are common in Nigeria's parliament but tensions rose last month over the proposed amendment of an electoral act that seeks to shift the presidential election date until after legislators and other branches have been elected.
Omo-Agege, a member of the ruling party was reported to have objected to the move which led to his suspension from the Senate.
President Buhari is seeking for re-election next year and tensions are set to rise ahead of the polls.
Nigerians are clamoring for change in areas including: security, the economy and corruption- promises made by President Buhari during his campaign before he took office in 2015.
Four years on, Buhari has faced criticism over increasing attacks by terror group Boko Haram in Nigeria's northeast, frequent attacks from Fulani herdsmen in the central region of the country and a growing separatist movement in the east of the country.