A military spokesman said Khalid al-Barnawi was captured in Lokoja, capital of the central state of Kogi.
The US had placed a $5m (£3.5m) bounty on his head after branding him one of three Nigerian "specially designated global terrorists" in 2012.
Ansaru is a splinter group of Nigeria's largest jihadist group, Boko Haram, known for kidnapping foreigners.
Ideologically aligned to al-Qaeda in the Islamic Maghreb, it is also accused of killing a number of Westerners.
Ansaru said it carried out an attack on a maximum security prison in the Nigerian capital Abuja in 2012, freeing dozens of inmates.
"Security agents made a breakthrough on Friday in the fight against terrorism by arresting Khalid al-Barnawi, the leader of Ansaru terrorist group in Lokoja," military spokesman Brigadier General Rabe Abubakar said.
"He is among those on top of the list of our wanted terrorists."