Two women carrying babies blew themselves up in the town of Madagali on 13 January, killing themselves, the infants and four others.
They passed a security checkpoint after being mistaken for civilians because they were carrying children, the BBC reports.
Two other women were stopped at a security checkpoint and detonated their explosives.
Officials told the broadcaster the use of babies could signal a "dangerous" trend.
Islamist group Boko Haram is widely suspected of being behind the attack.
The insurgent group has used scores of women and girls in suicide bombings, prompting suspicions some of those are among the many thousands they have kidnapped over the years.
In one particularly horrific example, a female suicide bomber carrying a baby on her back was shot by soldiers at a checkpoint on 28 November, detonating her explosives and killing the woman and the baby.
On New Year's Eve, a 10-year-old girl was used in a suicide bomb attack in the north-eastern city of Maiduguri.
Boko Haram's seven-year insurgency has killed more than 20,000 people, forced 2.6 million from their homes and created a massive humanitarian crisis.