State governor Nasir El-Rufai said 21,780 teachers, two-thirds of the total, had failed to score 75% or higher on assessments given to pupils.
He said 25,000 new teachers would be recruited to replace them.
Mr El-Rufai made the comments at a meeting with World Bank representatives in the state capital, Kaduna.
"The hiring of teachers in the past was politicised and we intend to change that by bringing in young and qualified primary school teachers to restore the dignity of education in the state," he was quoted as saying by Nigeria's Daily Trust newspaper.
Mr El-Rufai said problems with pupil-teacher ratios across the state would also be addressed.
He said some areas currently had one teacher for every nine pupils while in other areas the ratio was one to 100.