Russian forces have driven the Ukrainian army out of the biggest town in Russia's Kursk border region, officials claimed Thursday.