Chief government spokesman Osamu Fujimura announced that the talks will be held in Beijing on August 29 and cover "various pending issues."
Japan ruled the Korean Peninsula as a colony before and during World War II. Ties between Japan and North Korea remain chilly, and they do not have formal diplomatic relations.
The upcoming talks were scheduled after the two nations' Red Cross societies met in Beijing last week to discuss the repatriation of the remains of Japanese soldiers, Fujimura said.
The Red Cross talks, reportedly the first in 10 years, "helped us to deepen our understanding of each other's situation," Fujimura said. He said the two governments have sought since last week to deepen the discussion and agreed to hold a preliminary meeting on August 29.
The meeting will set an agenda for full-fledged talks expected soon afterward, Fujimura said.
He said Japan hopes the two countries will be able to comprehensively solve pending issues and settle their "unfortunate past" so they can establish diplomatic ties.
Japan and North Korea have not held government-level talks since August 2008 because of animosity over their past and disputes over the North's nuclear program and its kidnapping of Japanese citizens in the 1970s and 1980s.
Japan will push to have the kidnappings included in the talks' agenda, Fujimura said. North Korea has admitted abducting 13 Japanese nationals and using them to train spies. It pledged in the 2008 talks to reinvestigate the abductions, but has not yet done so.
In another sign of a thaw, Tokyo plans to issue special visas to North Korean soccer players to allow them to participate in the women's under-20 World Cup starting Sunday in Japan. Japan has banned trade and exchanges of people with North Korea under sanctions it imposed over the North's nuclear and missile programs, but sports and humanitarian visits are considered exceptions.