The pivotal moment in modern relations between the Arabian Peninsula's two largest countries came in the 1960s, when a coup brought an end to the Mutawakkilite Kingdom of Yemen and ushered in the first republic on the peninsula.
The transition was not smooth: a civil war broke out, with Egyptian troops supporting the republicans while Saudi Arabia and the United Kingdom backed the deposed royalists. In some form or another, the war rumbled on for years and was finally resolved only in 1970, when the Yemen Arab Republic was recognised internationally.
What is now often called the North Yemen civil war is one of the least understood parts of Arab history in the Nasser era.
Egypt's Gamal Abdel Nasser was keen to spread his brand of nationalism and saw Yemen, then the most populous country on the Peninsula, as the third leg of his strategy. Yemen had briefly joined the United Arab Republic (the short-lived experiment of joining Egypt to Syria), and Nasser appeared convinced he could tie the Yemen Arab Republic into his wider pan-Arab aspirations.
But the war dragged on for years, taking troops, money and political capital away from Egypt and stretching the Egyptian army - to its severe detriment in the 1967 war.
But the effects of the civil war were also keenly felt elsewhere in the Peninsula. According to Michael Horton of the Jamestown Foundation, a think tank, Yemen's republican coup had an impact on Saudi inner circles. "Reform minded factions within the Saudi royal family supported some of the republican/nationalist ideals and wavered in their support for the royalists. The political upheaval in Yemen led to a dramatic reshuffling of the government in Saudi Arabia."
In the years since, the relationship between Saudi Arabia and Yemen has been governed by this tension. It continued under the presidency of Ali Abdullah Saleh, whose erratic behaviour was often a thorn in the side of Saudi.
In one of the most disastrous decisions of a three-decade presidency marked by bad decisions, Mr Saleh, in 1990, sided with Saddam Hussein's invasion of Kuwait, leading Saudi Arabia to expel millions of Yemeni workers.
Since Mr Saleh never repented and remained president until he was toppled last year, there was no rapprochement with Saudi Arabia, and the Yemeni economy took a hit from which it is still suffering. The 300,000 Yemenis who could be deported from Saudi under new visa rules will have a similar effect on the struggling economy.
But neither Saudi Arabia nor Yemen is the same today as it was in 1970. The world has changed immeasurably - the rocketing of oil prices has brought an enormous disparity between the two countries.
Today, Saudi's GDP is 10 times that of Yemen's. Nasser died in 1970 and with him much of the pan-Arab revolutionary fervour he embodied. Political pluralism (such as it is) and universal sufferage in Yemen are no longer a threat to Saudi influence. And with Mr Saleh finally gone, there is a chance of a new relationship between Saudi Arabia and Yemen.
That is what is needed now. The visa expulsions demonstrate that Saudi still sees Yemen as just another country whose population wants access to its labour market.
But that is a mistake, because the long-term stability of the whole Peninsula depends to a large extent on what happens in Yemen. Locking the two countries into mutual development would be beneficial for the whole GCC, which needs manpower (which Yemen has). And Yemen's government needs investment (which Saudi can provide).
The visa issue is part of a wider trend of viewing Yemen as a problem to be solved rather than a solution to some of Saudi's problems.
Viewed from that angle, there is a straightforward policy for Saudi Arabia, one that was the norm up until 1990: allowing Yemen's workers free access to the Saudi labour market.
Before the siding of Mr Saleh with Saddam Hussein, Yemenis were given access to the labour market in preference to people from other Arab countries. The Saudi government reasoned, correctly it turns out, that it was preferable to have people with a common language, culture and religion - not to mention significant family ties - working in Saudi Arabia. That remains the case. And indeed, opening Saudi's labour market to Yemen is a much better policy than endless aid to Yemen, cash that tends to end up in the hands of local leaders without trickling down to the Yemeni people.
The situation in Yemen is dangerous, for Yemenis and for the whole of the GCC. The national dialogue that is currently in progress could provide some political stability.
But it is economic growth, ultimately, that will solve or at least ameliorate the worst of Yemen's woes. It is economic growth that will ensure the problems of poverty - increased militarism and a lack of law and order - will not spill over the long, porous border with Saudi Arabia.
The policy of Yemen's northern neighbour should reflect a belief that, over the long term, an open-door policy is far better than a locked gate.
On Twitter: @FaisalAlYafai
