You are correct in most countries the heir automatically becomes the sovereign once the reigning monarch dies or abdicates. In Belgium the heir only becomes monarch by being sworn in.
When Boudewijn died unexpectedly the official heir was his brother Albert but his perceived heir and the one Boudewijn had been training was Philippe.
It's rumoured that Albert spoke to Juan Carlos and Beatrix before deciding to keep his son out of the limelight for a while and step in himself. I have no doubts that his sister Josephine-Charlotte and her husband Jean also gave their opinion.
For most of his reign Albert II was a successful monarch moving away from his brother's personal rule to a more democratic, constitutional monarch. At the end of his reign things got worse because of the situation about his extra-marital daughter.
In the Dutch constitution abdication is equal to death so the effects are exactly the same. So the succession is clear no matter if the monarch dies or abdicates. Abdication is a constitutional death.
I believe other countries have similar rules that equate death and abdication. The succession rules never really stipulate that things are different but history has taught that (forced) abdications have been used to place someone other than the direct heir on the throne. Think in England of Henry IV succeeding Richard II or Edward IV succeeding Henry VI or France where Charles X and Louis XIX abdicated but Louis-Phillipe succeeded instead of Henri V.
In Russia Nicholas II abdicated but wasn't succeeded (technically) by his son Alexei II but his brother Michael II.
I've read that this country is one of the exceptions to the rule on automatic succession, insofar as the accession of a new sovereign has to be confirmed by Parliament.
Nobody expected King Baudouin to die so soon, or his brother to survive him for so many years: they were only four years apart in age. As is oftentimes the case when the throne passes to collateral lines in a house, the younger brother could easily predecease the older brother, and end up being only a conduit in the succession. That's why the expected successor to Baudouin had, for many years, been thought to be Philippe. In fact, I even heard that the uncle intended to abdicate the throne in favor of his nephew in 2005 (evidently he didn't want to reign until death).
And indeed, in the event of abdication, it's entirely possible to bypass heirs who are closer in line to the throne. This has been stressed in the British case of 1936. Earlier in European royal history, the woman we know as the Grand Duchess Charlotte of Luxembourg was not thought of as the automatic successor to her older sister, the Grand Duchess Marie Adelaide, who was forced for political reasons to abdicate the throne. Indeed, Charlotte almost got bypassed in the succession, for reasons that she was engaged to Prince Felix of Bourbon-Parma, brother of Empress Zita of Austria.
As it was, none of her younger sisters could have worked out, either -- as there were issues with all of them. So in the end, her claim prevailed (they somehow worked around the difficult situation with Charlotte's prospective husband).
Succession is tricky when the throne passes collaterally, as in the Benelux cases cited above, of when a throne is vacated through abdication, not death. I think the fact that Baudouin died before he could abdicate the throne made it easier to confirm his brother as his immediate successor. And indeed, had King Edward VIII of Great Britain suddenly died without any direct heirs, at any time after his accession in 1936, then the Duke of York would have automatically succeeded him on the throne: his status as the new king would not have had to be confirmed. As it was, he didn't die but rather, abdicated the throne, thereby creating a messy situation ...
Alternatively, there were no questions of succession after the 1848 abdication of King Ludwig I of Bavaria (as with the British king, for scandalous personal reasons) or the 1918 abdication of Tsar Ferdinand of the Bulgarians (as with the grand duchess of Luxembourg, for political reasons): in each case, the throne passed laterally, not collaterally. That is: the reigning monarch was succeeded by an heir apparent (the eldest son, known as the crown prince) -- meaning somebody who expected to eventually inherit the throne, anyway.