I think there may have been other motives for some to speculate about the sexuality of Albert and Friso. For Albert there were those who preferred Caroline as the heiress of their father.
It would require great ignorance of history if they believed homosexuality would necessarily oust one from one's place in the succession.