I don't know whether there was ever any reason to believe that Carl and Anne were attached to one another. In a biography of Diane which she co-authored with Jacqueline Malherbe, she says that Carl (and, at times, his entire family) spent vacations with her family, both in Portugal and in France. She also says that, once she turned 15, Carl came to visit more often. Of course, Diane's eldest brother married Carl's sister in 1957, which brought the families even closer together. The idea that Carl would marry Anne may have been wishful thinking on his parents' part. It is hard to know, since all the accounts I have seen come directly from Diane, and she definitely has an axe to grind (or more to the point, an axe to use against her sister Anne).