No one can deny the accomplishments of Prince Charles during his historically long reign as Prince of Wales. But Andrew Morton, Princess Diana's biographer, says Prince Charles will always be remembered for his failed marriage to the late Princess of Wales: "Just as Henry VIII will be remembered for having six wives, Prince Charles will be remembered for having his first! He will always be remembered for his first wife," Morton told PEOPLE.

Prior to his marriage to Diana on July 29, 1981, Prince Charles was considered the most attractive bachelor in the world. However, after only a few years of marriage, cracks began to appear, and after 11 years of marriage and affairs on both sides, especially with Charles and then-Camilla Parker-Bowles (now the Duchess of Cornwall and Charles' second wife Camilla), they separated in 1992. Charles and Diana's divorce was finalized in 1996.

Earlier this year, the Queen expressed her hope that Camilla would one day become Queen consort. It has been 17 years since Prince Charles and his wife Camilla were married and 25 years since Princess Diana died. Still, Princess Diana's legacy continues to rest on the future king.

"His tragedy is that no matter what he does, what he says, or how he acts, he will be remembered for one thing. 'It is the fact that his fairy-tale marriage is over. His life was defined by his marriage."

Charles does not speak publicly about his ex-wife, but according to PEOPLE, he assisted in an investigation in the early 2000s of her death in a car accident in 1997. He was questioned by British police in 2005 as part of Operation Paget, an investigation into various conspiracy theories surrounding Diana's death. Specifically, John Stevens, former director general of Scotland Yard, told the Daily Mail that Charles had written a note written by Diana in 1995, saying, "My husband was in my car to marry my son Prince William and Prince Harry's former nanny, Tiggy Legge-Burke, " He said he planned to have an "accident," lose the brakes, and suffer serious head injuries.

"Yes, there were allegations made about the Prince of Wales and other royals, but we had to find or examine [existing] evidence before we could pose formal questions to him," Stevens said. 'We found no other evidence to support the scenario suggested in Diana's notes.'

And although Diana ended her life 25 years ago this week, she remains in the public eye and inseparable from Charles' legacy.

"Diana's ghost truly haunted him at every turn," says royal biographer Ingrid Sward.