The reason: Lady Colin Campbell was being bullied!

Tony Hadley and Duncan Bannatyne were bullying Lady C, it has been claimed.
After the royal biographer dramatically walked out early in the morning, ITV claimed it was due to medical reasons, but it seems the real causes were other. Apparently she's in perfect health, and according to The Sun, last night a source said: "Lady C says she was being mercilessly bullied by Duncan and Tony. On the show it has always come across that she was the instigator of all the rows, but she says this is completely untrue and she was actually the victim."
Lady C claims ITV edited the footage to conceal the bits where this was seen, and instead always showing her as the ibad guy . Threats of calling lawers where even made, and finally ITV decided to let her go, and came up with the medical grounds excuse.
The incident is far from over, "when the socialite decides to talk, it's going to be explosive".