Louis Walsh is set to "settle a few scores" by writing a tell-all autobiography.
The 65-year-old music manager has been axed from "The X Factor" panel for a third time, and he said to be planning to write his memoirs, which could see him unveil some secrets from his time as a judge.
A source told the Daily Mirror newspaper: "He is going to settle a few scores."
Louis - who will reportedly be paid for the upcoming series because he has a two-year contract - was first axed from the talent show's judging panel in 2007 when he was replaced by Brian Friedman, but he was quickly drafted back in and the choreographer was appointed as the show's creative director instead.
Eight years later, Louis said he was quitting to focus on full-time music management, but he returned to the show by 2016.
Louis has appeared on all but one series since the programme's 2004 launch, but recently insisted it was time for a change.
He said: "I've had a fantastic 13 years on "The X Factor" but the show needs a change and I'm ready to leave."
Louis won't be leaving Thames and Syco for good though as while he won't be working on "The X Factor" anymore, he will still be the head judge on "Ireland's Got Talent", which returns in 2019.
He added: "I'm looking forward to another series of "Ireland's Got Talent", going back to my day job as a manager, writing my memoirs and seeing the world!"
Amelia Brown, Thames" managing director, added: "Louis has been a huge part of The X Factor's success since it launched and remains a good friend of the show.
"His contribution cannot be overstated. We would like to thank him for all he has given to the show over the years and look forward to continuing to work with him in the future."