It was claimed today that the Take That judge issued the ultimatum after growing tired of Walsh's "constant bitching and sniping".
"Gary is unsure about his future on the show," a source alleged. "The thought of another year with Louis is just too much for him to bear. He says he will only consider coming back if Louis leaves."
However, when questioned about the story on tonight's edition of The Xtra Factor, Barlow replied: "I don't read the newspapers. Is that what I said?"
After first joking that he did indeed want Walsh fired, Barlow clarified: "I tell you now, I wouldn't come back if he wasn't here."
The Irish music manager took the opportunity to gush about his love for Barlow, revealing: "You know what, I so appreciate this man. I grew up with his records, he's been a massive inspiration to me, more than he ever knows. And I really mean that."
The judges then thanked Xtra Factor hosts Olly Murs and Caroline Flack for their efforts and declared that they wanted them to return to the show next year.
"That's if we're back," Tulisa Contostavlos teased, prompting Barlow to jokingly warn once again that he would quit if new presenters were hired.
Little Mix were announced as winners of The X Factor 2011 earlier tonight.