Murs got his career breakthrough on Cowell's X Factor UK in 2009, earning instant praise from the notoriously tough judge.
In an interview USA Today, the singer said Cowell's encouragement was validation of his "talent".
"If there's one person that could see if I had talent, or if it's going to be rubbish, Simon was the person I was waiting to hear from," Murs recalled.
He continued: "When he said [at my audition] that I was the easiest 'yes' he's ever given on the show, that was an amazing moment for me!"
Murs - who ultimately finished as runner-up in The X Factor - has spent this summer opening for One Direction in the US.
"The [One Direction] fans are insane! Running after the bus, jumping from trash cans, just so infatuated with the guys," he joked.
"One Direction fans wanted to know who I was. American fans are brilliant!"
Murs previously revealed that he once caught One Direction fans rummaging through a bin in search of the band's trash.
Watch Olly Murs's video for 'Oh My Goodness' below: