The director revealed that he encouraged Jackson to bring the sadness of his Unbreakable performance to his portrayal of Nick Fury in the new superhero epic.
"The thing about Samuel L Jackson is that I always think there are two of him. He's famous for the sort of bravado of Pulp Fiction, he's the speechifying guy who can out-moxie anybody in the room," Whedon said, reports Flicks and Bits.
He further explained: "Of course as a huge Unbreakable fan, I'm also very much in love with the great well of sadness that he brings. I told him from the very beginning, my biggest note throughout the film was, 'Less Shaft, more Glass'.
"Because what I wanted to see was a guy who could, yeah, absolutely command a room with his voice, could absolutely be the guy you could never question who was in charge of this enormous organisation and everything around it.
"Could be a guy who could do stuff that was morally compromised, yet absolutely necessary. But at the same time would feel the burden of that. To be the leader means to separate yourself from everybody else."
Whedon previously revealed that The Dirty Dozen and Black Hawk Down served as inspirations for The Avengers.
The Avengers opens in UK cinemas on April 26, and May 4 in the US.
> Stellan Skarsgard on Avengers, Thor 2: 'I have fun in Marvel films'
> Scarlet Witch, Quicksilver could appear in Avengers, X-Men
Watch a trailer for The Avengers below: