The Mamma Mia! star, who plays Erik Selvig to Jackson's Nick Fury in the Thor sting, said that the filming process seemed to be over very quickly.
Skarsgard told Empire: "It was fun seeing him again. It was just a couple of months ago. I flew over there, went down into a basement, talked to Sam for a couple of minutes and went back home."
He also revealed that he is looking forward to working on the new Avengers movie, saying: "I know what's going to happen to my character in the film but I'm not allowed to say anything because I'll have to sell my house to pay the fines for that."
Thor director Kenneth Branagh recently said that he thought the sting was an "interesting way" to lead to "another story" in the future.
> Read our latest movie news
> Read our review of Thor
Watch Stellan Skarsgard talk about Samuel L. Jackon below: