The MMA champion, who won a bronze medal in judo at the last Games, revealed that the swimmer was distant during the Team USA celebrations.
"These NBA players are a bigger deal than this guy, and they're hanging out with us," Rousey told Yahoo of Phelps.
"We're teammates," she continued. "We're not a bunch of groupies. Come hang out with us."
Rousey went on to say the 27-year-old behaved no differently when members of Team USA visited Oprah Winfrey for an interview.
"I don't like being somebody's teammate and being treated like I'm a groupie," she said. "I didn't even want to go out of my way to say hello to that guy. Get over yourself.
"All you do is swim," Rousey added. "If someone slapped you every time you jumped in the pool, I might have a little respect, but I was a swimmer before I did this stuff, and you can't tell me that swimming is OH MY GOD, because it's not."
Phelps recently revealed that he gained 25lb after the Beijing Olympics.