@graf
Immagine that all the matter in the universe is a water in a baloon.

You are standing in an infinite hallway. But there is no water anywhere.

Then you take a needle and BIG BANG!!!

from then onward, the universe will expand into infinity, but the matter will remain the same.