Numbers are far from infinite. They actually only go up until 9 (or F in base 16). The fact that theoretically you can keep putting on numbers and reciting more of them is no different from any other aspect to which we plug in infinity.
Created objects can’t be infinite but conceptually there is nothing wrong with a concept being infinite. There is nothing real about it. While travel-able space might in fact be limited, theoretically space can go on and on. This is because the idea we are describing is not a created reality that is contained within its given limits. It is a form of measuring. It never actually happened.