Housing would still be built, but not as an asset for which appreciation would be expected. Your argument is like saying that unless cars appreciate in value, no cars would be built.
All else being equal, lower return on houses would result in fewer being built.
But all else wouldn't be equal, since regulatory barriers would be removed.
Indeed, your notion that houses would be spoiled as an investment, and fewer would be built, are contradictory. The former requires so many are built the market would be saturated, preventing appreciation.
Found the problem!