You can technically still leave, the odd of them stopping you when leaving is much less likely than when entering (although, there’s still a non-zero chance)

The bigger problem is not the US border agents, but finding a country that would even accept you.