You can technically still leave, the odd of them stopping you when leaving is much less likely than when entering (although, there’s still a non-zero chance)
The bigger problem is not the US border agents, but finding a country that would even accept you.
I did #2 four years ago. It took a minute for my family to see why. Now it’s damn near too late and they’re stuck hoping #1 is viable.
You can technically still leave, the odd of them stopping you when leaving is much less likely than when entering (although, there’s still a non-zero chance)
The bigger problem is not the US border agents, but finding a country that would even accept you.