proxymint
Proxy provider: residential, mobile, ISP and datacenter proxies. Pay per GB, no subscription.
We write about the parts of scraping that break: a 407 that only shows up in production, rotation that rotates too often, a request that works in curl and fails in the browser.
Some posts are scheduled with automation tools. Replies are written by the team.
- 9 Posts
- 4 Comments
proxymint@programming.devOPto
Programming@programming.dev•requests, httpx and curl give the same proxy stall three different names
2·10 days agoFair. We run proxies, so “proxy or site?” is the question we care about, and I didn’t trust the error messages to answer it.
The point is what you do after. If the proxy hung you rotate it out. If the site hung you slow down on that site. requests blames the site by name either way, so you can end up throttling a site that’s fine and keeping a dead proxy. That’s why.
proxymint@programming.devOPto
DevOps@programming.dev•Allowlisting an outbound IP doesn't need a static line, just a fixed exit
1·13 days agoit does when the other side allowlists a hostname. plenty of firewalls and saas allowlists only take an ip or a cidr though, and they don’t re-resolve, so the entry breaks the next time your isp renumbers you. it also only helps if the traffic actually leaves from that same connection.
git log -S'some_string' -pis the one that pays for itself. it finds the commits where that string was added or removed, so you can track down when a function disappeared without reading through history.-Gis the same idea with a regex, matching any changed line.and
git log --follow -p path/to/filekeeps going across renames, which plaingit log filewon’t do.




curl -x is fine for one look. in a retry loop you want 407 to stop and 502 to retry, thats why the script. noted on the formatting