Fucking idiot is too stupid to realize that local DRAM can be used for useful shit, as opposed to whatever he wants to use it for.

Yeah, I reflected on your comment for a bit and I can see what you’re saying. All software uses dependencies, but how they are shipped is different for a lot of languages. If we take an extreme example, Go has a lot of the same microdependencies that Python, JS, etc. have, (probably a lot more than Python even), but since programs are usually compiled into a standalone binary, you never think about them as a user. Interpreted languages on the other hand tend to not do this because they’re not compiled, and when you’re installing a dependency, you’re often installing the source code itself, rather than a shared library or .jar like you would in C++ or Java.

A lot of people do complain about virtual environments though, and while I don’t personally have a problem with them, I do wonder why nobody uses bundlers in Python. It seems that something like this already exists, but it doesn’t seem to be widely adopted or used for all the complaints I hear about virtual environments and such.

Unit tests are good IMO, but integration tests are also important. Unit tests are too discrete and don’t expose system wide issues that cross the boundaries of what you’re testing at the unit level. Examples: performance, stability, memory leaks, bugs that are caused by how multiple components interact with each other, etc.

The other thing is that I think people use tests to give a false sense of security. Tests cannot expose issues that aren’t explicitly covered by testing. Design validation is still important and people tend to overlook that because of the false sense of security they get from seeing a test suite pass.

Pretty much all software uses dependencies, my friend.

https://github.com/kubernetes/kubernetes/blob/release-1.23/go.mod

If you didn’t use a profiler, then how can you be sure that the issue was Python and not something else? If the slowness really is evenly distributed through the code as you suggest, then a profiler would show that almost instantly and it’s very easy to do that in Python using cProfile.

On top of that, the Python standard library is built for convenience, not performance. The nature of it being a standard library means that the implementation has to be simple. fastregex uses LLVM and SIMD for example, that’s something that’s wildly impractical for the Python stdlib to support. In contrast, I have no idea what regex or JSON library you used for your Rust code, but if it were outside the stdlib, chances are high that it had much more leeway to optimize performance because it didn’t have all the other maintenance obligations that stdlib code needs to support.

All of this is just pure speculation on my part, but ultimately I’m trying to make a point that it is hard to say that the difference was caused by a language issue and not something else. There are lots of other things that could be contributing to this difference and you can’t really say for sure without meaningful diagnostics.

As an example of this, look at Nicholas Nethercote’s excellent series of blogs on making the Rust compiler faster: https://nnethercote.github.io/

The “bottlenecks” they find result in 1% speedups.

The kinds of software that you and I write on a daily basis is nothing compared to pieces of software that get millions of man-hours per year. Assuming you’re not Linus Torvalds or Ted Ts’o hiding behind a funny alias, of course. I don’t think anyone is suggesting that it’s a good idea to write a compiler in Python (outside of prototyping purposes perhaps) since it’s obviously not I/O bound, but if you and I are trying to optimize something we wrote ourselves, we’re more likely to be in the “nice small hotspot” domain you described, rather than the 1% bottleneck speedups discussed in the link you posted.

Very cool link though by the way, I look forward to reading as much of it as I can…

Did you use a profiler to find the bottlenecks? Did you try using cjson to speed up the json parsing? I would also avoid using the stdlib regex library too. It’s probably more of a convenience/reference implementation, not something to be intended for high performance. Maybe there’s a re2 wrapper for Python or a similar regex library in C with Python bindings.

Rewriting in Rust is fine, but good Python devs know how to find bottlenecks in their software and replace them with fast implementations. I’ve written plenty of performant Python code, it’s not very hard to profile and optimize. Usually the performant stuff is already written in C/C++ or Fortran anyway (numpy, PyQt, etc.)

It’s solid because it’s so simple. And the documentation is great too - the handbook has everything you need!

Sounds like the distro is kinda of a clown show (see this comment: https://reddthat.com/comment/28643289), but it is really nice to see derivative distros coming off of Void. To me, it’s a sign that Void is gaining more relevance in the community which is great because I really think it’s the best distro out there these days.