Which Linux command or utility is simple, powerful, and surprisingly unknown to many people or used less often?
This could be a command or a piece of software or an application.
For example I’m surprised to find that many people are unaware of Caddy, a very simple web server that can make setting up a reverse proxy incredibly easy.
Another example is fzf. Many people overlook this, a fast command-line fuzzy finder. It’s versatile for searching files, directories, or even shell history with minimal effort.
I think a lot of people don’t realise that yt-dlp works for many sites, not just YouTube
I used it recently for watching a video from tiktok without having to use their god awful web UI and it was amazing
Since everyone keeps mentioning yt-dlp I gotta ask: what’s wrong with the original youtube-dl? I keep using it, it works, it’s still being updated.
yt-dlp has sponsorblock features, youtube-dl does not.
The huge list of sites can be found here https://github.com/yt-dlp/yt-dlp/tree/master/yt_dlp/extractor
It also supports ripping playlists. Fantastic to archive a set locally…
A few that I use every day:
- Fish shell
- Starship.rs
- Broot (a brilliant filesystem navigator)
- Helix editor (My favorite editor / IDE, truly the successor to vim IMO)
- Topgrade (updates everything)
I heard about helix from you and I’ve used it for a year and a half or so now, it’s by far the best editor I’ve used so far and I can definitely vouch for it
Nice!
Just commenting to give more love to helix. It’s my favorite “small quick edits” editor.
I’ve actually been testing with fish recently coming from zsh, though I might wait until 4.0 fully releases before I make a more conclusive decision to move or not.
With that said, I remember looking through omf themes and stumbled onto Starship that branched off one of the themes and really liked the concept.
One thing that holds people back sometimes is that bash scripts that set environment variables don’t work by default. https://github.com/edc/bass is an easy solution
Helix is great thanks
Once Helix gets plugin support and someone makes a Clojure REPL plugin as good as Conjure I am never touching
vimagain!It does have clojure lsp support, but you’ll probably have to use a command line for most repls.
Yeah the clojure lsp support is top notch, but there being no support for “jacking in” to a repl is the big thing keeping me from using helix full time. There’s a way of doing it if you use kitty, but it’s pretty janky.
Could you explain them in more depth? I opened them and don’t know
Helix is a terminal based text editor. It’s much like vim / neovim, but unlike those editors it’s good to go right out of the box, no configuration or plugins needed to make it work well.
Topgrade is one I haven’t used, but it looks like its intended purpose is to let you upgrade your apps with one command, even if you use multiple different package managers (I.e. if you were on Ubuntu, you could use it to upgrade your apt packages, at the same time as your snap packages, as well as flatpak, nix, and homebrew if you’ve added those.)
Thank you for explaining. I would never have understood topgrade without your example :)
grep goes crazy if you know your regex
I can never get grep to work consistently on Mac and Linux. Now, ripgrep OTOH…
That’s because Macs generally use BSD-based command line tools instead of GNU ones. You have to do a lot of Homebrew jiggery-pokery to approximate a GNU environment. Know Your Tools: Linux (GNU) vs. Mac (BSD) Command Line Utilities
Alas, doesn’t fit my purpose since it requires action by the script user. I usually just use perl in those situations
Check out my chapter on GNU grep BRE/ERE for those wanting to learn this regex flavor: https://learnbyexample.github.io/learn_gnugrep_ripgrep/breere-regular-expressions.html (there’s also another chapter for PCRE)
jq - super powerful json parser. Useful by hand and in scripts
I love jq, but I wouldn’t call it “surprising simple” for anything but pretty-formatting json. It has a fairly steep learning curve for doing anything with all but the simplest operations on the simplest data structures.
It’s not even pretty or accessible. 2-spaced indentation is incredibly hard to read.
It can also format minimized JSON from cURL API requests
jqandyqare both things I install on pretty much every machine I have.Combine with jc to process CSV files. This is how I get data into my plain text accounting system.
jq?
yq can do both JSON and YAML :)
https://github.com/johnkerl/miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
Funny how this was one of the first tools I learnt once I “seriously” started my linux journey, lol
The pipe (
|), which if you think about it is the basis for function composition.I’m a big fan of
screenbecause it will let me run long-running processes without having to stay connected via SSH, and will log all the output.I do a lot of work on customers’ servers and having a full record of everything that happened is incredibly valuable for CYA purposes.
I’d recommend
tmuxfor that particular use. Screen has a lot of extras that are interesting but don’t really follow the GNU mentality of “do one thing and do it well.”Tmux / Screen is like the emacs/vim of the modern day Linux I think.
Screen is more than capable, but for those who have moved to Tmux, they will absolutely advocate for it.
When tmux was first released I was already so used to screen that I never really considered switching. What would some convincing arguments be for me to make the effort to switch now?
The thing that got me to switch was being able to maintain my pane layout between connections. The various window and pane management niceties (naming, swapping, listing and the like) got me to stay. Now you can keep your screen, but you’d have to pry tmux from my cold, dead, tty.
Tmux was purpose built for terminal multiplexing. You can assign session names for organizing and manipulating multiple instances. Send keys to and read output from detached sessions. It’s easy to script.
Tmux was purpose built for terminal multiplexing.
Was screen not purpose built for terminal multiplexing?
Sorry, it was, just not for exploring all of those instances at once. Should have called out the tiling function. Screen also built in a serial terminal emulator and started playing with a few other things.
This was a few years ago so maybe it has improved, but I found that screen would crash and lose my session history and layout too often. That was bad enough, but when it happened it had some bullshit error message about a dungeon roof falling in. I don’t mind some comedy in code or even the interface, but don’t make light of the user losing their stuff. I tried tmux and it is much more stable than screen was.
I know everyone likes tmux but screen is phenomenal. I have a .screenrc I deploy everywhere with a statusbar at the bottom, a set number of pre-defined tabs, and logging to a directory (which is cleaned up after 30 days) so I can go back and figure out what I did. Great tool.
Woah screen is seeing active development again? There was like a decade where it stagnated. So much so that different distros were packaging different custom feature patches (IIRC only Ubuntu had a vertical split patch by default?) Looking at it now, the new screen maintainers had to skip a version to not conflict with forks that had become popular.
When tmux stabilized I jumped ship immediately and never looked back.
tmux with control mode in iterm is god mode for me on all my machines. Absolutely love it.
nohupis similarIt’s not as useful, sadly. Nohup disconnects standard input, output, and error. With screen or tmux, you can reattach them later.
Not powerful, but often useful,
column -taligns columns in all lines. EG$ echo {a,bb,ccc}{5,10,9999,888} | xargs -n3 a5 a10 a9999 a888 bb5 bb10 bb9999 bb888 ccc5 ccc10 ccc9999 ccc888 $ echo {a,bb,ccc}{5,10,9999,888} | xargs -n3 | column -t a5 a10 a9999 a888 bb5 bb10 bb9999 bb888 ccc5 ccc10 ccc9999 ccc888zoxide. It’s a fabulous
cdreplacement. It builds a database as you navigate your filesystem. Once you’ve navigated to a directory, instead of having to typecd /super/long/directory/path, you can typezoxide pathand it’ll take you right to/super/long/directory/path.I have it aliased to
zd. I love it and install it on every systemYou can do things like using a partial directory name and it’ll jump you to the closest match in the database. So
zoxide pawould take you to/super/long/directory/path.And you can do partial paths. Say you’ve got two directories named
datain your filesystem.One at
/super/long/directory/path1/dataAnd the other at
/super/long/directory/path2/dataYou can do
zoxide path2 dataand you’ll go to/super/long/directory/path2/dataYou can do
zoxide path2 dataI usually would just doz 2data. Yes, I’m lazy. It’s the perfect tool for lazy people.Nice! I guess I can be even lazier when navigating!
Better than fasd?
Sounds a lot like autojump
I’m not familiar with
autojump
I know
tmuxis incredibly popular, but a good use case for it that isn’t common is teaching people how to do things in the terminal. You can both be attached to the same tmux session, and both type into the same shell.tmux is my religion
Tmux is so much better than screen, and yes that is the hill I will die on
Specially when confined with tmuxp , it’s how I handle Game servers that can run headless to start at boot without losing access to giving commands to the server via its server console
yesThe most positive command you’ll ever use.
Run it normally and it just spams ‘y’ from the keyboard. But when one of the commands above is piped to it, then it will respond with ‘y’. Not every command has a true -y to automate acceptance of prompts and that’s what this is for.
Also, you can make
yesreturn anything:yes noI… did not know that. Thanks, TIL!
What’s the syntax here? Do I go
command && yes
I’m not sure if I’ve had a use case for it, but it’s interesting.
That will just wait for
commandto finish properly and then runyes.What you want to run is
yes | command, so it spams the command with confirmations.Also my favourite way to push a core to 100% CPU
yes > /dev/nullhow is that better than
cat /dev/zero > /dev/nullor
while true; do :; doneWho said it was better? It’s just my favourite.
Like my favourite shirt, it’s no better than the others, but it brings me a little joy :)
- on a serious note though, thank you for sharing your two examples - I didn’t know they existed.
truedelivers error level 0,falseerror level 1.yes && echo True || echo Falsewill always be True.false && echo True || echo Falsewill always be False.Common usage is for tools that ask for permissions and similiar.
yes | cp -ihas the same effect ascp --force(-i: prompt before overwrites).Sorry, I should have explained that. it’s
command | yesyes|command- Eg,yes|apt-get update(Not a great example since apt-get has -y, but sometimes that fails when prompting for new keys to accept)Edit: I got it backwards, thanks @lengau@midwest.social for the correction.
You’ve got it backwards - you need to pipe the output of
yesinto the input of the command:yes | command-that-asks-a-lot-of-questionsSo I did - thanks for the correction, edited.
For some cases I use “|| true”.
The idiom accepts that the preceding command might fail, and that’s OK.
For example, a script where mkdir creates a directory that might already exist.
mkdir -pwill not complain if the dir existsRight, it was an example of a pattern. In that case, -p could be used.
I figured as much. Just wanted to show another option.
Using rust rewrite of coreutils you can
cp -gto see progress. Set an alias :)Holy shit I was just talking about cp with progress today. Awesome
Where can this variant of coreutils be found? This is the first time I have heard of it.
bcIt’s a simple command line calculator! I use it all the time.
Very useful for shell scripts that need to do maths as well. I use it to make percentages when stdout has values between 0.0 and 1.0
I once wrote a bc script that calculated parameters for the Blackman window for a FIR filter. (Had formulas already so not that impressive) Upped the precision until it needed like 30 sec to calculate, completely unnecessarely :).
ddis probably well known, but one of the simplest and most powerful ways to accidentally delete all data on your hard drive.dd if=/dev/random of=/dev/sdaddrescue (or gddrescue) is a great version if you have a sick drive. It’ll try to copy the good areas first then go back to hammer on the sick areas.
Not perfect as it doesn’t know about the file system so it tries to copy the entire surface, but generally a good tool.
yq is crazy cool for converting between different text-based data formats such as yaml, json, xml, csv and others, and it has a super nice pretty-printing function as well. I use it all the time!
Just be aware that your distroy might come with a yq variant too, but possibly one that isn’t as powerful as the one I linked. I know this to be true at least for Ubuntu.
See also https://github.com/TomWright/dasel
I used jq for something similar before, recently I’ve discovered Nu Shell and have been using that for converting and analyzing data since a full shell is a lot more powerful than a command (e.g. open a yaml, for each element on key X grab the first element of list Y and export to a CSV)
