I’ve been trying to upgrade from 10.10 to 10.11 for a while now, as the Android TV app keeps nagging me, and every attempt ended with impossibly long library scan times.
After some thorough investigation, it appeared that a Home Videos type collection causes unending scan (yet to be solved), but also that Jellyfin does a lot of writes to the config directory (either database or metadata or both). Mine’s on spinning media part of a ZFS pool. I tried a few performance tuning options, such as testing the config dir with recordsize (similar to block size) of 4K, 8K, 64K, 128K and library scans fell from 30-40 minutes down to 8-13min with 4K-64K. The ZFS tuning wiki suggest 64K recordsize with LZ4 compression for SQLite workloads such as Jellyfin. That seems to work as well as 4K and 8K but likely is faster when reading thumbnails and such.
Note that upgrading to 12-rc3, which is supposed to speed up library scans did not improve scan times for me. Optimizing config/database write speed did. I cross-checked the culprit by experimenting with moving the config dir to NVMe and RAM. Both of those got the scan times down to 8-9 minutes compared to the optimized spinning media’s 12-13.
So if you had upgraded (or about to) to 10.11 your library scans are (about to get) dog slow and your Jellyfin’s config dir resides on spinning media, optimize its write performance for SQLite.
Isn’t it ironic that you post this meme along with complex instructions to fixing a problem in jellyfin?
With great power come great … uuh … :D
Also the meme is just bait so you read abt the problem and see if you need to solve it for your setup.
complications 😅
Complications, yes!
Excellent bait
Yeah but tell me about this tool for Plex remote streaming…
That’s the nice thing about Plex, you just stream your Plex library using Plex, and Plex is also the only place you and anyone you share with need to log in.
Also doesn’t jellyfin not support remote streaming?
You just need to setup a reverse proxy. My Jellyfin instance has been accessed from all over the world
It absolutely does but you have to do it yourself, they dont accept any liability for doing that and nor should they, so if you dont know how to do it with at least the minimal level of safety and security and accept that risk yourself then you really shouldn’t try. Ideally you would only do it via a private VPN to trusted users.
Plex has some features built in to make just opening the port reasonably safe, though I’d still recommend against that. They also have a reasonably secure account verification system, and the software itself has proven to be somewhat safe.
Jellyfin has very little security built in and is not intended to be public facing, it also has the double edged sword of being open source, so you could read through all the code and pick it apart for flaws and exploits if you wanted to and we all just trust that when exploits or flaws are found they are reported properly.
JF is great but you are on your own if you want to make it available via WAN.
Jellyfin does support remote streaming and it doesnt have to be through the web ui.
If you use folders a lot, the library will get faster with v12 https://github.com/jellyfin/jellyfin/issues/15141
OP said this didn’t improve their situation though.
Not sure if this is the problem I was hitting with my Home Videos library type. It’s got directories with YouTube videos. I tried re-adding it in 12-rc3, which was released a month after the nightly mentioned in the issue thread. Still couldn’t complete. I ended up re-adding the media as Shows library. Works okay with the directory structure I have.
The write performance was recorded without the problematic library (deleted).
How large is the database? Because there is a way to fully load SQLite into memory, which would speed everything up drastically for a scan. At the end of the scan, you could just write back everything to the disk, then it is just one blob of sequential write, which is fast enough on spinning rust.
Interesting, I figured people would use MySQL instead if possible.
600MB. Would that require changes to Jellyfin? They’re on the path to introducing Postgres support.
I just checked the code and unfortunately, they do not support loading the SQLite file into memory yet. Even worse, the change would be substantially. It appears they currently abuse the database connection a bit. They basically create a new connection for each operation. This does not matter if you can read/write in parallel, but that’s only the case with NVMe and certain drivers. So all SATA (SSD and HDD) suffer from the current mechanic. So just loading it into memory is not going to solve the issue. Jellyfin needs to change architecturally, so that a database connection is reused. I don’t know why or who implemented the current logic, but it is clearly done by someone not understanding how databases work.
Depending on the mood and time, I will come up with a solution that is solid. For now, you are stuck with the current logic.
Thanks for looking into it. As far as I read they moved to an ORM with 10.11 as a prereq to supporting other db engines. I’m guessing the db connection code would be undergoing major changes when they do that so I wouldn’t rewrite the current implementation. I’d either follow that work and help them get it right or rework the implementation after they’re done. We’ll just suffer through the mean time. 😄
Thanks, will take a look a my setup
I’ve always kept my apps and databases on solid state media, putting them on spinning rust, even in a well tuned ZFS pool with lots of disks is way too slow for their (admittedly inefficient) database.
This is true but it’s nice to have the whole app, data and database in one place, going together, snapshotted together, “backupable” together. It’s slower for sure. That said it can be reasonably fast with a large pool (more disks). With the magic of SSD cache, database reads fly and reads are the majority of the loads in my heads. In the future I would put root on ZFS as well and either do SSD cache or have root on ZFS SSD pool that gets send/recved regularly onto the spinning pool so it’s easy to restore when needed.
Hey not bad for the developer…he vibe coded it right
Just the desktop client. Not the server.
Removed by mod
Doesn’t jellyfin need a static IP.
Static IP for what? As long as you know the ip address of your jellyfin server you can connect to it. A static IP is just good to have for ease of remembering, using your own domain (not that you can’t do it with Dynamic addresses) or if you are integrating other services with it. This applies to any service you host.
Most ISP connection are behind CGNAT(ipv4), and in case of ipv6 to have a secure connection wouldn’t that require SSL certificate and a domain?
I have Pangolin setup on a VPS so I and my family can access it externally. It “bypasses” my router’s firewall. Tailscale was a stupid solution because a lot of people will be using it and I don’t want to maintain another service and be tech support for their tailscsle client. So the only thing I did to restrict access was by geoblocking.
Jellyfin expects you to actually host it yourself, instead of baking in a proxy and selling you the service.
There are plenty of free subdomain providers, and let’s encrypt gives out free certs.
If you’re not hosting through a VPN you should consider those a requirement so you’re not sending an unencrypted password to your jellyfin instance over the internet. Regardless of your ISP situation…
I’m hearing a common theme to poor application security here.
- host through vpn
- don’t expose publicly
- use a free subdomain provider
No.
To function? No.
Useful? Yes.
