• 1 Post
  • 27 Comments
Joined 3 years ago
cake
Cake day: June 20th, 2023

help-circle
  • The best and recommended way to connect to copyparty (either from windows, linux, or macos) is with WebDAV – this will give you much higher performance. WebDAV is also a MUCH safer choice when connecting over the internet, since it is just https after all. Meanwhile, exposing SMB to the internet is generally a recipe for, well… nasty surprises :-)

    There are also very copyparty-specific reasons to not use the SMB-server, and these are explained in the readme. That’s also why the SMB-server is not possible to enable in any of the official copyparty distributions without manually obtaining the necessary dependencies for that (impacket).


  • There were comments about security risks though, based on being a small project with a LOT of integrations.

    time will show, but the only thing i actively regret adding was the support for discord embeds (the “og” option); opengraph is an awfully designed concept and, unrelatedly, it has been a source of a handful of bugs in how it was implemented in copyparty (that one’s on me). Keeping that disabled avoids a lot of edgecases, most of which are decreed by the opengraph spec.

    That said, there’s no features keeping me up at night; i think for the most part things are fine – just don’t enable the smb server 😁



  • the intention with that statement was that seafile, by default, places all the files inside its own proprietary file container thing, where the files are not easily accessible from the server’s actual filesystem, using regular linux utilities. My knowledge of seafile is really minimal, so this could be wrong – in which case I’ll fix that right away! or, at the very least, try to clarify what I meant to avoid this confusion.

    in case you happen to know – are you aware if it’s possible to use Seafile while having it just place all the files and folders on the disk like any other program would?



  • No worries, good question :>

    The problem with bidirectional filesync is that it’s an absolutely massive can of worms, very easy to mess up, and the consequences of messing up are usually the worst kind (loss of data). There’s an insane amount of edgecases to keep in mind, and you need to get every edgecase right every single time, otherwise you might wipe someone’s vacation photos, or suddenly downgrade someone’s keepass database to an older version… And stuff like syncing multiple devices to the same server makes it balloon further.

    I’ve started becoming more confident in copyparty’s filesystem-index database, but it’s still just a hint/guideline, with the filesystem being the only source of truth – it’s still not something I’d trust with tracking sync-state against one or more clients.

    The bigger guys who offer bidirectional sync (nextcloud, syncthing, etc.) have spent years perfecting their logic, so I’d like to leave this in their capable hands.



  • sooo this is one of the things that started with someone saying “wouldn’t it be funny if…”

    if you open copyparty-sfx.py in a text editor, you’ll see how – but please make sure to use an editor which is able to handle about 600 KiB of comments which contain invalid utf8 / binary garbage 😁

    I ended up rolling my own packer since I wanted optimal encoding efficiency, and everything I could find would do stuff like base85 or ucs2 tricks, but it turns out python is perfectly happy with binary garbage in comments if you declare that the file is latin-1 so it realizes all hope is lost :D

    the only drawback of the sfx.py is that it needs to extract to $TEMP before running, so that’s the slight advantage of the zipapp (the .pyz alternative), but that suffers from some performance reduction in return, and is more hermetic (doesn’t let you swap out the bundled dependencies with fresh versions as easily if necessary)



  • Yep! Depending on what your home connection looks like, you have a few options:

    if you are lucky enough to have your own private IP-address and are able to open ports, then you’re almost done already – you can put copyparty on some port (or keep the default 3923), and then anyone could connect to it by going to https://your.ip.address:3923/

    (with this approach, you will want to create your own HTTPS certificate so the traffic is properly encrypted – the best option here is to get a domain and get a certificate for the domain)

    however, if you are behind CGNAT, meaning your internet provider has given you a shared IP-address, then people cannot connect directly to your home-PC. One way around that issue is by setting up a machine somewhere on the internet which bridges the gap back home to your PC. Cloudflare offers this as service, and this is explained in the copyparty readme – see the “at home” section for one way to do that.

    if you are against using Cloudflare for idealistic reasons (they are becoming quite powerful since they run a whole lot of the internet), then you can set up a cheap VPS which serves the same purpose. That’s my setup, and how you are accessing the copyparty demo server right now – I have the cheapest VPS you can get from Hetzner. The VPS is running nginx, and it forwards the traffic to my homeserver through an SSH tunnel. I haven’t documented this approach in the copyparty readme, but I have a feeling a lot of other people have :>




  • BTRFS and ZFS support real deduplication via copy on write, and would eliminate all current disadvantages of symlink and hardlink deduplication. It just works.

    yeah that’s a good point, I’ll add an option to take advantage of this if you know you’re running on a filesystem where that works as intended.

    Why have it be one huge python source file?

    oh don’t worry, it’s all separate files during development – there’s a build-stage which bundles everything up into a single file for distribution. But thanks for the concern :D



  • If all you need is basic remote file storage such as a samba server, and especially if you need samba in particular, then your current solution is probably a better fit for you.

    Copyparty’s main selling points is the large number of features in one package, and being pretty good at receiving file uploads (usually faster than other alternatives), but it does not have good samba support. Instead of samba, copyparty has WebDAV support, so you can still connect to it from your file manager – but the performance will be different; depending on your access pattern and the type of files, it could be faster or slower than samba.



  • So I realize the following does not directly apply in this specific case, since we are talking about a full android app. But in general, there are strictly technical limitations which absolutely requires you to use https. This for example applies to PWA’s, and it also applies to apps which are WebView-based.

    Basically the w3c is disagreeing with you; there are several important javascript features which are forcefully disabled if you are not connecting over https. This is a decision made by the webbrowser itself, and not something you or the dev can disable or otherwise avoid.

    For example, it is impossible to use the browser’s built-in api for getting the sha512 hash of a file, which is why i had to go through great pains to do that in other suboptimal ways in one selfhosted service i made. Most devs rightfully wouldn’t bother, since those restrictions are arbitrary and effectively pointless, as there are (usually painful) workarounds.

    List of features which require https: https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts/features_restricted_to_secure_contexts


  • Mind if I suggest my own software, copyparty?

    Regarding authentication, someone who has an account (in this case just yourself) can create password-protected shares which other people can browse, or upload, or browse+upload to (configurable when creating the share).

    There is WebDAV support, and it should integrate well enough with shares, but I haven’t tested that specifically.

    It has photo and video thumbnails, and a basic image-viewer, and with some elbow-grease it can also show exif-tags (gps-coordinates etc).

    There is also optional file dedup, so if two people upload the same file, it’ll detect and skip that during the 2nd upload (doesn’t waste any bandwidth) and swap out the new file with a symlink to the existing one. Default disabled to avoid surprising someone with symlinks.

    I think the following command would be enough to get you started:

    wget https://github.com/9001/copyparty/releases/latest/download/copyparty-sfx.py
    python3 copyparty-sfx.py -a sintan:yourpassword -v .::A,sintan --shr=/shr -e2dsa -e2ts
    

    but since that’s entirely unreadable, you can do it with a config file instead,

    [global]
      e2dsa  # enable filesystem indexing 
      e2ts  # enable media indexing (music tags)
      shr: /shr  # enable shares under this url
    
    [accounts]
      sintan: yourpassword 
    
    [/]  # create a volume at this url
      /srv/share/partypics  # the filesystem path to share
      accs:
        sintan: A  # give sintan read-write-move-delete-admin
    

    and use it like this:

    python3 copyparty-sfx.py -c the.conf
    

    there’s another example here and here for inspiration.



  • good idea, but a slight correction - mDNS and SSDP are entirely different things, rather SSDP was Microsoft’s initial proprietary take on the idea, and mDNS was created as the new and improved standard.

    mDNS does multicast (send/receive) on 224.0.0.251 and ff02::fb on port 5353, while SSDP uses 239.255.255.250 on port 1900 as you mentioned.

    But I think OP’s issue is that they’re on different subnets; mDNS expects the server and the client to have a perfectly overlapping subnets inside the same LAN. If the server has 10.0.0.3/16 and 192.168.1.3/24 then the client must also have 10.0.0.7/16 and 192.168.1.7/24. Or, if you can tell the server software exactly which IPs to announce, then that might work too.

    there are workarounds to this, using avahi reflector, but that thing is buggy – specifically you need to disable NSEC on the server, and lock it to either IPv4 or IPv6.

    I made some note on additional pitfalls while i was writing my own mDNS and SSDP servers for fun, they’re at the bottom of this page: https://ocv.me/copyparty/helptext.html


  • Absolutely; if I was a company, or hosting something important, or something that was intended for the general public, then I’d agree.

    But I’m just an idiot hosting whimsical stuff from my basement, and 99% of it is only of interest for my friends. I know ~everyone in my target audience, and I know that none of them use a VPN for general-purpose browsing.

    As it is, I don’t mind keeping the door open to the general public, but nothing of value will be lost if I need to pull the plug on some more ASN’s to preserve my bandwidth. For example when a guy hopping through a VPN in Sweden decides to download the same zip file thousands of times, wasting terabytes of traffic over a few hours (this happened a week ago).