It’s one of the reasons I got solar!

My electric bill was higher than my loan payment so it just made sense for me.

Check out open webui 10/10 do recommend

I think they mentioned on their discord that it’s in the pipeline

I really wish there was an update feature built into OWUI!

I think you’re right. Saw a post on Reddit basically mentioning the same things I’m seeing.

It looks like autoawq supports it but it might be an issue with how oobabooga implements it or something…

I have two 3090 Turbo GPUs and it seems like oobabooga doesn’t split the load between the two cards when I try to run TheBloke/dolphin-2.7-mixtral-8x7b-AWQ.

Does anyone know how to make text generation webui use both cards? Do I need an nvlink between the two cards?

I’m sure that’s not that big of a deal to some people. For example, I’m mainly using LLMs for use in my home assistant instance

You should probably also sync them. I use orbital sync for this. https://github.com/mattwebbio/orbital-sync