Fooocus: loading loras into google colab

Fooocus: loading loras into google colab

by eddyizm

on February 27th 2024

This series will mainly be focused (pun intended) on google colab since that is the main place where I run my instance of Fooocus, an open source generate ai gui running SDXL (Stability AI) base checkpoint models. I am not going to cover installing it or loading it up on google colab. There's plenty of youtube videos for that. Specifically, we will be loading lora's from, which when using the main fork of Fooocus, it may not be clear on how that actually works. I am going to be using my fork, and my personal branch, but generally, the process can be repeated using the same steps.

First thing you want to do is find the lora you want to use. Generally you should stick with SDXL 1.0 loras, they are a bit tricky and don't often work as you expect, in particular the SD 1.5 loras. You search models and filter by model type loRA and checkpoint SDXL 1.0. I quickly found Glass Scuplture which looks neat so we will try that one.

Notices that the url I linked is - however that is not the url we will be using to download the file. Remember this because it has tripped up many people before. On the right side there will be downloads, details and file, click on the 1 File chevron and you will see LoRA with a file size. In this case, LoRA (162.64 MB)

lora image

If you mouse over the Download to the right, you can right click and click -> copy link. It will look like this:

Notice how it is different from the page link above?

You need to get your civitia api key from your account page - Details here. Save your key and keep it safe. Now to the magic.

First thing you will notice is the default notebook has everything in one block.

!pip install pygit2==1.12.2 %cd /content !git clone %cd /content/Fooocus !python --share

First thing I do is create a couple of new code blocks so I can seperate everything and do things in step. If something fails, killing the app and trying to rerun the whole block could be a real pain. I will use my fork/branch in this example. The first block has the pull/install and then change directory. In case I switch branches, you don't need that unless you are using my fork and even then it is not required.

!pip install pygit2==1.12.2 %cd /content !git clone %cd /content/Fooocus !git checkout personal

In a new block you we will now grab the lora. It's important this is done after the git clone because we are doing to download it directly to that folder. Note that we are adding this to the api download link &token=$CIVITIA_API_KEY Yours should look replace $CIVITIA_API_KEY with your actual key.

# glass lora !wget -q --show-progress -O 'models/loras/sdxl_glass.safetensors' "$CIVITIA_API_KEY"

Run this and let it download, you should see a success message

models/loras/sdxl_gl 100%[===================>] 217.87M 78.9MB/s in 2.8s

If you get an error, check your spelling and verify your url structure, making sure you used your full key, short one character and it will fail. Now in your third code block, you can launch Fooocus (In my case, using my fork/branch with a custom preset, you can use the default one)

!python --preset turbo --theme dark --share --always-high-vram --all-in-fp16

You should see the new gradio link and then see the LoRA in the drop down of the advanced, Next, go wild!

image of glass scultpure of man and women on a beach

Hope this helps and please feel free to leave a comment or a link to your creation.

A note about loras. Please check sure you read the instructions, many have trigger words that need to be included in the prompt. I made this mistake before, wondering why nothing was working. Check the examples, often it also needs to be closer to the beginning so it takes more effect, sometimes further back for subtle effect, each one is different and there are many other factors. Patients and experimentation. Also, because remember trigger words is hard as your lora collection grows, a good hack is to name your lora one of the trigger words. So in our example, I would change 'models/loras/sdxl_glass.safetensors' to 'models/loras/translucent.safetensors'

Categories:  Technology   GenerativeAI