Fooocus: loading loras into google colab
by eddyizm
on February 27th 2024
This series will mainly be focused (pun intended) on google colab since that is the main place where I run my instance of Fooocus, an open source generate ai gui running SDXL (Stability AI) base checkpoint models. I am not going to cover installing it or loading it up on google colab. There's plenty of youtube videos for that. Specifically, we will be loading lora's from civitai.com, which when using the main fork of Fooocus, it may not be clear on how that actually works. I am going to be using my fork, and my personal branch, but generally, the process can be repeated using the same steps.
First thing you want to do is find the lora you want to use. Generally you should stick with SDXL 1.0 loras, they
are a bit tricky and don't often work as you expect, in particular the SD 1.5 loras. You search models and
filter by model type loRA and checkpoint SDXL 1.0. I quickly found Glass Scuplture which
looks neat so we will try that one.
Notices that the url I linked is
https://civitai.com/models/11203/glass-sculptures?modelVersionId=177888 - however that is not the
url we will be using to download the file. Remember this because it has tripped up many people before. On the
right side there will be downloads, details and file, click on the 1 File chevron and you will see
LoRA with a file size. In this case, LoRA (162.64 MB)
If you mouse over the Download to the right, you can right click and click -> copy link. It will
look like this:
https://civitai.com/api/download/models/177888?type=Model&format=SafeTensor
Notice how it is different from the page link above?
You need to get your civitia api key from your account page https://civitai.com/user/account - Details here. Save your key and keep it safe. Now to the magic.
First thing you will notice is the default notebook has everything in one block.
!pip install pygit2==1.12.2
%cd /content
!git clone https://github.com/lllyasviel/Fooocus.git
%cd /content/Fooocus
!python entry_with_update.py --share
First thing I do is create a couple of new code blocks so I can seperate everything and do things in step. If something fails, killing the app and trying to rerun the whole block could be a real pain. I will use my fork/branch in this example. The first block has the pull/install and then change directory. In case I switch branches, you don't need that unless you are using my fork and even then it is not required.
!pip install pygit2==1.12.2
%cd /content
!git clone https://github.com/eddyizm/Fooocus.git
%cd /content/Fooocus
!git checkout personal
In a new block you we will now grab the lora. It's important this is done after the git clone
because we are doing to download it directly to that folder. Note that we are adding this to the api download
link &token=$CIVITIA_API_KEY Yours should look replace $CIVITIA_API_KEY with your
actual key.
# glass lora
!wget -q --show-progress -O 'models/loras/sdxl_glass.safetensors' "https://civitai.com/api/download/models/177888?type=Model&format=SafeTensor&token=$CIVITIA_API_KEY"
Run this and let it download, you should see a success message
models/loras/sdxl_gl 100%[===================>] 217.87M 78.9MB/s in 2.8s
If you get an error, check your spelling and verify your url structure, making sure you used your full key, short one character and it will fail. Now in your third code block, you can launch Fooocus (In my case, using my fork/branch with a custom preset, you can use the default one)
!python entry_with_update.py --preset turbo --theme dark --share --always-high-vram --all-in-fp16
You should see the new gradio link and then see the LoRA in the drop down of the advanced, Next, go wild!
Hope this helps and please feel free to leave a comment or a link to your creation.
A note about loras. Please check sure you read the instructions, many have trigger words that need to be included
in the prompt. I made this mistake before, wondering why nothing was working. Check the examples, often it also
needs to be closer to the beginning so it takes more effect, sometimes further back for subtle effect, each one
is different and there are many other factors. Patients and experimentation. Also, because remember trigger
words is hard as your lora collection grows, a good hack is to name your lora one of the trigger words. So in
our example, I would change 'models/loras/sdxl_glass.safetensors' to
'models/loras/translucent.safetensors'
