Adventures in AI - My first week with Cuebric
A few weeks back, I was honoured to be one of a few selected artists to try out Cuebric. I finally got hold of my copy at the beginning of week. It’s been a busy week, but I managed to get a few hours in.
First impressions: Cuebric runs as a web interface. It is a great introduction to image generated AI. It has integration with XR stages (#Disguise) and can be used to create 2 1/2 D images in #Unreal. It has a GUI that is relatively intuitive although some features are somewhat hidden (like exporting a depth map of an image). Part of it uses layers in a similar way to Photoshop and you can hide and merge layers. There is easy access to the tutorials and it is relatively straightforward to figure out. Prompting and the tools have links to tutorials in the GUI which is also very helpful. The AI generation part seems to be similar to the basic text to image part of most diffusion systems, but in a simpler and more user-friendly interface. The build seems to change daily and even during the week I noticed some improvements.
The web-based software is pretty stable but does crash sometimes especially when upscaling and segmenting. Also it gets unstable when you leave it running for too long.
One thing I really loved was the history that can call up all of your images with the prompts you originally used. This feature really helps in branching from something that you created that was interesting and should be almost a universal thing so you can keep track of your prompts.
For this one month experiment I am working on a personal project - the story of my father (which involves most of the history of the nuclear bomb). Much of my work this week was based on things that happened between 1925-1945 in China and probably because of the time period and the limited number of images I can see the limits of the AI diffusion as many of the images that are being generated are relatively similar and stereotypical in style and seem to draw from a limited pool of sources that don’t take in the cosmopolitan nature of Shanghai and the French and international concessions in to account. A lot of results also came out in black and white until I put it in to the negative prompts.
Where the image generation seems to struggle - at least this first week - and this may be due to bad prompting from me, or just the limited amount of training it has had in prewar Shanghai images, is with people. I don’t think Cuebric was necessarily thought of as a software for this kind of image generation but more as a tool to create backgrounds to integrate with real actors on an XR stage, but since I am using it as a concept generation/trailer generation software I do need some people and found that they often were misformed or cloned. I also felt that the images were less cinematic than I would have liked even though I kept prompting for wide and extreme wide shots.
My first prompts were related to the old city of Shanghai where my father grew up. The images were not bad, but a lot of them were small variations on the same theme. Changing the seed, GFC scale, sample method or prompt gave me very similar results.
Where Cuebric shines and where it stands out from other AI generation platforms is in creating and cutting out quick masks. Cuebric has a workflow that is to create an image you like. You then cut out the elements you want to have as foreground, middle and background. This is very easy and until recently something that was very painful roto work - Cuebric makes it simple by cutting by object or using a depth map. I would have loved to be able to be a bit more precise on these masks - often small things like leaves or fingers are left out and a way to add or subtract in object mode would have helped a lot to avoid this. To be able to do a parallax effect you have to fill in the holes in the background plate once you have cut out the elements you want to have as foreground and middle ground. You can do this using the inpaint tool. I didn’t find the results that great, so I ended up doing it in photoshop and reimporting that image in Cuebric (the import function is very useful).
Cuebric generates all the images in a relatively low resolution. The process is to first create an image you like, then slice it up in layers, fill in the background, and then up-rez each layer in a process called superscaling. Superscaling actually works really well (but can be a bit buggy). Very much like Topaz AI - you can take a layer and scale it up to 4x the resolution (which still is less than 4k). Strangely enough you don’t seem to be able to scale more than one layer at a time so if you have several layers it can take some time to do and that almost always was buggy. I didn’t seem to be able to super scale my original image first and then do the cut outs - but instead had to start from the lower resolution image, cut out, and then scale each layer.
You can then test what it looks like by using something called parallax preview that slides the layers over each other - creating a “fake” 3d effect. The same tool can also give you a depth map which you can use in Unreal or any other software to create an extrusion of your image.
You can then export the layers to Unreal, After Effects or whatever software you are using. I ended up using the depth map as an atmospheric mist and created my depth using Zoedepth using the image generated in Cuebric adjusted in Photoshop and then put it all together in AE.
This was one of my first tries. Image generated in Cuebric (Old city, Shanghai, inner courtyard) - Image is generated in Cuebric and layers are cut out in Cuebric. The background was fixed in Photoshop. The depth is from Cuebric, but the 3d depth was done in Zoedepth. All animated quickly in AE.
Thoughts so far: It is a fast process to create something that is “good enough” to show an idea. It can be a great tool for creating animatics - you could, if you wanted, drop in your own images and quickly uprez and segment them for example. It also combines a few things in to one “app” - such as segmenting, creating depthmaps etc. However I miss the control I get in ComfyUi and Photoshop.
I’m grateful to have a chance to try out this tool. It allowed me to catch up on AI and allowed me to play around a bit with Pika and ComfyUI. As for Cuebric I understand it is still being tinkered on and I think it very quickly can become a lot better. So far I am a bit underwhelmed with diffusion and how it handles people and the more stereotypical results of period China could be a lot better, but that could be down to bad prompting, learning using the software, or just the type of project I am working on. Just for context I tried using ComfyUI and some images came out “better” and others came out “better” Cuebric. See below
The plan is to get a few images I am happy with and then try running them through Runway or Pika and see what happens.
——
An image from the 128 incident, the Battle of Shanghai Shanghai January 28, 1932, Cuebric
Another one from Cuebric with weird cloned kids in the background
Here is the same prompt in ComfyUI (similar issues with limbs but more interesting lighting)
I had a bit more success with Aurora College in the French Concession (although it does not quite look like it should). The results from Cuebric were better than the ones from ComfyUI (without any “fixes”)
