As Project Gold was temporarily on hold, we had some spare time on our hands and Rik decided to make a 2d animated shot with Grease Pencil.

In the first part, Rik will explain his process, followed by Beau's description of the steps he took to create the rich environment.

Animation

In previous projects like Sprite Fright and Wing it!, we’ve been using Grease Pencil to draw additional effects as 2d animation, however this time I wanted to combine 2d character animation with a 3d background and a moving camera. I saw some other examples like Kairos fan art, by Daniel Lara and the art of Dedouze by using Blender to create these awesome 2d/3d hybrids and that got me inspired to try something similar: a single shot of a cat running, sliding and jumping in a dynamic environment. That sounds fun! This resulted in a useful test case that puts both 2d and 3d tools in Blender to the test. Along the way, I’ve collaborated with Beau who did the environment modeling, shading, rendering and compositing. We’ve also got help from Falk David who’s the developer working on Grease Pencil 3.

I’m really happy with the result and things we’ve learned along the way. Enjoy the read!

Blocking a 3d environment and camera motion

From the beginning, I wanted to make a dynamic shot where the cat was jumping and running swiftly while still maintaining clear readability. Often, 2d animated content distort their backgrounds to enhance the dynamic effect by creating exaggerated perspectives. I wanted a similar effect by basically exaggerating the horizon. Instead of modeling everything on a flat surface, I started out with a massive torus. I placed a camera inside of the ‘tunnel’ but made sure it was constrained to an empty that was located exactly at the center of the torus. This way, when I rotated the empty, I could move the camera along the torus, creating a bended tunnel effect. I deleted the outer faces of the torus, resulting in an infinite waterslide! Keying the empty on frame 1 and 200 while rotating the X axis with linear interpolation, and the base for the camera motion was set.

I could already imagine the cat jumping from object to object, so it was time to place simple primitives to create some proxy geometry. I was imagining some sort of a sewer environment with pipes and machinery, but what was important at first was to create a nice parkour where our furry friend could be traversing through. I took into account composition, dynamic animation and enough variety in motion where the cat could be jumping, running and sliding. This was an iterative process. First adding objects within the torus base and tweaking them to get a nice variety but maintaining a good composition throughout the shot. Then replaying the shot over and over again to see where more adjustments should be made to create a nice parkour path with enough depth and variety. While still using primitives for fast iteration, experimenting with different shapes and composition made it relatively easy to come up with a solid blocking of the 3d environment and the main camera movement.

Blocking animation with a 3d cat

As I’m not a 2d animator by trade, I wanted to use the benefits of 3d animation to set a good foundation for the timing, spacing, perspective and volume retention. Therefore I used a simplified version of the ‘Cat’ rig from one of our previous movies, Wing it!. With only its head and body I was able to animate roughly the main beats of the cat jumping and running. It was fairly quick to set up and it gave me a good sense of what kind of beats I was looking for. I could as well have used a simple bouncing ball, but since I was planning to use roughly the same proportions of the Cat from Wing it, it was an easy choice. Once I had blocked out the 3d cat, I had a solid foundation that helped me a lot with drawing the 2d cat, maintaining the correct volume, proportions and perspective.

Grease Pencil object in 3d space

Different options were considered for setting up the Grease Pencil object. I wanted to have the object as close as possible to the 3d animated cat, so it would be in the correct 3d space, respecting overlapping geometry and maybe 3d lighting in the future.

At first, I made a ‘Child of’ constraint, connecting the grease pencil object to the 3d cat, but only the location channels enabled. This way it would always face the camera and ignore the rotations of the 3d cat. The issue with this setup: onion skinning would not work, because it’s generated in object space. However, with the new Grease Pencil update coming up (Grease Pencil 3), this will be possible in the near future. I did try other options, where the Grease Pencil object would only be constrained to the Y location. But this resulted in inconsistent positioning of the strokes, and did not result in accurate onion skinning either.

Another approach was to have the Grease Pencil object only move every 20 frames or so, with constant keys. This way it would give relatively accurate onion skinning, but would change positions of the canvas once the character moved significantly in 3d space. The main issue with this approach was that on any location update, you would see a significant change in stroke thickness. Then there was the option of making the 2d animation in camera space, drawing in front of the 3d environment. This would ultimately get the best onion skinning result, but would have to be corrected on every frame, losing the benefit of the 3d environment.

In hindsight, I think there’s not one option perfect; I could have drawn everything in camera space, masking out the overlapping geometry and it would work fine. But with staying true to the idea of merging 2d and 3d, I chose to go for the first approach: using a ‘Child of’ constraint with only location enabled. This gave me the flexibility to tweak the animation of the Grease Pencil object later on, although sacrificing the onion skinning feature.

Sketching key poses.

The initial pass of 2d animation felt a bit daunting at first, but having the 3d Cat as a point of reference really helped the drawing process. I didn’t care about details yet, no eyes, just broad silhouette and perspective. I started drawing the head in the same position as the 3d cat then adding the body, limbs and tail to it. I do have experience animating quadrupeds with tails, so I knew what kind of poses I was looking for. And the great thing about rough 2d animation is that it’s very forgiving. Your mind tends to fill in the blanks, stitching drawings together easily. It’s like squinting your eyes when looking at an impressionistic painting; your brain is very eager to make motion out of flipping drawings. It was nice to already see something moving on the first pass, giving me a good direction of where to improve timing, perspective and silhouette.

Refining and inbetweening

After some tweaking of the first pass of 2d animation, I created a new Grease Pencil layer and started drawing inbetweens and additional details like the tail, eyes and better defined limbs. As an extra pass I added a shirt to the cat which was on a separate layer as well.

Shading with Grease Pencil

For shading the, cat I followed a somewhat traditional shading method for 2d animation by creating a 3 layer shading structure: ‘Base Color’, ‘Shadow Color’ and ‘Highlight Color’. I first drew the base color on every frame so I could clearly see the silhouette of the cat. I made a separate layer for the shadow and highlights which were both using the ‘Base Color’ layer assigned as a mask. This made drawing within the boundary of the base color very convenient. I figured the main light source would come from above, so that’s how I would draw my shadows and highlights accordingly.

Additionally, I experimented with 3d lights, however they were not responding well with each layer, causing weird artifacts in the colors. I guess 3d lights could be interesting with more static scenes, but were rather hard to control, with a cat that’s running in 3d space in a fake way. Ultimately, it was more convenient to use an airbrush with a massive radius to emulate bounce lights/colors to give it that extra atmosphere based on the environment shading. This gave the best control over the color and the areas that were influenced by it.

---

Environment

Defining the style

Rik started this project to discover how to incorporate 2D Grease Pencil animation and a 3D background. After his initial animated cat I joined the project, and we have been experimenting for the last five to six weeks to produce this shot. My goal was to design a world that would fit the animation but also making it feel cohesive.

Before starting the modeling process in Blender it was necessary to create a moodboard to make some initial decisions. I personally love cozy and cute illustrations from artists such as Rien Poortvliet, which wasn’t the main source of visual inspiration but a feeling I wanted to communicate. Another element was the idea of stacking houses with different shapes together as clusters where the inhabitants improvised such structures. And finally the video game Stray came to mind where industrial and civilian structures are overgrown beautifully with vegetation.

After collecting the moodboard we made a clear distinction between the amount of detail we wanted to add. Similar to our movie Wing It, smaller details don’t contribute geometrically as much as the bigger stylized shapes. Any details are drawn in with grease pencil or by texture with some exceptions. The main idea up to this point is that the cat is running through the sewer filled with houses that can be described as medieval or classic in nature.

Initial pass

Rik sketched out some ideas in the environment which was a nice way to get the conversation started. The next thing was to create the houses and instancing these around the environment, and by adding materials to balance/judge the elements in frame better. Using shader nodes and the outline setup from Wing It - made by Simon Thommes – helped to speed things up. Check out this blog for more on shading of Wing It.

The first iteration to where the blocked materials received an initial material pass and some houses are placed

To make more informed decisions about the environment, I added a simple lighting setup to convey the feeling of midday. The sewer received a green hue and the windows warm lights. We are using Eevee for rendering since Wing It tech is created with this in mind.

As the project progressed I added a geometry nodes setup to scatter two variations of houses around the wall of the sewer pipe. This saved me quite some time and by adding a Simple Deform modifier to stretch the houses in the distance, it increased the amount of stylization too.

These images show the early development of the environment and style.

The houses, pipes and Grease Pencil elements are separated into their own collections, this is useful for creating holdouts (masking using the geometry) but also to neatly organize the scene

Adding Grease Pencil elements

After spending time working on the houses, we wanted to add more Grease Pencil elements to enhance the 2D look. These are the vegetation, fences, smaller details on the houses, wires, specific building outlines, adding dirt, and metal panel lines. The main grease pencil element are the patches of vegetation that are placed throughout the environment. They are hand painted using the tint tool for better integration with the lighting.

Vivien, our Art Director at the studio (also during Wing It) helped out with creating a couple of drawovers to suggest certain stylistic choices. This helped me a lot creating a coherent image and get a conversation going.

Experience using Grease Pencil

Another aspect of working with Grease Pencil is the option to receive lighting. But this has a main drawback in our situation due to the screen space method which caused colors to fade in and out of view. This also depends on the bounding box of the object, which is the reason I split all of the vegetation to initially avoid this issue.

Placing Grease Pencil objects made me conscious of which depth order to use 2D or 3D to create a stable image without elements cutting each other off. But also to decide the visual appearance of certain elements

Adding an outline around the vegetation was quite simple by duplicating the drawn layer and masking it. By increasing its thickness and tinting the second layer created a good looking outline.

When I needed to repaint an already tinted and animated Grease Pencil object, I enabled the Multiframe icon and selected all of the keyframes to recolor the entire object without inconsistencies.

The little scribbles on the vegetation are drawn in and later animated to match the perspective of the camera, since these are actually floating and don’t sit on any surface. This was not a very efficient but more artist friendly method of adding scribbles.

The clouds in the last frames have large Grease Pencil strokes to match them stylistically with the rest of the scene.

To add more depth to the Grease Pencil fences it was a matter of duplicating the fences and Tinting the duplicated object. This way you can add rim or shadow effects

Lighting

The lights are placed in designated collections and have different functions. As you can see in the video below, light objects are added around the sewage and at the top looking down, the sewer fill lights to "glue" the top and bottom part of the environment. Most large fill lights are instanced, meaning that a change made on one is propagated to others. I recommend working this way to create a consistent look.

A large shadow caster is used to block the light and have more control on the shadows and lighting: these are invisible planes that only cast a shadow, without interfering with the rest. Another aspect in creating a more 2D look is to uncheck the option 'Soft Shadows' as they ultimately enhance a 3D feel and look which is something we try to avoid - same with Ambient Occlusion.

FX

Smaller effects were added to complete the final look: these are flowing sewage, chimney smoke, falling debris, and floating dust particles. Rik also hand animated some nice scratches and smoke effects using Grease Pencil. The toxic sewage coming out of the pipe is an animated texture and so are the ripples in the sewage, these are animated on twos using the Math node set to Snap.

Compositing

Rik started shading the cat after we approved and locked the lighting. Once the shading was done, I added masks in the compositor to integrate the cat with the scene. This is mostly due to the Grease Pencil elements not being able to properly mask the cat with the compositing setup.

The final compositing setup is created to mask specific aspects of the scene. Some of them were visual errors due to the screen space nature of Eevee or layer hierarchy. Also adjusting the colors using RGB Curves and Hue Correct nodes based on the depth of the scene. Lastly I added a vignette that brightens the image in the center and animated noise set to Overlay.

---

Conclusion

What Worked Well

The final result successfully combines the animated cat into the 3D environment in terms of movement, colors and lighting. The inherent movement of the camera and the parallax are clear signs of the 3D nature. We had a brief conversation about this and accepted this aspect of the final result. The shading and texturing of some of the houses could use a more distinct and art directed design to push the final look a bit more.

Could Be Better

The Grease Pencil objects elevate the quality and look in the final version, but the smaller intersections on the geometry are difficult to remove or mask with more Grease Pencil, which is not always desirable. If this was a production with multiple shots it is important to create some tech to easily add scribbles, instead of manually placing them.

---

Credits: The Studio team and especially Julien Kaspar for in-depth tips and tricks.

Read & learn about the shading used on this blog post by Simon Thommes

Download the blend file here: