As Project Gold was temporarily on hold, we had some spare time on our hands and Rik decided to make a 2d animated shot with Grease Pencil.
In the first part, Rik will explain his process, followed by Beau's description of the steps he took to create the rich environment.
In previous projects like Sprite Fright and Wing it!, we’ve been using Grease Pencil to draw additional effects as 2d animation, however this time I wanted to combine 2d character animation with a 3d background and a moving camera. I saw some other examples like Kairos fan art, by Daniel Lara and the art of Dedouze by using Blender to create these awesome 2d/3d hybrids and that got me inspired to try something similar: a single shot of a cat running, sliding and jumping in a dynamic environment. That sounds fun! This resulted in a useful test case that puts both 2d and 3d tools in Blender to the test. Along the way, I’ve collaborated with Beau who did the environment modeling, shading, rendering and compositing. We’ve also got help from Falk David who’s the developer working on Grease Pencil 3.
I’m really happy with the result and things we’ve learned along the way. Enjoy the read!
From the beginning, I wanted to make a dynamic shot where the cat was jumping and running swiftly while still maintaining clear readability. Often, 2d animated content distort their backgrounds to enhance the dynamic effect by creating exaggerated perspectives. I wanted a similar effect by basically exaggerating the horizon. Instead of modeling everything on a flat surface, I started out with a massive torus. I placed a camera inside of the ‘tunnel’ but made sure it was constrained to an empty that was located exactly at the center of the torus. This way, when I rotated the empty, I could move the camera along the torus, creating a bended tunnel effect. I deleted the outer faces of the torus, resulting in an infinite waterslide! Keying the empty on frame 1 and 200 while rotating the X axis with linear interpolation, and the base for the camera motion was set.
I could already imagine the cat jumping from object to object, so it was time to place simple primitives to create some proxy geometry. I was imagining some sort of a sewer environment with pipes and machinery, but what was important at first was to create a nice parkour where our furry friend could be traversing through. I took into account composition, dynamic animation and enough variety in motion where the cat could be jumping, running and sliding. This was an iterative process. First adding objects within the torus base and tweaking them to get a nice variety but maintaining a good composition throughout the shot. Then replaying the shot over and over again to see where more adjustments should be made to create a nice parkour path with enough depth and variety. While still using primitives for fast iteration, experimenting with different shapes and composition made it relatively easy to come up with a solid blocking of the 3d environment and the main camera movement.
As I’m not a 2d animator by trade, I wanted to use the benefits of 3d animation to set a good foundation for the timing, spacing, perspective and volume retention. Therefore I used a simplified version of the ‘Cat’ rig from one of our previous movies, Wing it!. With only its head and body I was able to animate roughly the main beats of the cat jumping and running. It was fairly quick to set up and it gave me a good sense of what kind of beats I was looking for. I could as well have used a simple bouncing ball, but since I was planning to use roughly the same proportions of the Cat from Wing it, it was an easy choice. Once I had blocked out the 3d cat, I had a solid foundation that helped me a lot with drawing the 2d cat, maintaining the correct volume, proportions and perspective.
Different options were considered for setting up the Grease Pencil object. I wanted to have the object as close as possible to the 3d animated cat, so it would be in the correct 3d space, respecting overlapping geometry and maybe 3d lighting in the future.
At first, I made a ‘Child of’ constraint, connecting the grease pencil object to the 3d cat, but only the location channels enabled. This way it would always face the camera and ignore the rotations of the 3d cat. The issue with this setup: onion skinning would not work, because it’s generated in object space. However, with the new Grease Pencil update coming up (Grease Pencil 3), this will be possible in the near future. I did try other options, where the Grease Pencil object would only be constrained to the Y location. But this resulted in inconsistent positioning of the strokes, and did not result in accurate onion skinning either.
Another approach was to have the Grease Pencil object only move every 20 frames or so, with constant keys. This way it would give relatively accurate onion skinning, but would change positions of the canvas once the character moved significantly in 3d space. The main issue with this approach was that on any location update, you would see a significant change in stroke thickness. Then there was the option of making the 2d animation in camera space, drawing in front of the 3d environment. This would ultimately get the best onion skinning result, but would have to be corrected on every frame, losing the benefit of the 3d environment.
In hindsight, I think there’s not one option perfect; I could have drawn everything in camera space, masking out the overlapping geometry and it would work fine. But with staying true to the idea of merging 2d and 3d, I chose to go for the first approach: using a ‘Child of’ constraint with only location enabled. This gave me the flexibility to tweak the animation of the Grease Pencil object later on, although sacrificing the onion skinning feature.
The initial pass of 2d animation felt a bit daunting at first, but having the 3d Cat as a point of reference really helped the drawing process. I didn’t care about details yet, no eyes, just broad silhouette and perspective. I started drawing the head in the same position as the 3d cat then adding the body, limbs and tail to it. I do have experience animating quadrupeds with tails, so I knew what kind of poses I was looking for. And the great thing about rough 2d animation is that it’s very forgiving. Your mind tends to fill in the blanks, stitching drawings together easily. It’s like squinting your eyes when looking at an impressionistic painting; your brain is very eager to make motion out of flipping drawings. It was nice to already see something moving on the first pass, giving me a good direction of where to improve timing, perspective and silhouette.
After some tweaking of the first pass of 2d animation, I created a new Grease Pencil layer and started drawing inbetweens and additional details like the tail, eyes and better defined limbs. As an extra pass I added a shirt to the cat which was on a separate layer as well.
For shading the, cat I followed a somewhat traditional shading method for 2d animation by creating a 3 layer shading structure: ‘Base Color’, ‘Shadow Color’ and ‘Highlight Color’. I first drew the base color on every frame so I could clearly see the silhouette of the cat. I made a separate layer for the shadow and highlights which were both using the ‘Base Color’ layer assigned as a mask. This made drawing within the boundary of the base color very convenient. I figured the main light source would come from above, so that’s how I would draw my shadows and highlights accordingly.
Additionally, I experimented with 3d lights, however they were not responding well with each layer, causing weird artifacts in the colors. I guess 3d lights could be interesting with more static scenes, but were rather hard to control, with a cat that’s running in 3d space in a fake way. Ultimately, it was more convenient to use an airbrush with a massive radius to emulate bounce lights/colors to give it that extra atmosphere based on the environment shading. This gave the best control over the color and the areas that were influenced by it.
Rik started this project to discover how to incorporate 2D Grease Pencil animation and a 3D background. After his initial animated cat I joined the project, and we have been experimenting for the last five to six weeks to produce this shot. My goal was to design a world that would fit the animation but also making it feel cohesive.
Before starting the modeling process in Blender it was necessary to create a moodboard to make some initial decisions. I personally love cozy and cute illustrations from artists such as Rien Poortvliet, which wasn’t the main source of visual inspiration but a feeling I wanted to communicate. Another element was the idea of stacking houses with different shapes together as clusters where the inhabitants improvised such structures. And finally the video game Stray came to mind where industrial and civilian structures are overgrown beautifully with vegetation.
After collecting the moodboard we made a clear distinction between the amount of detail we wanted to add. Similar to our movie Wing It, smaller details don’t contribute geometrically as much as the bigger stylized shapes. Any details are drawn in with grease pencil or by texture with some exceptions. The main idea up to this point is that the cat is running through the sewer filled with houses that can be described as medieval or classic in nature.
Rik sketched out some ideas in the environment which was a nice way to get the conversation started. The next thing was to create the houses and instancing these around the environment, and by adding materials to balance/judge the elements in frame better. Using shader nodes and the outline setup from Wing It - made by Simon Thommes – helped to speed things up. Check out this blog for more on shading of Wing It.
To make more informed decisions about the environment, I added a simple lighting setup to convey the feeling of midday. The sewer received a green hue and the windows warm lights. We are using Eevee for rendering since Wing It tech is created with this in mind.
As the project progressed I added a geometry nodes setup to scatter two variations of houses around the wall of the sewer pipe. This saved me quite some time and by adding a Simple Deform modifier to stretch the houses in the distance, it increased the amount of stylization too.
These images show the early development of the environment and style.
The houses, pipes and Grease Pencil elements are separated into their own collections, this is useful for creating holdouts (masking using the geometry) but also to neatly organize the scene
After spending time working on the houses, we wanted to add more Grease Pencil elements to enhance the 2D look. These are the vegetation, fences, smaller details on the houses, wires, specific building outlines, adding dirt, and metal panel lines. The main grease pencil element are the patches of vegetation that are placed throughout the environment. They are hand painted using the tint tool for better integration with the lighting.
Vivien, our Art Director at the studio (also during Wing It) helped out with creating a couple of drawovers to suggest certain stylistic choices. This helped me a lot creating a coherent image and get a conversation going.
Experience using Grease Pencil
Another aspect of working with Grease Pencil is the option to receive lighting. But this has a main drawback in our situation due to the screen space method which caused colors to fade in and out of view. This also depends on the bounding box of the object, which is the reason I split all of the vegetation to initially avoid this issue.
Placing Grease Pencil objects made me conscious of which depth order to use 2D or 3D to create a stable image without elements cutting each other off. But also to decide the visual appearance of certain elements
Adding an outline around the vegetation was quite simple by duplicating the drawn layer and masking it. By increasing its thickness and tinting the second layer created a good looking outline.
When I needed to repaint an already tinted and animated Grease Pencil object, I enabled the Multiframe icon and selected all of the keyframes to recolor the entire object without inconsistencies.
The little scribbles on the vegetation are drawn in and later animated to match the perspective of the camera, since these are actually floating and don’t sit on any surface. This was not a very efficient but more artist friendly method of adding scribbles.
The clouds in the last frames have large Grease Pencil strokes to match them stylistically with the rest of the scene.
To add more depth to the Grease Pencil fences it was a matter of duplicating the fences and Tinting the duplicated object. This way you can add rim or shadow effects
The lights are placed in designated collections and have different functions. As you can see in the video below, light objects are added around the sewage and at the top looking down, the sewer fill lights to "glue" the top and bottom part of the environment. Most large fill lights are instanced, meaning that a change made on one is propagated to others. I recommend working this way to create a consistent look.
A large shadow caster is used to block the light and have more control on the shadows and lighting: these are invisible planes that only cast a shadow, without interfering with the rest. Another aspect in creating a more 2D look is to uncheck the option 'Soft Shadows' as they ultimately enhance a 3D feel and look which is something we try to avoid - same with Ambient Occlusion.
Smaller effects were added to complete the final look: these are flowing sewage, chimney smoke, falling debris, and floating dust particles. Rik also hand animated some nice scratches and smoke effects using Grease Pencil. The toxic sewage coming out of the pipe is an animated texture and so are the ripples in the sewage, these are animated on twos using the Math node set to Snap.
Rik started shading the cat after we approved and locked the lighting. Once the shading was done, I added masks in the compositor to integrate the cat with the scene. This is mostly due to the Grease Pencil elements not being able to properly mask the cat with the compositing setup.
The final compositing setup is created to mask specific aspects of the scene. Some of them were visual errors due to the screen space nature of Eevee or layer hierarchy. Also adjusting the colors using RGB Curves and Hue Correct nodes based on the depth of the scene. Lastly I added a vignette that brightens the image in the center and animated noise set to Overlay.
The final result successfully combines the animated cat into the 3D environment in terms of movement, colors and lighting. The inherent movement of the camera and the parallax are clear signs of the 3D nature. We had a brief conversation about this and accepted this aspect of the final result. The shading and texturing of some of the houses could use a more distinct and art directed design to push the final look a bit more.
The Grease Pencil objects elevate the quality and look in the final version, but the smaller intersections on the geometry are difficult to remove or mask with more Grease Pencil, which is not always desirable. If this was a production with multiple shots it is important to create some tech to easily add scribbles, instead of manually placing them.
Credits: The Studio team and especially Julien Kaspar for in-depth tips and tricks.
Read & learn about the shading used on this blog post by Simon Thommes
[download the blend file here](cat_purkour.zip )
Join to comment publicly.
14 comments
Congrats Team..this is amazing works.
Super awesome, Rik & team! I love the technique you used to rough in the cat in 3D first before going in with Grease Pencil for the hand-drawn frames. Really smart! I could see using that process to streamline any number of 2D animation projects. I'm also a sucker for all the NPR cartoon effects you do. Great work!
@Dirk I. Tiede Thanks Dirk!
That was super insightful, thank you! So did I read this correctly that onion skins should be working with grease pencil objects that were transformed in object space in the future? That'd definitely be incredibly helpful since this was on of the main problems I had when trying to animate 2D characters across 3D space. Using the canvas around the 3D cursor was a workaround for that but an addon is needed to remember where you put the cursor for every frame which makes things compicated.
@KSDN Exactly!
@Rik Schutte Oh that is super exciting! Can't wait ^^ Do you know if Greasepencil 3 is still planned for the 4.2 release or if it'll be postponed?
@KSDN it's scheduled to be released with Blender 4.3. More information can be found here: https://devtalk.blender.org/t/2024-04-22-grease-pencil-module-meeting/34318/2
Beautiful!
Love this!
Sooo Nice! The .blend file will be online?
Hey @Daniel Martinez Lara, the .blend is downloadable at the bottom of the article.
Sooo Nice!
Will you be posting the .blend?
Hey @jmyers, you can download it at the bottom of the article.