Documentation: C4D 3d animation and 360 video Integration in After Effects

This documentation is about the process of integrating animated 3d model created in C4D and 360 video footage in After Effects. Honestly, I am not an expert in either of them, but had some experience in this area because of my school project. I found that there is a limited number of tutorials about this workflow, so I decided to write down my exploration to help people who might need to do the same thing save time on finding the right resource, and also for myself, as the old Chinese notion said, 好记性不如烂笔头: The faintest ink is better than the best memory.

I benefit from watching these two tutorials: Using the VR Comp Editor in AE by PremiumBeat.com , 3D Tracking & Compositing Tutorial (After Effects & Cinema 4D) by Sean Frangella

Process:

stabilize footage > track motion again > compositing >>Success!

Stabilize footage

1. In After Effects, import the footage, and drag the footage create to a new composition that matches the footage setting. (This is important to prevent the analysis solve failed issue)

360 video Assets download here: https://www.premiumbeat.com/blog/vr-comp-editor-360-video/

1. Open VR Comp Editor — add 3d edit, add 3d properties, then apply AE 3d camera tracker to initialized the motion tracker.
2. Go to effect panel, click create camera to see the target point when the analysis is done.
3. Add 3d properties: stabilize footage.

Track motion Again

1. Create a new composition using the stabilized footage.
2. Apply camera tracker again, add camera, then generate all the tracker points. (you could also change the size of these target points in the effect panel)
3. Go to the beginning of the composition timeline, then select all the target points — right click to create hundreds of nulls, export the file to C4D
4. Delete all the nulls in AE (as we already have them in C4D), pre-compose the output footage.

Compositing

1. Open the file in C4D, you could see all the nulls as a sort of point cloud.

point cloud (target points, nulls) in C4D

1. Import your 3d models and put them at the right place, or simply select a null and attach the 3d model to it.

3D Model download here: https://www.turbosquid.com/3d-models/origami-paper-3d-model/512536

1. Optional: create a background rendering the same footage you are using in after effect just to have an idea about where is where. It might not be aligned exactly to the position in After effect, but once you send this c4d file back to AE, everything should be good!

a background rendering the footage in C4D

3. Import the c4d file in After effect, drag to the top on the precomposed footage. Render it using software final.

4. Render the file using media encoder, then HERE YOU GO!

5. Watch your 360 video using go pro player

final render

We are dealing with something so new that there is still no established way or formula to follow. To figure out one major problem, we have to separate the problem into multiple steps and find out the solution for each of them. Then, summarize and connect these solutions to solve the major issue, just like pearl threading, at least this is how I solve problems and learn about new technologies.