OpenTimelineIO in Blender

In this article you will learn about OpenTimelineIO and how we are using it in production at Blender Studio on the Sprite Fright short film.

---

What is the goal?

During the post-production of Sprite Fright, we are going to collaborate with a sound designer and a colorist who work with different software packages than Blender. In terms of sound design, a large chunk of the work was already done in Blender. Hjalti assembled more than 1500 audio files inside of the Blender Sequence Editor. It would be a shame if sound design had to redo all of that. In order to avoid this, we need a way to transport the edit from Blender into external software such as ProTools and DaVinci Resolve.

Introducing OpenTimelineIO

OpenTimelineIO is a format to describe an edit. It was started at Pixar and is available as an open source project. You can think of it as an editorial interchange formats like XML, AFF or EDL. However, OTIO is a whole toolset that also includes an API for reading, writing, and manipulating editorial data. On top of that, it ships with several adapters and plugins that can read, write and convert across most of the known editorial interchange formats.

So from the point of view of a pipeline TD who doesn't know much about the detailed specifications of XML, EDL or AAF, it's quite straightforward to create /one/ OTIO timeline with the python API and export it to any of these formats with the adapters that ship with OTIO.

---

OpenTimelineIO in Blender

Spoiler alert: I successfully exported the entire edit of Sprite Fright from the Blender Sequence Editor to Adobe Premiere, ProTools and DaVinci Resolve!

But instead of sharing a step by step tutorial on how to achieve it, I'd like to present the whole process and give more insights on what problems I faced and how I solved them.

Research

The first step was to find out if there are any solutions or add-ons already. While there were some old add-ons flying around the Internet that can export EDLs out of Blender, they weren't really an option for us because of the EDL restriction to one video track and only a handful of audio tracks. Our edit uses every available track in the Sequence Editor.

It turns out Félix David already created the vse-io blender add-on that uses the OpenTimelineIO package to export and import editorial interchange formats.

Installing the OTIO package in Blender

In order to get the add-on to work you need the opentimelineio python package available in Blender. Blender does not ship with that package by default, so you need to install it. The vse-io add-on explains one way of doing that in the readme.

Here is an alternative approach, without changing the content of your Blender installation.

1. Create a virtual environment with a python version that matches the one used by Blender as closely as possible:

   python3 -m venv /path/to/venv

2. Activate the virtual environment:

   source /path/to/venv/bin/activate

3. And then use the python packaging manager to install opentimelineio:

   pip3 install opentimelineio

   One side note: If you want the latest changes of OpenTimelineIO you can also use pip to directly install it from a git repository. Even a remote one:

   pip3 install git+https://github.com/PixarAnimationStudios/OpenTimelineIO@master

4. Create a little bash or bat script that starts Blender and modifies the PYTHONPATH environment variable before. Make sure to append the --python-use-system-env argument otherwise Blender ignores the PYTHONPATH. The content of this script would look something like this:

     export PYTHONPATH="$PYTHONPATH:/path/to/venv/lib/python3.9/site-packages"
     /path/to/blender --python-use-system-env

Now we can start Blender with the script and use the vse_io add-on to export our first .otio file!

---

Preparing the Edit

It is a good idea to reduce your edit as much as possible for the export. Even tough OTIO can handle nested metastrips we unpacked them. We also made sure that we only had video and sound strips and deleted all unused strips. The last thing to check is that you don't mix video and sound strips in one channel. Keep them separated. Most major editing softwares have dedicated audio and video tracks and if you mix them you might get unexpected results.

---

OTIO Useful Tools

The OpenTimelineIO package comes with a couple of useful command line tools that we can use in the virtual environment.

With otioview /path/to/file we can view any editorial exchange format in the otioviewer, which looks something like this for the Sprite Fright edit.

The metadata view on the right is quite handy to do some debugging.

With otioconvert -i /path/to/edit/format -o /path/to/other/edit/format we can convert any of the supported formats by OTIO to any other format via the commandline.

---

AAF Adapter

Unfortunately, major editing packages like Adobe Premiere, DaVinci Resolve or Avid Media Composer don’t seem to have adapted OTIO, yet. Most of them support AAF, XML or EDL. That means we need to use one of the adapters that ship with OTIO to convert our edit in one of these formats.

My initial thought was to use AAF because it is quite well supported everywhere. That lead to some problems.

AAF uses a unique identifier called a "MobID" or "SourceID" to link a clip to a specific media object "mob". So these mob IDs are quite important to link the right media to the right clip. But OTIO cannot create these MobIDs from scratch. It has some strategies to find and extract these MobIDs from existing AAF files but in our scenario they never existed.

We can still export an AAF but let OTIO create dummy MobIDs for each clip. The media of these clips will be offline, and need to be relinked within the application that imports the AAF. For that we need to pass an option to the export function via python:

otio.adapters.write_to_file(timeline, filepath, use_empty_mob_ids=True)

Or via the otioconvert command we can pass the argument:

-A use_empty_mob_ids=True.

To learn more about working with AAFs and OTIO, check this article.

When I tried to import the AAF in Adobe Premiere to test it, I noticed an issue: The edit comes in correctly, all the clips are there at the right place and time, the media is offline (which was to be expected) but Adobe Premiere appended a suffix to the clip names.

And that makes it impossible to relink the media by filename. And deleting this suffix for 1800 clips would be really cumbersome.

After getting some advice from the Academy Software Slack channel (they have an #opentimelineio channel with very helpful people), I found out that the recommended format to get edits in to premiere is Final Cut Pro XML.

XML Adapter

After some tweaks to the vse-io add-on, I was able to export a XML out of Blender and import it in to Adobe Premiere. Now I also had a lot more information available in the Link Media dialogue and could easily relink the missing media.

At this point the whole edit was successfully transported to Adobe Premiere.

Almost. What maybe is not quite clear through the picture is that two things didn't transport well. The audio levels of each clip and all audio clips got imported as mono clips.

---

OTIO Metadata

As Hjalti already pre-balanced the audio levels of all clips in Blender, it would be a waste to not transport those down the line. Further, each clip got imported as a mono track, which was not always correct, and would mean unnecessary work for the sound designer to change to stereo.

For the following section it is helpful to understand how the OTIO data structure works on a very basic level. We have a timeline object that holds tracks, which hold clips, which can have effects and more. More details are available in this article.

The beauty of OTIO is its simple and flexible design. You can append metadata to different entities, like the whole timeline, tracks or individual clips and it gets picked up by most of the adapters. So what we needed to find out is what metadata should be provided for the XML format to transport audio levels, and if an audio clip should be loaded as mono or stereo.

The workflow for that was to reverse engineer it. Which meant exporting a XML file out of Adobe Premiere, convert it to .otio and then identify what metadata was relevant. This was a lot of trial and error.

Audio Levels

To transport audio levels to Adobe Premiere via XML we need to append this metadata to each clip in the OTIO timeline.

'fcp_xml': {
    'effectcategory': 'audiolevels',
    'effectid': 'audiolevels',
    'effecttype': 'audiolevels',
    'mediatype': 'audio',
    'parameter': {
        '@authoringApp': 'PremierePro',
        'name': 'Level',
        'parameterid': 'level',
        'value': '1',
        'valuemax': '3.98109',
        'valuemin': '0'
        },
    'pproBypass': 'false'
}

And the value can then be set based on the volume of the sound strip in Blender. valuemax and valuemin are based on test exports out of Adobe Premiere.

If we re-import the XML file with the new metadata in to Adobe Premiere we can see that the audio levels got transported.

Mono and Stereo

To fix the issue that all the clips get imported as mono, we need to append more metadata. But first we need to find out if a sound clip is actually stereo or mono. How can we do that?

This information is currently not exposed in the Blender Python API so we need to check it another way. FFMPEG has a command called ffprobe that gives us information about a media file, including the number of audio channels. By wrapping this in a simple Python function, we can easily extract this information for each clip:

def get_nmbr_of_audio_channels(file_path: Path):
    cmd = (["ffprobe", "-v" , "quiet", "-print_format", "json", "-show_format", "-show_streams", file_path.as_posix()])
    output = subprocess.check_output(cmd)
    obj = json.loads(output)
    for stream in obj["streams"]:
        if stream["codec_type"] == "audio":
            return int(stream["channels"])
    return 0

Depending on the amount of audio channels we will update the metadata of each OTIO clip:

clip.metadata["fcp_xml"]["@premiereChannelType"] = "stereo" / "mono".

Each OTIO clip also contains a media_reference attribute which is its own object. We also need to append some metadata to the clip.media_reference.metadata:

"fcp_xml": {
    "media": {
        "audio": {
            "channelcount": 2 / 1,
        }
    }
}

At long last each OTIO track object (contains all clips of one channel in the Sequence Editor) needs to get some additional metadata so Adobe Premiere interprets it as a stereo track by default (that can also contain mono clips tough).

"fcp_xml": {
    "@currentExplodedTrackIndex": "0",
    "@premiereTrackType": "Stereo",
    "@totalExplodedTrackCount": "2",
    "enabled": "TRUE"
}

With all of that fuzz the clips finally import correctly as stereo or mono in Adobe Premiere!

Timeline Settings

Now, as a final touch to also make sure that the exported XML has the right FPS and screen resolution, we set this metadata on the root OTIO timeline object and adjust it to our needs:

"fcp_xml": {
    "media": {
        "video": {
            "format": {
                "samplecharacteristics": {
                    "anamorphic": "FALSE",
                    "fielddominance": "none",
                    "height": "856",
                    "pixelaspectratio": "square",
                    "rate": {"ntsc": "FALSE", "timebase": "24"},
                    "width": "2048",
                }
            }
        },
    },
    "rate": {"ntsc": "FALSE", "timebase": "24"},
}

---

Bringing it all to ProTools

For ProTools, the recommended import format is embedded AAFs. As we learned in the previous section it's problematic to get the MobIDs correct from scratch with OTIO. As far as I know there is also no way to create embedded AAFs either. So what we need to do is use Adobe Premiere as a bridge. Here is a document that explains the best settings for the export from Adobe Premiere to ProTools.

Bringing this all together our current pipeline to get an embedded AAF to ProTools looks like this:

---

Getting our edit in to other software was quite a journey! But OpenTimelineIO really was the toolset that made it possible in the end, thanks to its simplicity and working XML adapter. A great way to get started is with the official documentation.

Check out the vse-io add-on by Félix David which was a tremendous help to make this work. I did a modified version of the add-on, that has some patches for the XML export and adds all the metadata for Adobe Premiere. For now you can download it here:

We are in contact with the creator of the add-on to eventually merge some changes.