How to extract video (mp4) and photos (png) from moving photos (HEIC) using Python3

Overview

Much like other mobile users, I have come to enjoy the feature referred to as moving photos. For those of you who might not be familiar, it is a feature which adds a short video to your photos as you take the photo. When viewing the photo, it gives you the option to also view the short video clip associated with that photo. The challenge I was facing was related to what happens to the video portion after the photos has been edited, or more specifically, how to get the video out of the photo before losing it. For example, if you send a moving photo to someone on an application like WhatsApp then it gets converted and the moving part is gone. Doing a bit of searching to understand this, I found posts saying that Android phones should make use of Google photos for backups as this will then keep the video part as part of the photo. Not being satisfied with this answer I changed my focus from backups to, how does this even work, and can I get the video somehow?

How do moving photos work if it is only a HEIC or JPG

HEIF on Wikipedia:

High Efficiency Image File Format (HEIF) is a container format for storing individual digital images and image sequences. HEIF can store images encoded with multiple coding formats, for example both SDR and HDR images. HEVC is an image and video encoding format and the default image codec used with HEIF. HEIF files containing HEVC-encoded images are also known as HEIC files. Such files require less storage space than the equivalent quality JPEG.

The first question was, where is that video coming from as the file has an image format? Well, as it turns out, it is not in some Android Harry Potter level folder to make it work. The video is inside the image file itself.

The next step was, can I view these files on my PC once I copy it? Well, in short, no. Windows do natively read HEIC files, but if you try to open one it'll give you the option to download a Microsoft store addon... for a price. Just the fact that there is a price tag, made me close the store window. Looking a bit deeper land me on this forum post, which has a bunch of apps to extract videos from JPG moving photos. I downloaded it to have a look and found that there are way too many unknowns for me to be running this on my device, that and it is not automatable which is the bigger problem. I finally found this stack overflow post which explains how to find the video data inside the image file. Technically speaking, one need to do a byte seek/string search for the characters ftypmp4 or ftypisom in the file, move back 4 bytes and everything from that point is the video data. With this knowledge, I made use of today’s lazy technology and asked OpenAI to create a Python3 script which extract the video from the image. What followed worked like a charm.

The code

Below is the base code snippet, which will work for both JPG and HEIC files.

import os
import re

def get_offset(file):
    with open(file, 'rb') as f:
        content = f.read()
    match = re.search(b'ftypmp4', content)
    return match.start() - 4 if match else None

for filename in os.listdir('.'):
    if filename.endswith(".heic"):
        ofs = get_offset(filename)
        if ofs:
            with open(filename, 'rb') as old_file:
                with open(f'{filename[:-4]}.mp4', 'wb') as new_file:
                    old_file.seek(ofs)
                    new_file.write(old_file.read())

Bonus snippet, here we extract the image from a HEIC and save the image as a PNG.

from pathlib import Path
from PIL import Image
import pillow_heif

if __name__ == "__main__":
    pillow_heif.register_heif_opener()
    image_path = Path("test.heic")
    img = Image.open(image_path)
    img.save(f"{image_path.stem}.png", quality=100, save_all=True)
    img.close()

Conclusion

Now that we can mass extract videos and images from HEIC file formats, it is not a concern to keep taking photos in HEIC format. Additionally, one could now include these in almost any automation process.