DeepMotion

Overview

DeepMotion provides AI-driven tools for 3D character animation and motion capture, primarily through its Animate 3D platform. Established in 2014, the company focuses on converting 2D video input into 3D motion data, allowing users to create animations without traditional motion capture suits or extensive manual keyframing. This approach aims to reduce the time and resources required for character animation in various applications.

The platform is designed for a range of users, including indie game developers, virtual production studios, and animators seeking to accelerate their workflows. Animate 3D supports the generation of full-body, hand, and facial animation from standard video files. Users can upload video footage, which the system then processes using its Motion Brain AI to extract skeletal data and generate a 3D animation file. This output can be exported in formats compatible with common 3D software and game engines.

DeepMotion's utility extends to rapid prototyping, where developers can quickly iterate on character movements for game mechanics or cinematic sequences. It also serves animators who may not have access to expensive motion capture hardware, offering an accessible alternative for generating realistic character motion. The platform's emphasis on accessibility is further supported by a free tier with limited features, allowing users to evaluate its capabilities before committing to a paid plan. Compliance with GDPR standards is also noted, addressing data privacy concerns for users in relevant regions.

For integration into development pipelines, DeepMotion offers a REST API, enabling programmatic access to its motion capture and animation generation services. Additionally, dedicated SDKs for Unity and Unreal Engine are available, simplifying the integration of generated animations directly into game development environments. These tools aim to streamline the process of rigging, retargeting, and applying motion data to 3D character models within these popular engines. While some AI animation tools focus on generative text-to-animation, DeepMotion's primary method relies on video input, focusing on direct translation of human movement. This differentiates it from platforms like Plask, which also offer video-to-animation but may emphasize different aspects of the animation pipeline.

Key features

• Animate 3D: Core platform for converting 2D video into 3D character animation data.
• Motion Brain AI: Proprietary AI engine that processes video input to extract motion and generate animation.
• Full-body, hand, and face capture: Capabilities to extract detailed motion data for various parts of a character.
• Multiple output formats: Supports exporting animations in formats such as FBX, BVH, and GLB for compatibility with 3D software.
• Custom character support: Allows users to upload and animate their own 3D character models.
• Physics simulation: Includes options for secondary motion and physics-based animation for realism.
• REST API: Programmatic access to motion capture and animation generation services for custom integrations.
• Unity SDK: Software development kit for integrating DeepMotion animations into Unity projects.
• Unreal Engine Plugin: Plugin for direct integration of DeepMotion features within Unreal Engine workflows.
• Real-time previews: Provides immediate visual feedback on generated animations within the platform.
• Motion retargeting: Tools to adjust captured motion to fit different character rigs.

Pricing

DeepMotion offers a tiered pricing structure, including a free plan with limited features for evaluation purposes. Paid plans scale based on animation minutes, export options, and access to advanced features.

For the most current pricing details and specific plan inclusions, refer to the official DeepMotion pricing page.

Common integrations

• Unity: Direct integration via the DeepMotion Unity SDK for importing and applying animations.
• Unreal Engine: Plug-in available for integrating generated animations into Unreal Engine projects.
• Blender: Exported FBX or BVH files can be imported for further editing and rendering.
• Autodesk Maya: Supports import of animation data for professional animation workflows.
• Custom applications: Integration through the REST API for bespoke workflows and platforms.

Alternatives

• Plask: An AI motion capture and animation platform offering video-to-3D animation conversion and collaborative features.
• Move.ai: Specializes in markerless motion capture from multiple video inputs, aiming for high fidelity.
• Kinetics.ai: Provides AI tools for generating and refining character animations.
• RunwayML: Offers a broader suite of AI creative tools, including features for video-to-animation and generative video.

Getting started

To get started with DeepMotion's Animate 3D using their REST API, you would typically authenticate, upload a video, and then retrieve the processed animation. The following Python example illustrates a basic workflow for submitting a video for processing. This assumes you have an API key and the necessary video file.

import requests

API_KEY = "YOUR_DEEPMOTION_API_KEY"
VIDEO_FILE_PATH = "path/to/your/video.mp4"

# Step 1: Upload the video file
# This example simplifies the upload process; in a real scenario, you'd handle large files and progress.
with open(VIDEO_FILE_PATH, 'rb') as f:
    files = {'video': f}
    upload_response = requests.post(
        'https://api.deepmotion.com/v1/upload-video',
        headers={'Authorization': f'Bearer {API_KEY}'},
        files=files
    )

if upload_response.status_code == 200:
    upload_data = upload_response.json()
    video_id = upload_data['videoId']
    print(f"Video uploaded with ID: {video_id}")

    # Step 2: Request animation processing
    animation_config = {
        'videoId': video_id,
        'outputFormat': 'fbx',
        'characterType': 'humanoid' # Or 'quadruped', etc.
    }
    process_response = requests.post(
        'https://api.deepmotion.com/v1/process-animation',
        headers={'Authorization': f'Bearer {API_KEY}', 'Content-Type': 'application/json'},
        json=animation_config
    )

    if process_response.status_code == 200:
        process_data = process_response.json()
        job_id = process_data['jobId']
        print(f"Animation job started with ID: {job_id}")

        # Step 3: Poll for job status (simplified)
        # In a real application, you'd have a loop or webhook to check status.
        status_response = requests.get(
            f'https://api.deepmotion.com/v1/animation-status/{job_id}',
            headers={'Authorization': f'Bearer {API_KEY}'}
        )
        print(f"Initial job status: {status_response.json()}")

        # Once status is 'completed', you can download the animation.
        # Example of a simplified download URL (actual URL will be in the status response)
        # download_url = status_response.json().get('downloadUrl')
        # if download_url:
        #     animation_data = requests.get(download_url).content
        #     with open('output_animation.fbx', 'wb') as out_f:
        #         out_f.write(animation_data)
        #     print("Animation downloaded.")

    else:
        print(f"Error processing animation: {process_response.status_code} - {process_response.text}")
else:
    print(f"Error uploading video: {upload_response.status_code} - {upload_response.text}")

For detailed API endpoints, request bodies, and error handling, refer to the DeepMotion API Reference.