Parameters

423.5B

Context Window

131K

tokens

Max Output

16K

tokens

Input Price

$0.42

/1M tokens

Output Price

$1.25

/1M tokens

Overview

Ernie 4.5 Vl 424B A47b is a chat model by Baidu. It has 423.5B parameters. It supports a 131K token context window. Supports vision, reasoning.

Model Card

ERNIE-4.5-VL-424B-A47B-Base

Name: ERNIE 4.5 VL 424B A47B Base PT
Brand: Baidu
SKU: baidu/ernie-4.5-vl-424b-a47b
Price: 0.4200 USD
Availability: InStock

NOTE: Note: "-Paddle" models use PaddlePaddle weights, while "-PT" models use Transformer-style PyTorch weights.

ERNIE 4.5 Highlights

The advanced capabilities of the ERNIE 4.5 models, particularly the MoE-based A47B and A3B series, are underpinned by several key technical innovations:

Multimodal Heterogeneous MoE Pre-Training: Our models are jointly trained on both textual and visual modalities to better capture the nuances of multimodal information and improve performance on tasks involving text understanding and generation, image understanding, and cross-modal reasoning. To achieve this without one modality hindering the learning of another, we designed a heterogeneous MoE structure, incorporated modality-isolated routing, and employed router orthogonal loss and multimodal token-balanced loss. These architectural choices ensure that both modalities are effectively represented, allowing for mutual reinforcement during training.

Scaling-Efficient Infrastructure: We propose a novel heterogeneous hybrid parallelism and hierarchical load balancing strategy for efficient training of ERNIE 4.5 models. By using intra-node expert parallelism, memory-efficient pipeline scheduling, FP8 mixed-precision training and finegrained recomputation methods, we achieve remarkable pre-training throughput. For inference, we propose multi-expert parallel collaboration method and convolutional code quantization algorithm to achieve 4-bit/2-bit lossless quantization. Furthermore, we introduce PD disaggregation with dynamic role switching for effective resource utilization to enhance inference performance for ERNIE 4.5 MoE models. Built on PaddlePaddle, ERNIE 4.5 delivers high-performance inference across a wide range of hardware platforms.

Modality-Specific Post-Training: To meet the diverse requirements of real-world applications, we fine-tuned variants of the pre-trained model for specific modalities. Our LLMs are optimized for general-purpose language understanding and generation. The VLMs focuses on visuallanguage understanding and supports both thinking and non-thinking modes. Each model employed a combination of Supervised Fine-tuning (SFT), Direct Preference Optimization (DPO) or a modified reinforcement learning method named Unified Preference Optimization (UPO) for post-training.

To ensure the stability of multimodal joint training, we adopt a staged training strategy. In the first and second stage, we train only the text-related parameters, enabling the model to develop strong fundamental language understanding as well as long-text processing capabilities. The final multimodal stage extends capabilities to images and videos by introducing additional parameters including a ViT for image feature extraction, an adapter for feature transformation, and visual experts for multimodal understanding. At this stage, text and visual modalities mutually enhance each other. After pretraining trillions tokens, we obtained ERNIE-4.5-VL-424B-A47B-Base.

Model Overview

ERNIE-4.5-VL-424B-A47B-Base is a multimodal MoE Base model, with 424B total parameters and 47B activated parameters for each token. The following are the model configuration details:

| Key | Value |
| --------------------------------- | ------------- |
| Modality | Text & Vision |
| Training Stage | Pretraining |
| Params(Total / Activated) | 424B / 47B |
| Layers | 54 |
| Heads(Q/KV) | 64 / 8 |
| Text Experts(Total / Activated) | 64 / 8 |
| Vision Experts(Total / Activated) | 64 / 8 |
| Context Length | 131072 |

Quickstart

vLLM inference

VLLM>=0.11.2

# 80G * 16 GPU
vllm serve baidu/ERNIE-4.5-VL-424B-A47B-Base-PT --tensor-parallel-size 16 --trust-remote-code

License

Citation

If you find ERNIE 4.5 useful or wish to use it in your projects, please kindly cite our technical report:

@misc{ernie2025technicalreport,
      title={ERNIE 4.5 Technical Report},
      author={Baidu ERNIE Team},
      year={2025},
      eprint={},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={}
}

Features & Capabilities

Mode	chat
Context Window	131,072 tokens
Max Output	16,000 tokens
Function Calling	-
Vision	Supported
Reasoning	Supported
Web Search	-
Url Context	-

Technical Details

Architecture	Ernie4_5_VLMoeForConditionalGeneration
Model Type	ernie4_5_moe_vl
Languages	en, zh
Library	transformers

API Usage

from openai import OpenAI

client = OpenAI(
    base_url="https://api.haimaker.ai/v1",
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="baidu/ernie-4.5-vl-424b-a47b",
    messages=[
        {"role": "user", "content": "Hello, how are you?"}
    ],
)

print(response.choices[0].message.content)

Frequently Asked Questions

What is the context window of ERNIE 4.5 VL 424B A47B Base PT?

ERNIE 4.5 VL 424B A47B Base PT (baidu/ernie-4.5-vl-424b-a47b) has a 131,072-token context window and supports up to 16,000 output tokens per request.

How much does ERNIE 4.5 VL 424B A47B Base PT cost?

ERNIE 4.5 VL 424B A47B Base PT is priced at $0.42 per 1M input tokens and $1.25 per 1M output tokens when accessed via the haimaker.ai OpenAI-compatible API.

What features does ERNIE 4.5 VL 424B A47B Base PT support?

ERNIE 4.5 VL 424B A47B Base PT supports vision, reasoning.

How do I use ERNIE 4.5 VL 424B A47B Base PT via API?

Send requests to https://api.haimaker.ai/v1/chat/completions with model "baidu/ernie-4.5-vl-424b-a47b" using any OpenAI-compatible SDK. Authentication uses a Bearer API key from https://app.haimaker.ai.