中文  ｜ English

🤗 Hugging Face  |    ModelScope  |  

🖥️ Official Website  |   🕹️ Demo    

GITHUB

The Hunyuan-MT comprises a translation model, Hunyuan-MT-7B, and an ensemble model, Hunyuan-MT-Chimera. The translation model is used to translate source text into the target language, while the ensemble model integrates multiple translation outputs to produce a higher-quality result. It primarily supports mutual translation among 33 languages, including five ethnic minority languages in China.

• In the WMT25 competition, the model achieved first place in 30 out of the 31 language categories it participated in.
• Hunyuan-MT-7B achieves industry-leading performance among models of comparable scale
• Hunyuan-MT-Chimera-7B is the industry’s first open-source translation ensemble model, elevating translation quality to a new level
• A comprehensive training framework for translation models has been proposed, spanning from pretrain → continue pretraining (CPT) → supervised fine-tuning (SFT) → translation rl → ensemble rl, achieving state-of-the-art (SOTA) results for models of similar size

• 2025.9.1 We have open-sourced Hunyuan-MT-7B , Hunyuan-MT-Chimera-7B on Hugging Face.

Technical Report

把下面的文本翻译成<target_language>，不要额外解释。

<source_text>

Translate the following segment into <target_language>, without additional explanation.

<source_text>

Analyze the following multiple <target_language> translations of the <source_language> segment surrounded in triple backticks and generate a single refined <target_language> translation. Only output the refined translation, do not explain.

The <source_language> segment:
```<source_text>```

The multiple `<target_language>` translations:
1. ```<translated_text1>```
2. ```<translated_text2>```
3. ```<translated_text3>```
4. ```<translated_text4>```
5. ```<translated_text5>```
6. ```<translated_text6>```

First, please install transformers, recommends v4.56.0

pip install transformers==4.56.0

The following code snippet shows how to use the transformers library to load and apply the model.

we use tencent/Hunyuan-MT-7B for example

from transformers import AutoModelForCausalLM, AutoTokenizer
import os

model_name_or_path = "tencent/Hunyuan-MT-7B"

tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map="auto")  # You may want to use bfloat16 and/or move to GPU here
messages = [
    {"role": "user", "content": "Translate the following segment into Chinese, without additional explanation.\n\nIt’s on the house."},
]
tokenized_chat = tokenizer.apply_chat_template(
    messages,
    tokenize=True
    add_generation_prompt=False,
    return_tensors="pt"
)

outputs = model.generate(tokenized_chat.to(model.device), max_new_tokens=2048)
output_text = tokenizer.decode(outputs[0])

We recommend using the following set of parameters for inference. Note that our model does not have the default system_prompt.

{
  "top_k": 20,
  "top_p": 0.6,
  "repetition_penalty": 1.05,
  "temperature": 0.7
}

If you need to fine-tune our Instruct model, we recommend processing the data into the following format.

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Why is seawater salty?" },
    {"role": "assistant", "content": "Seawater is primarily saline due to dissolved salts and minerals. These substances come from the chemical materials in rocks and soil on the Earth's surface, which are carried into the ocean over time. When seawater evaporates, the water vapor leaves, but the salts and minerals remain, making the seawater saltier. Therefore, the salinity of seawater is determined by the amount of salts and minerals it contains."}
]

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("your_tokenizer_path", trust_remote_code=True)
train_ids = tokenizer.apply_chat_template(messages)

In the following chapter, we will introduce how to use LLaMA-Factory to fine-tune the Hunyuan model.

Verify installation of the following dependencies:

• LLaMA-Factory: Follow official installation guide
• DeepSpeed (optional): Follow official installation guide
• Transformer Library: Use the companion branch (Hunyuan-submitted code is pending review)

  pip install git+https://github.com/huggingface/transformers@4970b23cedaf745f963779b4eae68da281e8c6ca
  

We need to prepare a custom dataset:

1. Organize your data in json format and place it in the data directory in LLaMA-Factory. The current implementation uses the sharegpt dataset format, which requires the following structure:

[
  {
    "messages": [
      {
        "role": "system",
        "content": "System prompt (optional)"
      },
      {
        "role": "user",
        "content": "Human instruction"
      },
      {
        "role": "assistant",
        "content": "Model response"
      }
    ]
  }
]

Refer to the Data Format section mentioned earlier for details.

2. Define your dataset in the data/dataset_info.json file using the following format:

"dataset_name": {
  "file_name": "dataset.json",
  "formatting": "sharegpt",
  "columns": {
    "messages": "messages"
  },
  "tags": {
    "role_tag": "role",
    "content_tag": "content",
    "user_tag": "user",
    "assistant_tag": "assistant",
    "system_tag": "system"
  }
}

1. Copy all files from the llama_factory_support/example_configs directory to the example/hunyuan directory in LLaMA-Factory.
2. Modify the model path and dataset name in the configuration file hunyuan_full.yaml. Adjust other configurations as needed:

### model
model_name_or_path: [!!!add the model path here!!!]

### dataset
dataset: [!!!add the dataset name here!!!]

3. Execute training commands: *​​Single-node training​​ Note: Set the environment variable DISABLE_VERSION_CHECK to 1 to avoid version conflicts.

   export DISABLE_VERSION_CHECK=1
   llamafactory-cli train examples/hunyuan/hunyuan_full.yaml
   

   *Multi-node training​​ Execute the following command on each node. Configure NNODES, NODE_RANK, MASTER_ADDR, and MASTER_PORT according to your environment:

   export DISABLE_VERSION_CHECK=1
   FORCE_TORCHRUN=1 NNODES=${NNODES} NODE_RANK=${NODE_RANK} MASTER_ADDR=${MASTER_ADDR} MASTER_PORT=${MASTER_PORT} \
   llamafactory-cli train examples/hunyuan/hunyuan_full.yaml
   

We used our own AngleSlim compression tool to produce FP8 and INT4 quantization models. AngleSlim is a toolset dedicated to creating a more user-friendly, comprehensive and efficient model compression solution.

We use FP8-static quantization, FP8 quantization adopts 8-bit floating point format, through a small amount of calibration data (without training) to pre-determine the quantization scale, the model weights and activation values will be converted to FP8 format, to improve the inference efficiency and reduce the deployment threshold. We you can use AngleSlim quantization, you can also directly download our quantization completed open source model to use AngelSlim.

For deployment, you can use frameworks such as TensorRT-LLM, vLLM, or SGLang to serve the model and create an OpenAI-compatible API endpoint.

image: https://hub.docker.com/r/hunyuaninfer/hunyuan-7B/tags

We provide a pre-built Docker image based on the latest version of TensorRT-LLM.

We use tencent/Hunyuan-7B-Instruct for example

• To get started:

docker pull docker.cnb.cool/tencent/hunyuan/hunyuan-7b:hunyuan-7b-trtllm

docker run --privileged --user root --name hunyuanLLM_infer --rm -it --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 --gpus=all hunyuaninfer/hunyuan-7B:hunyuan-7b-trtllm

• Prepare Configuration file:

cat >/path/to/extra-llm-api-config.yml <<EOF
use_cuda_graph: true
cuda_graph_padding_enabled: true
cuda_graph_batch_sizes:
- 1
- 2
- 4
- 8
- 16
- 32
print_iter_log: true
EOF

• Start the API server:

trtllm-serve \
  /path/to/HunYuan-7b \
  --host localhost \
  --port 8000 \
  --backend pytorch \
  --max_batch_size 32 \
  --max_num_tokens 16384 \
  --tp_size 2 \
  --kv_cache_free_gpu_memory_fraction 0.6 \
  --trust_remote_code \
  --extra_llm_api_options /path/to/extra-llm-api-config.yml

Please use vLLM version v0.10.0 or higher for inference.

First, please install transformers. We will merge it into the main branch later.

pip install git+https://github.com/huggingface/transformers@4970b23cedaf745f963779b4eae68da281e8c6ca

We use tencent/Hunyuan-7B-Instruct for example

• Download Model file:

  • Huggingface: will download automicly by vllm.
  • ModelScope: modelscope download --model Tencent-Hunyuan/Hunyuan-7B-Instruct

• model download by huggingface:

export MODEL_PATH=tencent/Hunyuan-7B-Instruct

• model downloaded by modelscope:

export MODEL_PATH=/root/.cache/modelscope/hub/models/Tencent-Hunyuan/Hunyuan-7B-Instruct/

• Start the API server:

python3 -m vllm.entrypoints.openai.api_server \
    --host 0.0.0.0 \
    --port 8000 \
    --trust-remote-code \
    --model ${MODEL_PATH} \
    --tensor-parallel-size 1 \
    --dtype bfloat16 \
    --quantization experts_int8 \
    --served-model-name hunyuan \
    2>&1 | tee log_server.txt

• After running service script successfully, run the request script

curl http://0.0.0.0:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
"model": "hunyuan",
"messages": [
    {
        "role": "system",
        "content": [{"type": "text", "text": "You are a helpful assistant."}]
    },
    {
        "role": "user",
        "content": [{"type": "text", "text": "请按面积大小对四大洋进行排序，并给出面积最小的洋是哪一个？直接输出结果。"}]
    }
],
"max_tokens": 2048,
"temperature":0.7,
"top_p": 0.6,
"top_k": 20,
"repetition_penalty": 1.05,
"stop_token_ids": [127960]
}'

This section describes the process of deploying a post-quantization model using vLLM.

Default server in BF16.

Deploying the Int8-weight-only version of the HunYuan-7B model only requires setting the environment variables

Next we start the Int8 service. Run:

python3 -m vllm.entrypoints.openai.api_server \
    --host 0.0.0.0 \
    --port 8000 \
    --trust-remote-code \
    --model ${MODEL_PATH} \
    --tensor-parallel-size 1 \
    --dtype bfloat16 \
    --served-model-name hunyuan \
    --quantization experts_int8 \
    2>&1 | tee log_server.txt

Deploying the Int4-weight-only version of the HunYuan-7B model only requires setting the environment variables , using the GPTQ method

export MODEL_PATH=PATH_TO_INT4_MODEL

Next we start the Int4 service. Run

python3 -m vllm.entrypoints.openai.api_server \
    --host 0.0.0.0 \
    --port 8000 \
    --trust-remote-code \
    --model ${MODEL_PATH} \
    --tensor-parallel-size 1 \
    --dtype bfloat16 \
    --served-model-name hunyuan \
    --quantization gptq_marlin \
    2>&1 | tee log_server.txt

Deploying the W8A8C8 version of the HunYuan-7B model only requires setting the environment variables

Next we start the FP8 service. Run

python3 -m vllm.entrypoints.openai.api_server \
    --host 0.0.0.0 \
    --port 8000 \
    --trust-remote-code \
    --model ${MODEL_PATH} \
    --tensor-parallel-size 1 \
    --dtype bfloat16 \
    --served-model-name hunyuan \
    --kv-cache-dtype fp8 \
    2>&1 | tee log_server.txt

We also provide a pre-built Docker image based on the latest version of SGLang.

We use tencent/Hunyuan-7B-Instruct for example

To get started:

• Pull the Docker image

docker pull lmsysorg/sglang:latest

• Start the API server:

docker run --entrypoint="python3" --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    --ulimit nproc=10000 \
    --privileged \
    --ipc=host \
     lmsysorg/sglang:latest \
    -m sglang.launch_server --model-path hunyuan/huanyuan_7B --tp 4 --trust-remote-code --host 0.0.0.0 --port 30000

If you would like to leave a message for our R&D and product teams, Welcome to contact our open-source team . You can also contact us via email ([email protected]).