7. Key Quality Metrics#

Performance-related metrics must remain key acceptance criteria throughout Stages 3-6. Stage 3 focuses on operator performance, Stage 4a on communication throughput and linearity, and Stage 4b on training/inference accuracy and end-to-end performance. Before advanced cooperation, Day 0, distribution, or showroom plans, the efficiency target relative to the NVIDIA baseline under equivalent compute must be confirmed. The exact percentage will be defined later according to the KT2 requirements.

Stage

Acceptance criteria

Pass standard

Required hardware

Stage 0

MOU signing

Effective after both parties sign or seal

N/A

Stage 1

Environment readiness

Chip is recognized by the OS and PyTorch can recognize the backend

1 development board

Stage 2

Triton kernel compilation pass rate

>=95%, CI machine availability >=99%

1 CI machine

Stage 3

FlagGems operator test pass rate

>=80%, core operators 100%

1 CI machine and 1 debug machine

Stage 4a

Homogeneous collective communication throughput ratio

>=99% of the native library

2-machine multi-card cluster

Stage 4b

Model accuracy deviation

<2% relative to the NVIDIA baseline

Reuse Stage 4a devices

Stage 5

CTS core tests passed

100% passed or waiver approved

1 CI machine and 2 SVT machines

Stage 6

Number of models released through FlagRelease

>=10 models

1 dedicated FR machine

Advanced cooperation

Equivalent-compute performance comparison

Efficiency reaches X% of the NVIDIA equivalent-compute baseline

Multi-card cluster, FR machine, or CI