Új hozzászólás Aktív témák

• #65188 S_x96x_S addikt

  Új Válasz 2024-08-07 00:39:14 #65188

  Új hozzászólás

  Összes hozzászólása itt Válaszok az összes hozzászólására itt Válaszok erre a hozzászólásra

  Privát üzenet küldése

  

  S_x96x_S

  addikt

  MI300 - FP8-as támogatás

  AMD hopes to unlock MI300’s full potential with fresh code
  Devs invited to ROCm out with FP8 precision, quantize to their heart's delight
  https://www.theregister.com/2024/08/06/amd_unleashes_mi300s_full_potential/

  "
  That FP8 support is a big deal since it was one of the flagship specs when AMD's MI300A and X APUs and GPUs arrived in December. The AI-centric MI300X boasted 2,614 teraFLOPS of dense FP8 performance (double that with sparsity) putting it well ahead of Nvidia's H100 and H200 at 1,979 teraFLOPS.
  Unfortunately, tapping into that performance wasn't exactly easy as, at least at the time, vLLM, AMD's preferred model runner, didn't support FP8 data types. This has since changed, at least in the ROCm branch of vLLM and with the launch of ROCm 6.2 Instinct, developers can now take advantage of the data type.
  Other optimizations include support for multi-GPU execution, and 8-bit key-value caches."

Új hozzászólás Aktív témák