The latest version of Bitsandbytes now includes support for XPU on Windows and Linux. It should therefore be installed by default when an Intel GPU is detected.
In addition, there is now a third-party package containing Metal kernels. It can be implemented separately using code from the upstream integration PR (bitsandbytes#1853). This will make NF4 quantization available on all platforms except ROCm on Windows, although OSX may require a minor patch for Diffusers (diffusers#13361).
Separately from Bitsandbytes, there is also a new package with Metal kernels for FP4 and FP8 available here. It was designed for ComfyUI but hooks into Torch, and should therefore be Diffusers compatible as well.
Updated 2/5: MPS Bitsandbytes kernels
Updated 3/23: MPS FP8/FP4 kernels
The latest version of Bitsandbytes now includes support for XPU on Windows and Linux. It should therefore be installed by default when an Intel GPU is detected.
In addition, there is now a third-party package containing Metal kernels. It can be implemented separately using code from the upstream integration PR (bitsandbytes#1853). This will make NF4 quantization available on all platforms except ROCm on Windows, although OSX may require a minor patch for Diffusers (diffusers#13361).
Separately from Bitsandbytes, there is also a new package with Metal kernels for FP4 and FP8 available here. It was designed for ComfyUI but hooks into Torch, and should therefore be Diffusers compatible as well.
Updated 2/5: MPS Bitsandbytes kernels
Updated 3/23: MPS FP8/FP4 kernels