feat: Add bfloat16 to default dtypes list (#44)

Co-authored-by: Vinay Umrethe <vinayumrethe99@gmail.com>
2025-11-25 12:22:52 +05:30
parent 1efc4ee9e1
commit 63fc0e7d5a
1 changed files with 2 additions and 0 deletions
@@ -34,6 +34,8 @@ class Settings(BaseSettings):
            "auto",
            # If that doesn't work (e.g. on pre-Ampere hardware), fall back to float16.
            "float16",
+            # If float16 fails (e.g. due to range issues) and float32 is too large, try bfloat16.
+            "bfloat16",
            # If that still doesn't work (e.g. due to https://github.com/meta-llama/llama/issues/380),
            # fall back to float32.
            "float32",