Detected NVIDIA NVIDIA GeForce RTX 4090 D GPU, which is not yet supported in this version of the container

[mailliw2010]

Description

nvidia-smi

Thu Jan  2 12:07:31 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.120                Driver Version: 550.120        CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090 D      Off |   00000000:01:00.0 Off |                  Off |
|  0%   39C    P8             23W /  425W |    1706MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      1536      G   /usr/lib/xorg/Xorg                             19MiB |
|    0   N/A  N/A      1741      G   /usr/bin/gnome-shell                           10MiB |
|    0   N/A  N/A   1980732      C   python                                        828MiB |
|    0   N/A  N/A   1980858      C   /usr/local/bin/python                         828MiB |
+-----------------------------------------------------------------------------------------+

kubectl describe node xxx

Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests    Limits
  --------           --------    ------
  cpu                1165m (3%)  500m (1%)
  memory             660Mi (0%)  6322Mi (4%)
  ephemeral-storage  0 (0%)      0 (0%)
  hugepages-1Gi      0 (0%)      0 (0%)
  hugepages-2Mi      0 (0%)      0 (0%)
  nvidia.com/gpu     1           1

kubectl logs triton-inference-server-tritoninferenceserver-544f6c6d77-78pgq

===============================
== TensorRT Inference Server ==
===============================
NVIDIA Release 19.09 (build 8086825)

Copyright (c) 2018-2019, NVIDIA CORPORATION.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION.  All rights reserved.
NVIDIA modifications are covered by the license terms that apply to the underlying
project or file.
WARNING: Detected NVIDIA NVIDIA GeForce RTX 4090 D GPU, which is not yet supported in this version of the container
ERROR: No supported GPU(s) detected to run this container

NOTE: The SHMEM allocation limit is set to the default of 64MB.  This may be
   insufficient for the inference server.  NVIDIA recommends the use of the following flags:
   nvidia-docker run --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 ...

I0102 04:04:05.631646 1 metrics.cc:160] found 1 GPUs supporting NVML metrics
I0102 04:04:05.636899 1 metrics.cc:169]   GPU 0: NVIDIA GeForce RTX 4090 D
I0102 04:04:05.637062 1 server.cc:110] Initializing TensorRT Inference Server
E0102 04:04:05.637134 1 main.cc:943] error: creating server: INTERNAL - Unable to create GCS client. Check account credentials.

Expected behavior
triton-inference-server can run success in rtx4090D