Instance type g6.xlarge

[silvacarl2]

AWS has a new instance type g6.xlarge. Its is extremely cost effective. its is 0.8048 per hour with these specs:

vCPUs: 4
Memory: 16 GiB
GPU: 1 NVIDIA L4 Tensor Core GPU
GPU Memory: 24 GiB
Processor: 3rd Generation AMD EPYC with a base clock speed of 2.6 GHz
Network Performance: Up to 10 Gigabits per second (Gbps)
Instance Storage: 1 x 250 GB NVMe SSD
EBS-Optimized: Yes, with a maximum bandwidth of 5,000 Mbps

but for some reason it gets stuck on this step when trying to build it on the g6.xlarge:

[ggerganov]

Did you try waiting it out - this step can take several minutes to complete.

[silvacarl2]

will let it wait a lot longer and check back with you. it made it to 83% now.

llama.cpp worked fine.

this is running on ubuntu 22, CUDA 12.6

[silvacarl2]

so we ran it for a few hours and eventually it killed the server. which makes no sense because it works great on other EC2 instance types like g4dn and g5.

if you would like i can let you borrow one a g6.xlarge?