15% off your first three monthsJoin now

Netvaer AI Suite

What does the GPU memory of a plan stand for?

It sets the upper limit for model size and determines how much context and how many concurrent requests are possible. As a rule of thumb, a model at 4-bit needs about 0.6 GB per billion parameters, and a quarter of the memory stays free for context. That gives up to 15B parameters in Suite S, 30B in Suite M, 60B in Suite L and 120B in Suite Enterprise, which gets a whole GPU.

Back to the FAQ

More on Models & technology

All questions on Models & technology

Still have a question?

Tell us briefly what your team works with, and we will answer for your specific case.

Talk to us