Developers discuss running 35B parameter models on 16GB GPUs

**Summary**

Users on Reddit are discussing "poorman inference" — techniques for running large language models (LLMs) on consumer-grade GPUs with limited VRAM. The specific focus is on running a 35B parameter Mixture-of-Experts (MoE) Qwen model on a 16GB GPU for coding tasks. Related discussion includes hardware comparisons for local LLM inference and training, such as weighing RTX 5060 Ti configurations against RTX 3090 setups.

**Dominant Framing**

The discussion frames this as a practical problem-solving exercise: how to maximize inference capability within hardware constraints. Users are treating "poorman inference" as a legitimate optimization challenge rather than a workaround or compromise. The emphasis is technical — specific model sizes, VRAM requirements, and hardware configurations — suggesting an audience of practitioners trying to run models locally on budget or existing hardware.

**Notable Gaps**

Engagement is minimal (single-digit comments and scores), and discussion appears confined to Reddit with no mainstream media coverage. The term "poorman inference" itself has not gained traction beyond niche technical communities. There is no visible disagreement in the available posts, though the limited sample size makes it difficult to assess whether consensus exists or discussion simply hasn't developed enough to surface tensions.