Jon Allman | Puget Systems https://www.pugetsystems.com/bios/jonallman/ Workstations for creators. Tue, 01 Apr 2025 19:02:36 +0000 en-US hourly 1 https://wordpress.org/?v=6.7.1 https://www.pugetsystems.com/wp-content/uploads/2022/08/Puget-Systems-2020-logomark-color-500-48x48.png Jon Allman | Puget Systems https://www.pugetsystems.com/bios/jonallman/ 32 32 Exploring Hybrid CPU/GPU LLM Inference https://www.pugetsystems.com/labs/hpc/exploring-hybrid-cpu-gpu-llm-inference/ https://www.pugetsystems.com/labs/hpc/exploring-hybrid-cpu-gpu-llm-inference/#respond Thu, 20 Mar 2025 20:41:02 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=35198 A brief look into using a hybrid GPU/VRAM + CPU/RAM approach to LLM inference with the KTransformers inference library.

The post Exploring Hybrid CPU/GPU LLM Inference appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/exploring-hybrid-cpu-gpu-llm-inference/feed/ 0 NVIDIA GeForce RTX 5090 & 5080 AI Review https://www.pugetsystems.com/labs/articles/nvidia-geforce-rtx-5090-amp-5080-ai-review/ https://www.pugetsystems.com/labs/articles/nvidia-geforce-rtx-5090-amp-5080-ai-review/#respond Sat, 22 Feb 2025 00:16:26 +0000 https://www.pugetsystems.com/?post_type=article&p=34053 How do NVIDIA's new GeForce RTX 5090 and 5080, released with fanfare regarding their new features and capabilities, perform in real world AI applications?

The post NVIDIA GeForce RTX 5090 & 5080 AI Review appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/nvidia-geforce-rtx-5090-amp-5080-ai-review/feed/ 0 Exploring GPU Performance Across LLM Sizes https://www.pugetsystems.com/labs/articles/exploring-gpu-performance-across-llm-sizes/ https://www.pugetsystems.com/labs/articles/exploring-gpu-performance-across-llm-sizes/#respond Thu, 16 Jan 2025 20:49:25 +0000 https://www.pugetsystems.com/?post_type=article&p=33242 Does the size of a Large Language Model affect relative performance when testing a variety of GPUs?

The post Exploring GPU Performance Across LLM Sizes appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/exploring-gpu-performance-across-llm-sizes/feed/ 0 What’s the deal with NPUs? https://www.pugetsystems.com/labs/hpc/whats-the-deal-with-npus/ https://www.pugetsystems.com/labs/hpc/whats-the-deal-with-npus/#respond Fri, 25 Oct 2024 19:55:03 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=31072 An introduction to NPU hardware and its growing presence outside of mobile computing devices.

The post What’s the deal with NPUs? appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/whats-the-deal-with-npus/feed/ 0 LLM Inference – NVIDIA RTX GPU Performance https://www.pugetsystems.com/labs/articles/llm-inference-professional-gpu-performance/ https://www.pugetsystems.com/labs/articles/llm-inference-professional-gpu-performance/#respond Thu, 22 Aug 2024 16:41:27 +0000 https://www.pugetsystems.com/?post_type=article&p=29660 How do a selection of GPUs from NVIDIA's professional lineup compare to each other in the llama.cpp benchmark?

The post LLM Inference – NVIDIA RTX GPU Performance appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/llm-inference-professional-gpu-performance/feed/ 0 LLM Inference – Consumer GPU performance https://www.pugetsystems.com/labs/articles/llm-inference-consumer-gpu-performance/ https://www.pugetsystems.com/labs/articles/llm-inference-consumer-gpu-performance/#respond Thu, 22 Aug 2024 16:41:26 +0000 https://www.pugetsystems.com/?post_type=article&p=30017 How do a selection of GPUs from NVIDIA's GeForce series compare to each other in the llama.cpp benchmark?

The post LLM Inference – Consumer GPU performance appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/llm-inference-consumer-gpu-performance/feed/ 0 Tech Primer: What hardware do you need to run a local LLM? https://www.pugetsystems.com/labs/articles/tech-primer-what-hardware-do-you-need-to-run-a-local-llm/ https://www.pugetsystems.com/labs/articles/tech-primer-what-hardware-do-you-need-to-run-a-local-llm/#respond Mon, 12 Aug 2024 21:34:43 +0000 https://www.pugetsystems.com/?post_type=article&p=29116 What considerations need to be made when starting off running LLMs locally?

The post Tech Primer: What hardware do you need to run a local LLM? appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/tech-primer-what-hardware-do-you-need-to-run-a-local-llm/feed/ 0 Effects of CPU speed on GPU inference in llama.cpp https://www.pugetsystems.com/labs/articles/effects-of-cpu-speed-on-gpu-inference-in-llama-cpp/ https://www.pugetsystems.com/labs/articles/effects-of-cpu-speed-on-gpu-inference-in-llama-cpp/#respond Mon, 01 Jul 2024 17:20:22 +0000 https://www.pugetsystems.com/?post_type=article&p=28690 What effect, if any, does a system's CPU speed have on GPU inference with CUDA in llama.cpp?

The post Effects of CPU speed on GPU inference in llama.cpp appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/effects-of-cpu-speed-on-gpu-inference-in-llama-cpp/feed/ 0 Puget Mobile 17″ vs M3 Max MacBook Pro 16″ for AI Workflows https://www.pugetsystems.com/labs/articles/puget-mobile-17-vs-m3-max-macbook-pro-16-for-ai-workflows/ https://www.pugetsystems.com/labs/articles/puget-mobile-17-vs-m3-max-macbook-pro-16-for-ai-workflows/#respond Tue, 28 May 2024 19:17:34 +0000 https://www.pugetsystems.com/?post_type=article&p=27770 How does the new Puget Mobile 17" compare to the MacBook Pro M3 Max 16" in performance across a variety of AI-powered workloads?

The post Puget Mobile 17″ vs M3 Max MacBook Pro 16″ for AI Workflows appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/articles/puget-mobile-17-vs-m3-max-macbook-pro-16-for-ai-workflows/feed/ 0 Local alternatives to Cloud AI services https://www.pugetsystems.com/labs/hpc/local-alternatives-to-cloud-ai-services/ https://www.pugetsystems.com/labs/hpc/local-alternatives-to-cloud-ai-services/#respond Thu, 11 Apr 2024 20:07:33 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=26768 Presenting local AI-powered software options for tasks such as image & text generation, automatic speech recognition, and frame interpolation.

The post Local alternatives to Cloud AI services appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/local-alternatives-to-cloud-ai-services/feed/ 0