HPC Posts Archive | Puget Systems https://www.pugetsystems.com/all-hpc/ Workstations for creators. Thu, 20 Mar 2025 20:41:04 +0000 en-US hourly 1 https://wordpress.org/?v=6.7.1 https://www.pugetsystems.com/wp-content/uploads/2022/08/Puget-Systems-2020-logomark-color-500-48x48.png HPC Posts Archive | Puget Systems https://www.pugetsystems.com/all-hpc/ 32 32 Exploring Hybrid CPU/GPU LLM Inference https://www.pugetsystems.com/labs/hpc/exploring-hybrid-cpu-gpu-llm-inference/ https://www.pugetsystems.com/labs/hpc/exploring-hybrid-cpu-gpu-llm-inference/#respond Thu, 20 Mar 2025 20:41:02 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=35198 A brief look into using a hybrid GPU/VRAM + CPU/RAM approach to LLM inference with the KTransformers inference library.

The post Exploring Hybrid CPU/GPU LLM Inference appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/exploring-hybrid-cpu-gpu-llm-inference/feed/ 0 What’s the deal with NPUs? https://www.pugetsystems.com/labs/hpc/whats-the-deal-with-npus/ https://www.pugetsystems.com/labs/hpc/whats-the-deal-with-npus/#respond Fri, 25 Oct 2024 19:55:03 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=31072 An introduction to NPU hardware and its growing presence outside of mobile computing devices.

The post What’s the deal with NPUs? appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/whats-the-deal-with-npus/feed/ 0 Local alternatives to Cloud AI services https://www.pugetsystems.com/labs/hpc/local-alternatives-to-cloud-ai-services/ https://www.pugetsystems.com/labs/hpc/local-alternatives-to-cloud-ai-services/#respond Thu, 11 Apr 2024 20:07:33 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=26768 Presenting local AI-powered software options for tasks such as image & text generation, automatic speech recognition, and frame interpolation.

The post Local alternatives to Cloud AI services appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/local-alternatives-to-cloud-ai-services/feed/ 0 AMD Zen4 Threadripper PRO vs Intel Xeon-w9 For Science and Engineering https://www.pugetsystems.com/labs/hpc/amd-zen4-threadripper-pro-vs-intel-xeon-w9-for-science-and-engineering/ https://www.pugetsystems.com/labs/hpc/amd-zen4-threadripper-pro-vs-intel-xeon-w9-for-science-and-engineering/#respond Thu, 07 Mar 2024 19:56:57 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=25940 The performance improvement with the new Zen4 TrPRO over the Zen3 TrPRO is very impressive!
My first recommendation for a Scientific and Engineering workstation CPU would now be the AMD Zen4 architecture as either Zen4 Threadripper PRO or Zen4 EPYC for multi-socket systems.

The post AMD Zen4 Threadripper PRO vs Intel Xeon-w9 For Science and Engineering appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/amd-zen4-threadripper-pro-vs-intel-xeon-w9-for-science-and-engineering/feed/ 0 Benchmarking with TensorRT-LLM https://www.pugetsystems.com/labs/hpc/benchmarking-with-tensorrt-llm/ https://www.pugetsystems.com/labs/hpc/benchmarking-with-tensorrt-llm/#respond Fri, 16 Feb 2024 18:06:51 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=23187 Evaluating the speed of GeForce RTX 40-Series GPUs using NVIDIA's TensorRT-LLM tool for benchmarking GPU inference performance.

The post Benchmarking with TensorRT-LLM appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/benchmarking-with-tensorrt-llm/feed/ 0 Experiences with Multi-GPU Stable Diffusion Training https://www.pugetsystems.com/labs/hpc/multi-gpu-sd-training/ https://www.pugetsystems.com/labs/hpc/multi-gpu-sd-training/#respond Mon, 29 Jan 2024 23:22:19 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=22714 Results and thoughts with regard to testing a variety of Stable Diffusion training methods using multiple GPUs.

The post Experiences with Multi-GPU Stable Diffusion Training appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/multi-gpu-sd-training/feed/ 0 LLM Server Setup Part 2 — Container Tools https://www.pugetsystems.com/labs/hpc/llm-server-setup-part-2-container-tools/ https://www.pugetsystems.com/labs/hpc/llm-server-setup-part-2-container-tools/#respond Mon, 20 Nov 2023 22:15:10 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=21032 This post is Part 2 in a series on how to configure a system for LLM deployments and development usage. Part 2 is about installing and configuring container tools, Docker and NVIDIA Enroot.

The post LLM Server Setup Part 2 — Container Tools appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/llm-server-setup-part-2-container-tools/feed/ 0 LLM Server Setup Part 1 – Base OS https://www.pugetsystems.com/labs/hpc/llm-server-setup-part-1-base-os/ https://www.pugetsystems.com/labs/hpc/llm-server-setup-part-1-base-os/#respond Wed, 15 Nov 2023 23:42:42 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=20669 This post is Part 1 in a series on how to configure a system for LLM deployments and development usage. The configuration will be suitable for multi-user deployments and also useful for smaller development systems. Part 1 is about the base Linux server setup.

The post LLM Server Setup Part 1 – Base OS appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/llm-server-setup-part-1-base-os/feed/ 0 Can You Run A State-Of-The-Art LLM On-Prem For A Reasonable Cost? https://www.pugetsystems.com/labs/hpc/can-you-run-a-state-of-the-art-llm-on-prem-for-a-reasonable-cost/ https://www.pugetsystems.com/labs/hpc/can-you-run-a-state-of-the-art-llm-on-prem-for-a-reasonable-cost/#respond Mon, 17 Jul 2023 23:44:37 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=15672 In this post address the question that's been on everyone's mind; Can you run a state-of-the-art Large Language Model on-prem? With *your* data and *your* hardware? At a reasonable cost?

The post Can You Run A State-Of-The-Art LLM On-Prem For A Reasonable Cost? appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/can-you-run-a-state-of-the-art-llm-on-prem-for-a-reasonable-cost/feed/ 0 Note: How To Setup Apache on Ubuntu 22.04 For User public_html https://www.pugetsystems.com/labs/hpc/note-how-to-setup-apache-on-ubuntu-22-04-for-user-public_html/ https://www.pugetsystems.com/labs/hpc/note-how-to-setup-apache-on-ubuntu-22-04-for-user-public_html/#respond Fri, 02 Jun 2023 15:37:23 +0000 https://www.pugetsystems.com/?post_type=hpc_post&p=14961 This is a short note on setting up the Apache web server to allow system users to create personal websites and web apps in their home directories.

The post Note: How To Setup Apache on Ubuntu 22.04 For User public_html appeared first on Puget Systems.

]]> https://www.pugetsystems.com/labs/hpc/note-how-to-setup-apache-on-ubuntu-22-04-for-user-public_html/feed/ 0