January 12, 2016

Upgrading R730 with NVIDIA K1 GRID Card

I recently ordered a couple of Dell R730 servers and then got a subsequent request to add a little graphics horsepower for our VDI environment in the form of some K1 GRID cards. Turns out, the process to add these suckers in to an existing server that wasn't specifically built out for them has a few catches - you can't just drop them in to the server and take off. It's easy to do, but there isn't much official documentation from Dell on it, so here's a quick guide:

First, the requirements:

  • Compatible R720 or R730 series server
  • 1100W PSUs
  • Compatible processor - there's a specific list of procs that are certified for use depending on your chassis, which relates to power consumption with the GPUs.

Start by removing the shroud cover:


The first problem we have to fix is the airflow issue - the factory setup for an R730 uses heat sinks that capture all of the air under the shroud and will restrict the amount of fresh air going to the GPUs.:


If you ordered for K1 cards from Dell, you should also have received an "upgrade kit" consisting of a new low-profile heat sink and a 6-pin PCIe power cable. Loosen the four screws on the CPU heat sink (I'm installing two cards here and I have two CPUs, so I'll be replacing both), wiggle them a bit to get them to detach, and carefully clean all of the grey heat sink compound off the CPU:

Make sure you also replace any empty DIMM slots with blanks. Install your new low-profile heat sinks, replace the shroud, and you'll have an unrestricted path for airflow back to the PCIe risers (I removed a fan here):

Before you put in the card(s), connect the 6-pin power cable to the appropriate slot on the riser. If you are installing two cards, connect each to their respective riser - do not try to use one cable/riser to power both, even though they do have dual connectors on the end. Route the power cables through the notch in the shroud (marked in red below). Now remove your expansion slot blanks and carefully drop in your GRID cards. They should just barely clear any obstructions to get in there - it's a tight fit.


Install your second card if applicable and then make sure you press the buttons marked in blue to deploy the supports for the card(s):


If you already have 1100W power supplies equipped, you are all done. If you don't, you'll want to upgrade before the systems go in to production. I've used the K1 cards with 750W, but the system will complain about power usage and may not have enough juice at peak consumption or during a failure of one PSU.

Now just power on your server and follow the appropriate guide from NVIDIA for installing drivers: http://www.nvidia.com/object/virtual-gpus.html. Good luck!

1 comment:

  1. Hi Paul and thank you so much for this article, not sure if you are active on this site anymore but idk anyone that I could ask, I've installed the low profile cpu heatsinks, however the air shroud touches them and doesn't fit, did yours just fit ? or is there an air shroud replacement I'm not aware of.

    Thank you !

    ReplyDelete

Red Flags and the Value of Experience

One of the things I hear often said, and something I subscribe to as well, is the idea that a lot of technical knowledge in the world of IT ...