Jump to content

Archived

This topic is now archived and is closed to further replies.

jimishere

New Workstation Multi-GPU Question

Recommended Posts

Guest darby

Beware that GPU renderers don't generally achieve feature parity with CPU based renderers. Many of the shortcomings can be attributed to having access to less RAM.

Share this post


Link to post
Share on other sites

Dave - Thanks for the helpful reply. That thread is just the kind of discussion I was hoping to read.

 

It sounds like some other 3D tools can leverage the GPU a bit more easily than C4D. Perhaps I should figure out which software works the best with multi-GPU, rather than what hardware configuration works the best with C4D. I initially got into C4D as a natural extension of working in After Effects, but I'm really looking to push my 3D projects are far as the technology can take them. I am an artist and I want to create experimental video projects that combine cutting edge visual effects animations with dramatic film sequences. I'm enthusiastic about dynamic simulations, fluid effects, animations with tons of particles and moving objects - real compute heavy stuff.

 

I am extremely lucky to be able to build such a workstation. My background is in software engineering and I've built many machines. I sold my internet company so I have a decent budget to invest into passion projects. Give a computer geek some cash and it won't take long before they go and build a ridiculous supercomputer with it.

 

Any thoughts to building your own render farm?  Just wondering if you would be better off with one mid-range workstation (3.5 GHz processor, Titan black, 32 Gb DRAM) for your 3D work but offload the rendering to a bunch blades configured with 1.8MHz 4 core Xeon's in a rack.  The Xeon E5-2603 processors can be had for around $200 a piece ... they build racks that can hold 14 blades and each blade could support two processors giving you access to 112 cores and more than likely 10 times the rendering speed of two 3 GHz 10 core processors in a single workstation.

 

Again, this is a "money is no object" solution....but I have always wondered if building the cheapest blade possible, but having lots of them would be more effective than one super-computer.

 

Or you could consider this solution --- which can get up to 1920 cores.

 

Dave


Sorry...but I simply do not have enough faith to be an atheist.

Share this post


Link to post
Share on other sites


Brasco - Agreed on the CPU speed vs. cores. I am going to opt for a faster processor instead and focus on ways to really maximize the GPU rendering. Perhaps I'll start out with a single Titan Z for awhile, then see how it compares with two, before I consider 3 or more.

 

Just as a side note, DO NOT BUY A TITAN Z, slower more expensive ($3000 vs $2000) than dual Titan Blacks and they will utterly trounce it when overclocked. There's literally no real advantage since it's a 3 slot card, you don't even save space unless you're putting it in an ITX build.

 

cheers

brasc

Share this post


Link to post
Share on other sites
  • Topic Author
  • Any thoughts to building your own render farm?  Just wondering if you would be better off with one mid-range workstation (3.5 GHz processor, Titan black, 32 Gb DRAM) for your 3D work but offload the rendering to a bunch blades configured with 1.8MHz 4 core Xeon's in a rack.  The Xeon E5-2603 processors can be had for around $200 a piece ... they build racks that can hold 14 blades and each blade could support two processors giving you access to 112 cores and more than likely 10 times the rendering speed of two 3 GHz 10 core processors in a single workstation.

     

    Again, this is a "money is no object" solution....but I have always wondered if building the cheapest blade possible, but having lots of them would be more effective than one super-computer.

     

    Or you could consider this solution --- which can get up to 1920 cores.

     

    Dave

     

    I was wondering the same thing - if using a bunch of small computers would be better than one giant computer. It's actually pretty easy to spin up a cluster of amazon EC2 servers (http://aws.amazon.com/hpc/) to do this and pay by the hour rather than buying nodes. You can get as many cores as you care to pay for in this scenario.

     

    I researched Boxx and actually got a quote from one of their salespeople. The customer service was excellent, but they are terribly overpriced compared to other systems integrators. In fact it was after reviewing that quote that I decided to roll my own. On another note, thinkmate.com has comparable systems and they are reasonably priced - but their customer service was underwhelming. Go figure.

     

    I guess I was hoping to have a near realtime viewport rendering situation (ya know, so I can tweak this and that and watch what happens immediately) rather than having to upload my project across a network to a render farm for processing. It seems the multi-GPU panacea is *right* around the corner to enable this but it's not exactly arrived yet (at least with C4D).

    Share this post


    Link to post
    Share on other sites
  • Topic Author
  • Just as a side note, DO NOT BUY A TITAN Z, slower more expensive ($3000 vs $2000) than dual Titan Blacks and they will utterly trounce it when overclocked. There's literally no real advantage since it's a 3 slot card, you don't even save space unless you're putting it in an ITX build.

     

    cheers

    brasc

    Excellent point - I didn't realize it hogged three slots. But the Z does have twice the memory as the black. As I understand it, in a multi-gpu setup your memory is not really additive across cards. Two blacks is still effectively six gigs of memory, not twelve like the Z. Though to be honest I'm still unclear on exactly how important/helpful this memory is compared to raw number of cores in a GPU render scenario.

    Share this post


    Link to post
    Share on other sites

    C4D will only use one GPU, so throwing in multiple GPUs for 3D work is useless and a waste of money unless you're using some type of 3rd party render engine like Octane, Indigo, Bunkspeed, etc.

     

    The Nvidia maximus configuration is expensive (Quadro + Tesla) but its purpose is to take the stress off of the Quadro for CUDA processing. The Quadro would handle all OpenGL functions (driving the monitor, and any OpenGL stuff) while the Tesla would handle all CUDA stuff. I had the opportunity to use a machine for about 3 weeks that had the Maximus configuration (K5000, Tesla K20) and it really did not make a huge difference in anything other than Premiere, and Turbulence FD. The Tesla chewed through the simulation pretty fast.

     

    If you've got that kind of money to throw at a single machine, then my suggestion to you would be to build a single machine for working (i7, lots of RAM, GTX 780 or maybe a Titan) and then build or buy some render slaves, network them together (netrender) and that will greatly decrease render times.

     

    Also, stay away from dual-GPU cards. Most engines do not recognize both GPU's, thus leaving one GPU sitting there idle, never being used.

    Share this post


    Link to post
    Share on other sites

    Honestly, I would not get hung up on the fact that the Titan Z has 12 Gb of memory.  My understanding is that it uses 2 GPUs and 6 Gb is partitioned to each GPU.  That is why the Titan Z and the Titan Black and the Quadro K6000 (another 12 Gb GPU) all have the same NVidia Compute Capability Index of 3.5.  The clock speeds are about equal and while memory helps speed things up, the fact that the Titan Z has twice the memory, it is not speeding up at the 2X multipliers you are thinking.

     

    Interestingly enough, there is only one NVidia card out there that has a compute index higher than 3.5, and that is the GeForce GTX 750 and it only has 640 CUDA cores and 2Gb of memory.  But its clock speed is 15% faster than the Titan's (1020 MHz vs. 889 MHz).  With that said, you would expect a compute capability index 15% higher or less than the Titan's.  Instead, the GTX 750's compute index is 43% higher (5.0 vs. 3.5).  So based on specs, I have no idea what makes for a fast GPU: is it clock speed, memory, or CUDA cores?

     

    Now, I know that for Turbulence FD, it is the compute capability index more so than the number of CUDA cores for calculating the simulation.  For GPU renderers, I am not so sure if you want more CUDA cores or higher clock speeds. 

     

    Dave


    Sorry...but I simply do not have enough faith to be an atheist.

    Share this post


    Link to post
    Share on other sites
    Guest darby

    In the case of a 750 Ti vs. Titan comparison you'd have to consider that 750 Ti is based on the newest architecture (Maxwell) while most other 700 series and Titans are based on the older Kepler architecture.  The only Maxwell chip that has shipped thus far, though, is a lower end one as is clear from it being only in the 750 and 750 Ti.

     

    The 12GB in the Titan Z is split, 6GB for each chip.  The 12GB in the Quadro K6000 as well as the Tesla K40 is not split; those are single GPU cards.  That's the highest amount of memory you can get on an nvidia board these days.

    Share this post


    Link to post
    Share on other sites

    Dave - Thanks for the helpful reply. That thread is just the kind of discussion I was hoping to read.

     

    It sounds like some other 3D tools can leverage the GPU a bit more easily than C4D. Perhaps I should figure out which software works the best with multi-GPU, rather than what hardware configuration works the best with C4D. I initially got into C4D as a natural extension of working in After Effects, but I'm really looking to push my 3D projects are far as the technology can take them. I am an artist and I want to create experimental video projects that combine cutting edge visual effects animations with dramatic film sequences. I'm enthusiastic about dynamic simulations, fluid effects, animations with tons of particles and moving objects - real compute heavy stuff.

     

    I am extremely lucky to be able to build such a workstation. My background is in software engineering and I've built many machines. I sold my internet company so I have a decent budget to invest into passion projects. Give a computer geek some cash and it won't take long before they go and build a ridiculous supercomputer with it.

     

    As far as the CUDA question, it seems you are right about Titan. While NVIDIA claims that the Maximus configuration is the best of the best, I haven't found any real world arguments that show a compelling difference from a rendering perspective. Titans have more CUDA cores per card and at a cheaper price and that seems to be all that really matters - provided you are leveraging the GPU at all, that is.

     

    Brasco - Agreed on the CPU speed vs. cores. I am going to opt for a faster processor instead and focus on ways to really maximize the GPU rendering. Perhaps I'll start out with a single Titan Z for awhile, then see how it compares with two, before I consider 3 or more.

    This sounds as if you should check out what tool you are most comfortable with first. What help is an über machine that lets an application scream that does not do what you want, or that you just plain don't like to use?

    First: set your goals, your personal goals regarding your work, nothing technical

    Second: check what tools fit you and come to terms with them

    Third: Optimize production time by adding computing power. Which hardware you need will be easy to tell once you know your workflow/pipeline.

    Share this post


    Link to post
    Share on other sites

    I would buy one Titan z and see how that goes.  With that you would have incredibly fast rendering using Thea Render and other software that uses the GPU.  If you wanted more power you could add more cards later as needed.  To overcome the three slots it takes you could go with water cooling, but you have to be careful here as if done wrong the water could bust on your computer destroying it.  Warranties don't cover water damage.

    For Adobe software you have to take a look and see what cards are supported.  The Quadro might be the only one that works there and it might also not have any speed up with more than one card.  Thus the best set up would be three Titan z and one Quadro card.  That would be so much power it would be like a render farm in a box.  You would also need a 2000 Watt or so power supply and a dedicated 20 amp power plug at 110 volt for your house or 10 amp fuse at 220 volt.  If anything else is on that plug the computer it will blow the fuse to your house.  It will also make your room very hot!

     

    The amount of power from 3 Titan Z's would probably be too much for one person.  It would be more affordable and cheaper to use a render farm.  You can get over 63 days of farm time on Newton Render for $9,000.  With the speed of the farm equal to about 15x Titans and a 3x Titan z equal to 6 Titans you would get (15/6)x63= 157.5 equivalent days of rendering on the farm for the same $9,000.  The rendering would also take 63 days instead of 157.5 days.

    Share this post


    Link to post
    Share on other sites

    Excellent point - I didn't realize it hogged three slots. But the Z does have twice the memory as the black. As I understand it, in a multi-gpu setup your memory is not really additive across cards. Two blacks is still effectively six gigs of memory, not twelve like the Z. Though to be honest I'm still unclear on exactly how important/helpful this memory is compared to raw number of cores in a GPU render scenario.

     

    The guys have said it already, but I'll reiterate, it's two GPUs running 6GB per GPU, no software (currently) can address both sets of RAM, so it would always use the lowest (6GB in this case). The clock speeds are also massively lower than Titan Blacks due to trying to get the ridiculous card under TDP targets.

     

    @darby + @Dave, yes the 750ti is a Maxwell chip, but it's very low power and entry performance, great for efficiency not so great if you need to do anything, if someone wants a Maxwell card I'd recommend anyone to wait for GM210/200 which will hopefully be this Q3/4. The compute capability numbers (3.5 v 5.0) is just a software version, it's not really an indicator of performance.

     

    cheers

    brasc

    Share this post


    Link to post
    Share on other sites
  • Topic Author
  • Thanks for all the great responses folks. It's been awesome to get all this helpful feedback. FYI - after some more consideration I ordered the Quadro K6000+Tesla (Maximus) configuration. Seems to be the only way to get true 12GB memory and there are a few pro bells and whistles with that setup.

     

    I'll be tinkering with the various plugins and experimenting with ways to speed up my new GPU powered workflow.

     

    Happy rendering!

    Share this post


    Link to post
    Share on other sites
    Guest
    This topic is now closed to further replies.

    • Recently Browsing   0 members

      No registered users viewing this page.

    YOUTUBE CHANNEL:

    ABOUT US:

    C4D Cafe is the largest CINEMA 4D community. We provide facilities for discussion, showcasing and learning our favorite software :)
    ×
    ×
    • Create New...