Jump to content

Archived

This topic is now archived and is closed to further replies.

Guest kittonian

Creating a Render Farm (what hardware?)

Recommended Posts

Guest kittonian
  • Topic Author
  • Chris,

     

    Thank you for all of that. The reason for the way overkill on the P/S is because it is silent to up to 40% usage (i.e. the fan doesn't even come on). This is another way to keep the noise down. It's not just about the components, it's about a build that is extremely powerful, yet as quiet as possible.

     

    With regards to the CPU cooling, that's a good point. I would have though that the manufacturers considered dual processor boards when desigining their coolers so it didn't occur to me to look at space requirements.

     

    UPDATE:

    Just read a lot on the Fractal chassis and it seems that the Corsair is every bit as good if not better at making things as silent as possible. There's really only so much you can do before moving to an ISO box.

     

    ----------

     

    Srek,

     

    Thank you for your thoughts but if you read the thread you will see that a noisy system isn't in the cards.

    Share this post


    Link to post
    Share on other sites

    Guest Chriscorr
  • Topic Author
  • Fair enough on that PSU feature, although it still seems overkill to pay that much more for it. PSUs are very silent to begin with, CPU coolers and case fans will always be heard first. But who knows, maybe you'll move to GPU rendering down the line and buy a few high-end GPUs, then you will certainly need that much power. =)

     

    There are motherboards with much better spacing between sockets, but they are SSI EEB form factor. You can install bigger and more silent coolers on each CPU on many of those boards. They are more expensive, harder to find, and the form factor will make it harder to find a case for it as well... This stuff is really annoying sometimes because there are quite a few details to consider. And the further away from mainstream we move the easier it is to overlook "details".

     

    About the cases, the Carbide 330R is a great case. I prefer the the Define R5 for a few reasons that include internal configuration flexibility, more cooling mounting options and slightly better ergonomics. But these might not matter for you since it's not a workstation.

    Share this post


    Link to post
    Share on other sites

    If the render node is going into a server room, why worry so much about the noise. Is it a real server room or closet? Does the room have it's own cooling? As Srek points out, these machines will generate a lot of heat.

     

    Have you calculated your current render power (Cinebench Points) vs your expected CB from the new build? These values will approximate how much your renders will be sped up or how many more machines you'll need to achieve the render times that are acceptable. To make sure you are getting the most for your money, you should compare the cost of different systems by the Cost/Cinebench Point. This can vary quite a bit.

     

    There are tons of articles on the web for building your own farm. I've been researching the options myself for quite some time.

     

    http://www.fuchsundvogel.de/blog/2015-02-03-building-your-own-renderfarm

    http://www.braverabbit.de/playground/?page_id=630

    Share this post


    Link to post
    Share on other sites
    Guest kittonian
  • Topic Author
  • JimH:

     

    We work out of our house and have a room on the second floor dedicated to being the "server room" but it's just a big bedroom where all our CAT6 runs come in. So, although there is a door and it's a fairly large house, if the fan noise is loud we can hear it in other spaces. We have a big server rack and a before doing some updates to more silent options had some loud 4U chassis running our servers. Once we shut those off and got some silent systems the noise was no longer audible. That's why we're so concerned about the noise coming from these new systems. We have two air conditioners in the house (one that serves the first floor and one that serves the second floor), so the room stays pretty cool even in the summer.

     

    With regards to the Cinebench scores, I have done the research. Considering we are currently running a Mac Pro 1,1 and a Macbook Pro 6,2 and the CB scores are extremely low, this new machine I'm spec'ing out is far and away more powerful (CPU score around 2500). I have to imagine that with DDR4 memory and dual Xeon E5 2670 v3 processors (totalling 24 cores and 48 threads) that we will see quite an improvement in render times. Plus, as was mentioned earlier in this thread, the idea of having a single powerful machine vs multiple lower end machines alleviates noise, extra possible maintenance, more components that could fail, electricity costs, etc.

     

    Chris:

     

    You keep making great points that I didn't consider regarding the motherboard, processors, and cooling. Can you recommend a great motherboard in the SSI EEB form factor that would allow for two of these Arctic cooling systems (or perhaps another top end cooling solution that is virtually silent)? I can always find a chassis. The most important thing is ensuring we have the power we need to do these renders and that the CPUs stay as cool as possible so they don't slow themselves down.

    Share this post


    Link to post
    Share on other sites
    Guest kittonian
  • Topic Author
  • Chris,

     

    After watching some videos and reading a bunch on your Noctua suggestion, it seems that if I swap out the two Arctic components (cooler and thermal paste) for the Noctua NH-U9DXi4 90mm SSO2 CPU Cooler which seems to come with thermal paste, I can keep all the rest of the components I have listed and I'm good to go.

     

    Is this correct?

    Share this post


    Link to post
    Share on other sites
    Guest Chriscorr
  • Topic Author
  • Well, after some digging I believe this is the best option I can find:

     

    http://www.asus.com/Commercial_Servers_Workstations/Z10PED8_WS/overview/

     

    The sockets have decent separation between them and the ILM is square, although the RAM slots are pretty damn close to the sockets. One option is water cooling. Closed loop cooling systems can provide really quiet and cool operation and in tight spaces like this they can be a really good solution. They aren't without pitfalls however, it's water inside your case after all, no matter how safe and advanced these units have become the risk is higher. It would look something like this: 

     

    http://www.houseofdreams.be/wp-content/uploads/2013/12/pic_disp1.jpg

     

    http://www.synchrostudios.com/wp-content/uploads/2013/02/Synchro-Station.jpg

     

    If you're interested in that you'll need a case that supports nice systems and their radiators, like the NZXT Kraken X41. RAM clearance with these shouldn't be a problem because the tubes go straight upward.

     

    If you go with air cooling it's always hard to tell if they will fit, even when reading all the specs and looking at pics. 

     

    http://www.tweaktown.com/reviews/6120/asus-z9pe-d8-ws-workstation-intel-c602-motherboard-review/index5.html

     

    These guys used two Noctua NH-U9DX i4 coolers, but there is plenty of space left between the sockets, so I think you could use the NH-U12DX i4 without any clearance issues, which is the 120mm version (cooler and quieter). (btw I realise they are using the Z9PE and not the Z10PE board, but they are identical in layout as far as I can tell)

     

    As for the Arctic i30s you wanted. Take a look:

    http://www.avadirect.com/forum/Message/5975-WORKSTATION-PC-Enermax-Fulmo-GT-ASUS-Z9PED8-WS-2-x-Xeon-E52609-32GB-ECC-Tesla-C2070/

     

    At first glance it looks like they fit nicely on this board. But if you look closer, you'll see that the fan mount blocks the RAM slots, and the builder of this system had to put the RAM modules all on the opposite side, which I'm pretty sure will not allow for Quad Channel operation. So don't get the i30.

     

     

     

    EDIT:

     

    Hehe this is pretty weird. I didn't make any Noctua suggestions until this post, which I posted before I saw your post about me making Noctua suggestions, which were the exact same you mentioned... Cue the twilight zone theme. Anyway I suggest the Z10PE. The Z10PA is a compact design and will only make fitting the coolers a bigger unknown.

    Share this post


    Link to post
    Share on other sites
    Guest kittonian
  • Topic Author
  • Lol! No Twilight Zone music necessary. I think you forgot. Up above you posted:

     

    There are a few "big name" manufacturers that make Xeon specific coolers as well, such as this one:

     

    http://www.newegg.com/Product/Product.aspx?Item=9SIA0AJ2MW9893

     

    That links to the Noctua cooler I mentioned I took a look at.

     

    So basically go with the Noctua and the larger Asus motherboard and I'm good to go :)

    Share this post


    Link to post
    Share on other sites
    Guest Chriscorr
  • Topic Author
  • Yes, I think that's it. =) Although I don't think the Carbide 330R supports the SSI EEB format Z10PE board. 

    Share this post


    Link to post
    Share on other sites
    Guest kittonian
  • Topic Author
  • Well, foiled again. Seems that board you mentioned is discontinued. There's the Z10PE-D16 which is also an SSI EEB motherboard, but I'm wondering if I use the Noctua coolers if that will really be necessary? When I watched a video with a guy who was building a dual xeon mid-tower system with the Noctua he didn't seem to have any issues installing both of them on the motherboard. Just curious as the Corsair case doesn't support SSI-EEB and when I looked for SSI-EEB chassis it seems most of them are 4U rackmount which of course brings me back to how to keep things as quiet as possible.

     

    Update:

    Ha! Just saw you posted the same info. At some point we just jump into a IRC chatroom right? JK.

    Share this post


    Link to post
    Share on other sites
    Guest Chriscorr
  • Topic Author
  • You mean using the Z10PA-D8? I think you can. Both the CPU and RAM sockets are really close together, from the looks of it you'll have to rotate the cooler 90º so the fans are aligned vertically otherwise they will intersect each other. The fans on the Noctua seems to have enough clearance for the low profile memory you'll buy.

     

    I was suggesting the Z10PE-D8 because you really want to keep it quiet, which means you simply need more space for bigger coolers with larger fans. The question now is if these Noctua 90mm fans and your case are quiet enough for you. Honestly I can't imagine that you'll be able to hear anything at all using this configuration and a Carbide 330R or a Define R5 from another room!

    Share this post


    Link to post
    Share on other sites
    Guest kittonian
  • Topic Author
  • That's what I was thinking (noise-wise) and it allows me to stick with an ATX board which makes everything easier. If the system itself it basically whisper quiet and it's on the second floor behind a closed door, I shouldn't be able to hear anything when it's running (fingers crossed).

     

    Going through the specs last night I realized my math was a bit off. I had 64GB of memory in there and I actually needed 96GB (2GB x 48 threads).

     

    So now the whole thing is costing me $5645.

     

    We're thinking that since this is such a crazy powerful machine we would run Team Render Server and Team Render Client on it so that we can do a Save Project to the repository directory on this new machine and still have full use of our workstations (since they aren't much help with huge renders). Then, if need be, we can build another one of these puppies and speed up our renders even more.

    Share this post


    Link to post
    Share on other sites
    Guest Chriscorr
  • Topic Author
  • I didn't know about the 2GB per thread guideline and can't find much about it. But to get exactly 96GB you'll have to have uneven module capacity, for example 8x4 + 16x4. I don't know if this will affect quad channel operation.

    Share this post


    Link to post
    Share on other sites

    • Recently Browsing   0 members

      No registered users viewing this page.

    YOUTUBE CHANNEL:

    ABOUT US:

    C4D Cafe is the largest CINEMA 4D community. We provide facilities for discussion, showcasing and learning our favorite software :)
    ×
    ×
    • Create New...