Hi IBK.
Do you have some documentation on setting up a remote server for D6 or is it too experimental yet?
Thanks in adavance!
paul
D6 Remote server setup
Moderatoren: ghorwin, fechner, ruisinger
Re: D6 Remote server setup
Hi Paul,
I've uploaded the documentation, the server and client releases to our download page.
https://bauklimatik-dresden.de/downloads.php (very far down at utiliities)
I'll also edit the DELPHIN doc page later with some info on the Remote Solver Server functionality.
A note on the queues: originally, we only had single-core solvers and planned to run several jobs in parallel on the server machine. However, now that DELPHIN is parallelized, actually running n Jobs with p threads each *after* another is faster, than running those n jobs in parallel with 1 thread each (memory bottleneck issue).
Hence, for use with DELPHIN I would recommend configuring the queues to run only 1 or at max 2 jobs in parallel (to avoid stalling the queue) and allow each process to use as many threads as available.
Example config for our 16 core ThreadRipper:
1 job in the fast queue - using 10 cores
2 jobs in the slow queue - using 10 cores each
So, when all jobs are busy, we have 30 out of 32 (hyperthreaded) threads running, leaving 2 for system jobs.
In the normal case only 2 slow jobs are running, summing up to 20 threads (which means each CPU is running full-time, but jobs do not block each other much).
-Andreas
I've uploaded the documentation, the server and client releases to our download page.
https://bauklimatik-dresden.de/downloads.php (very far down at utiliities)
I'll also edit the DELPHIN doc page later with some info on the Remote Solver Server functionality.
A note on the queues: originally, we only had single-core solvers and planned to run several jobs in parallel on the server machine. However, now that DELPHIN is parallelized, actually running n Jobs with p threads each *after* another is faster, than running those n jobs in parallel with 1 thread each (memory bottleneck issue).
Hence, for use with DELPHIN I would recommend configuring the queues to run only 1 or at max 2 jobs in parallel (to avoid stalling the queue) and allow each process to use as many threads as available.
Example config for our 16 core ThreadRipper:
1 job in the fast queue - using 10 cores
2 jobs in the slow queue - using 10 cores each
So, when all jobs are busy, we have 30 out of 32 (hyperthreaded) threads running, leaving 2 for system jobs.
In the normal case only 2 slow jobs are running, summing up to 20 threads (which means each CPU is running full-time, but jobs do not block each other much).
-Andreas
Re: D6 Remote server setup
Thanks!
Any recommendations on the hardware configuration?
If the memory is the bottleneck, is there a sensible number of max cores?
We were thinking of getting the 24 core threadripper. Should the CPU clock be preferred over cores?
How about drives? NVMe or NVMe RAID?
As some of our group will still require windows for their software, can the remote solver run on windows too? (Haven't gone too deep into it, but microsoft was supposed to embed a linux subsystem in windows too)
Any recommendations on the hardware configuration?
If the memory is the bottleneck, is there a sensible number of max cores?
We were thinking of getting the 24 core threadripper. Should the CPU clock be preferred over cores?
How about drives? NVMe or NVMe RAID?
As some of our group will still require windows for their software, can the remote solver run on windows too? (Haven't gone too deep into it, but microsoft was supposed to embed a linux subsystem in windows too)
Re: D6 Remote server setup
Threadripper is a good compromise between speed and money, so go for it. If you can afford it, always take the faster variant - turbo speed still is the dominating factor.
Mind, DELPHIN parallelization benefits most until 6-8 Cores, after that the increase in speed is not as great (not all cores can be fully utilized). If you want to run several jobs in parallel, more cores are better (though mind the memory bus bottle neck).
We have a threadripper 1950x which is ok for most of our simulations our, the 2950x will do good as well (with 16 cores, 32 threads). Also, these are 180W CPUS - newer Threadrippers take 280W or more (but have more cores, of course).
NVMe is a must, i would say. Normal jobs, that you'll need a highend workstation for, are not really output-driven (unless you do optimization stuff, where you run thousands of small jobs to see the progress of your parameter optimization - here, ssd-speed rules). You can take the "evo" or consumer variants - the expensive pro variants are really not needed since simulation workstations are not database servers.
A raid system is IMHO not necessary for simulation jobs - if the ssd crashes, just reinstall and re-run the simulation (no backup - no pity
). Normally, pre- and postprocessing is done on a separate machine anyway. Save the money and put it into the CPU.
As for overclocking memory, I would go with regular normal brand memory chips - the overclocking ram modules with head spreader aren't really worth it (even put the system at risk).
Lastly, take a good case with decent ventilation, we have a "fractal design define r6" - quiet and very good air circulation. As CPU cooler we have a CORSAIR Cooling Hydro Series H100i v2 - which works very well so far. A large CPU fan should be ok, too.
Mind, DELPHIN parallelization benefits most until 6-8 Cores, after that the increase in speed is not as great (not all cores can be fully utilized). If you want to run several jobs in parallel, more cores are better (though mind the memory bus bottle neck).
We have a threadripper 1950x which is ok for most of our simulations our, the 2950x will do good as well (with 16 cores, 32 threads). Also, these are 180W CPUS - newer Threadrippers take 280W or more (but have more cores, of course).
NVMe is a must, i would say. Normal jobs, that you'll need a highend workstation for, are not really output-driven (unless you do optimization stuff, where you run thousands of small jobs to see the progress of your parameter optimization - here, ssd-speed rules). You can take the "evo" or consumer variants - the expensive pro variants are really not needed since simulation workstations are not database servers.
A raid system is IMHO not necessary for simulation jobs - if the ssd crashes, just reinstall and re-run the simulation (no backup - no pity
). Normally, pre- and postprocessing is done on a separate machine anyway. Save the money and put it into the CPU.As for overclocking memory, I would go with regular normal brand memory chips - the overclocking ram modules with head spreader aren't really worth it (even put the system at risk).
Lastly, take a good case with decent ventilation, we have a "fractal design define r6" - quiet and very good air circulation. As CPU cooler we have a CORSAIR Cooling Hydro Series H100i v2 - which works very well so far. A large CPU fan should be ok, too.