add arrow-down arrow-left arrow-right arrow-up authorcheckmark clipboard combo comment delete discord dots drag-handle dropdown-arrow errorfacebook history inbox instagram issuelink lock markup-bbcode markup-html markup-pcpp markup-cyclingbuilder markup-plain-text markup-reddit menu pin radio-button save search settings share star-empty star-full star-half switch successtag twitch twitter user warningwattage weight youtube

Comments

Comments

Comment reply on sysdmp's Completed Build: Reaper v2 (Gentoo Linux threadripper workstation)

  • 12 months ago
  • 2 points

I used awesomewm for a long time, takes too much time to set up and maintain configs =)
dwm is set and forget for many years. plasma is also nice lately, I mean veeery nice. and takes almost same amount of ram as xfce (not kidding). not that I care about RAM, but some folks do.

kernel times depend on the configuration A LOT.

make defconfig
make -j64

takes about 30 seconds.

my configuration (very static, everything possible stripped out) takes 15-20, sometimes 40 seconds, but I do module signing and debuginfo, it adds some delay.

libreoffice takes 10-14 minutes.

I run emerge with 19 niceness, and somewhat throttled io

PORTAGE_NICENESS=19
PORTAGE_IONICE_COMMAND="ionice -c 3 -p \${PID}"

also have some qa/binary checks on top that, with disabling all that stuff above times can be improved further. But it does not really bother me a lot, current speed is more than enough.

Comment reply on sysdmp's Completed Build: Reaper v2 (Gentoo Linux threadripper workstation)

  • 12 months ago
  • 1 point

I got them from serversupply. just arrived today.
M391A2K43BB1-CRC are the sticks, i got them for $200 each, used quote button on their website.

all 128gb working fine on 2933 so far, just bumped frequency and set voltage to 1.35. without bumping voltage system crashed.
haven't played with timings yet.
moderate latency, something like 55 in passmark, I'll try to squeeze a bit more from those bad boys and will update the post with timing and all the related info.

Comment reply on sysdmp's Completed Build: Reaper v2 (Gentoo Linux threadripper workstation)

  • 12 months ago
  • 1 point

yeah, those drives are good for the task. Perfect drives for slog, not so perfect for the wallet =)
and scratch space is ok, I meant don't put OS there, or any non-temorary, constantly accessed data.

measure, measure and test several times before you settle. for me average win was 10%, but it's not enough for me to care. your usecase may be different.
I strongly recommend against any layer (software or hardware raid) below zfs if you care about data, but you are the boss =)
btw, removing and adding slog/l2arc is a dynamic operation. and loosing slog you don't loose the pool, only single txg commit (amount of data you can write at max speed per 5 sec by default). and slog does not handle async writes by default, make sure you play with sync parameter. and of course slog/zil can be striped, which will reduce redundancy but can improve performance even more. latency is everything with slog.

good luck with zfs.

PS ordered samsung ECC sticks, upgrades coming tomorrow =)

Comment reply on sysdmp's Completed Build: Reaper v2 (Gentoo Linux threadripper workstation)

  • 12 months ago
  • 2 points

Hi!

Yeah, plasma5 is nice, but I could not stand plasma4, it was awful.
but on laptops I use dwm or i3 for speed and efficient workflow.

distcc is for several x86 laptops/boards running gentoo. saves so much time on upgrades. since CPUs in all the devices are different, I don't use binpackages, just let the big boy compile stuff for the rest of the crowd.

Well, system zpool is system zpool, it hosts main gentoo system, literally hundreds of auto-rotating snapshots (15min, hourly, daily, weekly, monthly, yearly ).
Special script creates and deletes backups using tower of hanoi algorithm.
Also my /home lives there.
So it's just separate zfs pool and I like to keep it that way. Just 2 mirrored disks for the system. Data and systems should always be separated.

I had no benefit using slog or l2arc ( I tried attaching nvme partitions to the big zfs pool), it all boils down to how you use the storage. My big pool is just for storing stuff, it's inactive 80% of the time.

If I had more spinning drives they'd be in raidzn for sure. But I only have 2 of them hence the mirror.

Also it does not make sense mixing different type of drives in the pool (aside from slog/l2arc) because zfs does not support tiering and you'll be limited by slowest drive in the pool.

I considered adding bcache layer under zfs, but in the end I went with less complex setup, because I value my data and don't want to loose it)
zfs just werks, I use it since t's inception on Solaris and later freeBSD and never lost a single bit of data, I'm so happy it works on linux now.
I did experiment with btrfs and all experiments ended in fatal data loss...

Before adding slog, do some research.
slog can kill ssd drive really quickly, because it just keeps writing to the same blocks over and over.
And you really want to have mirrored slogs. And since slog device is rather small thing (I bet you don't need anything over 8gb), it gets really stupid. basically you need 2x fast small ssd drives with high endurance.
slog always should be a separate drive (2x preferably), don't even think having slog as a partition on ssd drive that is used for something else.

As for l2arc, it's a hot cache, it needs to be warmed up before it starts working, and if you reboot - all the cache gets invalidated. It's very useful for filers constantly serving a lot of data, but I not for my use-case.

the double-nvme volume is kinda temporary fast storage, sometimes it's a mirrored or striped zpool with zvols, sometimes it's mdraid stripe, sometimes just several volumes, sometimes it's ccache. I destroy and re-create it every once in a while, depending on my current needs.
sometimes wine behaves funky on zfs, so I need to have ext4 fs nearby.

But lately I just put everything in ram/tmpfs, planning on replacing memory with 128G of ECC sticks.
Apparently certain Samsung 2400 ECC sticks can be overclocked to 3200, since it's the same B-die as in all the overclock-able memory around and ryzen looooves B-die.
https://www.youtube.com/watch?v=1NxSZil8KS8

I used to run the system from nvme pool but the difference is not very noticeable compared to SATA drives I have. I do not care about boot times. Post is long anyway, otherwise system boots in about 6 seconds on SATA, 4 seconds on NVME.

Comment reply on sysdmp's Completed Build: Reaper v2 (Gentoo Linux threadripper workstation)

  • 13 months ago
  • 2 points

thanks. idk, don't play or own GTA so can't answer.

Comment reply on sysdmp's Completed Build: Reaper v2 (Gentoo Linux threadripper workstation)

  • 13 months ago
  • 7 points

gentoo is nice, but can be brutal and gruesome if you are linux novice. Think of it like a more stable Arch with friendlier community, much more customizable and configurable. But you have to know what you want and how to achieve that.
Cons are :
1) little bit less packages in repos compared to arch+AUR (but main repo is still larger that arch main repo) and no single repo of user-submitted packages like AUR, but there are hundreds of "overlays" with extra packages (like multiple smaller AURs).
2) much longer package install time.
3) requires significant effort to learn and some effort to maintain( but maintenance is not that bad as soon as you are comfortable with it).

steam, wine is there and work just fine.
you can actually have all imaginable wine versions on single system with different patchsets (d3d9, staging, dxvk, vkd3d, vulkan etc) and can even combine patchets into single wine binary. and you can switch between wine versions and patchsets on the fly.

Comment reply on sysdmp's Completed Build: Reaper v2 (Gentoo Linux threadripper workstation)

  • 13 months ago
  • 2 points

both lamp and desk are IKEA.
desk is GERTON Tabletop. It is just a slab of raw wood, heavy as hell, kinda like big butchers block and you have to oil it to protect it and give some visual texture. I used Watco brand danish oil (mix of evaporating oil and varnish), 3 coats of it and it came out nice and even repels liquids. danish oil can come with some colors, so you can even make it dark, like cherry or walnut. it's really affordable tabletop of great quality if you are ready to spend some time oiling it properly. google the name, there are tons of DIY examples how to make this desk awesome. lamp is "hansakogg", but seems they no longer sell it.

Comment reply on sysdmp's Completed Build: Reaper (Gentoo threadripper workstation)

  • 17 months ago
  • 4 points

Bro I don't give a flying f about it being $100 gpu, it works and it's already an overkill for things I do on this machine right now.
it's not a gaming rig.

nvidia support on nix is wonky, the only scenario I'll get nvidia card is for tensorflow/pytorch CUDA support.

Comment reply on sysdmp's Completed Build: Reaper (Gentoo threadripper workstation)

  • 18 months ago
  • 1 point

I had similar thing once resetting CMOS or reflash may help:

Try resetting CMOS first, if it does not help:

Clear your CMOS again.
Reflash latest BIOS using the USB BIOS Flashback procedure (it's in the manual)
shutdown
Clear CMOS again
Once in the BIOS, load the optimized defaults

Comment reply on sysdmp's Completed Build: Reaper (Gentoo threadripper workstation)

  • 18 months ago
  • 1 point

just fine. I'd say effortlessly.

nvme support is in kernel, full EFI boot, no CSM/legacy mode. boots straight off nvme drives. tried refind, grub, systemd-boot,efistub - all work just fine if you configure properly. nothing special. gpt only.

nvme tool support smart status and temperature monitoring. latest smartmontools also support nvme, but you have to enable explicitly.

hddtemp does not yet support reading nvme temps but can be forced to read anyway.

Comment reply on sysdmp's Completed Build: Reaper (Gentoo threadripper workstation)

  • 18 months ago
  • 2 points

That cooler (esp with in push-pull) cools better than most generic AIOs out there. Even AIOs with threadripper specific plate size are on par with Notcua and cost 1.5x the money.

Only custom loop will beat it and will cost at least 2-3x more and will require maintenance. So why pay more to get less? Noctua does it's job perfectly with enough room for stable 4k overclock. I like it.

Comment reply on sysdmp's Completed Build: Reaper (Gentoo threadripper workstation)

  • 19 months ago
  • 3 points

The thing is a beast. Here are latest build times I get so far. I cleared build log some time ago.
The greatest achievement so far is full chromium build in less than 20 minutes.

Also I no longer overclock it. Not worth it.

Not that some packages are multilib, so they are actually being built twice, once as x86 and another as x64

user@reaper ~> qlop -gH gcc llvm clang libreoffice thunderbird webkit-gtk glibc icedtea
gcc: Mon Mar  5 23:31:26 2018: 30 minutes, 5 seconds
gcc: Tue Mar  6 03:34:52 2018: 28 minutes, 43 seconds
gcc: Tue Mar  6 05:00:16 2018: 27 minutes, 13 seconds
gcc: Tue Mar  6 06:28:36 2018: 27 minutes, 13 seconds
gcc: Mon Mar 12 01:37:53 2018: 28 minutes, 26 seconds
gcc: 5 times
llvm: Tue Mar  6 09:44:44 2018: 3 minutes, 30 seconds
llvm: Fri Mar  9 10:06:08 2018: 4 minutes, 25 seconds
llvm: 2 times
clang: Tue Mar  6 13:49:26 2018: 4 minutes, 43 seconds
clang: Fri Mar  9 10:10:33 2018: 5 minutes, 58 seconds
clang: Tue Mar 20 16:27:02 2018: 5 minutes, 18 seconds
clang: 3 times
libreoffice: Sun Mar 18 22:27:22 2018: 16 minutes, 26 seconds
libreoffice: Mon Mar 19 12:56:41 2018: 17 minutes, 14 seconds
libreoffice: 2 times
thunderbird: Tue Mar  6 21:41:34 2018: 6 minutes, 30 seconds
thunderbird: Fri Mar  9 10:16:40 2018: 6 minutes, 25 seconds
thunderbird: 2 times
webkit-gtk: Thu Mar  8 20:41:59 2018: 16 minutes, 37 seconds
webkit-gtk: Sun Mar 18 01:23:37 2018: 17 minutes, 44 seconds
webkit-gtk: 2 times
glibc: Mon Mar  5 22:34:25 2018: 3 minutes, 43 seconds
glibc: Tue Mar  6 04:35:41 2018: 3 minutes, 20 seconds
glibc: Tue Mar  6 06:04:55 2018: 3 minutes, 23 seconds
glibc: 3 times
icedtea: Tue Mar  6 22:11:57 2018: 7 minutes, 34 seconds
icedtea: Thu Mar  8 17:09:35 2018: 7 minutes, 48 seconds
icedtea: Thu Mar  8 23:06:33 2018: 11 minutes, 47 seconds
icedtea: 3 times

Comment reply on sysdmp's Completed Build: Reaper (Gentoo threadripper workstation)

  • 21 months ago
  • 1 point

I wanted to get this card originally https://pcpartpicker.com/product/cCfp99/msi-video-card-r64502gd3hlp

but decided to get something modern, instead of 7 year old card.

might get it anyway, because it's cheap, works with every linux and bsd around and can drive a screen or two at regular resolution.

my main concern was that it's unable to drive 4k+ screens, which I might buy in near future. I'm spoiled by apple's retina screens and really miss it on current screen.

and this build is not for games at all, the only game that might ever run here is quake3, which can be run on calculators now I believe.

Comment reply on sysdmp's Completed Build: Reaper (Gentoo threadripper workstation)

  • 21 months ago
  • 2 points

thanks!
I've also tried running FreeBSD-current with drm-next-kmod and it works just fine. I'm not going to use it as my os this time, but it was nice to see some progress from FreeBSD folks.

Comment reply on sysdmp's Completed Build: Reaper (Gentoo threadripper workstation)

  • 21 months ago
  • 1 point

yep. wil will try it.
for now I'm benchmarking, and just tinkering with the system.
4.15 looks really promising for this machine.
I'll get amdgpu, freesync, cpu temp, sensors and a lot more interesting stuff.

Sort

add arrow-down arrow-left arrow-right arrow-up authorcheckmark clipboard combo comment delete discord dots drag-handle dropdown-arrow errorfacebook history inbox instagram issuelink lock markup-bbcode markup-html markup-pcpp markup-cyclingbuilder markup-plain-text markup-reddit menu pin radio-button save search settings share star-empty star-full star-half switch successtag twitch twitter user warningwattage weight youtube