TrueNAS and Other Custom NAS Builds

[CaseySJ]

Here are the same disk performance numbers from the old TrueNAS SCALE system with five of the same 14TB Seagate Exos Enterprise drives in RAIDZ1 (name is Stratus):

# dd if=/dev/zero of=./test.dd bs=1024 count=256K
262144+0 records in
262144+0 records out
268435456 bytes (268 MB, 256 MiB) copied, 0.609739 s, 440 MB/s

# dd if=/dev/zero of=./test.dd bs=1024 count=512K
524288+0 records in
524288+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 1.2172 s, 441 MB/s

# dd if=/dev/zero of=./test.dd bs=1024 count=1M
1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.39099 s, 449 MB/s

In this system all hard drives are connected via SATA to SATA cables from backplane to motherboard SATA ports.

 

[CaseySJ]

Gallery of Build Images:

 

[CaseySJ]

** Parts List **

• Asus B760M-Plus WiFi motherboard (micro-ATX)
• Rack Choice 2U NAS chassis (supports Mini-ITX and Micro-ATX boards)
• Intel 13th Gen i5-13500 (6 P-cores and 8 E-cores for total of 14 cores and 20 threads with 65W TDP)
• Lian Li SP750 750W SFX power supply
• G.Skill Ripjaws 2x32GB 5600MT/s CL30 1.25V DDR5 memory (64GB total)
• Syba 8-Port SATA III PCIe x4 Expansion Card with two SFF-8087 connectors
• OIKWAN Mini SAS SFF-8087 to Right Angle SFF-8087 Cable (quantity 2)
• Noctua NH-L9i-17xx chromax.black low profile 37mm CPU cooler
• Gigabyte GC-AQC-113C 10GbE Network Card (not readily available) so try this one by NICGIGA

 

[CaseySJ]

** Geekworm X650 PiKVM Kit **

Every rack mounted server deserves IPMI (or remote KVM) capability. Instead of purchasing another PiKVM V4 Mini, I chose the Geekworm X650 Kit that requires us to bring our own Raspberry Pi Compute Module 4. The X650 has the virtue of fitting entirely inside a small-form-factor case. It comes with both standard-height and SFF mounting brackets.

Items Used:

• Geekworm X650 Kit -- about US$100
• Raspberry Pi Compute Module 4 -- about US$95
• REVOTECH Gigabit Type C PoE injector (5V, 3A)

The Compute Module 4 comes in several different configurations based on:

• Different amounts of on-board RAM
• Different amounts of on-board eMMC storage
• Without or without WiFi module

All that is needed is a Compute Module 4 with 2GB RAM and preferably 8GB eMMC. I typically avoid micro-SD cards. Due to constrained availability of several configurations, I chose a somewhat over-powered module with 8GB RAM, 32GB eMMC, and WiFi.

For $195 plus sales tax, this is about 30% cheaper than the PiKVM V4 Mini (US $275) and did I mention that it fits entirely within the chassis?

The X650 has been dry-fitted for testing purposes. Currently waiting for a couple of small parts to arrive before I tidy everything up, install the rack rails, and mount the chassis inside the rack.

And here it is...

 

[CaseySJ]

** Chassis Updates **

Some updates to the NAS chassis:

• Replaced Lian Li power supply with BeQuiet 600W SFX L PSU
  • Reasons:
    • Lian Li PSU does not come with a SFX to ATX mounting bracket that is needed for the 2U RackChoice NAS chassis
    • Lian Li PSU molex cable is too short to reach the backplane
  • BeQuiet 600W SFX L attributes:
    • Comes with SFX to ATX bracket
    • Molex cable easily reaches backplane
    • But it comes with only one 8-pin CPU power connector, which is okay because of the low TDP of i5-13600 and no over-clocking
    • Exhaust fan is on the opposite site, which requires flipping the unit when installing (see photos)
• Installed Geekworm X650 KVM into chassis
  • Fits nicely inside the 2U chassis
  • Cooling fan may not be necessary because there's plenty of air flowing in its direction from the 4 on-board chassis fans
    • Removing the fan will allow adjacent PCIe x1 slot to be used
    • This means all 3 slots on motherboard can be populated along with the X650
• Reinitialized my QNAP TS-h973AX, which means all data on all drives has been deleted and NAS is ready for a new set of drives
  • All five Seagate Exos 14TB drives have been moved to the new NAS
  • Both of the 1TB SATA SSDs have been moved to the new NAS

The new NAS has these pools:

• RAIDZ2 with 8 hard drives, each 14TB, for a usable capacity of 72.29 TiB
  • This is the main data warehouse
• MIRROR with 2 SATA SSDs, each 1TB, for a usable capacity of 899.25 GiB
  • This is for Apps as recommended by Truecharts

Currently in the process of transferring all 37 TiB of data from old TrueNAS to new TrueNAS using Data Protection -> Replication Tasks. The transfer is being done dataset by dataset because each dataset is a disk "volume" and needs to be handled separately.

Transfer is progressing surprisingly fast over a 10GbE network. It will probably take a whole day if run continuously, but I'll stagger the transfer over a couple of days.

 

[CaseySJ]

Status Update:

• Migration from old TrueNAS (Stratus) to new TrueNAS (Nimbus) was completed a couple of days ago
• 36.5 TiB of data was transferred over 10GbE network, which took less time than I had anticipated
• Data transfer was done using multiple Replication tasks, one for each dataset
• Apps have also been installed and configured on the new system
  • Collabora | NextCloud | Homepage | Jellyfin | DrawIO | FileZilla
• When setting up Jellyfin (on TrueNAS) and Infuse (on macOS), I ran into a problem with ACL permissions (access control list) for media datasets (Movies, Music) that are shared via NFS
  • Problem was fixed by setting dataset property ACL Type to SMB/NFSv4 and subsequently using ACL preset NFS4_OPEN

• We're already at 50% used, which is just fine since most of the space is consumed by Blu-ray movies
• I don't expect the remaining 50% to fill up any time soon

• Next step: Install chassis into rack using supplied rails
• Idle power consumption: 85W
• Four preinstalled chassis fans are exceptionally quiet; they're PWM fans connected to motherboard headers

 

[Leesureone]

Status Update:

• Migration from old TrueNAS (Stratus) to new TrueNAS (Nimbus) was completed a couple of days ago
• 36.5 TiB of data was transferred over 10GbE network, which took less time than I had anticipated
• Data transfer was done using multiple Replication tasks, one for each dataset
• Apps have also been installed and configured on the new system
  • Collabora | NextCloud | Homepage | Jellyfin | DrawIO | FileZilla
• When setting up Jellyfin (on TrueNAS) and Infuse (on macOS), I ran into a problem with ACL permissions (access control list) for media datasets (Movies, Music) that are shared via NFS
  • Problem was fixed by setting dataset property ACL Type to SMB/NFSv4 and subsequently using ACL preset NFS4_OPEN

View attachment 578391
View attachment 578392

• We're already at 50% used, which is just fine since most of the space is consumed by Blu-ray movies
• I don't expect the remaining 50% to fill up any time soon

View attachment 578390

• Next step: Install chassis into rack using supplied rails
• Idle power consumption: 85W

And then grant Leesureone access…just kidding of course. One heck of a setup there, nicely done as usual.

 

[CaseySJ]

Finally the rack mount chassis has been installed into the AV rack, which is appropriate because this NAS also serves music and movies throughout the house with both Jellyfin and Infuse.

Because there is no monitor, keyboard or mouse directly connected, the GeekWorm X650 PiKVM is essential.