New to OMV, some questions

  • Hi,


    I'm planning on building a NAS that can hold around 150TB or so, incrementally. This NAS will be mainly used to store my media files and used to stream to at least 5 devices simultaneous [1080p Content, at least 6Mb bitrate] Via Emby-Kodi. Parity/Redundancy is an important factor.


    Initial choice was to go with Freenas. Upon asking around, I found that SnapRaid suited my needs:

    • It allowed incremental addition of hard disks.
    • It has parity support [up to 6 disks].
    • I can lose all the parity disks + any failed disk and the remaining data on healthy disk still can be accessed. [Is this true? Some says yes, some says no.]
    • Cheaper compared to Freenas + easier to do overtime.

    I have several questions regarding SnapRaid in OMV.

    • The hardware requirement. The wiki states that Any i486 or amd64 + 1Gib or more RAM. What would be the best/optimum hardware for 4 simultaneous stream in Emby. I don't think trans-coding is involved. [Currently using Emby on my PC to Kodi on Raspberry Pi 2].
    • In the event of hard disk failure, I just need to remove the hard disk > insert new one and it will start it's resilvering process?
    • Is it better to build NAS with 3TB, 4TB or 6TB hard disks; factoring in the time to run Sync, Scrub and hard disk replacement.
    • As for extra backup, I can create another server and have this server backup to my new server right? As an onsite backup.
    • Do I need ECC RAM? Does SnapRaid automatically does error checksum if ECC-compatible hardware are used?
    • Since I'm using SnapRaid, I don't need to dabble with the Storage Option on the Sidebar of OMV, right? Any addition/removal of hard disk is through SnapRaid plugin, am I correct?
    • Drive Pooling. Is it a necessary thing? From my understanding, it just groups the drives to seem like one big drive, right? Do I lose anything by not drive pooling? If I need to drive pool, do I do that before SnapRaid or after?
    • Regarding Sync and Scrub. Just for clarification. Once I transfer data into drives, I run Sync and that creates the parity bits and the contents, correct? The Scrub functions checks for any error and fixes them, correct? Is the Scrub affected by whether ECC is used or not?

    Sorry for the long post. Any and every help/feedback is much appreciated.


    Thank you.

    • Offizieller Beitrag

    1. 4 streams without transcoding requires very little hardware. Any hardware you get to connect 150 TB can handle 4 streams.


    2. Yes. In snapraid, you just need to remove the old drive, name the new drive the same as the removed drive, then follow the manual for recovering.


    3. Good question. We don't have a lot of people with that much storage. So, you will have to tell us :) I would 6tb drives would be easier to implement.


    4. Yep. That is what I do. I have two OMV boxes. One runs the rsync job on a schedule and the other runs the rsyncd server. Works great.


    5. Most of my servers don't have ECC. With media and snapraid, I don't think you need it. One bad bit in a media file wouldn't be noticeable.


    6. Yes, you have to use it. When you install new drives including initial build, you need to create a filesystem (ext4 or xfs is fine) and mount it before you can select it in the snapraid plugin. Once that is done, you don't need to use it until you add or remove a drive.


    7. I wouldn't use Snapraid's pooling. It is read only as well. I would install the unionfilesystems plugin and add all data/content (non-parity) drives to a mergerfs pool. Then you share the pool in all plugins.


    8. Sync then scrub. Yep. I would add the snapraid-diff script to a scheduled task (on reboot). Then it will sync and scrub for you automatically. See earlier comment about ECC.


    While I don't see ECC as being necessary, in order to attach that much storage, you are going to probably end up with a motherboard/cpu that uses ECC.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    Einmal editiert, zuletzt von ryecoaaron ()

    • Offizieller Beitrag


    There's more to it than that. See:


    http://www.snapraid.it/manual Section 4.4 Recovering


    I guess you are correct. The additional steps have to be done from the command line as well.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    Einmal editiert, zuletzt von ryecoaaron ()

  • I may be mistaken, but I think that running a sync before running the fix procedure could wipe out the parity information for the failed disk making it impossible to recover that failed disk. Hopefully someone will correct me if I am wrong.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    • Offizieller Beitrag

    Maybe you are right. I don't remember. It doesn't help that I don't use snapraid. Following the manual is probably the best idea. Just forget what I mentioned before. I changed my post to say read the manual.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Zitat

    150 TB?
    Really?
    That would lead to something like that: aberdeeninc.com/abcatg/8U-Nehalem-Linux-NAS.htm


    Just setting an end point of 150TB. Most likely I wont or just reach half of that :P



    3. Still searching info on this, will post here if I find out either from other people or by testing :)


    4. Do you do any offsite/cloud backup as well? Can share your method/s?


    5. So, my data would still be okay without using any ECC?


    7. Do I need drive pooling? Do I lose anything by not pooling? If let's say, instead of OMV, i were to install in Windows 7; my drives would appear as A-Z but if I use drive pool, it only appears as A?



    Will consult the manual for more details on this :)


    Thank you.

  • Pooling with mergerfs is just a software layer over the physical file system that creates another filesystem in addition to the others. It doesn't touch the actual files on the drives, and you can pull them out and read/write each individual disk.

    • Offizieller Beitrag

    4. I rsync between servers and backup to LTO tape (which I haven't taken offsite yet). I would like to have a plugin for a cheap cloud service (like backblaze b2) but I haven't found one with an API I like yet.


    5. Yep, your data should be ok without ECC.


    7. Pooling just allows you to not worry about which drive you are putting the data on.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!