Migrating MHDDFS Pool from NTFS to EX4 and New OMV Host?

  • Hi, while I have been using OpenMediaVault for the last 6 months and have always been on the fence thus my usage of NTFS, it is only recently that I have decided to go more fully into it (helps a lot when I have another backup too on another computer!). So, I understand (and have seen via some quick tests) that using EX4 HDDs are way better and faster and I will like to convert my NTFS HDDs inside my MHDDFS pool into EX4 HDDs. After this, I may move the HDDs into a new machine too due to better airflow and consolidation e.t.c.


    My MHDDFS pool has the following HDDs:

    • 1x HGST HTS721010A9E630 (1TB) (2.5 inch)
    • 1x Hitachi HDS723030ALA640 (3TB) (3.5 inch)
    • 1x SAMSUNG HD103SJ (1TB) (3.5 inch)

    I have two empty Seagate Barracuda 7200.10 500GB HDDs (3.5 inch) and also a Toshiba MK3263GSX 320GB HDD (2.5 inch) that can be deployed to help in the migration. (I am even considering leaving the Barracudas in as part of my pool and taking out the HGST for my laptop instead. The 320GB will be eventually used for a different purpose)


    How should I best proceed? (I am trying not to use the backup as much as possible even though the data is more or less the same)


    Thanks!

    • Offizieller Beitrag

    Change pool to mergerfs (much better than mhddfs).
    Remove the drive from the pool.
    Convert it to ext4
    Add back to pool.
    Repeat for each drive.

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Hmm, the only issue then is whether the conversion to ex4 will destroy the data inside or not; trying to avoid that so as to minimise the amount of writes needed to rewrite the data e.t.c...

    • Offizieller Beitrag

    You can't convert directly from ntfs to ext4. You move the data off the drive, format as ext4, and put the data back on the drive. Moving to another system doesn't cause problems.

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • You can't convert directly from ntfs to ext4. You move the data off the drive, format as ext4, and put the data back on the drive. Moving to another system doesn't cause problems.


    Ah I see, I was under the impression that when adding new HDDs to the pool and then removing one of the HDDs containing the data, the data of the removed HDD will be automatically shifted to the new HDDs via mergefs (what's the proper software that can do this?)


    So, if I want to shift my pool of HDDs into a new machine/host, all I have to do is to move them (i.e. unplug from old machine and then connect them to the new one) and then set up the pool again in the new machine like how I first set up the pool?

    • Offizieller Beitrag

    mergerfs just pools the drives. It isn't a filesystem and doesn't manage filesystem changes or drive changes. Raid does what you are thinking.


    To change to a new filesystem, I would put a new drive in, rsync files from old drive to new drive, remove old drive from pool, add new drive to pool, reboot.


    Yes, that is all you have to do to move the drives.

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • mergerfs just pools the drives. It isn't a filesystem and doesn't manage filesystem changes or drive changes. Raid does what you are thinking.


    To change to a new filesystem, I would put a new drive in, rsync files from old drive to new drive, remove old drive from pool, add new drive to pool, reboot.


    Yes, that is all you have to do to move the drives.


    Thanks, almost done checking the data on this pool and my backup HDDs, will update here as I go through these steps.

  • Alright, I am done converting the HDDs to EX4; the mergefs pool is now a pure EX4 pool.


    I am seeing some speed improvements (e.g. initlal 90MB/s but then dropdown to 60MB/s instead of constant 40MB/s to 50MB/s). I am not too sure what to make out of this yet though; I may just want to shift this pool into a new machine first before pondering on the tweaking. Will this means that I should just mount the HDDs in the new machine and then create the pool again?

    • Offizieller Beitrag

    Yep, you can just move them and make a new pool. There is nothing on the drives that make it part of a mergerfs pool.


    As for performance, I would try using the direct_io flag. Put defaults,direct_io,allow_other in the options box to use.

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Thanks, I will try that in a while. Do I need to tweak SAMBA too (and will its effects work on Mac/Linux computers that read or write data from the OMV host?)?


    EDIT: Hmm, I can't seem to find a text box for me to use these flags in the unionfilesystem plugin. Does these need me to modify the fstab file by hand?

    • Offizieller Beitrag

    I don't have any extra options set for my samba servers.


    Ah, you are on OMV 2.x. OMV 3.x has the options box. You need to change it in /etc/openmediavault/config.xml then omv-mkconf fstab and reboot/remount

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

    • Offizieller Beitrag

    Maybe a little with more cpu. I don't think ram will make a difference.

    omv 7.1.0-2 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.2 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.5 | scripts 7.0.7


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Sorry for the delay, the shift took longer than usual. I have managed to shift the pool over to a new machine, but I am getting 40MB/s which is quite shockingly low.


    I think this is so as the OMV here is run as a VM instead of baremetal though.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!