Wednesday, December 29, 2010

Speed up MDADM RAID5 array using stripe_cache_size

edit /etc/rc.local
        add:
                echo 4096 > /sys/block/md0/md/stripe_cache_size

To make effect take place immediately:
echo 4096 > /sys/block/md0/md/stripe_cache_size

4096 was chosen after fairly extensive bonnie testing of various sizes from 256 to 8192 with and without NCQ enabled on the drives. Using this setting increased my write speeds by about 50%.

References:
NeilB's post: "You can possibly increase the speed somewhat by increasing the buffer space that is used, thus allowing larger reads followed by larger writes. This is done by increasing /sys/block/mdXX/md/stripe_cache_size"

Another one of NeilB's posts regarding this topic:
"Changing the stripe_cache_size will not risk causing corruption.
If you set it too low the reshape will stop progressing. You can then set it to a larger value and let it continue.
If you set it too high you risk tying up all of your system memory in the cache. In this case you system might enter a swap-storm and it might be rather hard to set it back to a lower value.
The amount of memory used per cache entry is about 4K times the number of device in the array."

NeilB is Neil Brown, the author of MDADM.

4 comments:

  1. Great blog! Very useful snippets and I very much like the way you are short and concise with your titles-- it makes finding information very easy, especially for fellow Linux server admins.

    This post was extremely useful to me. Using a stripe_cache_size of 8192, I was able to speed my reshape from 20+ hours to 8+. Thanks a lot and please keep on posting the stuff you encounter! :-)

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Hello,

    i run

    echo 4096 > /sys/block/md0/md/stripe_cache_size

    but it shows

    -bash: /sys/block/md0/md/stripe_cache_size: No such file or directory

    how can i correct it ?


    thank you

    ReplyDelete
    Replies
    1. You need to make sure the "md0" is your device. Change to "md0" in /sys/block/md0 to be your RAID Device.

      Delete