Software RAID And LVM - Perfomance Comparison

MEMO.T.C » Virtualization » Benchmarks For Vt » Software RAID And LVM - Perfomance Comparison

작성자: sio4 (최종수정: 21 Mar 2014 08:55)

가상화/클라우드 컴퓨팅을 위한 기반 기술 중, 로컬 디스크에 대한 사용성/성능 분석을 위한 시험 자료이다. 로컬 파일시스템 또는 In-House 스토리지 서버의 하부에 위치하게 될 로컬 디스크 구성에 있어서 성능과 신뢰성 측면에서 Software-RAID와 LVM의 성능을 비교한다.

Result of Test

In Details

Target Configuration

Configuration of LVM

+ show block

– hide block

vios@edward:~$ sudo pvs
  PV         VG   Fmt  Attr PSize   PFree  
  /dev/sda4  vg0  lvm2 a-   354.04G 354.04G
  /dev/sdb4  vg0  lvm2 a-   354.04G 354.04G

vios@edward:~$ sudo vgs
  VG   #PV #LV #SN Attr   VSize   VFree  
  vg0    2   0   0 wz--n- 708.08G 708.08G

vios@edward:~$ sudo lvcreate -L 10G -i2 -n stripe vg0
  Using default stripesize 64.00 KB
  Logical volume "stripe" created

vios@edward:~$ sudo lvcreate -L 10G -m1 -n mirror --alloc anywhere vg0
  Logical volume "mirror" created

vios@edward:~$ sudo lvs
  LV     VG   Attr   LSize  Origin Snap%  Move Log         Copy%  Convert
  mirror vg0  mwa-a- 10.00G                    mirror_mlog  35.59        
  stripe vg0  -wi-a- 10.00G                                              

vios@edward:~$ sudo pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda4
  VG Name               vg0
  PV Size               354.04 GB / not usable 1.16 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              90634
  Free PE               86793
  Allocated PE          3841
  PV UUID               Vxm9Qp-IdaL-0GlO-Ifvq-MIJA-TU1k-VfEcHO

  --- Physical volume ---
  PV Name               /dev/sdb4
  VG Name               vg0
  PV Size               354.04 GB / not usable 1.16 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              90634
  Free PE               86794
  Allocated PE          3840
  PV UUID               sEzTHt-2dVK-FAuG-kiIm-866A-OPxT-vAaNp5

vios@edward:~$ sudo vgdisplay
  --- Volume group ---
  VG Name               vg0
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  9
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               0
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               708.08 GB
  PE Size               4.00 MB
  Total PE              181268
  Alloc PE / Size       7681 / 30.00 GB
  Free  PE / Size       173587 / 678.07 GB
  VG UUID               IUpoME-ZrT7-KwAr-S0XF-Ldod-kliH-KcMeEq

vios@edward:~$ sudo lvdisplay
  --- Logical volume ---
  LV Name                /dev/vg0/stripe
  VG Name                vg0
  LV UUID                NZi0Cw-of5e-umP1-p97Q-FYRH-x1D4-R8gIcj
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                10.00 GB
  Current LE             2560
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     512
  Block device           252:0

  --- Logical volume ---
  LV Name                /dev/vg0/mirror
  VG Name                vg0
  LV UUID                9zT8Lc-kQ4y-ezos-yaqT-ZIMl-Wneo-nO92pN
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                10.00 GB
  Current LE             2560
  Segments               1
  Allocation             anywhere
  Read ahead sectors     auto
  - currently set to     256
  Block device           252:4

Test

Simple performance comparison with dd command

Write performance

write on shared memory : 1.2GB/s, 100% of CPU utilization

@edward:/tmp$ sudo time dd if=/dev/zero of=/dev/shm/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 0.851281 초, 1.2 GB/초
0.00user 0.99system 0:00.98elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+539minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/dev/shm/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 0.881937 초, 1.2 GB/초
0.00user 1.01system 0:01.02elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+536minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/dev/shm/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 0.830411 초, 1.3 GB/초
0.01user 0.96system 0:00.96elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+536minor)pagefaults 0swaps
vios@edward:/tmp$

write on software RAID, 177MB/s, 29% of CPU utilization

vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-raid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.98764 초, 175 MB/초
0.01user 1.71system 0:06.00elapsed 28%CPU (0avgtext+0avgdata 0maxresident)k
24inputs+2367488outputs (0major+536minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-raid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.99421 초, 175 MB/초
0.00user 1.90system 0:06.22elapsed 30%CPU (0avgtext+0avgdata 0maxresident)k
8inputs+2269184outputs (0major+537minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-raid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.8002 초, 181 MB/초
0.00user 1.77system 0:06.03elapsed 29%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2117816outputs (0major+538minor)pagefaults 0swaps
vios@edward:/tmp$

write on mirrored LVM, 156MB/s, 26% of CPU utilization

vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-mirror/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.74504 초, 155 MB/초
0.00user 1.67system 0:06.76elapsed 24%CPU (0avgtext+0avgdata 0maxresident)k
40inputs+2322432outputs (0major+537minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-mirror/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.61715 초, 158 MB/초
0.00user 1.91system 0:06.85elapsed 27%CPU (0avgtext+0avgdata 0maxresident)k
8inputs+2235824outputs (0major+536minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-mirror/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.73312 초, 156 MB/초
0.00user 1.84system 0:06.97elapsed 26%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2048000outputs (0major+538minor)pagefaults 0swaps

write on striped LVM, 300MB/s 48% of CPU utilization

vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-stripe/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 3.46133 초, 303 MB/초
0.00user 1.83system 0:03.58elapsed 51%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2269184outputs (0major+537minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-stripe/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 3.50917 초, 299 MB/초
0.00user 1.90system 0:03.75elapsed 50%CPU (0avgtext+0avgdata 0maxresident)k
32inputs+2244608outputs (1major+537minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-stripe/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 3.514 초, 298 MB/초
0.00user 1.66system 0:03.75elapsed 44%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2056512outputs (0major+538minor)pagefaults 0swaps
vios@edward:/tmp$

Read performance

Read from software RAID : 86.6 MB/s, 6% of CPU
Read from mirrored LVM : 76.4 MB/s, 13% of CPU
Read from striped LVM : 152 MB/s, 14% of CPU

vios@edward:/tmp$ for f in raid mirror stripe; do echo $f; echo 3 |sudo tee /proc/sys/vm/drop_caches; sudo time dd of=/dev/null if=/tmp/v-$f/1G bs=1M; done
raid
3
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 12.1129 초, 86.6 MB/초
0.00user 0.77system 0:12.15elapsed 6%CPU (0avgtext+0avgdata 0maxresident)k
2048552inputs+0outputs (6major+531minor)pagefaults 0swaps
mirror
3
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 13.7244 초, 76.4 MB/초
0.00user 1.90system 0:13.76elapsed 13%CPU (0avgtext+0avgdata 0maxresident)k
2048456inputs+0outputs (6major+530minor)pagefaults 0swaps
stripe
3
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.87971 초, 152 MB/초
0.01user 0.98system 0:06.92elapsed 14%CPU (0avgtext+0avgdata 0maxresident)k
2048504inputs+0outputs (6major+531minor)pagefaults 0swaps
vios@edward:/tmp$

Case of LVM on RAID

vios@edward:/tmp$ sudo pvs
  PV         VG   Fmt  Attr PSize   PFree  
  /dev/md0   vg1  lvm2 a-   325.96G 315.96G
  /dev/sda4  vg0  lvm2 a-   354.04G 339.04G
  /dev/sdb4  vg0  lvm2 a-   354.04G 339.04G
vios@edward:/tmp$ sudo vgs
  VG   #PV #LV #SN Attr   VSize   VFree  
  vg0    2   2   0 wz--n- 708.08G 678.07G
  vg1    1   1   0 wz--n- 325.96G 315.96G
vios@edward:/tmp$ sudo lvs
  LV     VG   Attr   LSize  Origin Snap%  Move Log         Copy%  Convert
  mirror vg0  mwa-a- 10.00G                    mirror_mlog 100.00        
  stripe vg0  -wi-a- 10.00G                                              
  onraid vg1  -wi-ao 10.00G                                              
vios@edward:/tmp$

vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/onraid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.0768 초, 173 MB/초
0.00user 1.92system 0:06.96elapsed 27%CPU (0avgtext+0avgdata 0maxresident)k
128inputs+2379776outputs (2major+536minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/onraid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.95193 초, 176 MB/초
0.00user 1.95system 0:06.18elapsed 31%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2244608outputs (0major+538minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/onraid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.81606 초, 180 MB/초
0.00user 1.67system 0:06.05elapsed 27%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2048000outputs (0major+537minor)pagefaults 0swaps
vios@edward:/tmp$

vios@edward:/tmp$ for f in onraid; do echo $f; echo 3 |sudo tee /proc/sys/vm/drop_caches; sudo time dd of=/dev/null if=/tmp/$f/1G bs=1M; done
onraid
3
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 12.2329 초, 85.7 MB/초
0.00user 0.82system 0:12.26elapsed 6%CPU (0avgtext+0avgdata 0maxresident)k
2048552inputs+0outputs (6major+531minor)pagefaults 0swaps
vios@edward:/tmp$

Eh? is it sector location performance? we need more test with same disk, same partition and sector location.

benchmark lvm performance softtware-raid storage tech-base

SiO4

a.k.a scinix, 용환의 보조기억장치

About…

Documents

Projects

Recent Blog Posts

Social

Tags