가상화/클라우드 컴퓨팅을 위한 기반 기술 중, 로컬 디스크에 대한 사용성/성능 분석을 위한 시험 자료이다. 로컬 파일시스템 또는 In-House 스토리지 서버의 하부에 위치하게 될 로컬 디스크 구성에 있어서 성능과 신뢰성 측면에서 Software-RAID와 LVM의 성능을 비교한다.
Result of Test
In Details
Target Configuration
Configuration of LVM
Test
Simple performance comparison with dd command
Write performance
write on shared memory : 1.2GB/s, 100% of CPU utilization
@edward:/tmp$ sudo time dd if=/dev/zero of=/dev/shm/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 0.851281 초, 1.2 GB/초
0.00user 0.99system 0:00.98elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+539minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/dev/shm/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 0.881937 초, 1.2 GB/초
0.00user 1.01system 0:01.02elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+536minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/dev/shm/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 0.830411 초, 1.3 GB/초
0.01user 0.96system 0:00.96elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+536minor)pagefaults 0swaps
vios@edward:/tmp$
write on software RAID, 177MB/s, 29% of CPU utilization
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-raid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.98764 초, 175 MB/초
0.01user 1.71system 0:06.00elapsed 28%CPU (0avgtext+0avgdata 0maxresident)k
24inputs+2367488outputs (0major+536minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-raid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.99421 초, 175 MB/초
0.00user 1.90system 0:06.22elapsed 30%CPU (0avgtext+0avgdata 0maxresident)k
8inputs+2269184outputs (0major+537minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-raid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.8002 초, 181 MB/초
0.00user 1.77system 0:06.03elapsed 29%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2117816outputs (0major+538minor)pagefaults 0swaps
vios@edward:/tmp$
write on mirrored LVM, 156MB/s, 26% of CPU utilization
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-mirror/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.74504 초, 155 MB/초
0.00user 1.67system 0:06.76elapsed 24%CPU (0avgtext+0avgdata 0maxresident)k
40inputs+2322432outputs (0major+537minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-mirror/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.61715 초, 158 MB/초
0.00user 1.91system 0:06.85elapsed 27%CPU (0avgtext+0avgdata 0maxresident)k
8inputs+2235824outputs (0major+536minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-mirror/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.73312 초, 156 MB/초
0.00user 1.84system 0:06.97elapsed 26%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2048000outputs (0major+538minor)pagefaults 0swaps
write on striped LVM, 300MB/s 48% of CPU utilization
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-stripe/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 3.46133 초, 303 MB/초
0.00user 1.83system 0:03.58elapsed 51%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2269184outputs (0major+537minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-stripe/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 3.50917 초, 299 MB/초
0.00user 1.90system 0:03.75elapsed 50%CPU (0avgtext+0avgdata 0maxresident)k
32inputs+2244608outputs (1major+537minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/v-stripe/1G bs=1M count=10001000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 3.514 초, 298 MB/초
0.00user 1.66system 0:03.75elapsed 44%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2056512outputs (0major+538minor)pagefaults 0swaps
vios@edward:/tmp$
Read performance
Read from software RAID : 86.6 MB/s, 6% of CPU
Read from mirrored LVM : 76.4 MB/s, 13% of CPU
Read from striped LVM : 152 MB/s, 14% of CPU
vios@edward:/tmp$ for f in raid mirror stripe; do echo $f; echo 3 |sudo tee /proc/sys/vm/drop_caches; sudo time dd of=/dev/null if=/tmp/v-$f/1G bs=1M; done
raid
3
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 12.1129 초, 86.6 MB/초
0.00user 0.77system 0:12.15elapsed 6%CPU (0avgtext+0avgdata 0maxresident)k
2048552inputs+0outputs (6major+531minor)pagefaults 0swaps
mirror
3
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 13.7244 초, 76.4 MB/초
0.00user 1.90system 0:13.76elapsed 13%CPU (0avgtext+0avgdata 0maxresident)k
2048456inputs+0outputs (6major+530minor)pagefaults 0swaps
stripe
3
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.87971 초, 152 MB/초
0.01user 0.98system 0:06.92elapsed 14%CPU (0avgtext+0avgdata 0maxresident)k
2048504inputs+0outputs (6major+531minor)pagefaults 0swaps
vios@edward:/tmp$
Case of LVM on RAID
vios@edward:/tmp$ sudo pvs
PV VG Fmt Attr PSize PFree
/dev/md0 vg1 lvm2 a- 325.96G 315.96G
/dev/sda4 vg0 lvm2 a- 354.04G 339.04G
/dev/sdb4 vg0 lvm2 a- 354.04G 339.04G
vios@edward:/tmp$ sudo vgs
VG #PV #LV #SN Attr VSize VFree
vg0 2 2 0 wz--n- 708.08G 678.07G
vg1 1 1 0 wz--n- 325.96G 315.96G
vios@edward:/tmp$ sudo lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
mirror vg0 mwa-a- 10.00G mirror_mlog 100.00
stripe vg0 -wi-a- 10.00G
onraid vg1 -wi-ao 10.00G
vios@edward:/tmp$
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/onraid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 6.0768 초, 173 MB/초
0.00user 1.92system 0:06.96elapsed 27%CPU (0avgtext+0avgdata 0maxresident)k
128inputs+2379776outputs (2major+536minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/onraid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.95193 초, 176 MB/초
0.00user 1.95system 0:06.18elapsed 31%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2244608outputs (0major+538minor)pagefaults 0swaps
vios@edward:/tmp$ sudo time dd if=/dev/zero of=/tmp/onraid/1G bs=1M count=1000
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 5.81606 초, 180 MB/초
0.00user 1.67system 0:06.05elapsed 27%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+2048000outputs (0major+537minor)pagefaults 0swaps
vios@edward:/tmp$
vios@edward:/tmp$ for f in onraid; do echo $f; echo 3 |sudo tee /proc/sys/vm/drop_caches; sudo time dd of=/dev/null if=/tmp/$f/1G bs=1M; done
onraid
3
1000+0 레코드 들어옴
1000+0 레코드 나감
1048576000 바이트 (1.0 GB) 복사됨, 12.2329 초, 85.7 MB/초
0.00user 0.82system 0:12.26elapsed 6%CPU (0avgtext+0avgdata 0maxresident)k
2048552inputs+0outputs (6major+531minor)pagefaults 0swaps
vios@edward:/tmp$
Eh? is it sector location performance? we need more test with same disk, same partition and sector location.