[mdlug] ESX3 IO Problems

Morris, Tim tmorris at ugs.com
Tue Aug 7 14:52:53 EDT 2007 

The problems I'm seeing are on the service console. I don't have any VMs
created. I know the console is limited, but to 5MB/s? ESX 2.5.2
performed much better in console I/O than that (on the same hardware).
The problem is that I have 400GB of data in VMs to move to this system,
so 5MB/s isn't going to cut it... I might have to revert to
OpenSuse/Vmware Server if that is the case. Vmkfstools doesn't do much
better. Is this Vmware's way of forcing you to buy VirtualCenter?


--  Tim Morris - UGS PLM Software

-----Original Message-----
From: mdlug-bounces at mdlug.org [mailto:mdlug-bounces at mdlug.org] On Behalf
Of Dan Pritts
Sent: Tuesday, August 07, 2007 1:13 PM
To: MDLUG's Main discussion list
Subject: Re: [mdlug] ESX3 IO Problems

I'm hardly a guru but you haven't said where you are doing the I/O,
within the service console, or within guests, or what.

I know that the service console is just not optimized to do disk I/O and
you can expect it to be slow when using standard unix commands.

I assume you've checked on the load using the vmware management console.
It can show you what each guest is doing i/o wise.

beyond that i don't know that i can offer much advice, but there seem to
be a lot of knowledgeable people on the forums at vmware.com.

On Fri, Aug 03, 2007 at 10:18:39AM -0500, Morris, Tim wrote:
> Are there any ESX gurus out there?
>  
> I have a bad I/O problem with our ESX 3.0.2 build 52542 server that I 
> thought I would ask about. The same problem occurs when using
> 3.0.1-32029 (unpatched).
>  
> Host Computer: 
> Dell Poweredge 2850 BIOS A06
> 2x 3.0 GHz Xeon Procs (HT disabled)
> 4GB RAM
> Dual Intel 8254NXX Gigabit Ethernet onboard PERC 4E/Di 2x 146GB SCSI3 
> (Maxtor) drives internal (RAID 1)
> 1 LD = ~139GB
> PERC 4E/Dc
> Powervault 220VS w/ 10x 146GB SCSI3 (Maxtor) drives (RAID 5 w/ 2
> hotspares)
> 1LD = 979MB
> All components have been updated to the latest availble firmware from 
> Dell (even all 12 drives!)
>  
> My problem is any type of file operation seems to hang every 50-200MB.
> cp, ftp, scp, everything.
> You can see it with an ftp hash especially. It hangs for a second, 
> then takes off again.
>  
> I have put together some transfer stats for you to look at (attached).
>  
> Any suggestions on where to go from here? 
> Any Performance testing/diagnostic utils/tips?
> Any more info needed?
> Anyone run into this before? 
>  
> I could really use some help on this...  Any suggestions would be 
> helpful.
>  
> --------------------------------------------------
> Tim Morris
> Systems Engineer
> Siemens Automation and Drives
> UGS PLM Software
>  

Content-Description: perf_test.txt
>
************************************************************************
***************
> **                               ESX3SRVR
**
> **********************************************************************
> ***************** dd if=/dev/zero bs=1000 count=100000 of=small_file 
> dd if=/dev/zero bs=1000 count=500000 of=medium_file dd if=/dev/zero 
> bs=1000 count=1000000 of=large_file # ls -l
> -rw-r--r--    1 root     root     1000000000 Aug  3 02:13 large_file *
(953.67MB)
> -rw-r--r--    1 root     root     500000000 Aug  3 02:10 medium_file *
(476.84MB)
> -rw-r--r--    1 root     root     100000000 Aug  3 02:09 small_file  *
(95.37MB)
> 
> CP RESULTS (performed in random order):
> (date && cp -f small_file small_file.00 && date) Test 1: Start: 
> 02:43:41 End: 02:43:50 * (10.60MB/s) Test 2: Start: 02:44:07 End: 
> 02:44:12 * (19.07MB/s) Test 3: Start: 03:09:47 End: 03:10:08 * 
> (4.54MB/s)
> 
> (date && cp -f medium_file medium_file.00 && date) Test 1: Start: 
> 02:44:30 End: 02:46:24 * (4.18MB/s) Test 2: Start: 02:52:56 End: 
> 02:54:49 * (4.22MB/s) Test 3: Start: 03:19:10 End: 03:21:04 * 
> (4.18MB/s)
> 
> (date && cp -f large_file large_file.00 && date) Test 1: Start: 
> 02:47:20 End: 02:51:09 * (4.16MB/s) Test 2: Start: 03:02:17 End: 
> 03:06:12 * (4.06MB/s) Test 3: Start: 03:10:52 End: 03:14:40 * 
> (4.18MB/s)
> 
> FTP (localhost) Results (performed in random order):
> large_file
> 1000000000 bytes received in 222 secs (4.4e+03 Kbytes/sec) * 
> (4.30MB/s) 1000000000 bytes received in 220 secs (4.4e+03 Kbytes/sec) 
> * (4.34MB/s) 1000000000 bytes received in 223 secs (4.4e+03 
> Kbytes/sec) * (4.28MB/s)
> 
> medium_file
> 500000000 bytes received in 111 secs (4.4e+03 Kbytes/sec) * (4.30MB/s)

> 500000000 bytes received in 106 secs (4.6e+03 Kbytes/sec) * (4.50MB/s)

> 500000000 bytes received in 109 secs (4.5e+03 Kbytes/sec) * (4.37MB/s)
> 
> small_file
> 100000000 bytes received in 20.2 secs (4.8e+03 Kbytes/sec) * 
> (4.72MB/s) 100000000 bytes received in 5.46 secs (1.8e+04 Kbytes/sec)
* (17.47MB/s)  ! cached?
> 100000000 bytes received in 13.6 secs (7.2e+03 Kbytes/sec) * 
> (7.01MB/s) 100000000 bytes received in 16.8 secs (5.8e+03 Kbytes/sec) 
> * (5.68MB/s)
> 
> 
>
************************************************************************
***************
> **   MYSRV (My OpenSuse 10.2 box at home... 2.8GHz Celery, 1 IDE Disk,
512MB RAM)    **
> **********************************************************************
> ***************** dd if=/dev/zero bs=1000 count=100000 of=small_file 
> dd if=/dev/zero bs=1000 count=500000 of=medium_file dd if=/dev/zero 
> bs=1000 count=1000000 of=large_file # ls -l
> -rw-r--r-- 1 root    root 1000000000 Aug  2 22:02 large_file  *
(953.67MB)
> -rw-r--r-- 1 root    root  500000000 Aug  2 22:01 medium_file *
(476.84MB)
> -rw-r--r-- 1 root    root  100000000 Aug  2 22:07 small_file  *
(95.37MB)
> 
> CP RESULTS (performed in random order):
> (date && cp small_file small_file.00 && date) Test 1: Start: 22:09:29 
> End: 22:09:29 * (-) Test 2: Start: 23:00:57 End: 23:00:59 * 
> (47.69MB/s) Test 3: Start: 23:06:49 End: 23:06:50 * (95.37MB/s
> 
> (date && cp medium_file medium_file.00 && date) Test 1: Start: 
> 22:09:47 End: 22:10:11 * (19.87MB/s) Test 2: Start: 23:01:12 End: 
> 23:01:44 * (14.90MB/s) Test 3: Start: 23:08:33 End: 23:09:03 * 
> (15.89MB/s)
> 
> (date && cp large_file large_file.00 && date) Test 1: Start: 22:10:29 
> End: 22:11:32 * (15.14MB/s) Test 2: Start: 23:03:25 End: 23:04:29 * 
> (14.90MB/s) Test 3: Start: 23:07:04 End: 23:08:09 * (14.67MB/s)
> 
> FTP (localhost) Results (performed in random order):
> large_file
> 1000000000 bytes received in 00:50 * (18.90 MB/s) 1000000000 bytes 
> received in 00:48 * (19.61 MB/s) 1000000000 bytes received in 00:51 * 
> (18.58 MB/s)
> 
> medium_file
> 500000000 bytes received in 00:23 * (19.93 MB/s) 500000000 bytes 
> received in 00:25 * (19.06 MB/s) 500000000 bytes received in 00:22 * 
> (20.89 MB/s)
> 
> small_file
> 100000000 bytes received in 00:03 * (27.69 MB/s) 100000000 bytes 
> received in 00:03 * (27.17 MB/s) 100000000 bytes received in 00:03 * 
> (28.56 MB/s)

> _______________________________________________
> mdlug mailing list
> mdlug at mdlug.org
> http://mdlug.org/mailman/listinfo/mdlug

danno
--
dan pritts
danno at umich.edu
734-929-9770
_______________________________________________
mdlug mailing list
mdlug at mdlug.org
http://mdlug.org/mailman/listinfo/mdlug

No virus found in this incoming message.
Checked by AVG Free Edition. 
Version: 7.5.476 / Virus Database: 269.11.8/940 - Release Date: 8/6/2007
4:53 PM

More information about the mdlug mailing list