[mdlug] [SUSPECTED SPAM] Re: ESX3 IO Problems

Tue Aug 7 15:11:52 EDT 2007

Does Dell have drivers that will work with VMware?  The reason I ask that is
because I know their Broadcom NICs have TCP hardware offloading that is only
activated when you use their drivers.  I wonder if there is something
similar for disk?

GReg

On 8/7/07 2:52 PM, "Morris, Tim" <tmorris at ugs.com> wrote:

> The problems I'm seeing are on the service console. I don't have any VMs
> created. I know the console is limited, but to 5MB/s? ESX 2.5.2
> performed much better in console I/O than that (on the same hardware).
> The problem is that I have 400GB of data in VMs to move to this system,
> so 5MB/s isn't going to cut it... I might have to revert to
> OpenSuse/Vmware Server if that is the case. Vmkfstools doesn't do much
> better. Is this Vmware's way of forcing you to buy VirtualCenter?
> 
> 
> --  Tim Morris - UGS PLM Software
> 
> -----Original Message-----
> From: mdlug-bounces at mdlug.org [mailto:mdlug-bounces at mdlug.org] On Behalf
> Of Dan Pritts
> Sent: Tuesday, August 07, 2007 1:13 PM
> To: MDLUG's Main discussion list
> Subject: Re: [mdlug] ESX3 IO Problems
> 
> I'm hardly a guru but you haven't said where you are doing the I/O,
> within the service console, or within guests, or what.
> 
> I know that the service console is just not optimized to do disk I/O and
> you can expect it to be slow when using standard unix commands.
> 
> I assume you've checked on the load using the vmware management console.
> It can show you what each guest is doing i/o wise.
> 
> beyond that i don't know that i can offer much advice, but there seem to
> be a lot of knowledgeable people on the forums at vmware.com.
> 
> On Fri, Aug 03, 2007 at 10:18:39AM -0500, Morris, Tim wrote:
>> Are there any ESX gurus out there?
>>  
>> I have a bad I/O problem with our ESX 3.0.2 build 52542 server that I
>> thought I would ask about. The same problem occurs when using
>> 3.0.1-32029 (unpatched).
>>  
>> Host Computer: 
>> Dell Poweredge 2850 BIOS A06
>> 2x 3.0 GHz Xeon Procs (HT disabled)
>> 4GB RAM
>> Dual Intel 8254NXX Gigabit Ethernet onboard PERC 4E/Di 2x 146GB SCSI3
>> (Maxtor) drives internal (RAID 1)
>> 1 LD = ~139GB
>> PERC 4E/Dc
>> Powervault 220VS w/ 10x 146GB SCSI3 (Maxtor) drives (RAID 5 w/ 2
>> hotspares)
>> 1LD = 979MB
>> All components have been updated to the latest availble firmware from
>> Dell (even all 12 drives!)
>>  
>> My problem is any type of file operation seems to hang every 50-200MB.
>> cp, ftp, scp, everything.
>> You can see it with an ftp hash especially. It hangs for a second,
>> then takes off again.
>>  
>> I have put together some transfer stats for you to look at (attached).
>>  
>> Any suggestions on where to go from here?
>> Any Performance testing/diagnostic utils/tips?
>> Any more info needed?
>> Anyone run into this before?
>>  
>> I could really use some help on this...  Any suggestions would be
>> helpful.
>>  
>> --------------------------------------------------
>> Tim Morris
>> Systems Engineer
>> Siemens Automation and Drives
>> UGS PLM Software
>>  
> 
> Content-Description: perf_test.txt
>> 
> ************************************************************************
> ***************
>> **                               ESX3SRVR
> **
>> **********************************************************************
>> ***************** dd if=/dev/zero bs=1000 count=100000 of=small_file
>> dd if=/dev/zero bs=1000 count=500000 of=medium_file dd if=/dev/zero
>> bs=1000 count=1000000 of=large_file # ls -l
>> -rw-r--r--    1 root     root     1000000000 Aug  3 02:13 large_file *
> (953.67MB)
>> -rw-r--r--    1 root     root     500000000 Aug  3 02:10 medium_file *
> (476.84MB)
>> -rw-r--r--    1 root     root     100000000 Aug  3 02:09 small_file  *
> (95.37MB)
>> 
>> CP RESULTS (performed in random order):
>> (date && cp -f small_file small_file.00 && date) Test 1: Start:
>> 02:43:41 End: 02:43:50 * (10.60MB/s) Test 2: Start: 02:44:07 End:
>> 02:44:12 * (19.07MB/s) Test 3: Start: 03:09:47 End: 03:10:08 *
>> (4.54MB/s)
>> 
>> (date && cp -f medium_file medium_file.00 && date) Test 1: Start:
>> 02:44:30 End: 02:46:24 * (4.18MB/s) Test 2: Start: 02:52:56 End:
>> 02:54:49 * (4.22MB/s) Test 3: Start: 03:19:10 End: 03:21:04 *
>> (4.18MB/s)
>> 
>> (date && cp -f large_file large_file.00 && date) Test 1: Start:
>> 02:47:20 End: 02:51:09 * (4.16MB/s) Test 2: Start: 03:02:17 End:
>> 03:06:12 * (4.06MB/s) Test 3: Start: 03:10:52 End: 03:14:40 *
>> (4.18MB/s)
>> 
>> FTP (localhost) Results (performed in random order):
>> large_file
>> 1000000000 bytes received in 222 secs (4.4e+03 Kbytes/sec) *
>> (4.30MB/s) 1000000000 bytes received in 220 secs (4.4e+03 Kbytes/sec)
>> * (4.34MB/s) 1000000000 bytes received in 223 secs (4.4e+03
>> Kbytes/sec) * (4.28MB/s)
>> 
>> medium_file
>> 500000000 bytes received in 111 secs (4.4e+03 Kbytes/sec) * (4.30MB/s)
> 
>> 500000000 bytes received in 106 secs (4.6e+03 Kbytes/sec) * (4.50MB/s)
> 
>> 500000000 bytes received in 109 secs (4.5e+03 Kbytes/sec) * (4.37MB/s)
>> 
>> small_file
>> 100000000 bytes received in 20.2 secs (4.8e+03 Kbytes/sec) *
>> (4.72MB/s) 100000000 bytes received in 5.46 secs (1.8e+04 Kbytes/sec)
> * (17.47MB/s)  ! cached?
>> 100000000 bytes received in 13.6 secs (7.2e+03 Kbytes/sec) *
>> (7.01MB/s) 100000000 bytes received in 16.8 secs (5.8e+03 Kbytes/sec)
>> * (5.68MB/s)
>> 
>> 
>> 
> ************************************************************************
> ***************
>> **   MYSRV (My OpenSuse 10.2 box at home... 2.8GHz Celery, 1 IDE Disk,
> 512MB RAM)    **
>> **********************************************************************
>> ***************** dd if=/dev/zero bs=1000 count=100000 of=small_file
>> dd if=/dev/zero bs=1000 count=500000 of=medium_file dd if=/dev/zero
>> bs=1000 count=1000000 of=large_file # ls -l
>> -rw-r--r-- 1 root    root 1000000000 Aug  2 22:02 large_file  *
> (953.67MB)
>> -rw-r--r-- 1 root    root  500000000 Aug  2 22:01 medium_file *
> (476.84MB)
>> -rw-r--r-- 1 root    root  100000000 Aug  2 22:07 small_file  *
> (95.37MB)
>> 
>> CP RESULTS (performed in random order):
>> (date && cp small_file small_file.00 && date) Test 1: Start: 22:09:29
>> End: 22:09:29 * (-) Test 2: Start: 23:00:57 End: 23:00:59 *
>> (47.69MB/s) Test 3: Start: 23:06:49 End: 23:06:50 * (95.37MB/s
>> 
>> (date && cp medium_file medium_file.00 && date) Test 1: Start:
>> 22:09:47 End: 22:10:11 * (19.87MB/s) Test 2: Start: 23:01:12 End:
>> 23:01:44 * (14.90MB/s) Test 3: Start: 23:08:33 End: 23:09:03 *
>> (15.89MB/s)
>> 
>> (date && cp large_file large_file.00 && date) Test 1: Start: 22:10:29
>> End: 22:11:32 * (15.14MB/s) Test 2: Start: 23:03:25 End: 23:04:29 *
>> (14.90MB/s) Test 3: Start: 23:07:04 End: 23:08:09 * (14.67MB/s)
>> 
>> FTP (localhost) Results (performed in random order):
>> large_file
>> 1000000000 bytes received in 00:50 * (18.90 MB/s) 1000000000 bytes
>> received in 00:48 * (19.61 MB/s) 1000000000 bytes received in 00:51 *
>> (18.58 MB/s)
>> 
>> medium_file
>> 500000000 bytes received in 00:23 * (19.93 MB/s) 500000000 bytes
>> received in 00:25 * (19.06 MB/s) 500000000 bytes received in 00:22 *
>> (20.89 MB/s)
>> 
>> small_file
>> 100000000 bytes received in 00:03 * (27.69 MB/s) 100000000 bytes
>> received in 00:03 * (27.17 MB/s) 100000000 bytes received in 00:03 *
>> (28.56 MB/s)
> 
>> _______________________________________________
>> mdlug mailing list
>> mdlug at mdlug.org
>> http://mdlug.org/mailman/listinfo/mdlug
> 
> danno
> --
> dan pritts
> danno at umich.edu
> 734-929-9770
> _______________________________________________
> mdlug mailing list
> mdlug at mdlug.org
> http://mdlug.org/mailman/listinfo/mdlug
> 
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.476 / Virus Database: 269.11.8/940 - Release Date: 8/6/2007
> 4:53 PM
>  
> _______________________________________________
> mdlug mailing list
> mdlug at mdlug.org
> http://mdlug.org/mailman/listinfo/mdlug