Exchange 2007 on a VM, and the case of the mysterious Isapi deadlock detected error.

 

 

Are you running Exchange 2007 on a VM?  Are you experiencing  odd warning events in the event log that look something like this?

Event Type: Warning
Event Source: W3SVC-WP
Event Category: None
Event ID: 2262
Date:  
Time:  12:28:18 PM
User:  N/A
Computer: [yourserver]
Description:
ISAPI ‘c:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\aspnet_isapi.dll’ reported itself as unhealthy for the following reason: ‘Deadlock detected’.

If you’ve answered yes to these questions, you’ve most certainly looked for the fix, and found other users in the same boat.  They try and try to fix the issue with adjustments in official documentation or otherwise, with no results.

That was me.  …until I ran across this link.

So, as suggested, I added a 2nd vCPU to my exchange server (running on Windows Server 2008 x64, in a vSphere cluster), and started it up.  These specific warning messages in my event log went away completely.  Okay, after several weeks of monitoring, I may have had a couple of warnings here and there.  But that’s it.  No longer the hundreds of warnings every day.

As for the official explanation, I don’t have one.  Adding vCPU’s to fix problems is not something I want to get in the habit of, but it was an interesting problem, with an interesting solution that was worth sharing.

 

Helpful links:

Microsoft’s most closely related KB article on the issue (that didn’t fix anything for me):
http://support.microsoft.com/kb/821268

Application pool recycling:
http://technet.microsoft.com/en-us/library/cc735314(WS.10).aspx

Restoring an Exchange 2007 mailbox using EqualLogic’s ASM/ME

 

 

Ask most IT Administrators in small to medium sized organizations to recover an Exchange mailbox, and you’ll get responses like, “how important is it to recover it?”  and “How much of it is gone?”  You might even get the slightly patronizing “Oh, well you were close to your storage quota anyway”   This is IT-speak for “I don’t want to recover it” (labor intensive), “I’m not sure if I can recover it” (it didn’t work the last time they tried), or “I can’t recover it, and don’t really want to tell you that.”  Trust me, I’ve been there.

The recovery process has ranged anywhere from non-existent (Exchange 4.0) to  supposedly easy (according to the glossy ad of the 3rd party solution you might have purchased, but could never get to work correctly), to cautiously doable, but an incomplete solution, with later versions of Exchange.  As each year passes, I’m always hoping that technology can make the process easier, without pushing through another big purchase.

I got my wish.  Technology has indeed improved the process.  I recently had a user mailbox’s contents vanish.  Who knows what happened, but I had to get the mailbox back fast, so I had to familiarize myself with the process again.  This time, since my Exchange 2007 Server is virtualized, and the Exchange databases and logs reside on guest attached volumes, I was able to take advantage of EqualLogic’s “AutoSnapshot Manager Microsoft Edition” or ASM/ME. 

ASM/ME allowed me to easily recover an Exchange Storage Group, then mount it as a Recovery Storage Group (RSG).  From that point I could restore just that single mailbox on top of the existing mailbox.  What a refreshing discovery to see how simple the process has become.  And that, it just worked.  No weird errors to investigate, no tapes to fiddle with.  It was a complete solution for a mailbox recovery scenario.  The best part of all, was that its a function available free of charge to any EqualLogic user who is using the Host Integration Toolkit (HITKit) on their Exchange server. 

Here is how you do it.

1.  On the Exchange Server, open up ASM/ME, highlight the smart copy collection that you’d like to recover.  I say “Collection” because I want to recover the volume that has the DB on it and the volume that has the Transaction Logs on it at the same time.

2.  Select option to “Create Recover Storage Group”

3.  Select the desired Storage Group

4.  It will prompt for two drive letters not being used.  This will represent the location of the restored volumes.  So if the Exchange databases are on E: and the Transaction Logs are on F:, it might prompt you to used “G:” and “H:” respectively.

5.  It will complete with the following message
image

6.  Close out of ASM/ME, and launch the “Database Recovery Management” in the Toolbox section of the Exchange Management Console.  This leads to the “Exchange Troubleshooting Assistant in the Exchange Management Console (EMC).

7.  Run through the restoration process.  It will restore the selected mailbox on top of the existing mailbox.

8.  Once it is complete, as the dialog above instructs, you will need to dismount and logoff the smart copy collection set with ASM/ME after the RSG is removed.

The process was fast, and worked the very first time without error.  I’d still prefer to never have to recover a mailbox, but it is nice to know that now, thanks to ASM/ME and the Exchange Database Recovery Management tool, that its really easy to do.

 

Helpful Links

A more detailed guide on using RSGs in Exchange:
http://www.msexchange.org/tutorials/Working-Recovery-Storage-Groups-Exchange-2007.html 

Working with a Recovery Storage group in the Exchange Management Shell (EMS) instead of the EMC.
http://technet.microsoft.com/en-us/library/bb125197(EXCHG.80).aspx

Side effects of upgrading VM’s to Virtual Hardware 7 in vSphere

 

New Years day, 2010 was a day of opportunity for me.  With the office closed, I jumped on the chance of upgrading my ESX cluster from 3.5 to vSphere 4.0 U1.  Those in IT know that a weekend flanked by a holiday is your friend; offering a bit more time to resolve problems, or back out entirely.  The word on the street was the upgrade to vSphere was a smooth one, but I wanted extra time, just in case.  The plan was laid out, the prep work had been done, and I was ready to pull the trigger.  The steps fell into 4 categories.

1.  Upgrade vCenter 
2.  Upgrade ESX hosts 
3.  Upgrade VM Tools and virtual hardware 
4.  Tidy up by taking care of licensing, VCB, and confirm that everything works as expected.

Pretty straightforward, right?  Well, in fact it was.  VMware should be commended for the fine job they did in making the transition  relatively easy.  I didn’t even need the 3 day weekend to do it.  Did I overlook anything?  Of course.  In particular, I didn’t pay attention to how the Virtual Hardware 7 upgrade was going to affect my systems at the application layer.

At the heart of the matter is that when the Virtual Hardware upgrade is made, VMware will effortlessly disable and hide those old NICs, add new NICs in there, and then reassign the addressing info that were on the old cards.  It’s pretty slick, but  does cause a few problems.

  1. Static IP addressing gets transferred over correctly, but the other NIC settings do not.  Do you have all but IPv4 disabled (e.g. Client for Microsoft networks, QoS, etc.) for your iSCSI connections to your SAN?  Do you have NetBIOS over TCP/IP shut off as well?  Well, after the Hardware 7 upgrade, all of those services will be turned on.  Do you have IPv6 disabled on your LAN NIC?  (no, it’s not a commentary on my dislike of IPv6, but there are many legitimate reasons to do this).  That will be turned back on.
  2. NIC binding order will reset itself to an order you probably do not want.  These affect services in a big way, especially when you factor in side effect #1.  (Please note that none of my systems were multi-homed on the LAN side.  The additional NIC’s for each VM were simply for iSCSI based storage access using a guest iSCSI initiator.)
  3. Guest iSCSI initiator settings *may* be different.  A few of the most common reactions I saw were the “Discovery” tab had duplicate entries, and the “Volumes and Devices” tab no longer had the drive letter of the guest initiated drive.  This is necessary to have in order for some services to not jump the gun too early.
  4. Duplicate reverse DNS records.  I stumbled upon this after the update based off of some errors I was seeing.  Many mysteries can occur with orphaned, duplicate reverse DNS records.  Get rid of ‘em as soon as you see them.  It won’t hurt to check your WINs service and clear that out as well, and keep an eye on those machines with DHCP reservations.
  5. In Microsoft’s Operating Systems, their network configuration subsystem generates a Global Unique Identifier (GUID) for each NIC that is partially based on the MAC address of the NIC.  This GUID may or may not be used in applications like Exchange, CRM, Sharepoint, etc.  When the NIC changes, the GUID changes.  …and services may break.

Items 1 through 4 are pretty easy to handle – even easier when you know what’s coming.  Item #5 is a total wildcard.

What’s interesting about this is that it created the kinds of problems that in many ways are the most problematic for Administrators; where you think it’s running fine, but it’s not.  Most things work long enough to make your VM snapshots no longer relevant, if you plan on the quick fix. 

Now, in hindsight, I see that some of this was documented, as much of this type of thing comes up in P2V, and V2V conversions.  However, much of it was not.  My hope is to save someone else a little heartache.  Here is what I did after each VM was upgraded to Virtual Hardware 7.

All VM’s

Removing old NICs

  1. Open up command shell option for the "run as administrator". In Shell, type set devmgr_show_nonpresent_devices=1, then hit Enter
  2. Type start devmgmt.msc then hit Enter
  3. Click View > Show hidden devices
  4. Expand Network Adapter tree
  5. Right click grayed out NICs, and click uninstall
  6. Click View > Show hidden devices to untoggle.
  7. Exit out of application
  8. type set devmgr_show_nonpresent_devices=0, then hit Enter.

Change binding order of new NICs

  1. Right click on Network, then click Properties > Manage Network Connections
  2. Rename NICs to "LAN" "iSCSI-1" and "iSCSI-2" or whatever you wish.
  3. Change binding order to have LAN NIC at the top of the list
  4. Disable IPV6 on LAN NIC
  5. For iSCSI NIC’s, disable all but TCP/IPv4. Verify static IP (w/ no gateway or DNS servers), verify "Register this connections address in DNS" is unchecked, and Disable NetBIOS over TCP/IP

Verify/Reset iSCSI initator settings

  1. Open iSCSI initiator
  2. Verify that in the Discovery tab, just one entry is in there; x.x.x.x:3260, Default Adapter, Default IP address
  3. Verify Targets and favorite Targets tab
  4. On Volumes and Devices tab, click on "Autoconfigure" to repopulate entries (clears up mangled entries on some).
  5. Click OK, and restart machine.

 DNS and DHCP

  1. Remove duplicate reverse lookup records for systems upgraded to Virtual Hardware 7
  2. For systems that have DHCP reserved addresses, jump into your DHCP manager, and modify as needed.

Exchange 2007 Server

Exchange seemed operational at first, but the more I looked at the event logs, the more I realized I needed to do some clean up

After performing the tasks under the “All VMs” fix, most issues went away.  However, one stuck around.  Because of the GUID issue, if your Exchange Server is running the transport server role, it will flag you with Event ID: 205 errors.  It is still looking for the old GUID.  Here is what to do.

First, determine status of the NICs

[PS] C:\Windows\System32>get-networkconnectioninfo

Name : Intel(R) PRO/1000 MT Network Connection #4
DnsServers : {192.168.0.9, 192.168.0.10}
IPAddresses : {192.168.0.20}
AdapterGuid : 5ca83cae-0519-43f8-adfe-eefca0f08a04
MacAddress : 00:50:56:8B:5F:97

Name : Intel(R) PRO/1000 MT Network Connection #5
DnsServers : {}
IPAddresses : {10.10.0.193}
AdapterGuid : 6d72814a-0805-4fca-9dee-4bef87aafb70
MacAddress : 00:50:56:8B:13:3F

Name : Intel(R) PRO/1000 MT Network Connection #6
DnsServers : {}
IPAddresses : {10.10.0.194}
AdapterGuid : 564b8466-dbe2-4b15-bd15-aafcde21b23d
MacAddress : 00:50:56:8B:2C:22

Then get the transport server info

[PS] C:\Windows\System32>get-transportserver | ft -wrap Name,*DNS*

image 

Then, set the transport server info correctly

set-transportserver SERVERNAME -ExternalDNSAdapterGuid 5ca83cae-0519-43f8-adfe-eefca0f08a04

set-transportserver SERVERNAME -InternalDNSAdapterGuid 5ca83cae-0519-43f8-adfe-eefca0f08a04

 

Sharepoint Server

Thank heavens we are just in the pre-deployment stages for Sharepoint.  Our Sharepoint Consultant asked what I did to mess up the Central Administration site, as it could no longer be accessed (the other sites were fine however).  After a bizarre series of errors, I thought it would be best to restore it from snapshot and test what was going on.  The virtual hardware upgrade definitely broke the connection, but so did removing the existing NIC, and adding another one.   As of now, I can’t determine that it is in fact a problem with the NIC GUID, but it sure seems to be.   My only working solution in the time allowed was to keep the server at Hardware level 4, and build up a new Sharepoint Front End Server. 

One might question why, with the help of vCenter,  a MAC address can’t be forced on the server.   Even though one is able to get the last 12 characters of the GUID (representing the MAC address) the first part is different.  It makes sense because the new device is different.  The applications care about the GUID as a whole, not just the MAC address.

Here is how you can find the GUID of the system’s NIC in question.  Run this BEFORE you perform a virtual hardware update, and save it for future reference in case you run into problems.  Also make note of where it exists in the registry.  It’s not a solution to the issue I had with Sharepoint, but its worth knowing about.

C:\Users\administrator.DOMAINNAME>net config rdr
Computer name
\\SERVERNAME
Full Computer name SERVERNAME.domainname.lan
User name Administrator
Workstation active on
NetbiosSmb (000000000000)
NetBT_Tcpip_{56BB9E44-EA93-43C3-B7B3-88DD478E9F73} (0050568B60BE)
Software version Windows Server (R) 2008 Standard
Workstation domain DOMAINNAME
Workstation Domain DNS Name domainname.lan
Logon domain DOMAINNAME
COM Open Timeout (sec) 0
COM Send Count (byte) 16
COM Send Timeout (msec) 250

 

 In hindsight…

Let me be clear that there is really not much that VMWare can do about this.  The same troubles would occur on a physical machine if you needed to change out Network cards.  The difference is, that it’s not done as easily, as often, or as transparently as on a VM.

If I were to do it over again (and surely I will the day when VMware releases a major upgrade again), I would have done a few things different.

  1. Note all existing application errors and warnings on each server prior to the upgrade, just so I don’t try to ponder if that warning I’m starring at had existed before the upgrade.
  2. Note those GUID’s before you upgrade.  You could always capture it after restoring from a snapshot if you do run into problems, but save yourself a little time and get this down on paper ahead of time.
  3. Take the virtual hardware upgrade slowly.  After everything else went pretty smooth, I was in a “get it done” mentality.  Although the results were not  catastrophic, I could have done better minimizing the issues.
  4. Keep the snapshots around at least the length of your scheduled maintenance window.  It’s not a get-out-of-jail card, but if you have the weekend or off-hours to experiment, it offers you a good tool to do so.

This has also helped me in the decision of taking a very conservative approach to implementing the new VMXNet3 NIC driver to existing VMs.  I might simply update my templates and only deploy them on new systems, or systems that don’t run services that rely on the NIC’s GUID.

One final note.  “GUID” can be many things depending on the context, and may be referenced in different ways (UUID, SID, etc).  Not all GUID’s are NIC GUID’s.  The term can be used quite loosely in various subject matters.   What does this mean to you?  It means that it makes searching the net pretty painful at times.

Interesting links:

A simple powershell script to get the NIC GUID
http://pshscripts.blogspot.com/2008/07/get-guidps1.html

Discovering AutoDiscover in Exchange 2007

 

In my post “Exchange 2007… Better Later than Never” I mentioned one of the post-deployment difficulties I faced was getting the "AutoDiscover” function to behave the way it was designed.  For those unfamiliar with the feature, it allows for automated discovery and configuration of various connectivity methods to an Exchange Server.  Exchange MAPI clients, Exchange HTTP/RPC clients, and mobile devices using ActiveSync all can use AutoDiscover in some form or another.

While it wasn’t critical for the transition itself, AutoDiscover was vital for our future deployments of “Outlook Anywhere” and “ActiveSync.”  I figured skimming over a few TechNet articles and blog postings, and I’d be quickly onto the next project.  That began my long ugly journey getting AutoDiscover to work.

It became clear that the ingredients for AutoDiscover to work correctly was a properly configured ISA Server, SSL certificates, namespace/DNS accommodations, and of course, Exchange.  What was really interesting about this particular project was that I was dealing with very mature products, yet, I never ran across so much contradicting information on how to make it work.  Perhaps some of that stems from so many valid topologies and configurations, or possibly big changes between the RTM versions of Exchange and ISA and their first service packs.  Still, it seemed odd.  I sifted through postings from desperate IT Administrators in similar situations who had no more hair to pull out.  You could sense the defeat in their words.  Now I understand.

One guideline mentioned quite often was the need for a special SSL certificate that allowed for more than one FQDN to be assigned to it.  You’ll see it referred to as a Unified Communication Certificate (UCC or UC) or a Subject Alternative Name (SAN) certificate.  The purpose is the same, but the names and the references are different.  While UC certificates are not technically a requirement, it is best to think of it that way.  For AutoDiscover, the names needed on a UC cert would look something like:

mypubliccompanyname.com
autodiscover.mypubliccompanyname.com
mail.mypubliccompanyname.com
internalmailservername
internalmailservername.myprivatelanname.lan

I went with a UC cert from DigiCert, but any of the larger commercial CA’s should work.  However, a word of warning.  Exchange doesn’t like self signed certificates, and many mobile phones have troubles with private certificates as well as those from smaller commercial CAs.  You should be fine if you run Certificate Services internally (or so I’m told), and your namespace checks out okay.  Don’t forget look at your ISA server and make sure you are running SP1 or later, due to limitations on how the RTM version handled UC certificates.

Speaking of namespaces, time for a thorn in my side to come back and sting me.  My internal namespace is not a name that we own (a legacy issue I should have taken care of long ago).  Certificate Authorities will not issue standard or UC SSL certificates to names you do not own for obvious reasons, even if the references are private.  Fortunately, I was able to work around this by making absolutely sure the simple name was used in any Exchange configuration settings that usually accepted the internal FQDN.  Disaster averted.

Now for the dirt on how I was able to make it work.  My as-built  design is modeled somewhat after Jason Jones’ method of Publishing Exchange 2007 Services with ISA 2006.  Following the construct of:

  • Not using the existing listener created for OWA, and creating a separate listener for Outlook Anywhere (OA)/Autodiscover, and binding UC cert to that listener. Using HTTP authentication with Integrated/Windows Auth (aka NTLM). This would provides HTTP/Integrated auth from the client to the FW, then basic auth from the FW to the Exchange server.
  • Allowing the ISA server to utilize Kerberos constrained delegation (KCD) by way of changes in AD.
  • Creating a single Publishing rule for OA , where KCD is used.
  • Setting internal and external URL’s to their respective internal and external locations (internalmailservername and autodiscover.mycompanyname.com)

After configuring it as above, AutoDiscover worked internally, but not externally.  Continually getting failures with the /rpc directory when testing internally (via test-outlookwebservices) and externally (via testexchangeconnecivity.com).  I found a post that gave the missing piece of the puzzle, and modified my configuration per the recommendations  http://forums.isaserver.org/m_2002041377/mpage_2/key_/tm.htm:

  • Create a 2nd Publishing rule for OA, sitting on top of primary OA publishing rule.
    • Only /rpc/* is published
    • Auth Delegation is set to "No Delegation, but client may authenticate directly"
    • Set to "all users" instead of "authenticated users"
    • Changing "EXPR" Outlookprovider to msstd:mail.mycompanyname.com so that the certificate mutual authentication test passes.

Under the conditions described above, Outlook Anywhere with Autodiscover functions as desired.

As Jason Jones put it best, “The reason for the need of a separate listener is that Windows Authentication (NTLM) and Forms Based Authentication (FBA) are mutually exclusive. It is not possible to use a single web listener for all Exchange 2007 publishing and achieve transparent authentication within Outlook anywhere.”  Thus the need to create a dedicated listener to be used exclusively for Outlook Anywhere and associated services.

I did have to make one other adjustment that is rarely brought up in the AutoDiscover deployment scenarios.  We know that AutoDiscover wants to look at your TLD name (e.g. yourpubliccompanyname.com) when doing it’s discovery process.  However, you may have simply had an “A” record of “yourpubliccompanyname.com” pointing to your web server to catch those users who forget to type in “www” before your domain name.  It’s also not a far fetch to assume you had an SSL certificate bound to that web server as well.  This is exactly what I had, so I had to make the following changes:

1.  Have our ISP (or whomever has authoritative control on the DNS zone file for “mypubliccompanyname.com”) change the “A” record from my public web server IP address, to my autodiscover address.

2.  In ISA, add a new “DENY (REDIRECT)” rule for mypubliccompanyname.com that does, well, a deny, and a redirect www.mypubliccompanyname.com.  This sits right above the web publishing rule for www.mypubliccompanyname.com

The original setup was a carryover from an earlier time.  The configuration above is the way I should have set it up.  Nice to do a little cleanup along the way.

I can’t tell you how relieved I was in getting this to work, no matter how many hoops I had to jump through.  I also have a complete set of as-built notes in case I need to recreate or debug the existing configuration.  It’s been stable since, but I have a feeling I’ll be looking at this again as soon as we transition to Exchange 2010. 

Other helpful links:

Microsoft Exchange Remote Connectivity Analyzer
https://www.testexchangeconnectivity.com/

Publishing Exchange 2007 Services with ISA Server 2006…
http://blog.msfirewall.org.uk/2008/07/publishing-exchange-2007-services-with.html

Technet white paper:  Exchange 2007 Autodiscover service:
http://technet.microsoft.com/en-us/library/bb332063.aspx

Generating SSL certificates for Exchange 2007 and ISA 2006:
http://www.isaserver.org/tutorials/Generating-SSL-Certificates-Exchange-2007-ISA-Server-2006.html 

Dr. Tom Shinder’s guides on Publishing Exchange 2007 OWA, activeSync, and RPC/HTTP using ISA 2006:
http://www.isaserver.org/tutorials/Publishing-Exchange-2007-OWA-Exchange-ActiveSync-RPCHTTP-using-2006-ISA-Firewall-Part1.html

A bulk discount on Tylenol.  …You’ll need it.
http://www.costco.com

Exchange 2007… Better late than never

 

Count me in as one of the many Administrators that finally got around to moving from Exchange 2003 to Exchange 2007.   Yes, it’s almost 2010, and probably just a few months away from the release of Exchange 2010.  I never disagreed with Microsoft’s decision to release Exchange 2007 as a 64 bit only application.  It’s just that it made it incredibly difficult to find a way to transition when your infrastructure is full of physical 32 bit servers.  I was constantly reminded that my Exchange 2003 server was brittle, a resource hog, and lacked many of the abilities that the end users and applications were demanding.  Twenty-five minute reboots, and inexplicable behaviors did not instill confidence.    It was time to make the move.

My timing couldn’t have been better.  I have a new virtualized infrastructure powered by some Dell blades, running VMWare ESX 3.5 using a Dell/ Equallogic PS5000 SAN to help with this transition.  It didn’t take away from the normal shrewd planning necessary on a project of this nature.  It just made it easier.  The other benefit to deploying 3 year old software is that issues, workarounds, and fixes to Exchange, as well as how to make it play nicely with other components (ISA, CRM, etc.) were well documented. 

With respect to running Exchange 2007 in a virtual environment, I noticed information on sizing is difficult to find.  Capacity planning guidelines still reflected deployments with physical servers (read: more is better), and white papers on virtualizing Exchange appeared to have the intended audience of large enterprise environments only.  My environment is relatively small, and a single server was going to be handling all Exchange roles.  Based on my environment of about 50 users and 100 mailboxes, I sifted through all the material I could find, threw in a few wild guesses, and decided with the following configuration.

  • 1 vCPU
  • 2.5GB of RAM
  • Primary OS resides in VMFS volume on the SAN
  • Using guest OS based iSCSI initiator with MPIO enabled, dedicated NTFS volume for Exchange database. 
  • Using guest OS based iSCSI initiator with MPIO enabled, dedicated NTFS volume for Exchange transaction logs.  

The transition occurred over one weekend, while the remainder of the outstanding issues were cleaned up throughout the week.  A moment of embarrassment occurred when I realized my thorough planning never took into consideration that the “move mailbox” function would hit the transaction logs so much.   I didn’t see it until it was too late.  The partition for the transaction logs filled up and the services shut down.  But, thanks to the ease of handling storage with the SAN, I was able to create a new LUN, initialize it in the OS, and change a couple of drive letters.  Ran a backup to commit the transaction logs to the database, and I was back in business.  The only puzzle that took me far too long to figure out was getting the AutoDiscover function in Exchange 2007 to work as intended.  It is worthy of it’s own post at another time.

As far as performance goes, I couldn’t be happier with it running as a VM.  I truly didn’t know how it would react under the settings I established, and was ready to make whatever changes necessary, but it has been performing very well.  Below are some very simple utilization numbers to give you an idea.

CPU:  Hovers around 50% utilization during the day, and around 15% utilization during off hours.  CPU Ready values range anywhere from 20 to 200 milliseconds.  This VM has just 1 vCPU assigned to it.  I wanted to keep it this way if possible, so that it could use the “Fault Tolerant” feature of vSphere when we upgrade.  Looks like I get my wish.

RAM:  Typically runs about 50% of the 2.5GB assigned.  No higher than 75% utilization during the busiest time of the day, and 15% utilization during off hours

Network:  Runs about 1.2 MBps.  Spikes of 40 MBps occur only because of some on-host backups occasionally occurring.  Bandwidth utilization is imperceptible during off hours.

Disk I/O:  Hard to gauge, but mostly because the need seems to be so low.  The OS partition coming from the VMFS volume might show 200KBps, but has bounced up to 7MBps on occasion.  No performance data yet on the drives connected via the guest iSCSI initiator.  I think they qualify as “fast” until I find anything that suggests otherwise.

Perception:  You won’t find this on any ESX or OS performance monitor.  But it’s importance cannot be underestimated.  I had about half of the staff comment about how much snappier their Outlook clients and OWA was working for them.  I never have staff stop by and offer random comments telling me how fast something is.  It’s a nice compliment to the improvements of Exchange 2007, and what it is running on.

No more painfully long restart times for me now.  It’s one of the more overlooked benefits I’ve noticed while moving my systems over to my new infrastructure.  Planned server restarts are a part of responsible management of IT systems.  Anything that can be done to reduce any interruption is appreciated.  A two minute restart is always welcome

It probably goes without saying that I took great joy decommissioning the old server.  Hopefully this will be me in a good position when Exchange 2010 comes off the presses.

Follow

Get every new post delivered to your Inbox.

Join 737 other followers