While working on an upgrade to vSphere 5.0U1 on a Cisco UCS environment, where the ESX hosts boot from SAN, I noticed one of the hosts was not registered correctly on the EMC VNX, as it showed up as unmanaged. Because the ESX hosts boot from SAN, the host has to be registered before it can auto register, and when it was registered manually the host was not able to update the registration. Continue reading
When I tried to upgrade my vCenter 5.0U1 Server to 5.1, all seemed to go well, up until the the moment vCenter tried to register with SSO.
I received an error message “Error 29107. The service or solution user is already registered. Check Vm_ssoreg.log in system temporary folder for details”
I checked this log, but it did not really point me in to the right direction.
Then I found a post in the 5.1 beta archive that said the unique identifier for a service to register with SSO is the Common Name from its certificate. Continue reading
In a vSphere environment I am working on we use VMware vShield Edge to do firewalling, NAT and terminate VPNs for customers.
On several occasions we where not able to make config changes to some of our VSE devices when we tried to publish the changes we made from within vShield Manager. Whenever we tried to publish the changes, we received an error message in vShield Manager it could not reach the vShield Edge device we where trying to configure.
Next to that, we noticed a lot off errors in the vShield Manager System Events tab for this specific Edge Device regarding “Multiple heartbeats missed from appliance”
An other thing we noticed was the VMware Tools for this specific VSE device did not seem to be running.
We decided to open a case at VMware and where told this is a know issue with the version of vShield we are running (5.0.1) and this will be fixed in a future version. (It is not fixed in version 5.0.2 that was released recently) Continue reading
A while ago I posted an article on LUN connectivity issues with Storage vMotion on EMC VNX when using VAAI we experienced.
Today I did received an e-mail from EMC they are able to reproduce our issues in their lab, which is an important step to get these issues resolved, since we can only do limited tests in our production environment. Great news to start the weekend. Will update again when I get more details on this.
Yesterday my good friend Gabrie van Zanten from Gabes Virtual World asked the following question on twitter:
How can I disable VAAI in vSphere5 on a per array basis? KB1033665 is not the solution #EMC #CX4 #VMware Dont want to disable for my VNX
— Gabrie (@gabvirtualworld) 18 juni 2012
My first reaction was “Why would Gabe want to disable VAAI on a per array basis isn the first place?” so I asked.
His answer was pretty simple and straight forward. He was working on an environment where ESX5 hosts had both EMC CX4s and VNXes connected, and VAAI was not supported on vSphere 5 for CX4, so he had to disable VAAI for the CX4’s and wanted to leave it on for the VNXes. Continue reading
Today I was working on upgrading some hosts in a vSphere 5 environment that is using Cisco Nexus 1000V virtual switches. I imported the extension bundle in Update Manager, created a baseline, and scanned the hosts. After a couple of seconds , I got a message in vCenter telling me the scan failed:
Scan entity <hostname> Host cannot download files from VMware vSphere Update Manager patch store. Check the network connectivity and firewall setup, and check esxupdate logs for details. Continue reading
The last few weeks I have been working on some serious issues in an environment where we used vSphere 5 with an EMC VNX storage array. All seemed to run fine, but whenever we started a Storage vMotion, we noticed all kinds of strange errors we where not expecting at all.
We saw messages regarding write-quiesced VMFS volumes, we lost paths. and in some cases, the Storage vMotions did not complete at all.
During these Storage vMotions we noticed datastore latency peaked at more then 5 seconds on the source and destination LUN’s.
Last week while working at a customer, we migrated their VM’s from a Standard Switch to a Distributed Switch using the “Migrate Virtual Machine Networking” wizard.
After the migration, some VM’s appeared to be connected to two Port Groups, even though they only had one Network adapter configured:
Recently I saw some unexplained LUN trespasses on an EMC VNX that is used in a vSphere 5 environment where we use VAAI.
Since we use pools on the VNX, it is advised to keep a LUN on the owning SP, to prevent unnecessary traffic over the internal bus between SPA and SPB. EMC says:
Avoid trespassing pool LUNs. Trespassing the pool LUNs to another SP may adversely affect performance. After a pool LUN trespass, a pool LUNs private information remains under control of the original owning SP. This will cause the trespassed LUNs I/Os to continue to be handled by the original owning SP. When this happens both SPs being used in handling the I/Os. Involving both SPs in an I/O increases the time used to complete an I/O. Continue reading
After upgrading an environment from ESX5 to ESX5U1, I noticed syslog stopped working. Since ESXi by default does not keep log messages across reboots, it is a must to either specify a syslog server to collect the log files, or set the log directory to a shared datastore.
In this environment I used syslog to log all messages from the ESX hosts to a central syslog server.
When checking the syslog server after the upgrade, I noticed there where no more log messages arriving at the syslog server, and confirmed there used to be log messages just prior to the upgrade. Continue reading