The ramdisk ‘var’ is full on ESXi host and fix (without maintenance mode or reboot)

I recently encountered an issue where vMotions on a host would fail, the host would disconnect from vCenter, and some other strange errors.

This was an HP host installed with the HP ISO image, but not sure if that is the cause of this issue.

When investigating the logs on the host I noticed that /var on the ramdisk was full

When issuing vdf -h available space for /var on the ramdisk was  0%

Looking in /var/log i noticed all logfiles where symlinks to /scratch except for the EMU directory, where some Emulex process seemed to fill up a log file …..

When removing the logfile /var/log/EMU/mili/mili2d.log and after restarting hostd, space was freed up on /var in the ramdisk, but the logfile /var/log/EMU/mili/mili2d.log returned and started filling up again.

Googeling I found a suggestion to remove the Emulex vibs when not using an Emulex HBA, but these hosts did have Emulex HBA’s

After some more research I found a fix that did not need a reboot or maintenance mode (which is great since vMotion stoped working on these hosts):

Continue reading

vSphere Distributed Switch refused to upgrade to version 6.5

Just after vSphere 6.5 was released I decided to upgrade my lab to 6.5. Most of the upgrade went pretty smooth, but two of my 3 distributed switches refused to upgrade. Googeling for a solution dit not help too much, probably since the product was released just a day before 🙂 When I tried to upgrade I got a message the vDS config could not be read. I also noticed I was not able to upgrade these switches to enhanced LACP.

I did find some kb articles regarding some wrong vCenter database entries for LACP in previous upgrades, so I had a feeling this was related to LACP (which I do not use) … Continue reading

Invalid credentials message when registering vCenter Server with external Platform Service Controller in vSphere 6

When vSphere 6 was released, I decided to delete my RTM version of my external Platform service Controller and vCenter Server appliances, to replace them by the GA versions.

Installing the PSC went fine, but when installing the vCenter appliance, I was not able to register it to the PSC. I kept getting the message “Invalid credentials” every time I entered the SSO administrator password. Redeployed the PSC several time, using different passwords, but no luck registering the VCSA.

Continue reading

Add RecoverPoint 4.1 with SRM RecoverPoint SRA 2.2 fails with error “SRA command ‘discoverArrays’ failed” UPDATED

During an installation and configuration of an SRM solution for a customer based on EMC RecoverPoint 4.1 I ran in to an interesting issue.

When I tried to add the RecoverPoint Clusters on both sites using the RecoverPoint SRA 2.2 I received the following error message:

Error

Continue reading

Batch file to start VSAN Observer on Windows vCenter Server

After reading the blog article by Erik Bussink on how to use the VSAN Observer software on a Windows vCenter Server, I quickly got annoyed by manually having to start the Ruby rvc script and the VSAN Observer every time

I created a little batch file that can be put on your desktop to launch the VSAN Observer

Just enter your credentials, vCenter server Name, Datacenter Name and Cluster Name and make sure the path to the rvc directory is correct, and of you go

After you started VSAN Observer, just connect to it via port 8010 on your vCenter Server, or add an exception for port 8010 in your Windows firewall to access VSAN Observer remotely, like described in Eriks’ blog

Enjoy

VSAN.Observer.bat

Tech Preview of Horizon View Desktops Plugin for the vSphere Web Client

During the Horizon View 5.2 Beta I noticed the View Desktops Tech Preview. Since the Beta was covered by an NDA I was not allowed to publish this, but since the product is GA for a while now here’s the story.

The plugin will let you search for usernames in the vSphere Web Client search field and shows you the desktop(s) for this user. Continue reading

Beta training in Frimley: VMware vCenter Operations Manager: Advanced Usage and Dashboard Design

May 8th – 10th 2013 VMware will host  a 3 day beta training at their Frimley UK HQ for their new VMware vCenter Operations Manager: Advanced Usage and Dashboard Design training.

This training course focuses on the advanced capabilities of VMware® vCenter™ Operations Manager™ 5.6, such as adapters, customization and management topics. You will learn how to use adapters, define custom super metrics, customize dashboards, and use the Custom user interface.

I have been fortunate enough to participate in an alpha training for this class and must say I enjoyed it very much. Very useful for people that want to customize their vCOps environment to show specific metrics or combined metrics

Cost for this training is around 50% of the normal price

See the attached info sheet, and if interested, register for this training here

Space is limited

beta announcement – vcops advanced 5 x

Continue reading

ESX hosts not registering on EMC VNX (and fix)

While working on an upgrade to vSphere 5.0U1 on a Cisco UCS environment, where the ESX hosts boot from SAN, I noticed one of the hosts was not registered correctly on the EMC VNX, as it showed up as unmanaged. Because the ESX hosts boot from SAN, the host has to be registered before it can auto register, and when it was registered manually  the host was not able to update the registration. Continue reading

EMC WebEx on VAAI support in ESX5 for CX4 at the 28th of June

Today I received a tweet from Chad Sakac, SVP Global SE at EMC, that he will be discussing some of the questions and concerns I raised in my blog post Challenges when upgrading environment with EMC CX4 to vSphere 5 and mixed CX4/VNX environment in next weeks Chad’s Choice WebEx, and in a blog post on his Virtual Geek blog.

Subjects for this call will be: Continue reading

Enabling VAAI on ESX5 on a per array base idea

Yesterday I wrote an article on issues when using a combination of CX4 and VNX in a vSphere 5 environment since ESX5 does not support on VAAI according to VMware and the VNX does. Simple solution would be to disable VAAI  on all ESX5 hosts, but in that case your VNXes would also loose VAAI. Victor Forde pointed me to a blog post by Chris Wahl titeled “Forcing the NMP Plugin for Microsoft Clustering LUNs on vSphere” (good post by the way)

When reading this article, I realized you can not only use the array vendor and model strings to assign the VAAI filter driver to an array, but you might also be able to use the location (the combination of adapter, target, channel and LUN) for a device to assign the VAAI filter and VAAI Plugin to a specific array.

In short, you would need to remove the default VAAI filter and VAAI plugin rules for vendor=DGC and model=* and replace them by claim rules for based on the location where you would use the target identifier to filter on, for the array that would need to have VAAI enabled.

Unfortunately I don’t have a VNX in my lab (Someone at EMC that wants to trade my VNXe and a case of beer for a VNX? 😉 ) so I am not able to test if this would really do the trick. Continue reading