Changes to what they announced on July 12th 2011…..

—————–

Today – August 3rd VMware announced that they will:

• Substantially raise the vRAM entitlements per vSphere edition from 48/32/24/24/24Gb to 96/64/32/32/32Gb where the versions are Enterprise+, Enterprise, Standard, Essentials+, Essentials, and Free Hypervisor (Note that the free hypervisor is limited to 32Gb of physical ram per physical server) (not that vRAM is the memory configured to a virtual machine and not the amount of memory in a physical server – assigning a certain amount of vRAM is a required step in the creation of a virtual machine)
• Cap the amount of vRAM counted per VM at 96Gb
• Calculate a 12 month average of configured vRAM rather than a high water mark to allow for short lived spikes in dev/test and DR type situations where transient VM increases are typically seen

vSphere 4.x vs vSphere 5:

• Licensing Units = both versions are processor based
• Core per Proc & Physical Ram per host is now unlimited in vSphere 5
• Pooled vRAM entitlements – new licensing restriction in vSphere 5

vRAM Concepts:

• Each vSphere 5 processor license comes with a certain amount of vRAM entitlement (see above for that amount)
• Pooled vRAM Entitlement is the sum of all processor license entitlements
• Consumed vRAM is the sum of vRAM configured into all powered on VMs
• Compliance = 12 month rolling average of Consumed vRAM < Pooled vRAM Entitlement
• vRAM is pooled among all vSphere hosts managed by vCenter or linked vCenter instance
• The monitoring tool is built into vCenter Server 5.0

vRAM pool can be extended by:

• Upgrading all CPUs to higher end vSphere Edition
• Adding processor licenses to the same set of CPUs
• Adding a new host with new licenses

Before upgrading to vSphere 5, partners can help you by running a utility that analyzes VI3 or vSphere 4 environment and determine the amount of vRAM consumed

After upgrading to vSphere 5, vRAM license monitoring and reporting is built into vCenter 5

New product vSphere Desktop Edition that is licensed per user – which is a NEW edition for VDI deployments only.  This new product:

• Can only be used for desktop virtualization
• Is for new purchases only
• Licensed on a total number of powered on desktop virtual machines
• Sold in pack size of 100 at a list price of $6,500.
• Unlimited vRAM entitlements
• All features and functions of vSphere Enterprise Plus

Existing VDI Customers with active vSphere licenses and SnS that is used for VDI may be upgraded to vSphere 5 and have unlimited vRAM entitlements that are managed by a different instance of vCenter

VMware View is still a Comprehensive end to end Desktop Virtualization Solution that is available as View Enterprise and View Premier

vSphere 5 licensing Model applies:

• For ELA Customers:
  • With active ELA will continue to be subject to the terms of their contracts for the duration of their contract, independent of which vSphere version they deployed
  • ELA customers may contact their VMware sales representatives (ME) to update the terms of their ELAs to the new vSphere 5 licensing model
• For customers without ELAs
  • The new model applies only to vSphere 5 licenses.
  • The new vSphere 5 licensing model will apply upon acceptance of the EULA
  • Customers who purchase vSphere 5 and decide to downgrade to older versions of vSphere will be subject to the EULA terms and licensing of the model of the vSphere they downgrade to
• Users with 4.1 licenses and prior or users who buy 5.0 licenses and downgrade to 4.x versions are governed by 4.0 EULA
• vSphere 4.x Advanced licenses can be upgraded to Enterprise for Free if SnS is still active

 

vSphere 5 has 5 versions:

• Essentials $83 per processor (limited to 192Gb physical ram)
• Essentials Plus $749 per processor (limited to 192Gb physical ram)
• Standard $995 per processor
• Enterprise $2,875 per processor
• Enterprise Plus $3,495 per processor

vSphere 5 Kits:

• Essentials Plus + vSphere Storage Appliance $7,995
• Essentials $495 6 CPUs 192Gb MAX, 32Gb vRAM
• Essentials Plus $4,495 6 CPUs 192Gb MAX, 32Gb vRAM
• Standard AK $10,000 8 CPUs 256Gb per kit, 32Gb vRAM
• Enterprise AK $17,495 6 CPUs 384Gb per kit, 64Gb vRAM
• Enterprise Plus AK $21,995 6 CPUs 576Gb per kit, 96Gb vRAM

VMware vSphere Hypervisor 5:

• Full featured hypervisor
• Based on ESXi
• Provides same performance, reliability and robustness of the ESXi included with paid versions of VMware vSphere
• Can not be managed with vCenter Server
• Can be upgraded to paid vSphere for centralized management and advanced capabilities
• Limited to 32Gb of physical ram

If so….VIRTUALIZATION is the primary cure!!!!

Below are pictures from a recently completed project.  Do you need help???

Before Virtualization – Several racks of equipment with no way to get to the server without risk of causing an issue because of bad wiring, old servers with no way to get spare parts, network rack with spaghetti wiring with no way to track source and destination, power cord death traps, and on and on and on:

AND NOW…..The AFTER – No detailed descriptions needed.

• The installation of several IBM high performance servers with full fault tolerance end to end
• Cables labeled and color coded depending upon traffic
• Redundant switches (fibre and ethernet)
• Management and Lan traffic physically seperate
• Redundant power
• Keyboard Video Mouse over IP
• Redundant Network based, Out of Band, Systems Management
• 8Gb Fibre Network
• High Speed Fibre Optic Tape Library
• Etc, etc, etc…….
• ALL data replicated intra and inter site…between 6 locations….where 2 locations are in a state that borders the Gulf of Mexico….and 4 locations are in states that border the Great Lakes…..ENJOY!

There is a new licensing model that is fairly complex but I can help walk you through that if you message me.  A lot of consultants are jumping on the web and posting all types of premature information about the new release.  I’ve wanted to set back and let everything get ironed out before posting.

I was at a briefing a week ago where one Technical Services Manager was certain he was right about something….to the point of arguing with one of VMware’s executive managers.  The point he was arguing about was mentioned in a beta release which never made it to production……ugh FAIL on his part.

Anyway the purpose of this post is to provide a set of links and a bit of information that will help you navigate this new product version release.

• Information on the new licensing model:  http://blogs.vmware.com/rethinkit/2011/07/understanding-the-vsphere-5-vram-licensing-model.html
• vSphere Licensing Overview:  http://blogs.vmware.com/euc/2011/07/vsphere-desktop-licensing-overview.html

If you need help with any of this message me.  It’s been a long few weeks!

AND NOW……

To the list of additions / updates:

http://vmguy.com/wordpress/index.php/archives/1674

With 25+ years of experience in the x86 arena and with 12 years experience tuning VMware the one thing that I have learned is that nothing is as it seems.  Working to systematically isolate the issue is what has to occur in every situation.  Each engagement has to start out with a broad view of your total environment with tenacious work to methodically narrow the extent of the research as possible sources of performance problems are eliminated.

I have worked with engineers who are experts in troubleshooting specific components such as storage or networks.  Whenever they experience a performance problem, they like to begin by taking a narrow look at the components where their level of expertise resides.  This expert knowledge of a narrow area oft times leads to them getting bogged down doing detailed analysis of one component of the whole environment, while the root cause of the issue is actually somewhere else in the infrastructure.  This is not to say that a detailed analysis of key components is not warranted, but in the vast number of cases, I find that a faster resolution to the issue is obtained by starting broad and narrowing the scope vs starting narrow and broadening the scope.

In this first post we will look at basic CPU troubleshooting.

Tools:

The tool that I normally use to troubleshoot is the vSphere client, which is my primary tool.  In some situations I will use the esxtop and resxtop utilities if I need detailed performance from a single ESX host.  Operating system specific tools inside a VM are generally a bad idea to use in troubleshooting performance especially in cases of over-commitment.

Resource Pool CPU Saturation:

• If you have resource pool’s configured in your virtual datacenter, check for CPU saturation esp if you have CPU limits set – are they close to the limit value?  If they are not close to the limit, you do not have a Resource Pool CPU saturation issue.  If yes check for high Ready Time.
• If the performance problem is specific to one VM in the resource pool, use that VM to check for high ready times.  If you find high CPU ready times you have a Resource Pool CPU saturation issue which needs to be corrected.

Host CPU Saturation:

• Check for high host CPU usage.  If average usage or peaks are below the norm you do not have host CPU saturation.
• If usage and peaks are above norm and you have a HIgh CPU ready times are high then you have host CPU saturation which needs to be corrected.

The normal root-cause of cpu saturation is simple.  Your host does not have enough CPU cycles to service the work loads of the virtual machines it is servicing.  There are a few scenarios where this typically happens with each having it’s own solution.

• Host with large number of VMs, all with low to moderate cpu requirements
• Host with few VMs, all with high cpu requirements
• Host with a mix of VMs with a similar mix of high and low cpu requirements

Guest CPU Saturation:

• Check for high guest cpu saturation.  If average usage and peaks are below the norm you do not have guest cpu saturation.
• If usage and peaks are above norm you have a guest cpu saturation issue which needs to be corrected.

Guest cpu saturation happens when operating system and application on a VM utilize all of the cpu cycles that the host machine provides it.  The identification of a guest cpu saturation does not necessarily mean there is a performance problem.  There are many cpu intensive applications that will use 100% of the cpu cycles it has access to, likewise many application experience peaks when under high workloads.  However, if you find that when your VM is processing a peak load it is experience cpu saturation then a performance issue is occurring which needs to be corrected.

Next post in the series:  Basic Memory Performance Troubleshooting

Web based System X Hardware Configurator:
https://www-01.ibm.com/products/hardware/configurator/americas/bhui/launchNI.wss

IBM BladeCenter Information Center
http://publib.boulder.ibm.com/infocenter/bladectr/documentation/index.jsp?

IBM Systems Software Information Center:
http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp

SAR Triggers and documents:
http://partners.boulder.ibm.com/src/assur30i.nsf/WebIndex/SA855

Parts lookup tool:
http://www-947.ibm.com/systems/support/supportsite.wss/partsform?brandind=5000008

Service Pac offerings:
http://www-935.ibm.com/services/us/its/html/servicepac_americas.html

IBM Virtual Briefing Center:
http://ibm-vbc.centers.ihost.com/

ServeRAID Adapter Quick Reference:
http://www.redbooks.ibm.com/abstracts/tips0054.html

speakers: Wilfredo Sotolongo VP of System X Sales North America

Bill Donohue describes this as THE most exciting announcement they have ever made in the x86 arena.

98 pct of servers in DC are x86

53 pct of spend is in the x86 arena

We can no longer throw processors at the problems. We need smarter architecture.

Its time for the next revolution. Its time to take PC out of the PC…IBM Created the first PC.

Biggest issue today is memory!

6 times the memory of what is out there today

2x the capacity of competitors forever

IBM is not releasing this technology to its competitors like it has done in the past

Announcement is about a new way of thinking about x86.

They have spend more in R&D in x5 than on any other technology.

Bob Zuber – Program Director of IBM World Wide High Performance

Out of all of this comes eX5

Flex proc node

Satish Gupta VP of Systems Hardware Development at STG

Working on changing the model of what a mainframe is!

IBM makes their own silicon. Their competitors do not.

IBM Invented the disk drive in 1956

You can now access storage at the same speed in memory.

IBM is the largest systems company with the broadest portfolio in the industry.

1/2 of IBMs R&D is in Systems, the other 1/2 is in software for those systems.

Acxiom runs 10,000 intel servers in production

Michael North – NFL

***The items in italics below should be substituted with specifics from your install***

esxcfg-vswif -l

esxcfg-vswif -d vswif0

esxcfg-vswif -a vswif0 -p Service\ Console -i hostipaddress -n subnetmask -b broadcastaddress

esxcfg-vswif -s vswif0

esxcfg-vswif -e vswif0

1. IBM BladeCenter E – This is their original chassis. The E is for Energy efficiency. This chassis supports 14 blades with 4 i/o ports in a 7U form factor.
2. IBM BladeCenter H – The H is for High Performance. This chassis supports 14 blades, 8 i/o ports in a 9U form factor. This is the chassis that many customers go with by default. However the other chassis absolutely have their sweet spots. This chassis was built to house big databases, utility and application servers, and is hands down the right choice for big enterprise datacenter virtualization and consolidation projects.
3. IBM BladeCenter S – The S IBM says is for Small and Medium Business as in SMB. I like to say it is for SWEET. This 7U chassis currently holds 6 blades, a KVM over IP solution, 6 i/o ports, and up to 24 sas or sata disks. As I indicated this chassis was originally aimed for the SMB community but when the larger enterprises caught wind of it they began deploying it in many areas that didn’t need the large number of blades, 8 i/o ports, or full end to end redundancy that the other four chassis are designed with. This chassis has full fault tolerance in all areas except for the management module if configured correctly and it offers the flexibility to have no fault tolerance if your project goals do not require. PLUS it will run on the same power as your coffee pot….120V….however if your datacenter is only 220V…it handles that with ease also. Personally I like to run it at 120v for demos or small office installations, but for big datacenter installs that use the fastest of processors and need fault tolerance with the electrical connections I like suggest 220v power.
4. IBM BladeCenter T – The T is for Telco. This chassis was designed to meet the stringent requirements of the telco industry with features such as air filtration and shock resistance. The funny thing to me was I see more of them installed in cruise ships than in telecommunication companies. The T chassis was modeled after the E chassis.  
5. IBM BladeCenter HT – The HT is for High Performance Telco. Again this was designed for the telco industry….but the military uses them in their ships and Hummers to control things such as missiles and communications.
   Bottom line is this ∂suite of chassis delivers

The bottom line is that this suite of chassis delivers major processing power whether your need is a few servers for a home office, a new development project, a large datacenter consolidation project, or to control missile flight paths for the strongest military in the world.

OK now for the I/O makeup of these chassis.

IBM BladeCenter E front and rear view followed by i/o topology. The chassis provides connections between blade slots and switch bays via a hard-wired dual redundant midplane (printed circuit board – PCB). Each blade slot has 4 dedicated I/O connections (two per midplane) connected to 4 switch bays. Thus, each switch bay has a total of 14 internal I/O connections (to 14 blade bays).

IBM BladeCenter H front and rear view followed by i/o topology. This chassis provides i/o paths between blade slots and switch bays through a hard-wired dual redundant midplane (printed circuit board PCB). The chassis has 10 I/O slots. The blade slots have 8 dedicated paths to the switch modules. This chassis has two types of fabrics. It has the standard fabric which is almost the same as what is found in the E chassis and it has a high speed fabric that is used to carry higher bandwidth i/o such as 10G ethernet or infiniband. The high-speed fabric is activated when a high speed i/o card is installed in a blade server. These high speed fabric cards have their own connections to the dual redundant midplane as well as a PCI Express socket on the blade server itself.

IBM BladeCenter S front and rear view followed by i/o topology. This chassis provides i/o paths between blade slots and switch bays through a hard-wired dual redundant midplane (printed circuit board PCB). The 6 server slots have 6 dedicated i/o paths to the each of the four switch bays. Switch module slot 1 and 2 have 2 connections to each blade slot. Switch module slots 3 and 4 have 1 connection to each blade slot. This means that any switch you install in switch slots 1 and 2 will have a total of 12 internal to the chassis hard wired connections (2 per blade slot) and switch modules 3 and 4 will have a total of 6 internal to the chassis hard wired connections (1 per blade slot). Typically switch module 1 will contain an ethernet switch, which provides 2 ethernet ports to each blade. Some customers will want redundancy at the switch level and will add an additional ethernet switch to switch module 2. If the customer wishes to utilize the storage module(s) in the chassis for san usage, in switch module bays 3 and 4 they will install sas switches. However there are other options for switch modules 3 and 4. For example if a customer wishes to utilize a fibre channel san or a dedicated iscsi san, they can install switch modules to access those storage solutions in switch modules 3 and 4. Obviously, if they do not utilize the sas switch modules in switch slots 3 and 4 they will not be able to use the internal storage modules.

IBM BladeCenter T front and rear view followed by i/o topology.

IBM BladeCenter HT front and rear view followed by i/o topology.

The BladeCenter H chassis is divided into two power domains each requiring two power supply modules to function. The BCH comes standard with 2x 2900w power supplies and can be upgraded to a total of 4. The power supply modules in the BladeCenter chassis must be connected to high voltage power only (between 200V and 240V AC). There are three options for power cords, 2x 60amp, 4x 30amp, or 6x 16amp. Power cords can be mixed as required for PDU and UPS, or direct wall power and UPS. Each pair of supplies is joined internally and there is one cord connector for each domain. Each power domain comes standard with 1 power supply. The rear of the chassis contains two Burndy connectors.

The 4 power supplies provide power following these guidelines:

• The two left power supplies (PS1 and PS2) come standard and operate as an N+N reduandant pair. They provide power for blades 1-7 as well as power for common chassis infrastructure
• The two right power supplies (PS3 and PS4) are optional and operate as an N+N reduandant pair. They provide power for blades 8-14 as well as power for common chassis infrastructure
• Power Supplies (PS1 and PS4) attaches to the first line cord via the right power port (as viewed from the rear of the chassis)
• Power Supplies (PS2 and PS3) attaches to second line cord via the left power port

IMPORTANT – Power for the blowers is not provided by the power supplies. Instead, the blowers are run on AC power coming directly from the power cords. This is a major difference from other vendors. This increases the power utilization by not wasting power that is normally wasted by the AC to DC conversion that other blade systems must use. The right power cord supplies AC power to the lower blower, and the left power cord supplies AC power to the upper blower. Note that what this means is even if you are only running 2 power supplies you still have to provide power to both left and right power inlets.

The dual power domain layout in the BCH is unique in that the common chassis components require power from both domains. Specifically power is provided in this manner:

• Blowers draw power directly by AC from the power cords and not from the power supplies
• Management Modules 1 & 2, Media Tray, midplane, and Switch Modules 1 & 2 are powered by both power domains from any power supply
• Switch Modules 3 & 4 draw power from power domain 1
• Bridge Modules 1 & 2 draw power from power domain 2
• High-speed Switch Modules 1 & 2 share power between the two domains but only from power supplies 1 & 3
• High-speed Switch Modules 3 & 4 share power between the two domains but only from power supplies 2 & 4

*diagram provided by IBM

Coming next wiring a BladeCenter H with wall power, PDUs, UPSs, and combos of each.

1. Check your hardware requirements. You want to make sure that your hardware is on the HCL before you begin the install. It will save you headaches later on. If you are ordering hardware from a vendor or a business partner they should have checked this for you but I like to do it once I am in front of the hardware just for the calming effect it gives me. Note that you want 64bit x86 CPUs with a minimum of 2Gb ram at a minimum. If you don’t have more than 2Gb of ram, you are not going to accomplish a lot but it will install with that little. url: http://www.vmware.com/resources/compatibility/search.php
2. Most modern servers come with boot optical media (CD\DVD) to help you set the date and time, create the boot raid array and partition, and in some cases install the operating system with the correct device drivers. If your hardware is listed in the HCL from above the correct device drivers are installed on the VMware media. Thus you do not need the boot CD for the OS and device drivers but I find it helpful for creating the boot device and the like. Once I am finished creating the boot partition I remove the manufactures media and then insert the VMware media to begin.
3. If you purchased the VMware software from IBM you will need to obtain the certificates you were shipped (most likely via Federal Express) and go to the VMware Partner Activation Code Registration Web Site at url: http://www.vmware.com/ibm/code and enter your codes from the Activation Certificates. You can enter up to 20 codes at once.
4. Before you begin an install know the dns structure and ip configs. Do not start without a solid dns infrastructure.
5. Now insert the VMware media and boot the server.   
6. For some reason I have been in locations and worked on hardware that would recognize the install media when booting but once I finished going through the install prompts on the vSphere 4 DVD it would tell me to insert the install media. I’m thinking WHAT? Why cant you get to the install media…your booted off of it! Well to get around this I pressed the F2 key, after restarting the install, at the initial install menu and added the “askmedia” option. This option allows you to interactively select the location of the ESX installation media. This option is required if the image is hosted at an HTTP, FTP, or NFS location. I used the http option since I keep the ISOs on my MacBook….and the rest is history. Of course you have to get the media in a location where it can be accessed. I read all over the web where ESX jockeys use 8G thumb drives to do the install via USB but I don’t have an available drive right now.
7. I like to use the following partition sizes when setting up a VMware server.
   • Mount point, Format, Size
   • /boot, ext3, 1100 MB (you can’t change / set this during interactive install)
   • /, ext3, min 5.00 GB – prefer 12.00 GB
   • , swap, min 1200 MB – prefer 1600 MB
   • /home, ext3, min 2.00 GB – prefer 4.00 GB
   • /tmp, ext3, min 2.00 GB – prefer 4.00 GB
   • /usr, ext3, 6.00 GB
   • /var, ext3, 5.00 GB
   • /var/log, ext3, 2.00 GB
   • /opt, ext3, min 2.00 GB – prefer 4.00 GB
   • /vmfs, vmfs3, the remainder of your disk
8. Post install to-do list:
   1. Before you start the install it is a good idea to disconnect all external shared storage. If you did then now you need to reconnect those connections.
   2. Connect vSphere Client to host (not VC) & add extra users (the sudo users) to Administrators group.
   3. Create user account & add to sudoer file (visudo – add to “user privilege specification”). Sometimes there exist a need to have the root account able to ssh directly to the physical host. nano /etc/ssh/sshd_config
      • “nano /etc/ssh/sshd_config” on the physical host and find the line that starts with PermitRootLogin and change the no to yes. You can find this line about 2 pages down from the top. Save the file by first pressing Ctrl-O and then Enter. Exit with Ctrl-X.
      • restart the sshd service: service sshd restart
      • if needed you can modify the host firewall to allow this access by entering: “esxcfg-firewall -e sshServer” press enter and then “esxcfg-firewall -e sshClient” and then press enter.
   4. Install Putty.exe and winSCP (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html – http://sourceforge.net/projects/winscp/files/WinSCP)
   5. Test cables are in correct VMNICs: $ “watch -n 1 ‘sudo /usr/sbin/esxcfg­nics –l’” Note that the first 1 is a one and the second l is a L)
   6. Rearrange VMNICs in /etc/vmware/esx.conf if required (reboot required).
   7. Configure vSwitches.
   8. Adjust Service Console memory to 800MB (reboot required).
   9. Configure NTP (time) settings (http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1339)
   10. Patch
   11. Configure storage (& set DiskMaxLUN as required).
   12. Connect vSphere Client to VC, add new host, move to required cluster.
   13. License host.
   14. Enable Web access if required.