ccolonbackslash

Just another WordPress.com site

Monthly Archives: January 2011

Step-By-Step p2v with Microsofts SCVMM R2

A step by step guide for p2v’ing servers, specifically HP Proliant DL/ML’s with SCVMMR2, where as mentioned in a previous post there can be issues during conversion. I’m pretty sure it can be applied to any server brand, but it’s mainly HP hardware that I deal with day to day. The document is based on our environment also hence references to ridiculously tiny c: partitions used on initial 2008 installs and conservative vm sizing references.

Introduction

Although not the most ideal long-term solution, p2v’ing a physical server can often be a sensible way to free up hardware, consolidate services and squeeze a little more life out of an aging installation.

Microsoft’s SCVMM R2 offers a robust p2v tool that fits most needs, however, like any p2v tools, used carelessly it can generate useless vm’s, damage other network services and even cause conflicts with its original physical host when that’s reused.

This document is intended to approach the P2V process from two perspectives; both processes are pretty much identical bar the last few steps.

  • P2v a production box in order that it continues to perform its original service with its original identity
  • P2v a production box in order to have a clone of a production machine on which patching, changes, upgrades etc. can be performed without impacting the production environment.

This guide is based on p2v’ing a file server, not an application server, there are additional considerations for these, particularly those that write to databases or provide messaging/real-time services.

(some) Virtualisation best practices:

 

Not all services are suitable for virtualization, but if hosts and services are chosen carefully and maintained thoroughly there is no reason a virtual machine can’t perform as well as its equivalent physical machine.

  • Clustered VM hosts cannot start without a DC – bear this in mind if you are considering virtualizing DC’s.
  • It is not usually a good idea to p2v a DC or a database server without significant planning and testing.
  • Ensure host layouts are planned so that potentially contentious machines are not sharing spindle sets or LUN’s unless they are designed appropriately.
  • Be conservative when initially allocating resources, they can always be added later if performance is inadequate.
  • Always plan for a way back with p2v, don’t reuse the physical box immediately, unplug it for a few business days first and make sure you have backups from before the start of the procedure.

P2v’ing a Production box for continued use:

Firstly check the environment is ready to host the existing production box,  that the host has sufficient processing capacity, memory and access to disk space in light of the other VM’s its hosting.

Secondly ensure that unwanted data is removed and that the box to be converted is healthy and patched. If you are remotely p2v’ing the machine ensure you have ilo access both to the donor physical machine and the VM host and that ILO is tested. MAKE A NOTE OF PHYSICAL MACHINES IP SETTINGS as they will be lost given physical nic will be removed and replaced by a new piece of (virtual) hardware.

 

Once the above is complete:

1.       Load SCVMM R2 and select “convert a physical server”, in the resulting dialogue select the machine you want to convert and enter suitable administrative credentials.

2.       Next, define the name of the converted machine (same as original) and add a description.

3.       Once this is complete, SCVMM will scan the machine to be converted and generate a summary of it’s storage, OS and network adaptors. Click Next.

4.       Now the SCVMM agent is installed It will assess the size of the data on the disks to be converted and give you the option to define the size of the VHD’s to host the data and whether or not they should be dynamically expanding. This is a good time to increase the size of the C: drive if it’s one of the early 2008 servers with a 30gb partition (see screenshot). For development or very low utilisation servers, dynamically expanding VHD’s are a good bet, for production it’s better to assign all space at the start. Just ensure you know the sum of all possible data that could fill a drive. I also prefer to select shut down physical machine when conversion is complete from the conversion options given we have ilo access if ther e is a need to switch it back on again.

5.       Next step is to choose the amount of memory and number of processor cores you are assigning to your virtual machine, again it depends on required performance, but for non-database servers I’d keep things modest – a single processor and 2-4gb RAM.

6.       SCVMM R2 will now attempt to recommend a hyper-v host server for your converted machine and flag any warnings. Choose the most appropriate host and click next.

7.       Now pick the volume on the host where the VM should be placed, if you click browse an unusually helpful dialog box appears outlining free space on each drive, ensure the path driveletter:\VirtualMachines is used for storage and add the path to the default paths for that host if offered.

8.       When p2v’ing it is better to start the machine with the nic’s disconnected from the lan, after conversion the converted machine has the same mac as the existing which can cause ARP conflicts and other problems, there are also a number of steps that need to be completed before the VM starts smoothly and can replace the existing physical machine and then be connected to the LAN.

9.       At this stage it is recommended that when defining actions for the physical host starting and stopping you select automatically start as disabled and that the machine is shut down when the physical host starts.

10.   Last step is that a quick issues check is completed; there should be no issues, if there are resolve them before continuing. Then click next.

11.   A summary page is now presented with the option to review the PowerShell script used to start the job and the option to automatically start the machine after converting, usually this isn’t necessary.

12.   Click “create” to start the task and a summary window is presented as below, you can monitor progress of the process here, particularly important is the completion of the transfer of the actual disk image and the point at which most HP Proliant conversions fail (around 58%).

13.   Once the disk transfer is completed, SCVMM will attempt to make the machine virtualizable and install vm components. If P2v’ing a proliant, I have found this will usually fail and eventually the machine will lock due to HP software conflicts/locks.  Log onto the physical host machine and open the hyper-v manager to check: be sure to switch off existing physical machine before continuing as SCVMM will not power it off independently if the conversion process does not finish – which in this case, it hasn’t.

14.   Once the conversion appears to have failed (it will stick on making OS virtualizable) power the VM off from the hyper-v console on the host and go into settings to change the NIC MAC from static to dynamic or your new vm will have the same mac as the existing (reusable….) physical server, do not connect the nic at this time.

15.   Now boot the converted machine from the hyper-v console on the host and hit f8 to boot it in safe mode.

16.   Once server comes up, log on as local admin (.\administrator) the mouse will not work as the integration services install failed and it’s in safe mode. When it does start it’ll install the vm integration drivers, eventually mouse will start working and the box will want to restart, do not restart now,  go into service manager and disable ALL HP services and restart the machine.

17.   Once complete the services viewer should display all HP services as disabled.

18.   Restart the server, log on as .\administrator, it should come up fine. Once logged in go to programs and features and carefully remove all HP software from top to bottom.

19.   Following this you should have a nice short list of installed apps.

20.   Now go to the NIC IP settings, they’ll be reset to dynamic as hardware is different. Set them back to appropriate static settings for server in question.

21.   Given an existing physical nic had the same settings you’ll be asked to confirm you want this to happen, obviously answer yes then shut the machine down.

22.   Go into the settings for the VM you’ve converted and change the nic settings from disconnected to appropriate LAN connection. Then start the machine.

23.   Now test everything works as expected, check event logs, backups, printing, access replication, whatever’s appropriate.

P2v’ing a Production box for use as a clone:

Firstly check the environment is ready to host the clone, that the host has sufficient processing capacity, memory and access to disk space in light of the other VM’s its hosting.

Secondly ensure that unwanted data is removed and that the box to be converted is healthy. MAKE A NOTE OF PHYSICAL MACHINES IP SETTINGS as they will be lost given physical nic will be removed and replaced by a new piece of (virtual) hardware.

Once the above is complete follow the instructions for p2v’ing a box for continued use up to the end of step 21 (do not connect box to production network).

1.       At this point it’s important to anonymise the machine and remove it from the network, SO, it’s here that we run sysprep on the machine in question.

2.       Go to Windows\System32\sysprep and run sysprep.exe

3.       You will be presented with a dialogue box, select oobe and generalize, then shut down

4.       Before booting the machine, connect the nic to the network, then proceed with basic setup, renaming and adding to the domain. By doing this you prevent both Mac conflicts and SID conflicts.


P2V using SCVMM R2 Introduction 

 

Although not the most ideal long-term solution, p2v’ing a physical server can often be a sensible way to free up hardware, consolidate services and squeeze a little more life out of an aging installation.

Microsoft’s SCVMM R2 offers a robust p2v tool that fits most needs, however, like any p2v tools, used carelessly it can generate useless vm’s, damage other network services and even cause conflicts with its original physical host when that’s reused.

This document is intended to approach the P2V process from two perspectives; both processes are pretty much identical bar the last few steps.

·P2v a production box in order that it continues to perform its original service with its original identity

·P2v a production box in order to have a clone of a production machine on which patching, changes, upgrades etc. can be performed without impacting the production environment.

This guide is based on p2v’ing a file server, not an application server, there are additional considerations for these, particularly those that write to databases or provide messaging/real-time services.

(some) Virtualisation best practices:

 

Not all services are suitable for virtualization, but if hosts and services are chosen carefully and maintained thoroughly there is no reason a virtual machine can’t perform as well as its equivalent physical machine.

·Clustered VM hosts cannot start without a DC – bear this in mind if you are considering virtualizing DC’s.

·It is not usually a good idea to p2v a DC or a database server without significant planning and testing.

·Ensure host layouts are planned so that potentially contentious machines are not sharing spindle sets or LUN’s unless they are designed appropriately.

·Be conservative when initially allocating resources, they can always be added later if performance is inadequate.

·Always plan for a way back with p2v, don’t reuse the physical box immediately, unplug it for a few business days first and make sure you have backups from before the start of the procedure.

P2v’ing a Production box for continued use: 

 

Firstly check the environment is ready to host the existing production box, that the host has sufficient processing capacity, memory and access to disk space in light of the other VM’s its hosting.

Secondly ensure that unwanted data is removed and that the box to be converted is healthy and patched. If you are remotely p2v’ing the machine ensure you have ilo access both to the donor physical machine and the VM host and that ILO is tested. MAKE A NOTE OF PHYSICAL MACHINES IP SETTINGS as they will be lost given physical nic will be removed and replaced by a new piece of (virtual) hardware.

Once the above is complete:

1.Load SCVMM R2 and select “convert a physical server”, in the resulting dialogue select the machine you want to convert and enter suitable administrative credentials.

2.Next, define the name of the converted machine (same as original) and add a description.

3.Once this is complete, SCVMM will scan the machine to be converted and generate a summary of it’s storage, OS and network adaptors. Click Next.

4.Now the SCVMM agent is installed It will assess the size of the data on the disks to be converted and give you the option to define the size of the VHD’s to host the data and whether or not they should be dynamically expanding. This is a good time to increase the size of the C: drive if it’s one of the early 2008 servers with a 30gb partition (see screenshot). For development or very low utilisation servers, dynamically expanding VHD’s are a good bet, for production it’s better to assign all space at the start. Just ensure you know the sum of all possible data that could fill a drive. I also prefer to select shut down physical machine when conversion is complete from the conversion options given we have ilo access if ther e is a need to switch it back on again.

5.Next step is to choose the amount of memory and number of processor cores you are assigning to your virtual machine, again it depends on required performance, but for non-database servers I’d keep things modest – a single processor and 2-4gb RAM.

6.SCVMM R2 will now attempt to recommend a hyper-v host server for your converted machine and flag any warnings. Choose the most appropriate host and click next.

7.Now pick the volume on the host where the VM should be placed, if you click browse an unusually helpful dialog box appears outlining free space on each drive, ensure the path driveletter:\VirtualMachines is used for storage and add the path to the default paths for that host if offered.

8.When p2v’ing it is better to start the machine with the nic’s disconnected from the lan, after conversion the converted machine has the same mac as the existing which can cause ARP conflicts and other problems, there are also a number of steps that need to be completed before the VM starts smoothly and can replace the existing physical machine and then be connected to the LAN.

9.At this stage it is recommended that when defining actions for the physical host starting and stopping you select automatically start as disabled and that the machine is shut down when the physical host starts.

10.Last step is that a quick issues check is completed; there should be no issues, if there are resolve them before continuing. Then click next.

11.A summary page is now presented with the option to review the PowerShell script used to start the job and the option to automatically start the machine after converting, usually this isn’t necessary.

12.Click “create” to start the task and a summary window is presented as below, you can monitor progress of the process here, particularly important is the completion of the transfer of the actual disk image and the point at which most HP Proliant conversions fail (around 58%).

13.Once the disk transfer is completed, SCVMM will attempt to make the machine virtualizable and install vm components. If P2v’ing a proliant, I have found this will usually fail and eventually the machine will lock due to HP software conflicts/locks. Log onto the physical host machine and open the hyper-v manager to check: be sure to switch off existing physical machine before continuing as SCVMM will not power it off independently if the conversion process does not finish – which in this case, it hasn’t.

14.Once the conversion appears to have failed (it will stick on making OS virtualizable) power the VM off from the hyper-v console on the host and go into settings to change the NIC MAC from static to dynamic or your new vm will have the same mac as the existing (reusable….) physical server, do not connect the nic at this time.

15.Now boot the converted machine from the hyper-v console on the host and hit f8 to boot it in safe mode.

16.Once server comes up, log on as local admin (.\administrator) the mouse will not work as the integration services install failed and it’s in safe mode (probably due to existing ILO or other HP drivers/software). When it does start it’ll install vm components, eventually mouse will start working and want to restart, DO NOT RESTART now go into service manager and disable ALL HP services and restart the machine.

17.Once complete the services viewer should display all HP services as disabled.

18.Restart the server, log on as .\administrator, it should come up fine. Once logged in go to programs and features and carefully remove all HP software from top to bottom.

19.Following this you should have a nice short list of installed apps.

20.Now go to the NIC settings, they’ll be reset to dynamic as hardware is different. Set them back to appropriate static settings for server in question.

21.Given an existing physical nic had the same settings you’ll be asked to confirm you want this to happen, obviously answer yes then shut the machine down.

22.Go into the settings for the VM you’ve converted and change the nic settings from disconnected to appropriate LAN connection. Then start the machine.

23.Now test everything works as expected, check event logs, backups, printing, access replication, whatever’s appropriate.

P2v’ing a Production box for use as a clone:

Firstly check the environment is ready to host the clone, that the host has sufficient processing capacity, memory and access to disk space in light of the other VM’s its hosting.

Secondly ensure that unwanted data is removed and that the box to be converted is healthy. MAKE A NOTE OF PHYSICAL MACHINES IP SETTINGS as they will be lost given physical nic will be removed and replaced by a new piece of (virtual) hardware.

Once the above is complete follow the instructions for p2v’ing a box for continued use up to the end of step 21 (do not connect box to production network).

1.At this point it’s important to anonymise the machine and remove it from the network, SO, it’s here that we run sysprep on the machine in question.

2.Go to Windows\System32\sysprep and run sysprep.exe

3.You will be presented with a dialogue box, select oobe and generalize, then shut down

4.Before booting the machine, connect the nic to the network, then proceed with basic setup, renaming and adding to the domain. By doing this you prevent both Mac conflicts and SID conflicts.

(semi) portable, externally accessible clustered Hyper-V lab

I’ve been using a portable lab for writing documentation and migration testing thats been working really well, the requirements were as follows:

  • Entire lab should weigh less than 12 kg
  • Storage solution be fast enough to be usable under load and support persistent reservation for CSV’s
  • Should be able to test pretty much any sensible scenario on two laptops.
  • Have storage of sufficient speed and quality that lab could be used for migrations or clones of production machines or for transferring machines between sites.

For the virtualisation platform the lab uses a pair of HP Elitebooks, one 2540p and an 8530w (until i can swap for something smaller), the 2540 is an i7 and the 8530 a Core2duo at 2.4 ghz, both have 8gb RAM and run Server 2008 R2 Datacenter.

For storage i purchased a Synology ds409 slim NAS that weighs about 700g and takes up to four 2.5″ hdd’s. I upgraded the firmware to the latest beta version of DSM which seems to support persistent reservations. I configured it with 3x500gb 7200rpm drives in RAID5 as a block target with a fourth disk for file storage, iso’s, sysprepped images etc. Very very pleased with this particular piece of kit. Not the cheapest but works fabulously for the size.

For connecting the lab together i found an 8 port SMC gigabit switch for around 300hkd.

Internet access is through a d-link DIR412 portable 3g/ethernet router, plugged into the above directly and to the internet with an unlocked Huawei hsdpa dongle. When traveling i purchase a pay as you go 3g sim for the lab that i also use with a flashed Orange San Francisco running as a pocket wifi hotspot for mobile data for laptop and blackberry instead of paying roaming fees.

Additionally i make use of two levelone usb ethernet adaptors when needed, the windows 7 drivers work fine on 2008 R2.

So far it’s been used to stage Exchange 2007 > 2010 migration, OCS 2007 R2 to Lync migration, TMG/UAG testing with Checkpoint R71, full DR lab for directory and exchange, an Orion/SCOM comparison, an SCCM image deployment lab, and a host of install documentation.

P2v of hp g6 proliant servers with SCVMM R2 failing on installing integration tools

Recently i’ve tried to p2v several servers for sandboxes and found all failed at around 70% when converting with SCVMM R2, specifically when installing the integration tools. All of these physical servers were based on G6 HP Proliant Servers, DL380’s and 360’s.

Each time the machines drives converted ok, but after a long time-out on the integration tools at 70% they’d fail. The VM would boot afterwards but with processor stuck at 12% when viewed through the management console and 30 second waits for responses to keypresses, additionally there was no mouse support, even on 2008 R2 server conversions with their native hyper-v tools. It’s related to the current Proliant support pack and the lack of HP hardware after the boot of the VM, but I specifically suspect the HP ILO support (with extra mouse/keyboard drivers) may be part of the cause but have yet to investigate properly.

In the meantime, best I’ve found to resolve this is to:

  • boot the newly created VM in safe mode using f8 when it’s switched on
  • Disable ALL HP services
  • Restart the server (you should now have mouse/keyboard access)
  • Uninstall all HP software
  • Install integration services.

Your converted VM should boot and perform fine now.

AZFF4YBXVU6

 

Freesco router for VMware

If you aren’t already using FREESCO in your sandbox it’s a tremendous application and allows you to stage  multisite application implementations and upgrades easily in fairly conservative virtual environments.

This link details setting the application up to work properly in vmware. Alternately just go here and download a working OVF: http://www.screencast.com/users/esloof/folders/FREESCO

http://dingetje.homeip.net/dokuwiki/freesco/howtos/vmware?DokuWiki=677cd5ee9680aa906eaec634c3f2b3fb

It’s really useful for firewall labs and topology changes etc. The VM itself uses practically no resources (its perfectly happy with 12-16mb), is hugely robust and has a mind-boggling array of features. Combined with the VMWare Workstations bandwidth limitation for the virtual networks possible to look at how applications perform over varying quality connections to other sites.

Other FREESCO links:

Home Pages: http://www.freesco.org/ http://www.freesco.info/

Download page: http://freesco.sourceforge.net/

 

Lync Server with Polycom phones VM Lab.

We got a couple of Polycom phones for Lync for review, set them up off the back of a lab in VMWare Workstation. Struggled to figure how to configure it physically given i needed it to be portable and all run off my laptop whilst i was also connected to my network, to get round this i used a usb ethernet adaptor bridged to one of the vmnet nics to connect a switch to the sandbox network and then the phones into that.

The voice quality seems superb, and the automation/integration with the Lync desktop client works lovely.

Free/Busy Exchange 2007 unavailable for single user due to infinitely recurring appointments

I’ve just spent two days dealing with a free/busy calendar viewing issue, combined with an inability to accept recurring appointments on an iPhone connected over ActiveSync to Exchange 2007.

The symptoms were this:

  • No one could view this particular execs free busy information from either Outlook 2010 or OWA on Exchange 2007. Everyone elses freebusy was viewable by all staff. This issue had been extremely long-standing, the users mailbox had been migrated up from Exchange 4 and we’d previously dealt with issues of non-inherited permissions caused by a cretinous admin giving him domain admin permissions to resolve “problems”.
  • Every so often when this users iPhone synced or he accepted an appointment he received “Synchronization with your iPhone failed for” blah blah “this appointment is still viewable in outlook or OWA”.
  • He was most distressed.

Initial attempts to solve this had been to look at logs, run outlook.exe /cleanfreebusy. This made no odds, as i later discovered this was due to this being a legacy command that cleans public folder free busy not availability service free busy (please correct me if I’m wrong).

I did some sniffing around without knowing about the availability services on Exchange 2007 and discovered that legacy free busy was supposed to be submitted to public folders and used outlook 2003 to try to view this users free busy and it worked fine.

Further investigation revealed that the only calendar items that were failing to be accepted on the iPhone were recurring appointments, further to that the only ones that seemed to be having an issue were those with no end date. I tried updating these to have an end date but then it seemed there was another issue here, when i tried to send the update i received a red pop up: “Can’t open this item. The property does not exist”. Clicking ok to this and then pressing send update seemed to work, but when i checked on the calendar items in list view it had updated the recurrence to end in the year 4500.

Googling revealed this dead thread:

Seemed similar but no conclusions. And this which was nearer.

I looked at the permissions on the mailbox and found NT Authority\Self was missing from both full access and send as, and corrected this using add-mailboxpermissions. I also cleaned up a load of other sledgehammer permissions fixes applied by someone in times of yore and restarted the IS.

This didn’t help so was nothing to do with the bodged permissions (seems silly to have thought it was now). I then increased the exchange logging for all of the availability services as it seemed both the iPhone issue and the freebusy were probably down to the same thing, corrupt or damaged calendar items. You no longer have to do this through powershell as of sp2, this post helped me here:

Exchange SP2 diagnostic logging config

As soon as i upped the logging and checked the freebusy i saw error 4009’s on the exchange server for my problem user:

“Exception returned is: Microsoft.Exchange.Data.Storage.ObjectNotFoundException: Cannot open embedded message. —> Microsoft.Mapi.MapiExceptionNotFound: MapiExceptionNotFound: Unable to open property 0x3701000D”

Of course it was not able to identify the problem calendar item, however that’s where this tremendously useful post from Nuttin But Exchange came in handy as it references a different problem with similar consequences but also links to a tremendously useful cmdlet from msexchangeteam.com that will scan a calendar and tell you on what days problem appointments are allowing you to go in and correct/delete. You can get this script here. You’ll also need EWS on the Exchange server if it’s not installed already and the machine running the cmdlet must be on PowerShell/WinRM 2.0. Given EWS has been updated since the cmdlet was written you’ll need to change the version in the path referring to ver 1.0 in the script to 1.1 before it will work, see below:

“C:\Program Files\Microsoft\Exchange\Web Services\1.0\Microsoft.Exchange.WebServices.dll”
All syntax is in the comments in the cmdlet.

So after running this script we found 25 bad days in the calendar, went through them all with his PA and every single one of them was a recurring appointment with no end (or the year 4500), once corrected, freebusy was available again and the user in question was able to view those meetings in their iphone.

Glad to put this one to bed. Not sure why more users weren’t affected as can’t imagine he was the only iphone user with infinitely recurring appointments, or why these appointments would also break freebusy? Any comments welcomed.