When I was building my System Center 2012 testing lab I installed SQL server in a virtual running on my HyperV systems I was controlling with SCVMM 2012 RC. I’ll admit it, it seemed like a good idea at the time. I put all of the System Center products into one database server with two instances running in a virtual on my HyperV systems controlled by SCVMM. The host that I put this database on was a quad processor AMD box with 8 gb of memory and pretty good drives so I figured this would be a good home for the various databases (it seemed like a good idea at the time but in retrospect this was Strike 1). The challenge hit me when I realized that I had bottlenecked on memory on my SQL server. At first I just increased the dynamic memory for that system but eventually it took all available memory which I had available from the host. I restricted the memory on the various SQL pieces but performance started really decreasing in the environment. My plan was to move this database to another HyperV system with more memory…. Here’s where my bad choices landed me in trouble. I do not have shared storage in my lab [Strike two] – each HyperV system is running it’s own local drives but they transfer pretty quickly because I have each of the hosts gigabit networked linked to each other. Now however I can’t migrate my virtual which has the SQL database from one node to another because the current host is an AMD system and my other HyperV hosts are all Intel systems. [Ouch, that was Strike 3]
Ok, at this point I’m pretty sure that this isn’t going to be good. I can’t move this virtual in SCVMM when it’s shut down because the SCVMM database is on it. And most likely I couldn’t move it in the first place because it’s the database which is running the SCVMM environment I’m trying to move it with. So I’ve got myself nicely in a catch-22.
What’s the resolution? It’s time to go old-school virtual moving.
0) Identify a host with sufficient resources (disk and memory in this case) to be the new home for this server.
1) Shut down all virtuals which will be accessing the database (including the SCVMM console)
2) Log into the HyperV system which is running the virtual, connect to the virtual with Server Manager and shut down the SQL Server.
3) Find where the VHD files are associated with this system and copy them to the new location (break for a long lunch here depending on the size of the VHD’s and the network connectivity levels).
[For the geek trivial pursuit, this is what it looked like on one of my hosts during the transfer in terms of the impacts on network and disk. As a geek I admit I like stress testing the network like this once in a while. It makes me feel better for actually having upgraded from a 100 MB to a 1 GB switch.]
4) In HyperV on the original server, renamed it to indicate it was the old version of the virtual by adding _OLD to the name.
5) Logged into the new HyperV server and created a new virtual with the same configuration as the original virtual.
6) Configured the new virtual to use Dynamic Memory and to start with 4 gb of memory and to expand to up to 12 gb of memory.
7) Started the SQL virtual on the new host. Logged in and configured the network adapter to the IP address which had originally been hard-coded for the system (replacing the original network adapter IP address).
8) Validated the SQL services were running and the SQL instances would connect through SQL Server Management Studio.
9) Restarted the SCVMM server from saved state and re-opened the SCVMM console. In SCVMM the original SQL database server appeared as _OLD on the original host. A little while later, the new database server appeared on the new host.
10) Re-activated all of the servers which were saved to disk prior to the migration. Validated functionality of the servers which were accessing the database and that performance was increased in the new configuration.
11) After validating functionality, the original VHD files for the database server could now be removed.
12) The next step will be to rebalance by moving some of my processor intensive virtuals from the current host which also has the SQL database server to the now empty HyperV host but that will wait a little while.
Start to finish on this my migration of this SQL server took about three hours. Another option would potentially have been to have disabled dynamic memory on the SQL virtual, exported it from one system and imported it to the other system. This might have simplified the process but would not have changed the time required significantly.
Summary: Even if you mistakenly put your SCVMM database on a virtual which SCVMM controls and you are unable to migrate the database server from one host to another in SCVMM that doesn’t mean that you can’t go back to the old way of moving around virtuals. It’s just not as much fun.