Tuesday, March 25, 2014

Microsoft Monitoring Agent & Windows Server 2012 R2? Some Attention Required

When you run SCOM 2012 R2 and are monitoring Windows Server 2012 R2 based systems some additional action is required.

Case
The SCOM 2012 R2 Agent (Microsoft Monitoring Agent) becomes greyed out on the Windows Server 2012 R2 based systems.

Cause
A lock in the OS code is hit when the PowerShell modules that ran a certain PS script are unloaded.

Solution
Apply the February 2014 update rollup as described in KB2919394. Read this posting on the SCOM Engineering Blog about this particular issue for more details.

!!!Warning!!!
Please be careful with applying this update rollup in your production environment since these updates might introduce new unwanted issues. So TEST them first before rolling them out in your production environment. I take no responsibility for any issues at all.

Free Ebook: Microsoft System Center: Network Virtualization & Cloud Computing

Yesterday Microsoft press released a new FREE ebook all about Microsoft System Center: Network Virtualization and Cloud Computing.
image

As many companies are moving into the cloud networking becomes even more crucial in order to facilitate a smooth running hybrid IT environment. This book will aid you in the first steps by identifying some key usage and deployment scenarios for cloud computing, based on Hyper-V Network Virtualization (HNV) and multi-tenant software-defined networking (SDN) solutions.

This FREE ebook is available in these formats: PDF, Mobi and ePub. IT can be downloaded from here.

Upgrading SCOM 2012 SP1 To SCOM 2012 R2? DON’T Forget The AV Exclusions…

When upgrading from SCOM 2012 SP1 to SCOM 2012 R2 it’s easily overlooked, to update the antivirus exclusions as well.

However, this is really necessary since otherwise the AV software will block the upgraded SCOM 2012 R2 components (Management Servers and Agents alike), resulting in an unstable SCOM 2012 R2 environment which is bad and might take way too much time to troubleshoot.

With SCOM 2012 R2 new installation paths are used, for ALL SCOM components, check this posting of mine for more information about the new SCOM 2012 R2 Agent.

In order to cover both environments (while you’re upgrading) it’s Best Practice to enforce the AV exclusions for BOTH SCOM 2012 x versions. When you’re SCOM environment is totally based on SCOM 2012 R2, you can remove the AV exclusions for SCOM 2012 SP1.

What are the differences?
As stated in this KB article you can see pretty much has changed when looking at the paths of the folders to be excluded. For your convenience I’ve made a small summary and made the differences in R2 red:

SCOM Management Servers

  • SCOM 2012 SP1 
    %Program Files%\System Center 2012\Operations Manager\Server\Health Service State\*
  • SCOM 2012 R2   
    %Program Files%\Microsoft System Center 2012 R2\Operations Manager\Server\Health Service State\*

SCOM Agents

  • SCOM 2012 SP1
    %Program Files%\System Center Operations Manager\Agent\Health Service State\*
  • SCOM 2012 R2
    %\Program Files%\Microsoft Monitoring Agent\Agent\Health Service State\*

The only thing which stays the same is the SCOM Gateway Server. Both versions use the same installation path so no need to change the AV exclusions for those SCOM 2012 R2 servers.

Monday, March 24, 2014

SCOM 2012: Please Monitor My Home Systems

As we already know SCOM allows you to monitor anything. Whether we’re talking about stock rates or the level of coffee in your machine. SCOM can do it! Now a new level of monitoring is added to SCOM, so keep on reading.

A good and well respected friend of mine is writing a whole series about how to enable SCOM to monitor your home systems like the Nest Thermostat and the Flukso Energy Meter. But there is more coming up and with every new home device being monitored he’ll post a new article on his blog.

So ladies and gentlemen, please give the stage to Dieter Wijckmans and be amazed what SCOM can do WITH some good MP authoring!

Postings so far (will update this posting accordingly when new postings come out):

  1. Monitor your home with SCOM;
  2. Series about the Nest Thermostat MP.

Awesome what Dieter is doing here and a good showcase of the capabilities of SCOM when matched with good MP authoring skills.

Available For Free: Veeam Visio Stencils For VMware & Hyper-V

Veeam released a free collection of VMware and Hyper-V virtualization Visio stencils that can be used by ESX administrators, system integrators and datacenter managers.

These Visio stencils contain:

  • ESX and Hyper-V hosts;
  • Datacenters;
  • SCVMM;
  • Local storage, shared storage;
  • LUN;
  • VMs with status sign;
  • NICs;
  • Networks;
  • & more!

These Visio Stencils can be downloaded for FREE from here.

Updated MP: Windows Client 2000/XP/Vista/Windows 7 MP

A few days ago Microsoft released and updated version of the Windows Client 2000/XP/Vista/Windows 7 MP, version 6.0.7120.0. This MP isn’t a big update at all but a bug fix for the aggregate reports.

MP can be downloaded from here.

Thursday, March 20, 2014

Solving Domain Controller Issues: The Reverse Way To Recover From An USN Rollback

Issue
Ouch! In my test lab I had a serious issue, caused by myself. In my test lab I ran 16 VMs, two of them DCs, DC01 and DC02. DC01 is the owner all FSMO roles and is also the Enterprise Root CA.

The disk of DC01 was based on Hyper-V 2008 R2, so it wasn’t a VHDX file. Time to fix that I thought. So I converted the disk to VHDX. But for some reasons I decided to roll it back to VHD (that file was still present). And that’s when the troubles started since the DC looked upon this action as an unsupported roll back, also known as an update sequence number rollback, or USN rollback. And to be frank, I can’t blame the DC, only myself.

But now my whole AD infra was broken. The Netlogon service was paused because the DC itself had added an additional regkey (DSA not writable) in order to prevent replication with DC02.

On top of it, DC01 had also disabled it’s in- and outbound replication, as displayed in the Directory Service event log on DC01:

  • Event ID: 1115 > Outbound replication has been disabled by the user.
  • Event ID: 1113 > Inbound replication has been disabled by the user.

So I had called upon myself a serious issue, even though it was my own test lab!

Cause
My own STUPID actions!

Case solved!
Since this happened late in the evening I decided to leave it like that and take a new fresh look at it another day and time when I was fresh again. So this evening I finally cracked it by following a reverse way it’s normally done.

  1. I removed the earlier mentioned regkey so Netlogon service wasn’t paused anymore after a reboot.
  2. First I tried to use the normal way which is transferring the FSMO roles from the defect DC (DC01) to DC02. But that didn’t work well, even though I succeeded. DC02 was the owner of all FSMO roles BUT since replication was broken, DC01 still thought it was the owner as well. And when I switched off DC01 everything came to a halt, so DC01 was still in charge, even though it was broken.
  3. So now I had TWO defective DCs! DC01 was totally isolated because of the replication blockage, but DC02 couldn’t function WITHOUT DC01. So I feared the worst by removing DC01 completely from AD, making things even worse.
  4. But enabling replication on DC01 would make things bad as well since BOTH DCs thought to be the owner of all FSMO roles. On top of it all, DC01 is the enterprise root CA, so breaking that server would wreck my CA as well. Ouch!
  5. Finally I concluded at least ONE DC had to go, no matter what. And DC02 was ‘only’ a DC and nothing more. So DC02 had to go, no matter what.
  6. So I ran a forced DC demotion on DC02 which worked great. Afterwards I switched it off and marked it in Hyper-V as a demoted DC.
  7. Now I had to clean up the meta data, referring to DC02 on DC01. In order to do that I used this article from www.Petri.co.il. Which worked great as well. And I also cleaned up DNS (forward and reverse lookup zones) and cleaned out the Sites.
  8. So far so good. After a reboot of DC01 far less errors were shown in the Directory Service event log. But still two Events worried me: EventID 1115 and EventID 1113 Sad smile.
  9. Soon I learned about a tool, repadmin. However it was an outdated article, referring to Windows Server 2000. After some searching I soon found about an updated version working up to Windows Server 2008 R2. This tool is found in the Windows Server 2003 SP1 Support Tools. I downloaded it, ran the installer and YES the tool was installed as well. Time for the next step.
  10. After searching on the internet I found this TechNet Library all about the Repadmin commands, also for Windows Server 2012! I know I run Windows Server 2012 R2, but it gave me hope all wasn’t lost. This outdated (based on Windows Server 2003!) article showed me the commands to force replication. Now it was time to put it together.
  11. So I started an elevated cmd-prompt and run these commands:
    1. repadmin /showreps in order to see the current status of the replication. This is what I got back:

      Default-First-Site-Name\DC01
      DC Options: IS_GC DISABLE_INBOUND_REPL DISABLE_OUTBOUND_REPL
      Site Options: (none)
      DC object GUID: 541ca80e-cc21-4cd9-98cf-94fd2e0a73c5
      DC invocationID: def47bba-aeeb-4ce4-9d0b-1ce8b9c71606

    2. Time to kick some ass! So I ran this command repadmin /syncall
    3. And now it was time to remove the constraints by running these two command, one after the other: repadmin /options dc01 -DISABLE_OUTBOUND_REPL and repadmin /options dc01 -DISABLE_INBOUND_REPL. Both commands got feedback telling me the constraints were removed!
    4. Time to check it by running this command again repadmin /showreps. This is what I got:

      Default-First-Site-Name\DC01
      DC Options: IS_GC
      Site Options: (none)
      DC object GUID: 541ca80e-cc21-4cd9-98cf-94fd2e0a73c5
      DC invocationID: def47bba-aeeb-4ce4-9d0b-1ce8b9c71606

  12. AWESOME! The restraints are really gone. I emptied the Directory Service event log and rebooted the DC. When it was back again NO MORE ERRORS!
  13. Soon I rolled out a new server which I promoted to DC (DC03) and all is just fine now.

Recap
As you can see, I didn’t remove the problematic DC but removed the other one instead. And it worked out. Took me some time to figure it out though but I am glad I solved it.

Learned my lessons these days, starting with not to fiddle around with DCs since you can wreck them no matter how rock solid Microsoft has made them. Simply because Microsoft can’t protect your environment against stupid actions like the one I did.

However, I also learned how to troubleshoot deep AD issues as well, so that’s good and now I am happy about everything what happened since I’ve learned many new stuff.

Like an old manager once said to me: ‘I don’t worry when my people make mistakes. I start worrying when they don’t make mistakes anymore because those are the moments they don’t work and more important, learn!’.

Thursday, March 13, 2014

SCCM 2012 R2 Endpoint Protection: Updated Predefined Antimalware Policies

Since I am taking a deep dive into SCCM 2012 R2 lately I am also working with Endpoint Protection. Even though Microsoft provides several predefined antimalware policies it turns out some of them require an update in order to reflect the latest product versions.

Therefore I’ve decided to add the updated versions of some of those policies, aimed at these products:

  1. SCCM 2012 R2;
  2. SCOM 2012 R2 – Management Server roles;
  3. SCOM 2012 R2 – Agents (Microsoft Monitoring Agent!);
  4. SQL Server 2012.

For your convenience you can download them from my OneDrive.

Remark:
Please know that ALL sections are still present in these antimalware policies. I have only modified the Exclusion settings section and didn’t touch any other sections. Adjust them to your needs and requirements so your other antimalware policies – already in place – aren’t negatively affected.

Wednesday, March 5, 2014

Windows 8.1 & Deduplication? YES We Can!

Deduplication is an awesome Windows Server 2012 (R2) feature. Sometime ago two IT experts advised me to enable it on my test lab – running Hyper-V based on Windows Server 2012 R2 - and I must say I am impressed!

The amount of disk space saved on both SSDs I use for running my VMs is impressive. I am talking here about 100+ GB per SSD drive! And until now I experience no performance hits what so ever.

Wouldn’t it be nice to have dedup on Windows 8.1 as well?
I run multiple test labs. Besides the one at home I am blessed with a real powerful notebook with an SSD as well which I solely use for my VMs, based on the Windows 8.1 hyper visor. And it runs very well. However, the SSD is running out of free space. So wouldn’t dedup work here as well?

Yes, of course! However, out of the box, dedup isn’t available on Windows 8.x machines, since it’s a Windows Server 2012 feature only…

Along came Teh Wei King…
But when searching the internet for a solution I soon bumped into the blog of Teh Wei King. He runs a blog all about geeky stuff Smile. And YES! He has posted two articles all about enabling dedup on Windows 8 clients and Windows 8.1 clients.

Both postings are required since the first posting (enabling dedup on Windows 8 clients) tells you all about how to use the code and required files found in the second posting, all about enabling dedup on Windows 8.1 clients.

The dedup job is still running on my notebook but already 10 GB of additional free disk space is added!

All credits go to Teh Wei King of course. Thanks for sharing Teh Wei King!

One additional comment
In order to make sure dedup processes EVERY file run this command for the volume where you want to enable dedup for. In thise example dedup is enabled for volume E:

Set-DedupVolume E: -MinimumFileAgeDays 0

Tuesday, March 4, 2014

Cross Post: SCOM 2012x Agents Become Unresponsive On WS 2012 R2 DCs

This is a know issue: SCOM 2012x Agents become unresponsive on Domain Controllers based on Windows Server 2012 R2. I’ve seen this issue on many occasions on the TechNet forums and on the MVP mailing list this was a top issue.

Finally there is a fix for it! Kevin Holman has written an excellent posting about the cause and how to solve it (installing an optional hotfix). On top of it all, this issue can happen on any Windows Server 2012 R2 OS.

Go here for more information.