The first thing you should do about it is that, don't do it. There are many limitations, bad implications and restrictions involved into it.

In case, you are in a state that you cannot avoid launching an interactive process from Windows Service then you might want to read this article.

Launching an interactive application from a Windows service is different in Windows Vista

Introduction

To maintain isolation and provide rich user experience Windows uses sessions. When a user logs in a Terminal Services enabled computer a new session is created with each logon. Each session is associated with Logon LUID.

The interactive window station, Winsta0, is the only window station that can display a user interface or receive user input.

In Windows Vista and later Windows service run in a separate session.

In all the interactive sessions Winsta0 is created by default. A Window station acts as container to the desktop. Each session has at least one Window Station (default windows station "winsta0") and a Window station have three desktops

• Default desktop
• Screensaver desktop
• Winlogon desktop

Name of the window station is case in-sensitive however name of desktops are case sensitive.

Windows station and desktop is provided to secure UI and GDI objects.  UI and GDI objects are associated with Windows station and desktops and Window station and desktops have DACLs associated with them. They are considered as securable objects in Windows.

Note:

1. Windows station and desktop are secured. A user or a process running under that user's context can access them only if the user has got appropriate access rights to access it.
2. Every session has its own isolated Window station and desktops. You cannot obtain handle of a Window station and subsequently of desktops of a session other than your session. Therefore you cannot modify the DACLs or security of the window station or desktops across session.

Effect of session on launching a process

A process is associated with the access token of a user. This access token is compared against securable objects DACLs when the process tries to access it.  There are 14 type of securable objects present in Windows for example Files, folders and registry keys. For a process it is must to have an access token which is derived from a user under whose context the process is launched.

There are three most effective ways to launch a process using Windows APIs.

1)      CreateProcess

a.       Access token is derived from the parent. The process is launched in the session of the parent and access the window station desktop of the parent by default.

b.      You can change the desktop accessed by this process by populating lpDesktop member of STARTUPINFO structure. Make sure that parent process has appropriate access to that window station and desktop and they are in the same session of the parent process.

2)      CreateProcessAsUser

a.       You need a primary access token to pass as the first parameter of this API.

b.      LogonUser API can be used to get the primary access token of a user. LogonUser expects credentials of the user in plain text.

3)      CreateProcessWithLogonW and CreateProcessWithTokenW

a.       The CreateProcessWithLogonW and CreateProcessWithTokenW functions are similar to the CreateProcessAsUser function, except that the caller does not need to call the LogonUser function to authenticate the user and get a token. CreateProcessWithLogonW cannot be called from Local System context. Therefore if you have a service running in Local System context you cannot use  CreateProcessWithLogonW.

Session information is kept in User's access token. A process gets a session depending on the token used to launch it. The session information of an access token can be changed.

Note: A process cannot change the session id of its own token.

Both 2^nd and 3^rd method requires a user to login in the local system. The newly created process's access token will be checked against the DACLs of Windows Station and desktop while it will try to attach itself to them.  Also, as mentioned above Windows create a session on each logon. While launching the process by using method 2 and 3 either you would need to create and specify windows station and desktop for lunching process or Windows by default will create a Window station and desktop

For 1^st method, process is launched in the same session of parent process with same access token. By default this child process will have access to the Window station and desktop of the parent. If you are changing these two you would need to make sure that parent process have access to the new window station and desktop.

Process to launch the interactive process

Until Windows Server 2003 the services and the first logged on user used to run in the same session, Session 0. Therefore, the Window station and desktop of the first logged on user was shared by services running in the same session. Therefore services were able to obtain the handle of the user's desktop and modify its DACL.

Up to and including Windows 2003 you check the check box saying "Interact with desktop" in Properties dialog box of the Service and you can launch the process interactively.

With security enhancement in Windows Vista and later versions of Windows, services run in a separate session called Session 0. Now, in Session 0 a process can only get handle of Window station and desktop associated with session 0.

Let's categorize the process in two by two cases; we will discuss each of them. These cases are listed in decreasing order of feasibility.

1)       Your service is running in LocalSystem account context.

A.      You are willing to launch the application as currently interactively logged on user.

B.      You are willing to launch the application as some user other that interactively logged on user.

2)       Your service is running in any account other than LocalSystem account.

A.      You are willing to launch the application as currently interactively logged on user.

You are willing to launch the application as some user other that interactively logged on user.         

1.A. ) Your service is running in LocalSystem account and you are willing to launch the application as currently interactively logged on user.

This is how you do it:

Get the user token by calling

WTSQueryUserToken (WTSGetActiveConsoleSessionId (), &hToken) ;

Use this token in CreateProcessAsUser.

This token will have the right session ID of the interactive user. You will not have to change anything you are good to go.

1.B. ) Your service is running in LocalSystem account and you are willing to launch the application as some user other than interactively logged on user.

For an interactive application we need two essential things

1)      An access token with right session ID

a.       You need an access token of that user. You would need to call LogonUser to get the primary token. This call will create a new session and associate the session ID with token. Now, your token does not have the right Session ID. You can change it by calling SetTokenInformation. After this call you have the right session ID in the token.

2)      You need a Window station and desktop to associate with your application.

Newly created application will not have access right to the interactive session’s window station and desktop. This application cannot associate itself with interactive desktop.

Therefore, even though we have good token the launched process will not survive. It will be killed by Windows desktop manager because the desktop it is trying to access is not accessible to it.

You can change the permissions of the Window station and desktop. The only way to change the permission is to get the handle of the desktop. The process cannot obtain the handle of the desktop of session other than their session. This implies that service which is running in session 0 cannot obtain the handle of the desktop in the interactive session.

To workaround this you would need to perform following steps

a)      Launch a hidden temporary process as explained in step 1.A.

                                I.            Get the user token by calling WTSQueryUserToken (WTSGetActiveConsoleSessionId (), &hToken) ;

                              II.            Use this token in CreateProcessAsUser to launch the temporary process.

Call  CreateProcessWithLogonW from the temporary process to create the target application. CreateProcessWithLogonW saves you from explicitly modifying the DACL of the interactive desktop. However, you need the credentials of the target user to provide in this API.

2.A.) Your service is running in any other account and you are willing to launch the application as currently interactively logged on user.

You cannot call WTSQueryUserToken. This API can only be called from services running under LocalSystem account.

There is no definite way to do this because there is no legitimate way to obtain the logged on user's token with right Session ID. Here is what you should never, never and never do.

You would need to steal the access token of a process running on interactive user’s desktop. The best process to steal the access token would be Explorer provided only one interactive user is logged in. 

a)      Enumerate the process to get the handle of the Explorer.

b)      OpenProcessToken should give you the access token.
Note : The account under which your service is running must have appropriate privileges to call this API and get process token.

c)       Once you have the token call CreateProcessAsUser with this token. This token already have the right session Id.

2.B.) Your service is running in any account other than LocalSystem and you are willing to launch the application as some user other that interactively logged on user.

Same text goes again:: Here is what you should never, never and never do.

You would again need follow the technique we followed in 1.B . Launch a hidden application by performing

2.A.a, 2.A.b and 2.A.c

Once hidden application is launched follow 1.B.a- 1.B.f to launch the target application.

When interactive user log off

I have seen people saying that my application is running in a user's context which is not same as interactively logged in user, should my application be running when the interactive user logs off ?

When the user logs off, the system destroys the session associated with that user. Therefore the Windows stations and the desktops associated with that user will be destroyed. Even if your application is running in a different user's context it is attached to the interactive user's desktop.

All the applications running on the interactive desktop will be terminated. So is yours.

 However, you can expect a WM_QUERYENDSESSION and WM_ENDSISSION for your application.

Where is the code example ??

You got to write your own!!!

-Prateek

Launching an interactive process from a service used to be straight forward.  You could either configure your service to be interactive or you could specify “Winsta0\\Default” as the desktop (in CreateProcess API) and as long as the launched process had the appropriate permissions to the desktop, the launched process would run interactively.

These techniques stopped working with the introduction of Windows VISTA and continues today on Windows 8.  Beginning with Windows VISTA, Windows Services were isolated in their own session.  (Session 0).  This meant if you launched a process from a Windows Service, the process was going to run in session 0 and processes running in session 0 are not interactive.

The session the launched process would run in was determined by the user’s token.  If you set a different session ID in the token, the process would launch in that session.  You’ll notice that you won’t be able to use CreateProcess() any more since it is going to launch in the same session as the service.  Since you need to specify a token, you’ll need to call CreateProcessAsUser().  The other issue you need to address in order to launch a process interactively is that the process needs FULL permissions to the interactive desktop which is “Winsta0\\Default”.  There is a lot of old sample code (http://support.microsoft.com/kb/165194 - I actually wrote this article) out there that demonstrates how to modify the security permissions to grant a user access to the Interactive desktop. Unfortunately this code will not work anymore since you can’t modify the permissions for the desktop that is located in a different session.  You can only modify the desktop in the session where your code is running in.

The recommend way to launch an interactive process is to determine the session ID of the session you are targeting and to obtain the token of the interactive user in that session.  If you know the session ID, you can call WTSQueryUserToken() to obtain the token.  You can then launch a process into this session as this user.  You do need to have the SeTcbPrivilege in order to make the call to WTSQueryUserToken() so it’s typically something only a highly privileged user should do.

If you want to launch a process as a different user (other than the interactive user running in the session), the best way to do this is to have a process running in that session call CreateProcessWithLogonW().  This is equivalent to using the runas command. 

The Process.Start method in System.Diagnostics is equivalent to CreateProcess() in Win32. If you want to 

launch a process as a different user, in Win32 you either need to call CreateProcessAsUser, 

CreateProcessWithLogonW or CreateProcessWithTokenW. In .NET using Process.Start, you can use the 

“runas” verb to the same thing.

If you have a .NET Service configured with the LocalSystem account, the “runas” verb doesn’t work and 

you’ll encounter an exception and a failure of “access denied”. 

Unhandled Exception: System.ComponentModel.Win32Exception: Access is denied
 at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
 at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)

Internally what is happening is that .NET is using CreateProcessWithLogonW() to implement the 

“runas” verb. If you take a look at the remarks for CreateProcessWithLogonW(), you’ll find the following, 

http://msdn.microsoft.com/en-us/library/windows/desktop/ms682431(v=vs.85).aspx :

“You cannot call CreateProcessWithLogonW from a process that is running under the "LocalSystem" 

account, because the function uses the logon SID in the caller token, and the token for the "LocalSystem" 

account does not contain this SID. As an alternative, use the CreateProcessAsUser and LogonUser functions.”

The reason this is happening is that CreateProcessWithLogonW() is assuming that the desktop of the 

launched process is “Winsta0\Default”. This is known as the INTERACTIVE desktop. The security on this 

desktop is based on the Logon SID associated with the INTERACTIVELY logged on user. (This is the 

user who logged onto the system via CTRL-ALT-DELETE). The desktop associated with a LocalSystem 

service is a hidden desktop. When CreateProcessWithLogonW() attempts to add the Logon SID from 

the current user (LocalSystem), it won't find a LOGON SID (since LocalSystem doesn't have one). 

This will cause CreateProcessWithLogonW() to fail with error code 5 or "access denied". Not only is 

this an issue, CreateProcessWithLogonW() is assuming it is modifying "Winsta0\Default" which it isn't.

You may run into a situation where you need to create a token to be used at a later time since you can’t keep the credentials around in your application.  The token representing the user is going to be used to launch a process.  In the past, this would be done by generating the token with LogonUser() and then launching the application with CreateProcessAsUser().  In order to launch an INTERACTIVE process, you need to modify the INTERACTIVE Window Station and Desktop (referred to as the INTERACTIVE desktop) to grant access to the user,   http://support.microsoft.com/kb/165194.  (The KB article was written by me).  The weakness in this technique is that you are responsible for the DACLs on the INTERACTIVE Window Station and Desktop.  Since this a SYSTEM named object, the SYSTEM (and other processes) may not be aware of the changes that you have made.   If the SYSTEM or another process makes a change to the DACLs and is unaware of your changes, your user will not have any more access to the INTERACTIVE desktop.

A better technique for granting access to the user is to use the technique used by CreateProcessWithLogonW().  CreateProcessWithLogonW() internally grants access to the user by obtaining the LOGON SID of the current INTERACTIVELY logged on user and adding this LOGON SID to the user specified in CreateProcessWithLogonW(). 

What you can do is launch a temporary process suspended.  CreateProcessWithLogonW() will go through the steps of creating the token with the appropriate permissions to the INTERACTIVE desktop.  You can then obtain this token via a call to OpenProcessToken().  You can then get your own copy of the token by calling DuplicateTokenEx().  Once you have the token, you can get rid of the suspended temporary process.

At a later time, when you need to launch a process as a specific user, you can call CreateProcessWithTokenW() to launch the process.

The above solution will not work on Windows XP since CreateProcessWithLogonw() isn’t supported. You may be asking why you don’t just call CreateProcessAsUser() instead of CreateProcessWithTokenW() since you have a token from CreateProcessWithLogonW().  The reason is because CreateProcessAsUser() requires the “Replace a Process Level Token” right for the caller which even Administrators do not have by default.  (This avoids granting any additional rights to the caller for this technique to work).

Follow us on Twitter, www.twitter.com/WindowsSDK.

Job Objects were introduced to Windows in Windows XP/Windows Server 2003 to allow an application to manage a group of processes.

One of the limitations of Job Objects is that a process can only belong to a single Job object.  This becomes an issue when you are attempting to manipulate a process which is already associated with a Job Object.  A process can be associated with a Job object through inheritance or via assignment (AssignProcessToJobObject API).  If you call AssignProcessToJobObject() to assign a Job to your process and it is already associated with a Job Object, the API will fail with error code 5 or "Access Denied".  (Note, error code 5 could have another meaning).

It turns out that Windows uses Job Objects for various scenarios such as:

• A process launched via the Run As Command (This is implemented by calling CreateProcessWithLogonW. CreateProcessWithTokenW() is a similar API except for it uses a token for the user's identity)
• Remote Desktop Client setting (mstsc.exe), "Start a Program"
• The Task Scheduler

Based on the way Job Objects are designed, it may not be possible to disassociate a process from a Job.  (This is referred to as BREAKING AWAY)

If the Job wasn't created with JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK or if the child wasn't launched with CREATE_BREAKAWAY_FROM_JOB + JOB_OBJECT_LIMIT_BREAKAWAY_OK, the process will not be able to disassociate from a Job.

If you can obtain a handle to the the Job Object, you could change the future behavior of the child process so that it could be broken away.  This requires the Job object to be named and the caller has to have the permissions to modify the Job object.  (This may not always be possible).

To address this limitation on Windows 8/Windows Server 2012, you can associate a process with more than 1 Job via a Nested Job.

See the following for more information on Nested Jobs:

http://msdn.microsoft.com/en-us/library/windows/desktop/hh448388(v=vs.85).aspx

You've probably seen this error before in Windows at one time or another.  Somebody launched calc.exe and it failed to run.  The error doesn't tell you much.  It provides you an hex error and your only option is to terminate the application.

I first encountered this error over 20 years ago and unfortunately, it is still around today. 

There are primarily 3 actions that can cause this error:

• Launching lots of applications
• Launching an application as a different user
• Launching an application to a different desktop

This can happen from the user's desktop (the user who logged on by entering their credentials) or from a Windows Service.  Programmatically, you launch applications using the family of CreateProcess APIs (CreateProcessAsUser(), CreateProcessWithLogonW(), CreateProcessWithTokenW()). Another thing you have to look out for in the call is whether, a desktop was specified in the STARTUPINFO struture:

typedefstruct _STARTUPINFO {
  DWORD  cb;
  LPTSTR lpReserved;LPTSTR lpDesktop;
  .......
} STARTUPINFO, *LPSTARTUPINFO;

I'll come back to this later.

You'll see the API will succeed but if you check the exit code of the launched process (with GetExitCodeProcess()), it will return 128 or ERROR_WAIT_NO_CHILDREN, "There are no child processes to wait for."

If you are wondering what the error 0xc0000142 means in the above Message Box, you can find the error in ntstatus.h.  It is STATUS_DLL_INIT_FAILED or "{DLL Initialization Failed} Initialization of the dynamic link library %hs failed. The process is terminating abnormally."

If you debugged the launched process and reviewed why Windows wasn't loading the DLL, it makes perfect sense that there was no error with CreateProcess (or CreateProcessAsUser) and the error was seen in the exit code of the launched process.  CreateProcess's responsiblity is to find the exe and provide it to the Windows Loader, if this succeeds then CreateProcess() was done at this point.  If the process can't load the DLLs that are required for it to run, this isn't the responsibility of the loader but is a failure with the application itself.

Why is this happening because of these actions?

It comes down to Windows Stations & Desktops.  Before moving on, if you are not familiar with these objects, I suggest you read the following BLOG posts:

http://blogs.msdn.com/b/ntdebugging/archive/2007/01/04/desktop-heap-overview.aspx
http://blogs.technet.com/b/askperf/archive/2007/07/24/sessions-desktops-and-windows-stations.aspx

You can also read up on them in MSDN:

https://msdn.microsoft.com/en-us/library/windows/desktop/ms687098%28v=vs.85%29.aspx?f=255&MSPPError=-214721739

There are 2 issues you have to consider with Window Stations & Desktops:

• Desktop Heap Exhaustion - There is a limited amount of desktop heap (memory).

• Desktop Heap Security - When moving between users or desktops from the calling process, you have to ensure that the launched process has he appropriate permissions to the targeted process.

Let's first start with desktop heap exhaustion.  There have been a lot of Blog Posts and KB articles on this topic.  I will let you review the following links but I'll mention the following:

1. Services (Session 0) have smaller desktop heaps than the INTERACTIVE desktop.
2. Desktop Heap has increased with x64 systems.
3. Desktop Heaps are smaller in Remote Desktop Sessions

There used to be a nice utility, Desktop Heap Monitor Version 8.1 to determine the amount of heap being used on your system.  The only issue is that it was only supported on Windows 2000 and Windows Server 2003.

https://www.microsoft.com/en-us/download/details.aspx?id=17782

http://blogs.msdn.com/b/alejacma/archive/2008/07/29/how-to-use-dheapmon-exe-to-troubleshoot-desktop-heap-issues.aspx

The only way to do this now is through the Kernel Debugger or through indirect methods such as creating as many Window Stations & Desktops to determine how much space is left.

http://blogs.msdn.com/b/ntdebugging/archive/2013/02/27/debugging-a-debugger-to-debug-a-dump.aspx

Desktop Permissions is the other issue where you could encounter this error.  I touched this topic indirectly (following this BLOG post, you shouldn't encounter this issue) on the Windows SDK Support BLOG.  "How to launch processes INTERACTIVELY from a service" See the following:

http://blogs.msdn.com/b/winsdk/archive/2013/05/01/how-to-launch-a-process-interactively-from-a-windows-service.aspx

Years ago, I wrote the following KB article on how to deal with the issue by directly modifying the permissions on the Window Station and Desktop Object

"User32.dll or Kernel32.dll fails to initialize"
https://support.microsoft.com/en-us/kb/184802

I wouldn't recommend using this method anymore.

Today, the INTERACTIVE desktop permissions are accessed using the technique implemented in CreateProcessWithLogonW().  I discussed this in the following BLOG post:

http://blogs.msdn.com/b/winsdk/archive/2013/11/12/runas-verb-process-start-doesn-t-work-from-a-localsystem-net-service.aspx

I would recommend doing this to avoid directly modifying the permissions on a Window Station and Desktop controlled by the system since the system could make changes and negate yours and cause your application to not have any more permissions.

I will continue more posts on this topic in the future.

Windows 10 introduces a new concept (for Windows anyway) called Virtual Desktops.  Currently, the guidance for this on MSDN states:

The user can group a collection of windows together to create a virtual desktop. Every window is considered to be part of a virtual desktop. When one virtual desktop is hidden, all of the windows associated with it are also hidden. This enables the user to create multiple working environments and to be able to switch between them. Similarly, when a virtual desktop is selected to be active, the windows associated with that virtual desktop are displayed on the screen.

To support this concept, applications should avoid automatically switching the user from one virtual desktop to another. Only the user should instigate that change. In order to support this, newly created windows should appear on the currently active virtual desktop. In addition, if an application can reuse currently active windows, it should only reuse windows if they are on the currently active virtual desktop. Otherwise, a new window should be created.

That’s good advice as it makes for the best user experience in most cases and as a developer lets you ignore virtual desktops altogether in most simple applications; however, if you have an application or scenario that wants to do something such as always stay on top even when the user changes virtual desktops, what can you do?

IVirtualDesktopManager

To go along with the addition of virtual desktops in Windows 10, a new shell interface was introduced called IVirtualDesktopManager.  It only has three functions, but those allow you to do many things with virtual desktops and your own application.  Attempting to say move a window to another virtual desktop with these functions will not work for windows that your process doesn’t own.  As this isn’t a scenario that should be common or desired behavior for most applications, there’s isn’t a notification that you can subscribe to so that you know that your application window’s virtual desktop is no longer visible or that your application window has been moved to a new virtual desktop.  However, if your window has focus when the user switches to another virtual desktop, you will be told that you’ve lost focus.

IsWindowOnCurrentVirtualDesktop will tell you if your window is on the current virtual desktop.  GetWindowDesktopId will give you the ID of the desktop the specified window is on.  MoveWindowToDesktop will allow you to move a specified window to a specified desktop.

But how do you know what the current desktop ID is if you don’t have any windows on the current desktop?  That one turns out to be pretty simple.  If you create a new window with no parent, it will be placed on the current virtual desktop.

Demonstration

Putting all of the above together, here’s a straightforward C# WinForms app as an example of an always on top window that can move itself between Virtual Desktops (csproj attached at the end):

using System;
using System.Runtime.InteropServices;
using System.Windows.Forms;

namespace VirtualDesktopSwitch
{
    /// <summary>
    /// Example form
    /// </summary>
    public partial class VDExampleWindow : Form
    {
        public VDExampleWindow()
        {
            InitializeComponent();
        }
        private VirtualDesktopManager vdm;
        private void VDExampleWindow_Load(object sender, EventArgs e)
        {
            //Create IVirtualDesktopManager on load
            vdm = new VirtualDesktopManager();
        }

        private void label1_Click(object sender, EventArgs e)
        {
            //Show details on click
            MessageBox.Show("Virtual Desktop ID: " + vdm.GetWindowDesktopId(Handle).ToString("X") + Environment.NewLine +
                "IsCurrentVirtualDesktop: " + vdm.IsWindowOnCurrentVirtualDesktop(Handle).ToString()
                );
        }
        //Timer tick to check if the window is on the current virtual desktop and change it otherwise
        //A timer does not have to be used, but something has to trigger the check
        //If the window was active before the vd change, it would trigger
        //the deactivated and lost focus events when the vd changes
        //The timer always gets triggered which makes the example hopefully less confusing
        private void VDCheckTimer_Tick(object sender, EventArgs e)
        {
            try
            {
                if (!vdm.IsWindowOnCurrentVirtualDesktop(Handle))
                {
                    using (NewWindow nw = new NewWindow())
                    {
                        nw.Show(null);
                        vdm.MoveWindowToDesktop(Handle, vdm.GetWindowDesktopId(nw.Handle));
                    }
                }
            }
            catch
            {
                //This will fail due to race conditions as currently written on occassion
            }
        }

        /// <summary>
        /// Required designer variable.
        /// </summary>
        private System.ComponentModel.IContainer components = null;

        /// <summary>
        /// Clean up any resources being used.
        /// </summary>
        /// <param name="disposing">true if managed resources should be disposed; otherwise, false.</param>
        protected override void Dispose(bool disposing)
        {
            if (disposing && (components != null))
            {
                components.Dispose();
            }
            base.Dispose(disposing);
        }

        #region Windows Form Designer generated code

        /// <summary>
        /// Required method for Designer support - do not modify
        /// the contents of this method with the code editor.
        /// </summary>
        private void InitializeComponent()
        {
            this.components = new System.ComponentModel.Container();
            this.label1 = new System.Windows.Forms.Label();
            this.VDCheckTimer = new System.Windows.Forms.Timer(this.components);
            this.SuspendLayout();
            //
            // label1
            //
            this.label1.Dock = System.Windows.Forms.DockStyle.Fill;
            this.label1.Font = new System.Drawing.Font("Microsoft Sans Serif", 13.875F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((byte)(0)));
            this.label1.Location = new System.Drawing.Point(0, 0);
            this.label1.Name = "label1";
            this.label1.Size = new System.Drawing.Size(1112, 368);
            this.label1.TabIndex = 0;
            this.label1.Text = "Example Contents";
            this.label1.TextAlign = System.Drawing.ContentAlignment.MiddleCenter;
            this.label1.Click += new System.EventHandler(this.label1_Click);
            //
            // VDCheckTimer
            //
            this.VDCheckTimer.Enabled = true;
            this.VDCheckTimer.Interval = 1000;
            this.VDCheckTimer.Tick += new System.EventHandler(this.VDCheckTimer_Tick);
            //
            // VDExampleWindow
            //
            this.AutoScaleDimensions = new System.Drawing.SizeF(12F, 25F);
            this.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Font;
            this.ClientSize = new System.Drawing.Size(1112, 368);
            this.Controls.Add(this.label1);
            this.FormBorderStyle = System.Windows.Forms.FormBorderStyle.Fixed3D;
            this.Name = "VDExampleWindow";
            this.Text = "VD Example";
            this.TopMost = true;
            this.Load += new System.EventHandler(this.VDExampleWindow_Load);
            this.ResumeLayout(false);

        }

        #endregion

        private System.Windows.Forms.Label label1;
        private System.Windows.Forms.Timer VDCheckTimer;

        [STAThread]
        static void Main()
        {
            Application.EnableVisualStyles();
            Application.SetCompatibleTextRenderingDefault(false);
            Application.Run(new VDExampleWindow());
        }
    }
    [ComImport, InterfaceType(ComInterfaceType.InterfaceIsIUnknown), Guid("a5cd92ff-29be-454c-8d04-d82879fb3f1b")]
    [System.Security.SuppressUnmanagedCodeSecurity]
    public interface IVirtualDesktopManager
    {
        [PreserveSig]
        int IsWindowOnCurrentVirtualDesktop(
            [In] IntPtr TopLevelWindow,
            [Out] out int OnCurrentDesktop
            );
        [PreserveSig]
        int GetWindowDesktopId(
            [In] IntPtr TopLevelWindow,
            [Out] out Guid CurrentDesktop
            );

        [PreserveSig]
        int MoveWindowToDesktop(
            [In] IntPtr TopLevelWindow,
            [MarshalAs(UnmanagedType.LPStruct)]
            [In]Guid CurrentDesktop
            );
    }

    public class NewWindow : Form
    {
    }
    [ComImport, Guid("aa509086-5ca9-4c25-8f95-589d3c07b48a")]
    public class CVirtualDesktopManager
    {

    }
    public class VirtualDesktopManager
    {
        public VirtualDesktopManager()
        {
            cmanager = new CVirtualDesktopManager();
            manager = (IVirtualDesktopManager)cmanager;
        }
        ~VirtualDesktopManager()
        {
            manager = null;
            cmanager = null;
        }
        private CVirtualDesktopManager cmanager = null;
        private IVirtualDesktopManager manager;

        public bool IsWindowOnCurrentVirtualDesktop(IntPtr TopLevelWindow)
        {
            int result;
            int hr;
            if ((hr = manager.IsWindowOnCurrentVirtualDesktop(TopLevelWindow, out result)) != 0)
            {
                Marshal.ThrowExceptionForHR(hr);
            }
            return result != 0;
        }

        public Guid GetWindowDesktopId(IntPtr TopLevelWindow)
        {
            Guid result;
            int hr;
            if ((hr = manager.GetWindowDesktopId(TopLevelWindow, out result)) != 0)
            {
                Marshal.ThrowExceptionForHR(hr);
            }
            return result;
        }

        public void MoveWindowToDesktop(IntPtr TopLevelWindow, Guid CurrentDesktop)
        {
            int hr;
            if ((hr = manager.MoveWindowToDesktop(TopLevelWindow, CurrentDesktop)) != 0)
            {
                Marshal.ThrowExceptionForHR(hr);
            }
        }
    }
}

Follow us on Twitter, www.twitter.com/WindowsSDK.

VirtualDesktopSwitch.zip

This post is a follow up to the FileSystemWatcher Follies post.  I received a lot of feedback that it would be useful to highlight what would be appropriate to guide against some of the pitfalls that I mentioned in that post.  I’ll cover several of the issues here over a couple of posts and propose things that could be done to detect that they are there before using the FileSystemWatcher class against them.  Though the code examples will all be in C#, there will be some P/Invoke involved here as not all of this functionality is exposed through .NET Framework classes at this time.

Using Change Journals

If you’ve already determined that your path is local and uses the NTFS or ReFS file system, a great alternative to the FileSystemWatcher is to use change journaling.  Change journals can be complicated, but they also give you very fine grained control over the information that you want.  However, your code must be running as an administrator or system in order to create or delete them, and they do take up some space on disk (the maximum amount that will be taken up can be specified).    Because change journals monitor an entire volume, if you’re designing an application to make optimal use of this functionality for consistent change monitoring, you may want to put the data that you’re consistently monitoring for changes on its own volume.

Other things to keep in mind when using change journals:

1. Changes for files and directories are not full paths; parent directories are identified by IDs and those directory names can be looked up by OpenFileByID amongst other methods
2. If BytesToRead is set to zero, it will immediately return with up to one entry; otherwise it will wait until that many bytes are filled in to the buffer or the specified timeout value.  If you want to get immediate notification
3. It does not work on network file paths.
4. All of the functionality works through the use of DeviceIOControl; consult the documentation for the structure type and enumeration value for additional details about how to use that value.

Basic Change Journal Wrapper

Below is sample code for a basic change journal class which monitors a volume for changes.  The changes monitored for are specifiable and the types available are included as an enumeration.  The values are hardcoded to only show file creation and delete events.  If the buffer size is set to anything less than 1024, the sample will use zero for BytesToRead to immediately return upon receiving each entry.

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.Win32.SafeHandles;
using System.Runtime.InteropServices;
using System.IO;
using System.Threading;

namespace ChangeJournal
{
    public class ChangeJournalHandle : SafeHandleMinusOneIsInvalid
    {

        [DllImport("kernel32", CharSet = CharSet.Unicode, SetLastError = true)]
        private static extern IntPtr CreateFileW(
            [MarshalAs(UnmanagedType.LPWStr)]
            string FileName,
            int DesiredAccess,
            FileShare ShareMode,
            IntPtr SecurityAttributes,
            int CreationDisposition,
            int FlagsAndAttributes,
            IntPtr hTemplateFile
            );

        [DllImport("kernel32", CharSet = CharSet.Unicode, SetLastError = true)]
        private static extern int GetVolumeInformationByHandleW(
  IntPtr hFile,
  StringBuilder lpVolumeNameBuffer,
  int nVolumeNameSize,
  out int lpVolumeSerialNumber,
  out int
     lpMaximumComponentLength,
  out int lpFileSystemFlags,
  StringBuilder lpFileSystemNameBuffer,
  int nFileSystemNameSize
);



        [DllImport("kernel32", CharSet = CharSet.Unicode, SetLastError = true)]
        private static extern int DeviceIoControl(
    IntPtr hDevice,
    int dwIoControlCode,
    IntPtr lpInBuffer,
    int nInBufferSize,
    IntPtr lpOutBuffer,
    int nOutBufferSize,
    out int lpBytesReturned,
    IntPtr lpOverlapped
    );

        [DllImport("kernel32", SetLastError = true)]
        private static extern bool CloseHandle(
            IntPtr handle);


        [DllImport("kernel32", SetLastError = true)]
        private static extern IntPtr OpenFileById(
  IntPtr hFile,
  ref FILE_ID_DESCRIPTOR lpFileID,
  int                 dwDesiredAccess,
  FileShare dwShareMode,
  IntPtr lpSecurityAttributes,
  int dwFlags
);
        [DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
        private static extern int GetFinalPathNameByHandleW(
  IntPtr hFile,
  StringBuilder lpszFilePath,
  int cchFilePath,
  int dwFlags
);
        [StructLayout(LayoutKind.Explicit)]
        public struct FILE_ID_DESCRIPTOR
        {
            [FieldOffset(0)]
            public int Size;
            [FieldOffset(4)]
            public int Type;
            [FieldOffset(8)]
            public long FileId;
            [FieldOffset(8)]
            public Guid ObjectId;
            [FieldOffset(8)]
            public Guid ExtendedFileId; //Use for ReFS; need to use v3 structures or later instead of v2 as done in this sample
        }

        public static int CTL_CODE(int DeviceType, int Function, int Method, int Access)
        {
            return ((DeviceType) << 16) | ((Access) << 14) | ((Function) << 2) | (Method);
        }

        protected override bool ReleaseHandle()
        {

            if(handle != IntPtr.Zero)
            {
                if(createdJournal == true)
                {
                    TryDeleteCurrentJournal();
                }
                StopListening(10);//this may cause a delay
                return CloseHandle(handle);
            }
            return false;
        }

        public const int FILE_DEVICE_FILE_SYSTEM = 0x00000009;
        public const int METHOD_BUFFERED = 0;
        public const int METHOD_IN_DIRECT = 1;
        public const int METHOD_OUT_DIRECT = 2;
        public const int METHOD_NEITHER = 3;
        public const int FILE_ANY_ACCESS = 0;

        public static int FSCTL_READ_USN_JOURNAL = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 46, METHOD_NEITHER, FILE_ANY_ACCESS);
        public static int FSCTL_ENUM_USN_DATA = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 44, METHOD_NEITHER, FILE_ANY_ACCESS);
        public static int FSCTL_CREATE_USN_JOURNAL = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 57, METHOD_NEITHER, FILE_ANY_ACCESS);
        public static int FSCTL_READ_FILE_USN_DATA = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 58, METHOD_NEITHER, FILE_ANY_ACCESS);
        public static int FSCTL_QUERY_USN_JOURNAL = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 61, METHOD_BUFFERED, FILE_ANY_ACCESS);
        public static int FSCTL_DELETE_USN_JOURNAL = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 62, METHOD_BUFFERED, FILE_ANY_ACCESS);
        public static int FSCTL_WRITE_USN_REASON = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 180, METHOD_BUFFERED, FILE_ANY_ACCESS);
        public static int FSCTL_USN_TRACK_MODIFIED_RANGES = CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 189, METHOD_BUFFERED, FILE_ANY_ACCESS);

        [StructLayout(LayoutKind.Sequential)]
        public struct USN
        {
            public long Usn;
        }

        [StructLayout(LayoutKind.Sequential)]
        public struct MFT_ENUM_DATA_V0
        {
            public USN Low;
            public USN High;
        }

        [StructLayout(LayoutKind.Sequential)]
        public struct MFT_ENUM_DATA_V1
        {
            public long StartFileReferenceNumber;
            public USN Low;
            public USN High;
            public short MinMajorVersion;
            public short MaxMajorVersion;
        }


        [StructLayout(LayoutKind.Sequential)]
        public struct CREATE_USN_JOURNAL_DATA
        {
            public long MaximumSize;
            public long AllocationDelta;
        }


        [StructLayout(LayoutKind.Sequential)]
        public struct READ_USN_JOURNAL_DATA_V0
        {
            public USN StartUsn;
            public int ReasonMask;
            public int ReturnOnlyOnClose;
            public long Timeout;
            public long BytesToWaitFor;
            public long UsnJournalId;
        }


        [StructLayout(LayoutKind.Sequential)]
        public struct READ_USN_JOURNAL_DATA_V1
        {
            public USN StartUsn;
            public int ReasonMask;
            public int ReturnOnlyOnClose;
            public long Timeout;
            public long BytesToWaitFor;
            public long UsnJournalId;
            public short MinMajorVersion;
            public short MaxMajorVersion;
        }


        [StructLayout(LayoutKind.Sequential)]
        public struct USN_TRACK_MODIFIED_RANGES
        {
            public int Flags;
            public int Unused;
            public long ChunkSize;
            public long FileSizeThreshold;
        }

        [StructLayout(LayoutKind.Sequential)]
        public struct USN_RANGE_TRACK_OUTPUT
        {
            public USN Usn;
        }

        public const int FLAG_USN_TRACK_MODIFIED_RANGES_ENABLE = 0x00000001;

        public class UsnRecordV2WithName
        {
            public USN_RECORD_V2 Record { get; set; }
            public string Filename { get; set; }
        }

        [StructLayout(LayoutKind.Sequential)]
        public struct USN_RECORD_V2
        {
            public int RecordLength;
            public short MajorVersion;
            public short MinorVersion;
            public long FileReferenceNumber;
            public long ParentFileReferenceNumber;
            USN Usn;
            public long TimeStamp;
            public int Reason;
            public int SourceInfo;
            public int SecurityId;
            public int FileAttributes;
            public short FileNameLength;
            public short FileNameOffset;
            //WCHAR FileName[1];

        }

        [StructLayout(LayoutKind.Sequential)]
        public struct USN_RECORD_V3
        {
            public int RecordLength;
            public short MajorVersion;
            public short MinorVersion;
            public Guid FileReferenceNumber;
            public Guid ParentFileReferenceNumber;
            USN Usn;
            public long TimeStamp;
            public int Reason;
            public int SourceInfo;
            public int SecurityId;
            public int FileAttributes;
            public short FileNameLength;
            public short FileNameOffset;
            //WCHAR FileName[1];

        }


        [StructLayout(LayoutKind.Sequential)]
        public struct USN_RECORD_COMMON_HEADER
        {
            public int RecordLength;
            public short MajorVersion;
            public short MinorVersion;
        }

        [StructLayout(LayoutKind.Sequential)]
        public struct USN_RECORD_EXTENT
        {
            public long Offset;
            public long Length;
        }

        [StructLayout(LayoutKind.Sequential)]
        public struct USN_RECORD_V4
        {
            public USN_RECORD_COMMON_HEADER Header;
            public Guid FileReferenceNumber;
            public Guid ParentFileReferenceNumber;
            public USN Usn;
            public int Reason;
            public int SourceInfo;
            public int RemainingExtents;
            public short NumberOfExtents;
            public short ExtentSize;
            public USN_RECORD_EXTENT Extents; //Extents[1]
        }


        public const int USN_PAGE_SIZE = (0x1000);
        public const int USN_REASON_DATA_OVERWRITE = (0x00000001);
        public const int USN_REASON_DATA_EXTEND = (0x00000002);
        public const int USN_REASON_DATA_TRUNCATION = (0x00000004);
        public const int USN_REASON_NAMED_DATA_OVERWRITE = (0x00000010);
        public const int USN_REASON_NAMED_DATA_EXTEND = (0x00000020);
        public const int USN_REASON_NAMED_DATA_TRUNCATION = (0x00000040);
        public const int USN_REASON_FILE_CREATE = (0x00000100);
        public const int USN_REASON_FILE_DELETE = (0x00000200);
        public const int USN_REASON_EA_CHANGE = (0x00000400);
        public const int USN_REASON_SECURITY_CHANGE = (0x00000800);
        public const int USN_REASON_RENAME_OLD_NAME = (0x00001000);
        public const int USN_REASON_RENAME_NEW_NAME = (0x00002000);
        public const int USN_REASON_INDEXABLE_CHANGE = (0x00004000);
        public const int USN_REASON_BASIC_INFO_CHANGE = (0x00008000);
        public const int USN_REASON_HARD_LINK_CHANGE = (0x00010000);
        public const int USN_REASON_COMPRESSION_CHANGE = (0x00020000);
        public const int USN_REASON_ENCRYPTION_CHANGE = (0x00040000);
        public const int USN_REASON_OBJECT_ID_CHANGE = (0x00080000);
        public const int USN_REASON_REPARSE_POINT_CHANGE = (0x00100000);
        public const int USN_REASON_STREAM_CHANGE = (0x00200000);
        public const int USN_REASON_TRANSACTED_CHANGE = (0x00400000);
        public const int USN_REASON_INTEGRITY_CHANGE = (0x00800000);
        public const uint USN_REASON_CLOSE = (0x80000000);

        [Flags]
        public enum UsnReasonType : int
        {
            USN_REASON_DATA_OVERWRITE = (0x00000001),
            USN_REASON_DATA_EXTEND = (0x00000002),
            USN_REASON_DATA_TRUNCATION = (0x00000004),
            USN_REASON_NAMED_DATA_OVERWRITE = (0x00000010),
            USN_REASON_NAMED_DATA_EXTEND = (0x00000020),
            USN_REASON_NAMED_DATA_TRUNCATION = (0x00000040),
            USN_REASON_FILE_CREATE = (0x00000100),
            USN_REASON_FILE_DELETE = (0x00000200),
            USN_REASON_EA_CHANGE = (0x00000400),
            USN_REASON_SECURITY_CHANGE = (0x00000800),
            USN_REASON_RENAME_OLD_NAME = (0x00001000),
            USN_REASON_RENAME_NEW_NAME = (0x00002000),
            USN_REASON_INDEXABLE_CHANGE = (0x00004000),
            USN_REASON_BASIC_INFO_CHANGE = (0x00008000),
            USN_REASON_HARD_LINK_CHANGE = (0x00010000),
            USN_REASON_COMPRESSION_CHANGE = (0x00020000),
            USN_REASON_ENCRYPTION_CHANGE = (0x00040000),
            USN_REASON_OBJECT_ID_CHANGE = (0x00080000),
            USN_REASON_REPARSE_POINT_CHANGE = (0x00100000),
            USN_REASON_STREAM_CHANGE = (0x00200000),
            USN_REASON_TRANSACTED_CHANGE = (0x00400000),
            USN_REASON_INTEGRITY_CHANGE = (0x00800000),
            USN_REASON_CLOSE = unchecked((int)(0x80000000))
        }

        [StructLayout(LayoutKind.Sequential)]
        public struct USN_JOURNAL_DATA_V0
        {
            public long UsnJournalID;
            public USN FirstUsn;
            public USN NextUsn;
            public USN LowestValidUsn;
            public USN MaxUsn;
            public long MaximumSize;
            public long AllocationDelta;

        }
        [StructLayout(LayoutKind.Sequential)]
        public struct USN_JOURNAL_DATA_V1
        {
            public long UsnJournalID;
            public USN FirstUsn;
            public USN NextUsn;
            public USN LowestValidUsn;
            public USN MaxUsn;
            public long MaximumSize;
            public long AllocationDelta;
            public short MinSupportedMajorVersion;
            public short MaxSupportedMajorVersion;
        }

        [StructLayout(LayoutKind.Sequential)]
        public struct USN_JOURNAL_DATA_V2
        {
            public long UsnJournalID;
            public USN FirstUsn;
            public USN NextUsn;
            public USN LowestValidUsn;
            public USN MaxUsn;
            public long MaximumSize;
            public long AllocationDelta;
            public short MinSupportedMajorVersion;
            public short MaxSupportedMajorVersion;
            public int Flags;
            public long RangeTrackChunkSize;
            public long RangeTrackFileSizeThreshold;
        }



        [StructLayout(LayoutKind.Sequential)]
        public struct DELETE_USN_JOURNAL_DATA
        {
            public long UsnJournalID;
            public int DeleteFlags;
        }

        public int USN_DELETE_FLAG_DELETE = (0x00000001);
        public int USN_DELETE_FLAG_NOTIFY = (0x00000002);
        public int USN_DELETE_VALID_FLAGS = (0x00000003);

        public UsnReasonType EventTriggerMask
        {
            get
            {
                return (UsnReasonType)rdata.ReasonMask;
            }
            set
            {
                rdata.ReasonMask = (int)value;
            }
        }

        public long Timeout
        {
            get
            {
                return rdata.Timeout;
            }
            set
            {
                rdata.Timeout = value;
            }
        }

        public bool TriggerOnCloseOnly
        {
            get
            {
                return rdata.ReturnOnlyOnClose != 0;
            }
            set
            {
                rdata.ReturnOnlyOnClose = value ? 1 : 0;
            }
        }
        private ReaderWriterLockSlim readBufferLock = new ReaderWriterLockSlim();

        private int readBufferSize = 8192;

        //This could hang if there is a long timeout value
        public int ReadBufferSize
        {
            get
            {
                readBufferLock.EnterWriteLock();
                try
                {
                    return readBufferSize;
                }
                finally
                {
                    readBufferLock.ExitWriteLock();
                }
            }
            set
            {
                readBufferLock.EnterWriteLock();
                try
                {

                    if (value > 0)
                    {
                        if (readBuffer == IntPtr.Zero)
                        {
                            readBuffer = Marshal.AllocHGlobal(value);
                        }
                        else
                        {
                            readBuffer = Marshal.ReAllocHGlobal(readBuffer, (IntPtr)value);
                        }
                    }
                    readBufferSize = value;

                }
                finally
                {
                    readBufferLock.ExitWriteLock();
                }
            }
        }
        public event Action<ChangeJournalHandle, UsnRecordV2WithName> OnChange;
        public event Action<ChangeJournalHandle, Exception> OnError;

        private bool shouldRun = false;

        private Thread thread = null;

        public ChangeJournalHandle(string path) : base(true)
        {
            //TODO:Handle taking non-volume paths
            handle = CreateFileW(path, unchecked((int)(0x80000000 | 0x40000000)),
                FileShare.ReadWrite, IntPtr.Zero, 3, 0, IntPtr.Zero);
            if (IsInvalid)
            {
                Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());
            }
        }

        public bool TryCreateJournal(long maxSize = (1024 ^ 2) * 500, long allocationDelta = 8192)
        {
            CREATE_USN_JOURNAL_DATA data = new CREATE_USN_JOURNAL_DATA();
            data.AllocationDelta = allocationDelta;
            data.MaximumSize = maxSize;
            int size = Marshal.SizeOf(data);
            IntPtr buffer = Marshal.AllocHGlobal(size);
            try
            {
                int bufSizeOut;
                int result = DeviceIoControl(handle, FSCTL_CREATE_USN_JOURNAL, buffer, size, IntPtr.Zero, 0, out bufSizeOut, IntPtr.Zero);
                if (result == 0)
                {
                    ReportLastError();
                    return false;
                }
                createdJournal = true;
                return true;
            }
            finally
            {
                Marshal.FreeHGlobal(buffer);
            }
        }
        public void CreateJournal(long maxSize = (1024 ^ 2) * 500, long allocationDelta = 8192)
        {
            CREATE_USN_JOURNAL_DATA data = new CREATE_USN_JOURNAL_DATA();
            data.AllocationDelta = allocationDelta;
            data.MaximumSize = maxSize;
            int size = Marshal.SizeOf(data);
            IntPtr buffer = Marshal.AllocHGlobal(size);
            try
            {
                int bufSizeOut;
                int result = DeviceIoControl(handle, FSCTL_CREATE_USN_JOURNAL, buffer, size, IntPtr.Zero, 0, out bufSizeOut, IntPtr.Zero);
                if (result == 0)
                {
                    Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());
                }
            }
            finally
            {
                Marshal.FreeHGlobal(buffer);
            }
        }
        public bool TryDeleteCurrentJournal()
        {
            USN_JOURNAL_DATA_V0 data = new USN_JOURNAL_DATA_V0();
            int size = Marshal.SizeOf(data);
            IntPtr buffer = Marshal.AllocHGlobal(size);
            try
            {
                int outSize;
                int result = DeviceIoControl(handle, FSCTL_QUERY_USN_JOURNAL, IntPtr.Zero, 0, buffer, size, out outSize, IntPtr.Zero);
                if (result == 0)
                {
                    ReportLastError();
                    return false;
                }
                data = Marshal.PtrToStructure<USN_JOURNAL_DATA_V0>(buffer);
            }
            finally
            {
                Marshal.FreeHGlobal(buffer);
            }
            DELETE_USN_JOURNAL_DATA d = new DELETE_USN_JOURNAL_DATA();
            d.UsnJournalID = data.UsnJournalID;
            d.DeleteFlags = 3;
            size = Marshal.SizeOf(d);
            buffer = Marshal.AllocHGlobal(size);
            try
            {
                Marshal.StructureToPtr(d, buffer, false);
                if (DeviceIoControl(handle, FSCTL_DELETE_USN_JOURNAL, buffer, size, IntPtr.Zero, 0, out size, IntPtr.Zero) == 0)
                {
                    ReportLastError();
                    return false;
                }
                return true;
            }
            finally
            {
                Marshal.FreeHGlobal(buffer);
            }
        }
        public bool TryDeleteJournal(long UsnJournalID)
        {
            //Note that overloads would be needed for different versions of the structure
            DELETE_USN_JOURNAL_DATA d = new DELETE_USN_JOURNAL_DATA();
            d.UsnJournalID = UsnJournalID;
            d.DeleteFlags = 3;
            int size = Marshal.SizeOf(d);
            IntPtr buffer = Marshal.AllocHGlobal(size);
            try
            {
                Marshal.StructureToPtr(d, buffer, false);
                if (DeviceIoControl(handle, FSCTL_DELETE_USN_JOURNAL, buffer, size, IntPtr.Zero, 0, out size, IntPtr.Zero) == 0)
                {
                    ReportLastError();
                    return false;
                }
                return true;
            }
            finally
            {
                Marshal.FreeHGlobal(buffer);
            }

        }
        public void DeleteAllJournals()
        {
            try
            {
                while (true)
                {
                    USN_JOURNAL_DATA_V0 data = new USN_JOURNAL_DATA_V0();
                    int size = Marshal.SizeOf(data);
                    IntPtr buffer = Marshal.AllocHGlobal(size);
                    try
                    {
                        int outSize;
                        int result = DeviceIoControl(handle, FSCTL_QUERY_USN_JOURNAL, IntPtr.Zero, 0, buffer, size, out outSize, IntPtr.Zero);
                        if (result == 0)
                        {
                            ReportLastError();
                            break;
                        }
                        data = Marshal.PtrToStructure<USN_JOURNAL_DATA_V0>(buffer);
                    }
                    finally
                    {
                        Marshal.FreeHGlobal(buffer);
                    }
                    DELETE_USN_JOURNAL_DATA d = new DELETE_USN_JOURNAL_DATA();
                    d.UsnJournalID = data.UsnJournalID;
                    d.DeleteFlags = 3;
                    size = Marshal.SizeOf(d);
                    buffer = Marshal.AllocHGlobal(size);
                    try
                    {
                        Marshal.StructureToPtr(d, buffer, false);
                        if (DeviceIoControl(handle, FSCTL_DELETE_USN_JOURNAL, buffer, size, IntPtr.Zero, 0, out size, IntPtr.Zero) == 0)
                        {
                            ReportLastError();
                            break;
                        }
                    }
                    finally
                    {
                        Marshal.FreeHGlobal(buffer);
                    }
                }
            }
            catch (Exception ex)
            {
                ReportException(ex);
            }
        }
        public void StartListening()
        {
            //See https://msdn.microsoft.com/en-us/library/windows/desktop/aa365736(v=vs.85).aspx
            if (!shouldRun)
            {
                thread = new Thread(ListenProc);
                shouldRun = true;
                thread.Start();
            }
        }

        public void StopListening(int timeout = int.MaxValue)
        {
            if (shouldRun)
            {
                shouldRun = false;
                if (thread != null)
                {
                    if(!thread.Join(timeout))
                    {
                        thread.Abort();
                    }
                    thread = null;
                }
            }
        }

        public string GetNameForId(long id)
        {
            try
            {
                FILE_ID_DESCRIPTOR fid = new FILE_ID_DESCRIPTOR();
                fid.FileId = id;
                fid.Size = Marshal.SizeOf(fid);
                IntPtr h = OpenFileById(handle, ref fid, unchecked((int)0x80), FileShare.ReadWrite | FileShare.Delete, IntPtr.Zero, 0);
                if (h == new IntPtr(-1))
                {
                    Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());
                }
                int size = 1024;
                StringBuilder sb = new StringBuilder(size);
                if (GetFinalPathNameByHandleW(h, sb, size, 0) == 0)
                {
                    int hr = Marshal.GetHRForLastWin32Error();
                    CloseHandle(h);
                    Marshal.ThrowExceptionForHR(hr);
                }
                CloseHandle(h);
                return sb.ToString();
            }
            catch(Exception ex)
            {
                ReportException(ex);
                return id.ToString("X");
            }

        }
        private READ_USN_JOURNAL_DATA_V0 rdata = new READ_USN_JOURNAL_DATA_V0() { ReasonMask = unchecked((int)0xFFFFFFFF) };
        private IntPtr readBuffer;
        private bool createdJournal = false;
        void ListenProc()
        {
            try
            {
                USN_JOURNAL_DATA_V0 data = new USN_JOURNAL_DATA_V0();
                int size = Marshal.SizeOf(data);
                IntPtr buffer = Marshal.AllocHGlobal(size);
                try
                {
                    int outSize;
                    int result = DeviceIoControl(handle, FSCTL_QUERY_USN_JOURNAL, IntPtr.Zero, 0, buffer, size, out outSize, IntPtr.Zero);
                    if (result == 0)
                    {
                        if(TryCreateJournal())
                        {
                            result = DeviceIoControl(handle, FSCTL_QUERY_USN_JOURNAL, IntPtr.Zero, 0, buffer, size, out outSize, IntPtr.Zero);
                        }
                        if(result == 0) ReportLastError();
                    }
                    if(result != 0) data = Marshal.PtrToStructure<USN_JOURNAL_DATA_V0>(buffer);
                }
                finally
                {
                    Marshal.FreeHGlobal(buffer);
                }
                rdata.UsnJournalId = data.UsnJournalID;
                rdata.StartUsn.Usn = 0;
                int rsize = Marshal.SizeOf(typeof(USN_RECORD_V2));
                size = Marshal.SizeOf(rdata);
                buffer = Marshal.AllocHGlobal(size);
                if (readBuffer == IntPtr.Zero)
                {
                    //Allocates the buffer if it's empty
                    ReadBufferSize = ReadBufferSize;
                }
                int usize = Marshal.SizeOf(typeof(USN));
                try
                {
                    List<UsnRecordV2WithName> records = new List<UsnRecordV2WithName>();
                    while (shouldRun)
                    {
                        records.Clear();
                        int outSize;
                        int result;
                        if(readBufferSize >= 1024)
                        {
                            rdata.BytesToWaitFor = readBufferSize;
                        }
                        else
                        {
                            //Returns immediately
                            rdata.BytesToWaitFor = 0;
                        }
                        Marshal.StructureToPtr(rdata, buffer, false);

                        readBufferLock.EnterReadLock();
                        try
                        {
                            result = DeviceIoControl(handle, FSCTL_READ_USN_JOURNAL, buffer, size, readBuffer, readBufferSize, out outSize, IntPtr.Zero);
                            if (result != 0 && outSize >= usize)
                            {
                                USN usn = Marshal.PtrToStructure<USN>(readBuffer);
                                rdata.StartUsn = usn;
                                int retbytes = outSize - usize;
                                IntPtr record = IntPtr.Add(readBuffer, usize);
                                while (retbytes > 0)
                                {
                                    USN_RECORD_V2 r = Marshal.PtrToStructure<USN_RECORD_V2>(record);
                                    UsnRecordV2WithName r2 = new UsnRecordV2WithName();
                                    r2.Record = r;
                                    r2.Filename = Marshal.PtrToStringUni(IntPtr.Add(record, r.FileNameOffset), (r.FileNameLength / 2));
                                    records.Add(r2);
                                    record = IntPtr.Add(record, r.RecordLength);
                                    retbytes -= r.RecordLength;
                                }
                            }
                            else
                            {
                                ReportLastError();
                            }
                        }
                        finally
                        {
                            readBufferLock.ExitReadLock();
                        }
                        foreach (var r in records)
                        {
                            ReportChange(r);
                        }
                    }
                }
                finally
                {
                    Marshal.FreeHGlobal(buffer);
                }
            }
            catch (ThreadAbortException tae)
            {

            }
        }

        void ReportChange(UsnRecordV2WithName record)
        {
            if (OnChange != null)
            {
                OnChange(this, record);
            }
        }

        void ReportLastError()
        {
            ReportException(Marshal.GetExceptionForHR(Marshal.GetHRForLastWin32Error()));
        }

        void ReportException(Exception ex)
        {
            if (OnError != null)
            {
                OnError(this, ex);
            }
        }



        private static void Cjh_OnError(ChangeJournalHandle arg1, Exception arg2)
        {
            Console.WriteLine("Error:/t{0}", arg2.ToString());
        }

        private static void Cjh_OnChange(ChangeJournalHandle arg1, UsnRecordV2WithName arg2)
        {
            //Note that it would be typically faster in the long run to build a dictionary
            //of directory names by IDs and reset it whenever the change journal resets
            //instead of looking up the directory each time
            //Also, note that if the directory is deleted before OpenFileById is called in GetNameById, it's going to fail with an out of range error
            Console.Write(arg1.GetNameForId(arg2.Record.ParentFileReferenceNumber));
            Console.Write("\\");
            Console.Write(arg2.Filename);
            Console.Write(":\t");
            Console.WriteLine(((ChangeJournalHandle.UsnReasonType)arg2.Record.Reason).ToString());
        }
        static void Main(string[] args)
        {
            string pathToVolumeToMonitor = @"\\?\C:";
            //This will filter to show only files that are deleted or created
            UsnReasonType reasonsToMonitor = UsnReasonType.USN_REASON_FILE_CREATE | UsnReasonType.USN_REASON_FILE_DELETE;
            using (ChangeJournalHandle cjh = new ChangeJournalHandle(pathToVolumeToMonitor))
            {
                cjh.OnChange += Cjh_OnChange;
                cjh.OnError += Cjh_OnError;
                cjh.EventTriggerMask = reasonsToMonitor;

                cjh.StartListening();
                Console.ReadLine();
                cjh.StopListening();


                cjh.OnChange -= Cjh_OnChange;
                cjh.OnError -= Cjh_OnError;
            }
        }
    }
}

Sample output:

\\?\C:\Windows\Temp\MSIc9528.LOG:       USN_REASON_FILE_CREATE

\\?\C:\Windows\Temp\MSIc9528.LOG:       USN_REASON_FILE_CREATE, USN_REASON_CLOSE

\\?\C:\Windows\Temp\MSIc9529.LOG:       USN_REASON_FILE_CREATE

\\?\C:\Windows\Temp\MSIc9529.LOG:       USN_REASON_FILE_CREATE, USN_REASON_CLOSE

\\?\C:\Windows\Temp\MSIc952a.LOG:       USN_REASON_FILE_CREATE

\\?\C:\Windows\Temp\MSIc952a.LOG:       USN_REASON_FILE_CREATE, USN_REASON_CLOSE

\\?\C:\Windows\Temp\MSIc952b.LOG:       USN_REASON_FILE_CREATE

\\?\C:\Windows\Temp\MSIc952b.LOG:       USN_REASON_FILE_CREATE, USN_REASON_CLOSE

Follow us on Twitter, www.twitter.com/WindowsSDK.

Jeff here from the Windows SDK team. Here is the list of October 2015 Hotfixes

[Drum roll, please…]

KB3030736   “550 The process cannot access the file” error when you try to download a file in Windows

KB3044546   An updated reservation may disappear on a DHCP failover cluster in Windows Server 2012 or Windows Server 2012 R2

KB3049591   Transparent areas are printed as black when you use a v4 XPS printer driver in Windows

KB3053667   Users can’t connect to virtual machines that are running Windows 8.1 or Windows Server 2012 R2 by remote desktop

KB3077354   Computer freezes when WFP leaks nonpaged pool memory in Windows Server 2012 R2 or Windows Server 2012

KB3084093   Child nodes under protected OU are deleted in Windows Server 2012 R2

KB3084426   System becomes unresponsive when file system minifilter drivers are installed in Windows 8 or Windows Server 2012

KB3084787   Event ID 4102 when DFS Replication cloning fails on a Windows Server 2012 R2-based cluster

KB3084953   OOBE crashes when you enter SIM card PIN in Windows 8.1

KB3084956   You can’t log on to a domain-joined computer in Windows 8 or Windows Server 2012

KB3084983   SetOptions or GetOptions method doesn’t work for PRINTER_PROPERTY features in Windows

KB3086644   System freezes after you start a backup task for virtual machines in Windows Server 2012 R2

KB3090322   Space doesn’t regenerate upon reallocation in Windows Server 2012 R2

KB3090973   Reenlist can’t be called when SQL Server receives transaction outcome from MSDTC in Windows Server 2012 R2

KB3091057   Cluster validation fails in the “Validate Simultaneous Failover” test in a Windows Server 2012 R2-based failover cluster

KB3091061   Update to add CSVFS tracing for performance issues during backup operations in Windows Server 2012 R2

KB3091342   Computer crashes after you install update 3000850 in Windows 8.1

KB3091402   Site-to-site VPN goes down when you set a VNet-to-VNet connection in Azure in Windows 8.1 or Windows Server 2012 R2

KB3091403   High CPU usage for 12 seconds on a multiple-network-adapters computer in Windows Server 2012 R2 or Windows Server 2012

KB3092002   Set-Acl cmdlet fails although delegated admins have “Change Permissions” enabled in Windows Server 2012 R2

KB3092003   Page loads repeatedly and authentication fails when users use MFA in Windows Server 2012 R2 AD FS

KB3092005   Group Policy settings are set back to factory settings in GPMC in Windows Server 2012 R2

KB3092006   Subfolders disappear from FSRM console after you rename the root folder in Windows Server 2012 R2 or Windows Server 2012

KB3092695   Licensing report file is corrupted in Windows Server 2012 R2 RDS environments for large report files

KB3093550   All ScriptProperty members are invoked when you run the Add-Member PowerShell command in Windows 8.1

KB3093571   Update to replicate multiple VM groups and VMs that use shared VHDs in Windows Server 2012 R2 or Windows Server 2012

KB3093803   Error 0x800704C9 occurs when you try to copy file to NFS share in Windows 8 or Windows Server 2012

KB3093899   VMs that run on CSVs fail if DCM can’t query volumes in Windows Server 2012 R2

KB3093900   0x50 Stop error and users can’t access documents on shared folders or home folders in Windows Server 2012 R2

KB3094197   Files aren’t fully optimized and a deduplication cache lock contention issue occurs in Windows Server 2012 R2

KB3094199   Application module can’t receive correct process status after security update 3045999 is installed in Windows

KB3094202   Directory listing fails when sharing violations occur on Windows Server 2012 R2 or Windows Server 2012-based NFS server

KB3094446   Authentication through proxy fails in Windows Server 2012 or Windows Server 2008 R2 SP1

KB3095308   VMs may not get additional memory although they’re set to use Dynamic Memory in Windows Server 2012 R2

KB3095319   You receive an error message when you use GPMC to manage audit policies in Windows 8.1 or Windows Server 2012 R2

KB3095663   VSS_E_PROVIDER_VETO error occurs when you restore a LUN from backup in Windows Server 2012 R2

KB3095711   Update to support LTO-7 tape drives in Windows Server 2012 R2 and Windows Server 2012

KB3095737   Azure Backup takes a long time to back up data with a guest OS that’s running Windows Server 2012

Happy patching…

/Jeff

Follow us on Twitter, www.twitter.com/WindowsSDK.

Jeff here from the SDK team. Here are the hotfixes for November 2015.

KB3040017   XPS driver is slower than GDI driver to print files in Windows

KB3063109   Virtual machine crashes and WAL consistency is not maintained for Guest clustered VMs in Windows Server 2012 R2

KB3078414   NFS service freezes on a Windows Server 2012 cluster and a client computer can’t access NFS share

KB3080141   Cluster service freezes on a Windows Server 2012 R2 or Windows Server 2012-based failover cluster

KB3086918   Original file is lost with new file not being saved in WebDAV folder in Windows 8.1

KB3091411   User connection fails when many connections are made to Windows Server 2012 R2-based RD Connection Broker

KB3092604   Network is corrupted between Guest OS and external network for VMs hosted on Windows 8.1 or Windows Server 2012 R2

KB3092688   UPD profiles corrupted when a network connectivity issue occurs in Windows Server 2012 R2

KB3095113   Update to enable WSUS support for Windows 10 feature upgrades

KB3095682   File Explorer shows thumbnail contents even though files are marked with offline flag in Windows

KB3098841   Application crashes with access violation error in Windows 7 or Windows Server 2008 R2

KB3100460   Video stops playing unexpectedly when another video pauses in the same application in Windows 8 or Windows Server 2012

KB3100474   Can’t connect to wireless network when you resume the computer from hibernate mode in Windows

KB3100477   A large file upload or a large repository clone fails on VSO in Windows Server 2012 R2

KB3100527   System becomes unresponsive and crashes on Windows Server 2012 R2-based file servers

KB3100530   Windows backup fails with no sufficient free space on target volume in Windows Server 2012

KB3101217   Client requests take a long time to execute or COM+ application freezes in Windows 7 SP1 or Windows Server 2008 R2 SP1

KB3101694   “0x000000D1″ Stop error in Pacer.sys when there’s heavy QoS traffic in Windows Server 2012 R2

KB3101705   Schema load failures or SPF failures in Windows Server 2012 R2 in Turkey

KB3101718   Application freezes when you switch the system from DC mode to AC mode in Windows 8.1 or Windows Server 2012 R2

KB3102236   Group membership removal operation fails when this operation is for a deleted user account in Windows Server 2008 R2

KB3102242   Computer crashes or restarts unexpectedly when it’s resumed from sleep mode in Windows

KB3102354   Hyper-V generation 2 virtual machines can’t start with some pass-through disks in Windows Server 2012 R2

KB3102770   The chkdsk command together with the spotfix option doesn’t fix extended volumes corruption in Windows Server 2012 R2

KB3103000   RemoteApp windows disappear and screen flickers when you switch between windows in Windows 8.1 or Windows Server 2012 R2

KB3103616   WMI query doesn’t work in Windows Server 2012

KB3105881   Can’t access applications when device authentication is enabled in Windows Server 2012 R2-based AD FS server

Happy patching.

/Jeff

Follow us on Twitter, www.twitter.com/WindowsSDK.

With the industry moving away from SHA1 including Microsoft (see http://social.technet.microsoft.com/wiki/contents/articles/32288.windows-enforcement-of-authenticode-code-signing-and-timestamping.aspx), many developers should start using SHA2 in their code.  There are resources on the internet that describe how to use SHA256 with SignedXml. This blog is meant to summarize it in one place.

First of all, we need to register a SignatureDescription class that defines the DigestAlgorithm as SHA256.  The .NET Cryptography namespace implements a class called RSAPKCS1SHA1SignatureDescription that supports SHA1.  So we need a similar class called RSAPKCS1SHA256SignatureDescription that supports SHA256.

If your code is based on .NET 4.5 or higher, there is an RSAPKCS1SHA256SignatureDescription class you can register.  You have to reference the System.Deployment assembly in your project.  The full namespace is System.Deployment.Internal.CodeSigning.RSAPKCS1SHA256SignatureDescription.  You must call CryptoConfig.AddAlgorithm to register the class.

Here’s the MSDN SignedXml sample modified to use SHA256:

This post is to help the product team spread the word on Windows (version 7 and higher) and Windows Server will no longer trust any code that is signed with a SHA-1 code signing certificate and that contains a timestamp value greater than January 1,2016, effective January 1, 2016.

For more information or to get the latest information on this topic, please check out the following WIKI on Microsoft Technet, http://aka.ms/sha1

Jeff here, from the SDK team with the Holiday Hotfixes. Happy patching.

KB2920591   High CPU usage and performance issues occur when access-based enumeration is enabled in Windows 7 Service Pack 1 or Windows Server 2008 R2 Service Pack 1

KB3084463   WSUSutil.exe csaimport fails upon import in Windows Server 2012 R2 or Windows Server 2012

KB3095319   You receive an error message when you use GPMC to manage audit policies in Windows 8.1 or Windows Server 2012 R2

KB3102997   Data is corrupted after iSCSI sessions or paths recover in Windows Server 2012 R2 or Windows Server 2012

KB3102998   Application can’t connect to iSCSI servers in Windows Server 2012 R2 or Windows Server 2012

KB3106296   Can’t connect to a wireless network when you resume the computer from hibernation

KB3107128   Search result is incomplete if search criteria contain digits in Windows

KB3108319   VSS backup of the PI Data server fails and the computer crashes in Windows 8.1 or Windows Server 2012 R2

KB3109093   Applications can’t communicate over TCP loopback path in Windows 8.1 or Windows Server 2012 R2

KB3109099   Update adds support for the slow timer in LACP in Windows Server 2012 R2

KB3109156   Applications may freeze when ADSI APIs waits infinitely for server to respond in Windows Server 2012 R2

KB3109600   Users can’t log on to Outlook Web App client from a browser in Windows

KB3109973   Backup fails with a “File Not Found” error on a Windows Server 2012 R2 cluster

KB3114133   Windows Server Backup fails when you back up multiple volumes in Windows Server 2012 R2

Happy Holidays,

/Jeff

Follow us on Twitter, www.twitter.com/WindowsSDK.

CryptCATAdminCalcHashFromFileHandle can fail when evaluating a file path to an executable while setting the last error code to 0x800700c1.  It’s an HRESULT instead of just an error code, but the relevant portion of the error code is 0xc1 (193L).  That is the error code for ERROR_BAD_EXE_FORMAT.  In some cases, this will happen even though the executable can run without an error.  So why would this function return ERROR_BAD_EXE_FORMAT when directly executing the file works without issue?

There are actually several reasons why this error can be returned, and only some of them would actually prevent an executable from being able to run on an unrestricted system.  All of them come from the portable executable (PE) header of the file.  The types used below can be found in the Windows SDK, primarily in winnt.h.  Here are most of the reasons:

• PIMAGE_DOS_HEADER->e_magic is an invalid value
• PIMAGE_DOS_HEADER->e_lfanew is an invalid value
• PIMAGE_NT_HEADERS->Signature is an invalid value
• PIMAGE_NT_HEADERS->FileHeader.SizeOfOptionalHeader is an invalid value
• PIMAGE_NT_HEADERS->FileHeader.Machine is an invalid value
• PIMAGE_NT_HEADERS->OptionalHeader.Magic is an invalid value
• PIMAGE_NT_HEADERS->OptionalHeader.FileAlignment is an invalid value
• Any of the populated members of PIMAGE_NT_HEADERS->OptionalHeader.DataDirectory have invalid values
• The certificate directory (IMAGE_DIRECTORY_ENTRY_SECURITY) has an offset that puts its data in an invalid location; see https://msdn.microsoft.com/en-us/windows/hardware/gg463180 for more details on what the standards for that are.

If you encounter this error on an executable file, these can be identified manually by looking at the output of a couple of tools.  If there’s a problem with the IMAGE_DIRECTORY_ENTRY_SECURITY section, then running SignTool.exe verify /v filename will output “SignTool Error: File not valid: filename”.  The rest of them can be identified by looking through the output of dumpbin.exe.  Dumpbin is available through Visual Studio and SignTool is available through the Windows SDK.

Follow us on Twitter, www.twitter.com/WindowsSDK.

Jeff here from the SDK team. This month there is only 1 public hotfix because of the holidays.

3047331 “0x00000050″ Stop error on a failover cluster that is running Windows Server 2012 R2 or Windows Server 2012

Happy patching.

/Jeff

Follow us on Twitter, www.twitter.com/WindowsSDK.

Hi guys,

Jeff here from the Windows SDK team. Here are the Windows hotfixes for February 2016.

KB3007507   “HTTP Error 500.19″ error when you browse an IIS 8.5 website in Windows

KB3090343   Cluster service stops during the VSS backup in a Windows Server 2012 R2 or Windows Server 2012-based Hyper-V cluster

KB3123593   A multi-site failover cluster goes into a split brain situation in Windows Server 2012 R2

KB3133689   UBPM doesn’t set environmental variables correctly when you run scheduled tasks in Windows Server 2012 R2

Happy patching!

/Jeff

Follow us on Twitter, www.twitter.com/WindowsSDK.

We recently had someone ask why their PowerShell script stopped working correctly when they enabled tracing with the command Set-PSDebug -Trace 2.  The part of the code in question that didn’t work basically boiled down to something like this:

function Execute-Query($connectionString)
{
$connection = New-Object System.Data.SqlClient.SQLConnection;
$connection.ConnectionString = $connectionString;
$connection.Open();
$cmd = $connection.CreateCommand();
$cmd.CommandText = "SELECT * FROM [dbo].[Table]";
$cmd.Connection = $connection;
$result = $cmd.ExecuteReader();
$result
}
$result = Execute-Query $connString
foreach($Record in $result)
{
$Record[1]
}

When this was ran with Set-PSDebug -Trace 2 turned on, the foreach loop’s body would not execute no matter how many records where returned by the query.  If Set-PSDebug -Off is then set and the code ran again, it would execute the body of the loop for each record that the query returned.

This happens because the output from Set-PSDebug -Trace 2 results in the object being generically enumerated before it gets to the foreach loop and the object only supports being enumerated through once.  It’s also an issue to some degree because said object is assigned to a variable before that variable is then written to the pipeline which results in the need for more trace output.  Assigning the collection result to a variable, outputting that variable from the function, assigning to a second variable, and operating the foreach loop against that variable while having the tracing enumerate through the collection each time in order to output its contents is what ultimately causes this to result in different behavior with the tracing enabled versus disabled.  That is not supported to behave differently in that scenario and thus will not be changed to accommodate that scenario.  Most collections can be enumerated through multiple times so this won’t come up with them, but if you do want to use Set-PSDebug and use classes which are enumerable but only once for a given object, you do have some options besides just turning Set-PSDebug off.

If you don’t need to do the multiple variable assignments or have the function return the collection directly, you can output the $cmd object and only call ExecuteReader() when your code actually needs the result of the query.

If you’re using PowerShell 4 or 5, this can be worked around by changing the lines “$result =$cmd.ExecuteReader(); $result” to “$result = $cmd.ExecuteReader().GetEnumerator(); $result” unless you need to use members of SqlDataReader that aren’t present in its IEnumerator.  That isn’t the only way to resolve it but it is the smallest code change that comes to mind while keeping the multiple variable assignments in tact.  You can see how the output differs in the trace output from Set-PSDebug -Trace 2 when calling ExecuteReader().GetEnumerator() versus ExecuteReader().  When just ExecuteReader() is assigned to the variable, the collection is enumerated through in the output and the type of the variable is SqlDataReader.  When ExecuteReader().GetEnumerator() is assigned to the variable,  just IEnumerator is seen in the trace output when the variable is set instead of the collections being enumerated through and the variable is of the type DbEnumerator.  If one doesn’t need the assignment to $result in the function, simply outputting “$cmd.ExecuteReader()” to the pipeline also works in this situation.  If you do need to assign the SqlDataReader type to a variable to use its members but only from within the function itself and don’t mind doing the query twice, you can assign $result to $cmd.ExecuteReader(), do what you need to do with $result, call $result.Close(), and then  call $cmd.ExecuteReader() again to output its result.

If the multiple variable assignments of the collection and working with Set-PSDebug -Trace 2 are necessary for one’s scenario, one thing that you can do that works at least as far back as Powershell 2 is to wrap the enumerable results into another collection object which supports multiple enumerations.  For instance, instead of assigning the result of $cmd.ExecuteReader() directly to result, you could make $result = New-Object System.Collections.ArrayList and then pipe the ExecuteReader function’s results to ForEach-Object { [void]$reader.Add($_) }.  That would result in an ArrayList containing the data record objects which could be enumerated through multiple times.  If you want to assign that collection to a variable to work with within a function before putting it to the pipeline from that function and using that variable in a foreach loop, you’ll want to wrap that in another collection as well if Write-Output with the NoEnumerate option isn’t available (e.g. your PowerShell version is older than 4).

Visual Studio Team Services and Personal Microsoft Accounts

10/5/2016 Update – Due to the unexpectedly high impact to many corporate VSTS accounts backed by Personal Microsoft accounts, VSTS accounts are temporarily exempted from the scenario described below. Personal Microsoft accounts created from VSTS invite links or on VisualStudio.com can now be created even if they have the same sign-in name as a Work or School account. This is temporary! Some time in 2017, Personal Microsoft accounts will be permanently blocked if the sign-in name is already in use by a Work or School account.

Please use this extra time to plan your move to one of the 3 options below, and if you are setting up a new VSTS account, please use one of the 3 options below from the start.

/Jeff

Jeff here from the Windows SDK team. I am on temporary assignment to the VSTS group, so this blog post is something a little different.

Microsoft operates two cloud-scale identity systems: Personal Microsoft accounts and Azure Active Directory (AAD).
1. Personal Microsoft accounts (formerly called Live ID). These accounts are meant to be created, managed and used by individuals and are bound to Microsoft’s consumer Terms and Privacy policy.
2. Work or school accounts in Azure AD. These are typically created by IT departments for their users (employees or students). They’re meant to be used to access corporate owned resources such as Office 365 services. They are governable by IT (meaning that IT gets to control and audit the use of these accounts).

When you create a Personal Microsoft account, you either get a new email address from Microsoft (@Outlook.com, @Hotmail.com, etc.) or you can use your own email address or phone number as a sign-in name. Some users have created a Personal Microsoft account with their Work or School email address as a sign-in name. If their organization uses Office 365 or any other business service from Microsoft that relies on Azure AD for authentication, these users will end up with two Microsoft identities with the same sign-in name. When logging on with one of these “Dopplegangers” you may see this dialog:

Starting on 9/15/2016, Microsoft blocked being able to create new Personal Microsoft accounts using a Work or School email address of a domain that’s configured in an Azure AD or Office 365. There are several problems that can arise when you have duplicate Personal/Work or School accounts with the same sign-in name. It is always confusing which account you need or are using, and in some situations it is not possible to choose the correct account. To address these problems, Microsoft no longer lets you create accounts that overlap. This is explained in this blog. Previously created Personal Microsoft accounts will continue to work, though.

Trying to create a duplicate account throws the error: “You can’t sign up here with a work or school email address.”

Impact to Visual Studio Team Services

VSTS has been greatly impacted by this change. Many VSTS accounts are “Personal Microsoft account backed,” this means that you can only logon to VSTS using a Personal Microsoft account. Many businesses have been adding users to Personal Microsoft account backed VSTS accounts by inviting User@company.com. VSTS allows you to invite any user, whether or not the account already exists. In this scenario, the VSTS admin invites a new corporate user, User@company.com, and they receive the email in their corporate inbox. After receiving the email invite, they are prompted to create a Personal Microsoft account when they try to access VSTS, if needed. Many VSTS administrators assume users will be able to create the new Personal Microsoft account, but since 9/15/2016 this has been blocked. Previously invitees to VSTS successfully created Personal Microsoft accounts with their @company.com address, however new invitees may not create a Personal Microsoft account that duplicated the sign-in name of their Work School account.

One of the VSTS program managers shared this with me, as to why using a Personal Microsoft account version of User@company.com (Work or School account) isn’t secure.

“Using a corporate email address to create a personal Microsoft account does not make [it] more or less governable than using a personal email address, but it creates a expectation/perception of governability that has hurt enough customers that we’ve decided to stop that practice. 

Unlike user accounts in Azure AD,  IT pros have no administrative control of personal Microsoft accounts, regardless of the email address those accounts were created with. They can’t prevent users from changing the account username and they can’t prevent users from continuing to use those accounts after they leave the organization. They can’t apply policies to them. They can’t delete them. They can’t exert any control over what applications are accessed by these accounts. They can’t audit usage and can’t validate to authorities any particular use of those accounts. They can’t ask Microsoft to release control of these accounts or of account data.”

Now that new duplicate Personal Microsoft accounts are blocked, the only ways for new users to access VSTS are:

1)      Invite users using a real Personal Microsoft account email account (@Hotmail, @Outlook).

2)      Invite users using an email account NOT associated with Azure/O365 (@Gmail, @Yahoo). These users will still need to create a Personal Microsoft account, if they haven’t already.

3)      Create a link between your Azure/O365 AAD and VSTS.

To use options 1 and 2 you need to be OK with @Hotmail, @Gmail, etc. accounts in your VSTS account list. Options 1 and 2 are the easiest, no planning needed.

If you need your VSTS users to only use their corporate email accounts, then you need to use option 3. Option 3 is the more difficult, and requires some thought and planning. Information on linking an AAD and VSTS can be found here.

Some things to watch out for if you link your AAD and VSTS.

1)      External accounts. When you link VSTS and an AAD, only users in the AAD can access VSTS. Corporate users will be fine, but if your VSTS account had any @Hotmail, etc. accounts you will need to add them to your AAD. (When adding a user, choose ‘User with an existing Microsoft Account’.)

2)      Guest accounts. When a VSTS account is backed by Personal Microsoft accounts, there is no concept of guest users, only when you link an AAD to your VSTS account does this show up. Corporate users in the AAD will be members so they are OK, but there are several gotchas with using external Personal Microsoft accounts (guests) with a VSTS account. You can see the user type in the new Azure portal (www.azure.com.)

1. Guests can’t search the AAD to add new users VSTS. See this blog for more information and how to use PowerShell to change Guests to Members. Please note: the error message has changed recently, instead of one about AAD guests searching the directory, you will receive the error, “No identities found.”
2. A guest can’t be assigned ownership of a VSTS account. A guest can still own it, if they owned it before the AAD was linked and you then added their account to the AAD. You could also use PowerShell to switch them to a guest in the AAD (but shouldn’t.)
3. Guests can be denied all access. There is an option on the settings tab for a VSTS account, External Guest Access. It is set to allow by default, if you set it to deny then no guests can access VSTS. It is possible to have an AAD with only guests in it and in this case no one will be able to access VSTS. You would need to use PowerShell to flip the VSTS owner to be a member, then logon to VSTS and set guest access back to Allow.
   

/Jeff
 

Backup and restore privileges allow some, but not all, types of access checks to be bypassed.  Typically those are sufficient to allow a backup utility to restore data to even sensitive folders.  However, there are some exceptions to that.  For example, at the time of this writing, much of the contents under %programfiles%\WindowsApps out of the box won’t grant sufficient privileges for restore operations that need to modify data on the disk.  If I enable auditing on my Windows 10 1607 system, I may see entries such as this when trying to restore data even when running from a service as local system:

Access Reasons:  READ_CONTROL: Unknown or unchecked
WRITE_DAC: Unknown or unchecked
WRITE_OWNER: Unknown or unchecked
SYNCHRONIZE: Unknown or unchecked
ACCESS_SYS_SEC: Unknown or unchecked
ReadData (or ListDirectory): Unknown or unchecked
WriteData (or AddFile): Denied by Process Trust Label ACE
AppendData (or AddSubdirectory or CreatePipeInstance): Denied by Process Trust Label ACE
WriteEA: Denied by Process Trust Label ACE
ReadAttributes: Unknown or unchecked
WriteAttributes: Unknown or unchecked

Access Mask:  0x11E0197
Privileges Used for Access Check: SeBackupPrivilege
SeRestorePrivilege

This is probably because of the SYSTEM_PROCESS_TRUST_LABEL_ACE present in an object’s SACL; while its meaning isn’t documented, you can reason that the SYSTEM_PROCESS_TRUST_LABEL_ACE, which you can see on the object through the use of common security APIs, corresponds to the message “Denied by Process Trust Label ACE.”  You can see that by running code such as (which could easily be modified to check for said ACE if one wanted to do so):