P3.NET

Default Task Scheduler

Firstly, the new Task API in .NET v4 is awesome.  It really makes working with threading easier.  Unfortunately you really have to understand how it works otherwise you will likely make the code harder to read and use.  Here’s an interesting case I ran into recently.

Attached is a simple WinForm application that is, I suspect, relatively common.  The application displays a simple UI with a button.  When the button is clicked a lengthy operation is performed.  To let the user know that the application is working a progress bar is shown.  The user can cancel the operation through a button at any time.  When the task completes a message box appears displaying success or failure.  Here is the important parts of the code used to start the lengthy work on another thread using the Task API.

private void StartWork ( )
{
   m_cancel = new CancellationTokenSource();

   var scheduler = TaskScheduler.FromCurrentSynchronizationContext();

   var task = Task.Factory.StartNew(DoWork, m_cancel.Token, m_cancel.Token)
                .ContinueWith(OnFinished, CancellationToken.None, TaskContinuationOptions.OnlyOnRanToCompletion, scheduler)
                .ContinueWith(OnCancelled, CancellationToken.None, TaskContinuationOptions.OnlyOnCanceled, scheduler)
                .ContinueWith(OnError, CancellationToken.None, TaskContinuationOptions.OnlyOnFaulted, scheduler);
}

Straightforward stuff – start a new task and, based upon the results, call one of several methods to finish up.  The cleanup methods are all called on the UI thread so we can update the UI.  DoWork just loops for 10 seconds checking the cancellation flag.  If you run the sample application then you should see the UI being responsive while the task runs.  Here’s the cancel method.

private void OnCancelled ( Task task )
{
   try
   {
      task.Wait();
   } catchh
   { /* Eat it *// };

   if (MessageBox.Show(“Task cancelled.  Try again?”;“Question”, MessageBoxButtons.YesNo) == DialogResult.Yes)
   {
      StartWork();
   };                        
}

Remember that the task resources remain allocated until you do something to let the framework know you are working with the completed task.  Such activities include checking the Exception or Result properties or calling Wait.  The cancel method calls Wait to complete the task and then asks the user if they want to try again.  If the user clicks yes then a new task is started.

Think about what is happening during this process.  Initially when the user clicks the button to start the task the UI thread calls StartWorkStartWork starts a new task on a non-UI thread and returns.  In the cancel method, which is run on the UI thread, the StartWork method is called to do the same thing.  They should behave identically – but they don’t.  Try running the program, starting the task, cancelling the task and then clicking Yes to restart the task.  The UI freezes while the (new) task is being run.  What is going on here?

I can honestly say I have no idea.  It appears as though the factory, when it is trying to figure out what scheduler to use, uses the current scheduler if there is already one available.  But this wouldn’t make any reasonable sense.  Nevertheless the sample program demonstrates this very behavior.  Fortunately the fix is easy.  You just need to make sure that you ALWAYS specify a scheduler when creating a new task.  In this case the default scheduler is fine.  If you modify StartWork to look like this then the problem goes away.

private void StartWork ( )
{
   m_cancel = new CancellationTokenSource();

   var scheduler = TaskScheduler.FromCurrentSynchronizationContext();

   //var task = Task.Factory.StartNew(DoWork, m_cancel.Token, m_cancel.Token)
   var task = Task.Factory.StartNew(DoWork, m_cancel.Token, m_cancel.Token,TaskCreationOptions.None, TaskScheduler.Default)   
                    …
}

The only difference is that we are now specifying that the default scheduler should be used.  You would assume that the default scheduler to use when no scheduler is defined would be the default scheduler but I guess not.  Nevertheless this is a really odd behavior that might or might not be a bug in the implementation. 

Extension Method Guidelines Rebuffed

With the additional of extension methods developers have a powerful new way of extending existing types.  As extension methods become more and more common guidelines will be needed to ensure that extension methods “fit in” with the general design of the library.  If you are not aware of what an extension method is then the quick summary would be that it is a method that appears as an instance method of a type but is, in fact, defined in a separate static class.  This allows us to extend existing types without modifying the type itself.  It really becomes a powerful feature when applied to interfaces.  We can enhance an interface to expose a lot of new functionality while not requiring the interface implementer to do any extra work.  LINQ is a good example of this extensibility.  I personally tend to view extension methods more as static methods on a type  that be called using the instance-method syntax.  This, to me, gives a more accurate picture of what is happening and, as you will see, what is allowed.

Currently there are a lot of “guidelines” floating around that people like to mention.  Unfortunately it is still too early to determine which of these are truly best practices and which are opinions.  One guideline in particular bothers me the wrong way: extension methods should throw a NullReferenceException if the source is null.

Let’s start with a regular method as an example.  This method will return everything to the left of a particular string in another string.

static string Leftof ( string source, string value )
{
   if (String.IsNullOrEmpty(source))
      return “”;

   if (String.IsNullOrEmpty(value))
      return “”;
   …
}

This is a pretty handy function so it makes sense to make it an extension method so we might do this.

static class StringExtension
{
   public static string Leftof ( this string source, string value )
   {
      if (String.IsNullOrEmpty(source))
         return “”;

      if (String.IsNullOrEmpty(value))
         return “”;
      …
   }
}

So far so good but it violates the aforementioned rule about throwing an exception if a null is passed in.  The rationale is that an extension method appears to the developer as an instance method and should therefore behave like one.  While that is a reasonable argument I don’t think it applies in all cases.  Why do we have static methods on types, such as String.IsNullOrEmpty?  The general reason is because the method does not need an instance to perform its work.  But if the method itself considers null to be valid then the method would also need to be static.  So syntactically we cannot allow an instance method to accept null but static methods (and by definition extension methods) we can.

One of the big arguments against this approach is that “extension methods should look and act like instance methods”.  Why?  The only real reason I can see is because they look like instance methods when used but we, as developers, are use to the actual code being quite different than what we expect.  The assignment operator, for example, we would never expect to throw an exception yet it can if the type implements the assignment operator and does something inappropriate.  Addition, subtraction and other operators are the same way.    How about the relational operators? 

Data value1 = null;
Data value2 = null;
            
bool result = value1 < value2;

We would never expect the above code to throw an exception and yet a method call is occurring under the hood.  If that method throws an exception then so does the above code.  As developers we have to be aware that we don’t know everything that is going on under the hood so we should expect errors anywhere.

Another problem I have with this guideline is the solution.  The guideline is that extension methods should behave like instance methods but that simply isn’t possible.  If you call an instance method with a null reference then you will get a NullReferenceException.  This particular exception is one of a handful of system exceptions.  A system exception is an exception that is raised by the runtime proper.  Normal code, aka ours, should not throw system exceptions.  If you explicitly throw a system exception then code analysis tools like FxCop will generate CA warnings about it.  That is what you want.

Instead we have “application” exceptions.  Hence when we receive an argument that is null and we do not allow them then we throw ArgumentNullException.  So to make our extension method behave like an instance method we would need to either throw a system exception explicitly (which we aren’t suppose to do) or reference the source variable.  Here’s what we’d have to do to our earlier example to get it to throw the appropriate exception while still making code analysis happy.

public static string Leftof ( this string source, string value )
{
   //Force a NullReferenceException
   int len = source.Length;

   if (String.IsNullOrEmpty(value))
      return “”;

   …
}

I have a really big problem writing code just to force something to happen like throw a specific exception.  If I really wanted the exception I’d just throw ArgumentNullException.  But if I do that then I’m no longer making my extension method act like an instance method.  This becomes even more of an issue if you can short-circuit the method return based upon other argument values. 

A final argument for the guideline is that if an extension method eventually becomes an instance method then your application should behave the same but the reality is that you should have unit tests to verify your application’s behavior so a major change like going from extension to instance should be thoroughly tested anyway.  Here are my guidelines for extension methods.  Take them with a grain of salt.

  • Extension methods should be placed in a static class called TypeExtension.  For interfaces the “I” can be left off.
  • Extension classes should be only for extension methods and should not contain non-extension code.
  • Each extension method’s first parameter should be this Type source.
  • Use an extension method only if the source parameter makes sense and is used in the method.  Use a normal static method otherwise.
  • An extension method should not throw ArgumentNullException for source.    Other arguments are fine.  If source cannot be null then referencing the value will be sufficient to generate the correct exception.
  • If null is not necessarily an invalid value then do not throw any exceptions.  An extension method does not need to follow instance method semantics.
  • Document the behavior if source is null if it is not going to throw an exception.

App.config Magic In Visual Studio

One area that causes lots of confusion to developers is the behavior of app.config while running within Visual Studio.  Because VS treats app.config specially it can be confusing when app.config does not work the way we expect.  This post will clarify how the config file is treated while running inside VS.  But first an aside.

Visual Studio Hosting Process

Debugging another process is not trivial.  There are lots of details to worry about.  Consequently debuggers are not simple pieces of code.  Starting a debugger can be expensive.  Beginning a few versions back VS started using the VS hosting process for debugging processes.  This process (vshost) is a wrapper around the debugger.  It allows the debugger to be started onced and then continue to run between debug sessions.  This helps to alleviate the overhead of the debugger and to also allow the debugger to maintain information between debugging sessions. 

The vshost process is started when VS loads a project that is using it.  You can see this by using Task Manager to see the running processes.  The actual process is called <project>.vshost.exe.  It is within this process that your code runs rather than through the normal <process>.exe.  Running through a host process is generally the best idea but it can cause some issues.  Therefore in the Debug properties of the executable’s project settings is an option to turn off the hosting process.

App.config

In .NET an application configuration file is required to be called <process>.exe.config.  This is the file that the configuration subsystem will load.  The file must reside in the same directory as the executable.  Because VS recreates the output directory each time you build you cannot just create this file and store it in the output directory.  Instead when you add an Application Configuration File item to your project it is added to the root of the project and called app.config.  This file is automatically copied to the output directory and renamed on each build.  Therefore after each build you will have a brand new configuration file.  Note that if you call the file anything else then this process will not work.  Rarely should you change the name.

For most development you will edit the configuration file in VS.  These changes will be automatically seen when the program runs under the debugger.  However if you make changes to the configuration file at runtime (while running under the debugger) the changes will only persist until the next build. At that time VS will overwrite the file with the root copy.  Therefore you CAN NOT run your program and modify the project’s configuration file.  The only way to emulate this behavior would be to copy the (modified) configuration back to the root project and rename it.  As an aside note that if you modify the configuration file at runtime but do not rebuild your code then the configuration changes may persist. 

Which Config Is Which?

One area of confusion is which configuration file your application is actually using while running under the debugger.  Many people will say it is the <process>.exe.config file but that is only partially correct.  Remember the vshost process of earlier?  It complicates things.  Remember that if you are using the VS host (the default) then your process is actually <process>.vshost.exe.  The configuration subsystem knows nothing about the VS host process so it expects to load the <process>.vshost.exe.config file.  When a project is configured to use the hosting process then VS will copy the standard <process>.exe.config file to <process>.vshost.exe.config when the debugger starts.  This gives the appearance that the hosting process is using the application’s configuration file when in fact it is VS that is just copying the file for us.  More importantly the configuration subsystem is not impacted by this change at all.  If the program modifies the configuration file while it runs then those changes are made to the <process>.vshost.exe.config file.  The actual process’s configuration and the project’s app.config files are not modified.   

DLL Config Files – Myth or Fact?

Many class libraries (DLLs) require configuration information.  Most devs like to rely on the configuration subsystem to store this information.  Makes sense.  If you add an application configuration to the class library project then VS will copy the app.config to the output directory and call it <project>.dll.config as you would expect.  It behaves just like a standard Windows application project.  Here’s the issue though – the configuration subsystem neither understands nor supports these files.  Let me say it again.  The configuration subsystem WILL NOT load .dll.config files. 

Technically you can change the mappings used by the subsystem to get it to load the files but that is beyond the scope of this article and beyond most applications.  The configuration subsystem will only load the <process>.exe.config file.  There are forum postings all the time from devs who are confused as to why their dll.config file contains some configuration section but the configuration subsystem won’t read it.  They believe, incorrectly, that the subsystem uses the configuration file associated with the assembly that is calling it.  That is false.  The subsystem only reads the application’s configuration file.

There are two possible workarounds to this issue.  The first workaround, the most common, is to merge the dll.config file into the application’s configuration.  This is generally the easiest approach and can be done at build time.  The alternative approach is to read the configuration file manually at runtime.  Some complex libraries follow this route but I cannot recommend it.  That is just too much code to write.

When Regular Expressions Go Bad

As a compiler guy I’m very comfortable with “standard” regular expressions (REs).  After all scanners for compilers are predominantly written using regular grammars (of which REs are derived).  REs have been around since the early days of computers and yet are still not truly standardized.  Each implementation adds its own twists and rules.  As a result an RE written for one implementation might or might not work (or behave) the same in another.  Therefore whenever REs are involved proper testing is important.  Excluding this detail REs are really, really useful for validating string formats.  While REs cannot validate all string formats they can handle a large subset.  Hence I tend to use REs for validating string formats whenever possible.  One of the big benefits of REs is speed.  Most parsing can be done through REs faster and easier than through hand-generated code.

In .NET The Regex class is available for writing REs.  Here is a sample piece of code I wrote to validate that a string is a properly formatted DNS name.

private static bool Validate ( string value )
{
   Match m = Regex.Match(value, @”^(?<label>[a-zA-Z]([w-]*[a-zA-Zd])?)(.(?<label>[a-zA-Z]([w-]*[a-zA-Zd])?))*$”);
   if ((m == null) || !m.Success)
      return false;

   //Validate the length of each label
   foreach (Group group in m.Groups)
   {
      if (group.Length > 63)
         return false;
   };

   return true;
}

In a nutshell a properly formatted DNS name obeys the following rules::

  • The name is divided into one or more parts. Parts are separated by a dot.
  • Each part must begin with a letter.
  • Each part must end with a letter or digit.
  • Parts may contain letters, digits or dashes (and underscores if you are supporting NetBIOS).
  • A part must be between 1 and 63 characters long.

The above code uses an RE to validate all but the last rule.  In .NET REs you can capture substrings out of the matched expression using capture groups.  In the above code I capture each part separately and then, if the RE validates, enumerate the parts to ensure they are of the appropriate length.   Overall the code is small and easy to read.  I tested the code (using an outside program) against valid and invalid strings to ensure all the rules were correct.  This code is about a year old and is used in my application as part of UI validation, amongst other things.

The other day I was writing some code that relied on the DNS validation code shown earlier.  Basically as the user typed in a textbox the string was validated and controls were enabled or disabled (along with error messages).  So far so good.  I was trying to test an aspect of the UI so I happened to enter a medium size string and accidentally added a semicolon to the end (dfafddafafadfafsadf;).  My application locked up.  Concerned I stepped through my code and found that it was locking up inside my validator (in the RE matching).  Hmm.  It appears that the RE caused an infinite loop.  To verify it was .NET and not something with my code I copied the RE and the input to an external program that happens to use .NET as well and it also locked up.

Actually it was not an infinite loop it just took a really, really long time to finish up.  That’s odd.  What makes the particular input interesting was that it was long and would ultimately fail validation because of an invalid character.  I checked my RE to determine if I was doing anything that might cause the parser to have to do backtracking.  Backtracking occurs when a parser gets into an error state and decides to back up the input until it can recover so it can try a different path.  As strings get longer this can become very time consuming.  With this particular input the parser would reach the semicolon and, in this case, deem the input invalid.  Nowhere in the RE is a semicolon legal.  In my mind the parser should not have to do any backtracking because the semicolon is not valid anywhere in the RE.  Backtracking would do nothing but waste time.  Nevertheless this does not seem to be the case with the .NET version.  Note that this doesn’t mean the parser has a bug but rather the implementation has some worse case paths for some grammars.

Theoretically I could probably modify my RE to use either aggressive parsing and/or different rules but this would require me to: look up how to do it in .NET, document this behavior for other devs who might need to maintain it and test it thoroughly since it might introduce a completely separate set of issues.  Instead I decided that the validation rules are simple enough that I could just ditch the RE and implement the logic myself.  Here is the new version sans RE.

private static bool Validate ( string value )
{
   string[] parts = value.Split(‘.’);
   foreach (string part in parts)
   {
      int len = part.Length;
      if (!Range.InRange(len, 1, 63))
         return false;

      //Validate the part
      if (!Char.IsLetter(part[0]) || !Char.IsLetterOrDigit(part, len – 1))
         return false;

      foreach (char ch in part.Skip(1))
      {
         if (!Char.IsLetterOrDigit(ch) && (ch != ‘_’;) && (ch != ‘-‘))
            return false;
      };
   };

   return true;
}

After implementing this new version I confirmed that my inputs (valid and invalid) still work.  Even better is that the performance is just as fast as the original RE.  Even better than that, the code is even more readable with no loss of functionality or time.  Note that I haven’t fully tested this code yet so if you borrow it then be sure to test it before using it.

Consider it a lesson learned.  Use REs when they meet your need.  When you do use REs be sure to create enough test cases to verify the behavior.  Specifically make sure you test invalid characters throughout strings of varying length and verify the parser does not take too long to run.  If you do run across worse case inputs then either special case these or switch to manually parsing the code.

Computer Science Education

Recently I read a blog post from Keith Ward about Computer Science education.  It struck a cord for a couple of reasons.  Firstly I’m a computer professional by day and a CS professor at night.  I teach both basic computer and programming courses at my college.  I’ve been teaching for years.  Secondly I’ve seen the arguments on both sides of the fence about the “best” way to teach students to ensure they are successful.  Thirdly I’ve interviewed hundreds of people over the years for various positions and I’ve seen the results of both good and bad educations.  Rather than posting a long message in Keith’s blog about my opinion I figured it would be better to post it here since I’ve been meaning to discuss it for a while.

Computer science is hard.  When I went to college we had math professors who learned to program and then taught us.  The teaching involved the professor going over some programming concepts and throwing up sample code in class and the students going home and working on programming labs for hours.  You might take a 2 semester programming course and then move on to the other computer topics.  In the early courses it was not unheard of to have a project due every 2 weeks.  You were on your own to learn your way around the computer, the IDE and the language.  Debugging was trial and error.  If you really needed help you could talk with the professor but otherwise you were on your own.  In later courses you might have to learn an entirely new language (C or C++ at the time) on your own in order to do the assignments in class because a professor would only accept certain languages. This was above and beyond the course material.  Back then computer programming was new, exciting and in high demand.  Folks were there to get a degree in something that was seen to be rewarding.

Fast forward to today where schools offer 2 or 3 different languages.  The core programming language (often C++) might be a 2 or 3 semester course.  The number of projects have dwindled down to 5 to 7.  Students are expected to get most of the information during class including how to debug, use the IDE, write code and test.  Needless to say that is a lot of material to cover in 15 weeks (a standard semester).  There is almost no time for professors to slow down to ensure the class is getting a particularly tough section.  There’s also no time to address any additional information that is outside the curriculum (IDE usage, debugging, etc).  The CS industry is becoming saturated.  CS is no longer a specialty career.  Students aren’t interested in working harder to get a degree when there are other, equally rewarding, careers available.

In order to write code you need to know several things: the language, the IDE, design methodologies and debugging.  Traditionally the language is taught first.  Consequently the IDE has to be learned by the student.  Debugging code is generally an on-the-fly skill learned by the student with assistance from the professor.  Proper design comes later after the programming courses are done.  So out of the box the student has to pick up the IDE and debugging themselves while learning a new language and, even more important, an entirely new thought process.  Statistically speaking most schools see either a low enrollment for the courses or a low retention rate for subsequent courses.  Is it any wonder?  There is so much work involved just to become a basic programmer that most students have neither the time nor the desire to continue.

What can we, the educators, do to solve this issue?  The (somewhat) popular approach is to dumb down the class; make only a few simple projects due, provide simple exams that test only rudimentary knowledge, and/or give large curves.  What does this do?  It encourages the students to continue in the program.  No matter how we dice the problem it eventually is going to boil down to the retention rate.  A department (CS or otherwise), no matter how good it is, cannot survive unless students enroll in the courses.  The harder the courses the lower the retention rate. 

I worked at a college (for one semester) that followed this philosophy wholesale.  We, as instructors, were told that if a student showed up for class then they passed.  It didn’t matter how they did on assignments (of which we should give few) or exams (which should be easy).  The important thing was that students wanted to come back the next semester.  Needless to say I didn’t follow that philosophy.  It was a rough semester for me.  I would never work there again nor did they ever call me back.  In this particular case the school put retention over the quality of the students’ education.  These students were going to pay a lot of money for a degree and then not be able to find a job.

As an interviewer (at least for entry positions) I’m interested in what concepts you can put into play.  It isn’t good enough that you can give me the definition of encapsulation if you can’t also provide me a reason why you would use it.  In my experience you will get one of three students in an interview (again, entry level): book smart, straggler and earner.  The book smart student can tell you the textbook definition of any concept you want.  The problem is they cannot identify it in the wild or know when to use it.  Even worse is any variations on a theme will cause confusion.  These students have been taught to read a book and answer review questions.  Their instructors failed to teach them the why’s and how’s of the concepts.  Such students will easily pass exams but struggle with real-world tasks.  This, in my opinion, is a failing of the educators who put too much emphasis on passing exams and too little on practical usage.

The straggler is the student who didn’t do well in most courses but managed to pass anyway.  They might or might not know the answers to even the most basic questions.  How someone such as this could get a degree is beyond me.  Early on the instructors should have properly tested the student and determined that they do not have sufficient knowledge to pass the course.  It is a fact of life that not everyone can be a CS person.  It is important for the student to identify this early on.  Yes the school will lose a student but it is better than having the student struggle 4 years and then walk away with a degree that they might not be able to use.  Where is the fault here?  In this case it can lie anywhere.  Some instructors are more concerned about passing students either to make the college or their approval ratings higher (http://www.ratemyprofessors.com/).  These instructors are only hurting their students.  I’d rather a student struggle through my class and barely pass but know the material well than pass with flying colors but know nothing.  However it might also be the student’s fault (gasp).  Some students are interested in easy grades so they’ll do whatever they can to do the bare minimal.  As an instructor I can’t force students to try and excel.  If they want to eek by then all I can do is grade fairly and warn them about the potential issues they will face down the road.

Finally the earner has done their best, graduated with a degree and knows the material.  The instructors have done their job and taught the material and the student has done their best to understand it.  This is the student who employers like because they can be molded into a productive employee.  Unfortunately in the modern era of quick degrees and lackluster colleges it is harder to find these students.  Not every student is going to understand every concept they were taught but an earner has done the work and demonstrated their desire to at least try.  Note that an earner isn’t necessarily a straight A student.  When I look at student grades (as an interviewer) I’m generally more interested in students who struggled through at least a couple of courses.  I’m especially interested in students who failed/withdrew from a course and then retook it.  That shows initiative and a desire to succeed.  Failures happen.  It is how you respond to them that tell’s me what type of person you are.

To wrap up I have to share my opinion of professor ratings.  There are sites available (see above) that allow students to rate professors.  Additionally most schools have mandatory student evaluations of their instructors.  For the school the evaluations determine which instructors to keep and which to let go.  For students it allows them to share their opinions.  The problem is that it is just that: an opinion.  It is a fact of life that not every student is going to like every professor and vice versa.  Unfortunately it is very easy to skew ratings/evaluations because of personality conflicts.  For this reason many instructors are impersonal to students to avoid conflicts (ironically, impacting their rating).  While I think the evaluation of instructors is important to weed out those who are not providing optimal education I think the entire picture must be taken into account.  A single set of bad reviews doesn’t necessarily mean the instructor is bad.  Finding patterns in the reviews is more likely to provide a better picture.  As an instructor I am always interested in hearing feedback about my courses.  Sometimes I get the summary information from the student evals but more often than not I never see them.  As a result I ask the students to provide me feedback as they see fit.  Dangerous?  Yes.  Helpful?  Most definitely.   The most discouraging thing about this is why I see a lot of bad reviews for an instructor on a site and yet that instructor is still teaching.  Are the students giving the school one set of evaluations while posting another?  I don’t know.  Perhaps the schools are not doing anything with these evaluations in which case it is a waste of everyone’s time.  More likely however the review sites are being filled with negative (or positive) comments by people who are posting their opinions irrelevant of the actual quality of the instructor.  As a result I’m still not convinced professor rating sites are a good thing but until student evals are publicly disclosed and acted upon by schools there seems to be little alternative for students who want the best educators.

Updating the UI On an Arbitrary Thread

(Originally published: 5 March 2007)

It is very common to have to perform work on a secondary thread.  This allows us to provide a more interactive UI to our users while still allowing us to do lengthy work.  Unfortunately this is not as easy as it first appears.  We will discuss the problem, the standard v1.x solution and the v2 solution to the problem.

Why Do We Need It

Windows has always had the cardinal rule that a UI element can only be interacted with on the thread that created it.  This is because internally Windows still uses a message queue to communicate with UI elements.  The message queue is per-thread.  It is created on the thread that created the element.  This ensures that all messages are serialized to the UI and simplifies our work. 

However it is often necessary to do work on secondary threads and still update the UI.  In order for this to work we have to actually send the request to the UI’s thread and then have it pushed onto the message queue.  There are a couple of different ways to do this.  We’ll look at the most common method in v1.x and v2.  .NET v3.x introduces the concept of synchronization contexts which can be used to execute code on specific threads.  Synchronization contexts are a lengthy topic that will not be covered in this article.

v1.x Solution – Invoke and InvokeRequired

Every UI element (a control, here on out) has a InvokeRequired property.  This property can be called on any thread and indicates whether the calling thread is the thread that created the control.  If it is then the property is false otherwise it is true.  When working with controls in a multithreaded environment you must check this property.  If it is true then you must marshal the request to the UI thread. 

You marshal the request to the UI thread using the Invoke or BeginInvoke/EndInvoke methods.  These methods can be called on any thread.  In each case they accept a delegate, switch to the UI thread and then call the delegate.

if (myControl.InvokeRequired) 

   myControl.Invoke(…);
else 
   …

To make things easier we generally wrap the above code in a method and then create a delegate with the same signature.  This allows us to use one function for two different purposes.  The first “version” is called in all cases and determines whether an invoke is required.  The second “version” actually does the work on the UI thread.  This is a nice way to hide the details from your clients.

public delegate void UpdateProgressDelegate ( int progress );

public void UpdateProgress ( int progress )
{
   //If not on UI thread
   if (myProgress.InvokeRequired)
      myProgress.Invoke(new UpdateProgressDelegate(UpdateProgress), new object[] { progress });
   else
   {
      //Do work here – called on UI thread
      myProgress.Value = progress;
   };
}

As far as your users are concerned this is a clean interface.  They simply call UpdateProgress.  Internally if an invocation is required to get the call on the UI thread then it does so.  However it calls the same method again.  The second time it is called it will be done on the UI thread.  The biggest downside to this code is that: a) you have to write it, and b) you have to define a delegate.  Here is how we might call it.

public void Run ( )
{
   Thread t = new Thread(new ThreadStart(DoWork)); 
   t.IsBackground = true
   t.Start();
}

public void UpdateProgress ( int progress ) 
{ myProgress.Value = progress; } 

private void DoWork ( ) 

   for (int nIdx = 1; nIdx <= 100; ++nIdx) 
   { 
      Thread.Sleep(50); 
      UpdateProgress(nIdx);
   }; 

v2 Solution – BackgroundWorker Component

Since this is such a common situation in v2 Microsoft has added the BackgroundWorker component (BWC).  This component’s sole purpose in life is to do work on a secondary thread and then update the UI.  The component requires that you identify the function to run.  It will then run the function on a thread pool thread.  Once it is complete the component will raise the RunWorkerCompleted event.

public void Run ( )
{
   BackgroundWorker bwc = new BackgroundWorker(); 
   bwc.DoWork += DoWork; 
   bwc.RunWorkerCompleted += OnCompleted; 
   bwc.RunWorkerAsync();    //Run it
}

private void OnCompleted ( object sender, RunWorkerCompletedEventArgs e ) 
{ MessageBox.Show(“Done”); } 

private void DoWork ( object sender, DoWorkEventArgs e ) 

   for (int nIdx = 1; nIdx <= 100; ++nIdx) 
   { Thread.Sleep(50); }; 
}

Notice a few things about the code.  Firstly it is smaller as we don’t have to worry about the thread management.  Secondly notice we do not have to worry about the UI thread.  All events exposed by BWC are raised on the UI thread automatically.  Note that you can drop BWC onto your form or control because it shows up in the designer.  Personally I prefer to create it on the fly.

There is a problem with the above code.  It does not actually update the progress while it is running.  In order to raise progress events we need to tell BWC we will be raising progress notifications.  We also need to hook up another function to handle the event.

public void Run ( )
{
   BackgroundWorker bwc = new BackgroundWorker(); 
   bwc.DoWork += DoWork; 
   bwc.RunWorkerCompleted += OnCompleted; 
   bwc.WorkerReportsProgress = true
   bwc.ProgressChanged += OnProgressChanged;

   bwc.RunWorkerAsync();    //Run it
}

private void OnProgressChanged ( object sender, ProgressChangedEventArgs e ) 
{ myProgress.Value = e.ProgressPercentage; }

private void OnCompleted ( object sender, RunWorkerCompletedEventArgs e ) 
{ MessageBox.Show(“Done”); } 

private void DoWork ( object sender, DoWorkEventArgs e ) 

   BackgroundWorker bwc = sender as BackgroundWorker; 
   for (int nIdx = 1; nIdx <= 100; ++nIdx) 
   { 
      Thread.Sleep(50); 
      bwc.ReportProgress(nIdx); 
   };
}

The changes are in bold.  Basically the UpdateProgress of earlier changes to OnProgressChanged.  Again, it is called on the UI thread.  We have to report progress using BWC.ReportProgress.

One final feature of BWC that was not in the original code we wrote is cancellation support.  It is often nice to allow users to cancel lengthy tasks.  BWC supports this automatically.  All you have to do is tell it that you will allow cancellation and then periodically check for a cancellation request.  For algorithms that loop you generally check at the beginning of each loop.  The more frequently you check the cancellation flag (CancellationPending) the faster your user will see the process be cancelled.

public void Run ( )
{
   BackgroundWorker bwc = new BackgroundWorker(); 
   bwc.DoWork += DoWork; 
   bwc.RunWorkerCompleted += OnCompleted; 
   bwc.WorkerReportsProgress = true
   bwc.ProgressChanged += OnProgressChanged;
   bwc.WorkerSupportsCancellation = true
   bwc.RunWorkerAsync();    //Run it
}

private void btnCancel_Click ( object sender, EventArgs e ) 
{ bwc.CancelAsync(); }

private void OnProgressChanged ( object sender, ProgressChangedEventArgs e ) 
{ myProgress.Value = e.ProgressPercentage; }

private void OnCompleted ( object sender, RunWorkerCompletedEventArgs e ) 
{
   if (e.Cancelled) 
      MessageBox.Show(
“Cancelled”); 
   else 
      MessageBox.Show(“Done”);

private void DoWork ( object sender, DoWorkEventArgs e ) 

   BackgroundWorker bwc = sender as BackgroundWorker; 
   for (int nIdx = 1; nIdx <= 100; ++nIdx) 
   { 
      if (bwc.CancellationPending)
      {
         e.Cancel = 
true;
         
return
      };

      Thread.Sleep(50); 
      bwc.ReportProgress(nIdx); 
   };
}

Here we’ve set the property notifying BWC that we support cancellation and we check for cancellation during each iteration of the loop.  We also modified the completion handler to detect whether the operation completed or was cancelled.  The only hard part here is actually hooking a control (such as a button) up to cancel the work.  If you use the designer to drop the BWC on your form then you can use the field that is created to access the BWC.  In the above code it was a local variable so you would have to somehow store the local variable somewhere where the cancel button’s click handler could get to it.  I leave that for you to figure out.

Why Not Do It Automatically

Some people ask why, if it is so common, Windows does not do this automatically or why the framework developers didn’t do it, or why they shouldn’t do it themeselves.  The answer is performance.  As common as it might be most UI interaction still occurs on the appropriate thread.  It is not really worth the performance hit to always handle the situation.  This is a common theme throughout the framework.  Another problem with threading messages is the order.  In general if you send two messages (one right after the other) then you expect the first message to be processed first and then the second message.  With asynchronous calls this might not be true.  Therefore more effort must be put into ensuring that the ordering of messages is not important. 

Most classes in the framework are not thread safe.  The reasoning is that it adds complexity and degrades performance.  A good multithreaded appliation does not allow just any object to be accessed across threads.  This would be a bad design.  Instead only a select few objects are shared and it is these objects that need to be thread safe, not the entire application.

Until Windows sheds its message queue behavior I’m afraid we are stuck with this limitation.  Although I believe WPF has somewhat remedied the situation…

Persisting Form Settings

(Originally published: 15 September 2007)

This article was originally written when .NET v2 was released.  Some of the information in here may be outdated with newer .NET versions but there is still some good information on persisting form settings and avoiding some pitfalls in the process.  Therefore this article is being posed unchanged from its initial form.

When working with Windows applications users expect that the application will remember its state when it is closed and restore that state when it is opened.  This allows the user to customize the layout of the application to fit their needs.  If Bill only works with a single application at a time he might like to have the application take up the whole screen so he can maximize his view.  Julie, on the other hand, runs several applications at once.  She wants to keep windows laid out in a certain order so she can efficiently copy and paste data between applications.  It doesn’t make sense to require Julie and Bill to share the same window layouts when their individual settings can be stored easily.

In the early betas of .NET v2 this functionality was available.  It seems to have been removed before the final release.  This article will discuss how to add in form state persistence to your WinForms applications.  This article will only deal with a single form and the basic form attributes but it can be extended to multiple forms with many different properties including tool windows and docking state.

Where to Store the Data

For persisting data on a per-user basis you basically have three common choices: registry, isolated storage and settings file.  The registry is commonly used for old-school programming.  The HKEY_CURRENT_USER key is designed for storing per-user settings.  It works a lot like a file system.  The registry is generally not recommended anymore.  It is designed for small pieces of data and has limited support for data formats.  It is, however, secure and requires more than a passing knowledge of Windows to use properly.  Therefore it is a good choice for settings that should generally be protected but not to big in size.  A big limitation of the registry is that it can’t be used in applications that don’t have registry access (like network applications or smart clients).

Isolated storage is a step up the file system hierarchy.  It looks like a file system (and actually resides in the system somewhere) but its actual location is hidden from applications.  Isolated storage allows any application to store data per-user.  The downside to isolated storage is that it can be a little confusing to use.  Additionally, since the actual path and file information is hidden, it can be hard to clean up corrupt data if something were to go wrong. 

Finally there is the settings file.  We are talking about the user settings file here, not the application settings file.  Each user can have their own settings file.  This file works similar to the application property settings file that you can use for application-wise settings.  The difference is that each user has their own copy and it is store in the user’s profile directory.

Before moving on it is important to consider versioning of the settings.  If you want a user running v1 of your application to be able to upgrade to v2 and not lose any of their settings then you must be sure to chose a persistence location that is independent of the version of the application.  The registry is a good choice here as isolated storage and user settings are generally done by application version.  Still it doesn’t make sense in all cases to be backward compatible with the settings file.  You will have to decide on a case by case basis.

FormSettings

Let’s start with the basic class we’ll use.  Ultimately, since we might have quite a few forms to persist we want to create a base class (FormSettings) that will take care of the details.  We can derive from this class for custom form settings as needed.  In this article we will use the user’s setting file so we derive from ApplicationSettingsBase.  If you want to use a different storage mechanism then you’ll need to make the appropriate changes. 

Since we want our class to work with multiple forms we need to make each form unique.  We will use the SettingsKey to make each form unique.  Each form must specify the key it will use.  Here is the start of our class.

public class FormSettings : ApplicationSettingsBase 

   public FormSettings ( string prefix ) 
   { 
      SettingsKey = prefix; 
   } 

   public void Load ( Form target )  />   {
      //Load
   }

   public void Save ( Form target ) />   {
      //Save
   }
}

When the form is loaded it will call Load to load its settings.  When the form is closed it will call Save.  Here is sample code for our form.

protected override void OnLoad(EventArgs e) 

    base.OnLoad(e); 

    m_Settings.Load(this); 

protected override void OnFormClosing ( FormClosingEventArgs e ) 

    base.OnFormClosing(e); 

    if (!e.Cancel) 
        m_Settings.Save(this);
}

private FormSettings m_Settings = new FormSettings(“MainForm”);

 

Persisting Properties

At a minimum a user would expect to be able to move the window around and resize it to fit their needs.  Therefore we need to load and save the following properties: DesktopLocation, Size and WindowStateDesktopLocation specifies the position, relative to the top-left corner of the desktop, of the top-left corner of the form.  The Size property indicates the width and height of the form.  Finally the WindowState is used to track when a window is minimized or maximized.  We will discuss this property shortly.

To load and save properties using the settings file we need to define a property for each item we want to save.  We need to mark the property as user-scoped and we need to get the value from and set the value to the settings file.  This is pretty straightforward so we will not dwell on the details.  Here is the code for getting and setting the property values.  One point of interest is that we use special values when we can not find the property values.  This will come back later when we talk about loading the settings.

public class FormSettings : ApplicationSettingsBase  
{
   … 

   [UserScopedSetting] 
   public Point Position  BR />   { 
      get 
      { 
         object value = this[“Position”]; 
         return (value != null) ? (Point)value : Point.Empty; 
      } 
      set { this[“Position”] = value; } 
   } 

   [UserScopedSetting] 
   public Size Size  R />   { 
      get 
      { 
         object value = this[“Size”]; 
         return (value != null) ? (Size)value : Size.Empty; 
      } 
      set { this[“Size”] = value; } 
   } 

   [UserScopedSetting] 
   public FormWindowState State  R />   { 
      get 
      { 
         object value = this[“State”]; 
         return (value != null) ? (FormWindowState)value : FormWindowState.Normal; 
      } 
      set { this[“State”] = value; } 
   } 
}

 

Saving the Settings

Saving the settings is really easy.  All we have to do is set each of the user-scoped property values and then call Save on the base settings class.  This will flush the property values to the user’s settings file.

public void Save ( Form target ) 

   //Save the values 
   Position = target.DesktopLocation; 
   Size = target.Size; 
   State = target.WindowState; 

   //Save the settings
   Save(); 
}

 

Loading the Settings

Loading the settings requires a little more work.  On the surface it is similar to the save process: get each property value and assign it to the form.  The only issue that comes up is what to do when no settings have been persisted (or they are corrupt).  In this case I believe the best option is to not modify the form’s properties at all and, therefore, let it use whatever settings were defined in the designer.  Let’s do that now and see what happens.

public void Load ( Form target ) 

   //If the saved position isn’t empty we will use it 
   if (!Position.IsEmpty) 
        target.DesktopLocation = Position; 
   //If the saved size isn’t empty we will use it 
   if (!Size.IsEmpty) 
     target.Size = Size; 

   target.WindowState = State; 
}

It seems to work.  This is too easy, right?  Now try minimizing the form and then closing it.  Open the form again.  Can’t restore it can you? 

Minimizing and Maximizing Forms

The problem is that for a minimized/maximized form the DesktopLocation and Size properties are not reliable.  Instead we need to use the RestoreBounds property which tracks the position and size of the form in its normal state.  If we persist this property when saving then when we load we can restore the normal position and size and then set the state to cause the form to minimize or maximize properly.  But there is another problem.  RestoreBounds isn’t valid unless the form is minimized or maximized.  Therefore our save code has to look at the state of the form and use DesktopLocation when normal and RestoreBounds when minimized/maximized.  Note that RestoreBounds.Size is valid in both cases although whether this is by design or not is unknown.  The load code remains unchanged as we will set the values based upon the form’s normal state and then tell the form to minimize or maximize.  Here is the updated code.

public void Save ( Form target ) 

   //Save the values 
   if (target.WindowState == FormWindowState.Normal) 
      Position = target.DesktopLocation; 
   else 
      Position = target.RestoreBounds.Location; 

   Size = target.RestoreBounds.Size; 
   State = target.WindowState; 

   //Save the settings 
   Save(); 
}

 

Disappearing Windows

The final problem with our code is the problem of disappearing windows.  We’ve all seen it happen.  You start an application and the window shows up in the Task Bar but not on the screen.  Windows doesn’t realize that the application’s window is off the screen.  This often occurs when using multiple monitors and we switch the monitors around or when changing the screen resolution.  Fortunately we can work around it.

During loading we need to verify that the window is going to be in the workspace of the screen (which includes all monitors).  If it isn’t then we need to adjust the window to appear (at least slightly) on the screen.  We can do this by doing a quick check to make sure the restore position is valid and update it if not. 

What makes this quite a bit harder is the fact that screen coordinates are based off the primary monitor.  Therefore it is possible to have negative screen coordinates.  We also don’t want the Task Bar or other system windows to overlap so we will use the working area of the screen which is possibly smaller than the screen size itself.  .NET has a method to get the working area given a control or point but it returns the closest match.  In this case we don’t want a match.  .NET also has SystemInformation.VirtualScreen which gives us the upper and lower bounds of the entire screen but it doesn’t take the working area into account.

For this article we’ll take the approach of calculating the working area manually by enumerating the monitors on the system and finding the smallest and largest working areas.  Once we have the work area we need to determine if the caption of the form fits inside this area.  The caption height is fixed by Windows but the width will match whatever size the form is.  We do a little math and viola.  If the caption is visible then we will set the position otherwise, in this case, we simply let the form reset to its initial position.  Here is the load code.

public void Load ( Form target ) 

   //If the saved position isn’t empty we will use it 
   if (!Position.IsEmpty) 
   { 
       //Verify the position is visible (at least partially) 
       Rectangle rcArea = GetWorkingArea(); 

       //We want to confirm that any portion of the caption is visible 
       //The caption is the same width as the window but the height is fixed 
       //from the top-left of the window 
       Size sz = (Size.IsEmpty) ? target.Size : this.Size; 
       Rectangle rcForm = new Rectangle(Position, new Size(sz.Width, SystemInformation.CaptionHeight)); 
       if (rcArea.IntersectsWith(rcForm)) 
          target.DesktopLocation = Position; 
   }; 

   //If the saved size isn’t empty we will use it 
   if (!Size.IsEmpty) 
       target.Size = Size; 

   target.WindowState = State; 

private Rectangle GetWorkingArea () 

   int minX, maxX, minY, maxY; 
   minX = minY = Int32.MaxValue; 
   maxX = maxY = Int32.MinValue; 

   foreach (Screen scr in Screen.AllScreens) 
   { 
      Rectangle area = scr.WorkingArea; 

      if (area.Bottom < minY) minY = area.Bottom; 
      if (area.Bottom > maxY) maxY = area.Bottom; 

      if (area.Top < minY) minY = area.Top; 
      if (area.Top > maxY) maxY = area.Top; 

      if (area.Left < minX) minX = area.Left; 
      if (area.Left > maxX) maxX = area.Left; 

      if (area.Right < minX) minX = area.Right; 
      if (area.Right > maxX) maxX = area.Right; 
   }; 

   return new Rectangle(minX, minY, (maxX – minX), (maxY – minY)); 
}

UpdateUserType (Visual Studio Addin)

Here is another addin I wrote for VS2008 and then updated for VS2010.  In C++ you can define your own custom keywords and VS will color them differently if you say so (ToolsOptions -> Fonts and Colors).  There are a couple of problems with the VS solution.

  1. You have to edit the UserType.dat file.
  2. You have to restart VS.
  3. You have only one set of keywords for all of C++.

The second and third problems are the killers here.  As you are developing code you will often find a set of keywords you’ll want to add.  To do so you’ll have to open the .dat file, add the keywords and then restart VS.  Even worse is that if you are working on an MFC project then you’ll likely want some MFC keywords but if you switch to Win32 then you want a different set of keywords.  To resolve this problem you’ll have to keep multiple .dat files that you can swap in and out.  Enter UpdateUserType.

UpdateUserType does a couple of things. Firstly it allows you to separate your keywords into different files (i.e. C++, MFC, Win32).  Secondly UpdateUserType can merge all these files into a single .dat file.  Thirdly the addin will detect when the .dat file changes and request VS to refresh its keyword list.  

Using the addin UI you can add a list of .txt files to be monitored.  Whenever one of these files is modified the addin merges them all into a new .dat file.  This allows you to separate keywords by area and enable only the keywords appropriate for your current project.  My personal technique is to keep one or more of them open in a separate editor so that I can add new keywords as I go through code.  Saving the files in the editor causes the .dat file to be regenerated.

When the .dat file changes the addin requests VS to refresh the keyword list.  When this happens VS will read the .dat file and begin coloring the keywords again.  This feature allows you to add keywords on the fly and have VS begin coloring them almost instantiately. 

Attached is the source and binaries for the VS2010 version of this addin.  Additionally I’ve included some starter .txt files that you can use.  A caveat is in order.  The .XML file where the addin stores the files to be merged supports relative or absolute paths.  However the UI and the code only supports absolute paths.  Therefore if you use the UI to change the file list then the paths are converted to absolute paths.

To install the addin copy the contents of the Bin directory to your %My Documents%Visual Studio 2010Addins directory.  You might need to enable the addin via ToolsAddin Manager.  If the addin does not show up (it’ll appear under the Tools menu) then you might need to force a re-registration.

Code

CodeHTMLer (Visual Studio Addin)

CodeHTMLer is a utility written by Weshaggard and available here.  The utility allows you to convert code to HTML for insertion into a website or blog.  It happens to be the tool I use for code on this site.  I know, I know.  There are many tools already available for this but I prefer CodeHTMLer for several reasons.

  1. It supports multiple languages.
  2. The HTML is partially configurable via the XML file it loads.
  3. Some code elements can be ignored rather than having to generate style information for everything.
  4. Can use either existing CSS styles or inline styles.
  5. It is fast.

The only issue I had with it was that I wanted to use it inside Visual Studio.  Being a developer I did the logical thing and wrote an addin for it.  The addin was originally written for VS2008 and the updated for VS2010.  The attached code runs under VS2010.  The addin wraps CodeHTMLer and adds the following features.

  1. Copy as HTML is exposed as an option in the editor.
  2. A UI is added to allow you to change the options in the XML file to control how HTML is generated.
  3. Uses the VS code model to determine which language definition to use for formatting so you can format VB one way and C# another, for example.

I only ran into a couple of problems with CodeHTMLer.  These mainly revolved around the manner in which it expected to be used.  I had to modify (and annotate) the areas I changed.  Furthermore I needed a few additional options for controlling formatting so I extended the language definition code to add a few extra attributes along with the serialization logic.  So the version that is included here is a modified version of the one available from CodePlex.  As a final note the attached language definition file is modified to support how VS names languages.  The original version did not use the same name for C# as VS.

IMPORTANT: I don’t take credit for CodeHTMLer or any of its source.  I only take credit for the addin code.  Please support the original author for CodeHTMLer.

The current project is for VS2010 but you can recompile the code if you want a VS2008 version.  To use this code extract the binaries to your addin directory (%My Documents%Visual Studio 2010Addins).  In VS you probably need to enable the loading of the addin via the ToolsAddin Manager command.  If it does not show up then you might need to modify the addin file to force a re-registration.  The addin shows up as an item in the Tools menu and as a context menu option when you right-click in the code area.

FAQ: Rectangular selection in Visual Studio

Q: I need to delete some code from several lines and the code is well formatted.  Can I use some sort of regular expression to replace the code?

A: You could but this might be overkill.  VS has supported rectangular selection for a while.  With rectangular selection you can select a block of code and then select that same block on several consecutive lines and then delete the selected code. 

Do the following to use rectangular selection.

  1. Place the caret on the column where you want to begin selection.
  2. Hold down the ALT key.
  3. Drag the mouse to select the region you want to select.  If you cross lines then the same columns are selected on all the lines.
  4. Release the ALT key and then press DEL to delete the columns from each line.

Starting with VS2010 some new capabilites were added.  They are discussed here.  My personal favorite is the ability to replace the same block of code on all lines (such as renaming private to public). 

To do that just select the region that you want to replace and then begin typing the replacement text.