Controlling console output from a windows task

Windows tasks continue to be very handy especially for automating batch operations that do not require constant human attention. But even such operations may require occasional human interference to troubleshoot problems.

Most of the Windows tasks execute command line programs that do not have graphical user interface but produce output in form of log files or console messages. While monitoring of log files can be relatively easy automated via different kinds of log collection services, the console output that is mostly meant for human eyes and could provide helpful information is usually lost as by default Windows task does not save console output anywhere.

There is however a relatively easy way to save console output produced by a Windows task in a text file. Windows command line environment provides with a function to redirect standard program output to stdout or stderr to a file using redirection commands '>' or '>>'. The first one redirects that output to a file and replaces the file if it exists with a new version and the second one appends the output to a file if it exists.

In order to use output redirection a Windows task action should be configured to execute a CMD shell instead of an actual cl program. So instead of

>myprogram.exe > output.txt

it should be configured as

>CMD /C "myprogram.exe" > output.txt

In the first example of direct calling a program Windows task will ignore output redirection and the entire output will not be saved. In the second example the redirection will be executed not by a Windows task but by a CMD shell instead and the output will be saved in a file.

404 Error in IIS for a Url with a plus + in the Path

Default Internet Information Server (IIS) behavior since IIS 7 rejects requests to Urls with a plus (+) sing in the path like:

www.somesite.com/one+two or
www.somesite.com/path/subpath/three+four

The behavior is considered a security feature and controlled by a setting called "Double Escaping Filtering". To override the default rejecting behavior the setting needs to be changed either via a web.config configuration:

<system.webServer>
  <security>
    <requestfiltering allowDoubleEscaping="true" />
  </security>
</system.webServer>

or an IIS Management user interface:

External Social Login/Authentication with Microsoft ASP.NET Identity Does Not Work

Well, it's painful as hell, when you spend many days developing custom implementation of ASP.NET Identity with a main goal to enable social/external authentication (as in Facebook, Google, Twitter, etc.) and at the end everything works but the social authentication itself.

The main reason of it not working is that inside ExternalLogin (or similarly named) action method of an Account controller the external login info object is always null:

      var info = await AuthenticationManager.GetExternalLoginInfoAsync();
      if (info == null)
      {
         return RedirectToAction("Login");
      }

Google search returns a lot of discussions on Stack Overflow and it takes a lot of time to go through them but eventually you'll find out (and will be very disappointed) that if you did everything right with your Identity implementation and did not make any stupid mishaps than it's totally not your fault and there is almost nothing in answers on Stack Overflow for you to figure out the problem.

Apparently it's an OWIN bug that is still not fixed (I tested with both Identity 2.0.x and 2.1 and it's still there and happening) as it was figured by smart people (https://stackoverflow.com/questions/20737578/asp-net-sessionid-owin-cookies-do-not-send-to-browser) and without going into obscure details there is a simple fix for it.

In a Login (or named similarly) action method (GET version) of your Account controller add the very first line of code as below:

      Session["dummy"] = "dummy";

The most disappointing part of this problem is that it will not necessarily surface while you are testing. First everything might work just fine and than suddenly out of thin air without any obvious reason it will stop working completely. If it's the case (you had it working and than it stopped) than this is your solution: just add that dummy code line as above and it will do the magic.

Let's see how long it will take Microsoft to fix this bug. Happy coding and thanks all the smart people for sharing their findings.

Upgrade TeamCity to enable support for Visual Studio 2013

Symptoms

If you are using TeamCity in your development process and had upgraded from Visual Studio 2010 to VS 2013 in your development environment you may experience unexpected build errors on TeamCity if its version is lower than 8.1.x similar to the following:

error MSB4019: The imported project "C:\Program Files (x86)\MSBuild\Microsoft\
VisualStudio\v11.0\WebApplications\Microsoft.WebApplication.targets" was not found. 
Confirm that the path in the declaration is correct, and that the file exists on disk.

The Story

The situation is that developers' work stations had been upgraded to Windows 8.x along with Visual Studio 2013 but a TeamCity build server still happily runs under Windows 2008 with .NET 4 installed and everything is just fine. Most likely the projects TeamCity builds target .NET 4.0 framework because they don't utilize newer features from .NET 4.5.x.

Next the decision is to switch your projects target framework to .NET 4.5 in order to take advantage of some new features, for example MVC 5, or ASP.NET Identity framework, etc. The projects have been successfully modified on developers' workstations and compiles and runs there but suddenly TeamCity is not happy because it cannot figure out how to properly build the project with the existing set of MSBuild tools. Apparently a build server software upgrade is required.

The Problem

In the error message above the key problem is that MSBuild is trying to import build targets from the wrong location "C:\...\MSBuild\...\v11.0\...". The exact problem is "v11.0" part. It comes from an environment variable that TeamCity sets when invokes MSBuild. The TeamCity sets it based on MSBuild tool version selected in its project definition. So technically it should be just enough to modify the MSBuild version to 2013 to fix the problem but TeamCity before 8.1.0 simply does not allow any other option to select from. Upgrade is required.

The Solution: Proper Build Server Set-up

Ideally a build server should resemble a production environment and should run the same versions of the build tools as developers have so obvious choice here is to install Windows 2012 with .NET 4.5 already on it and add Visual Studio 2013 build tools for compilation support. Then move TeamCity from the old build server to the new one and everything should be just fine. But there is a catch.

On the old build server TeamCity build steps have been configured to run MS Build v4.0 on .NET 4.0 Framework and MS Build would not be able to find required build targets because they are now in a different location corresponding to Visual Studio 2013. Installing Visual Studio 2013 in addition to the MS Build Tools 2013 would not help resolve the problem (and it's absolutely not required) since the problem is not with the .NET environment at all: the problem is that TeamCity does not know about Build Tools version 12.0.

To fix the problem TeamCity must be upgraded to the latest version but not below 8.1 because support for Visual Studio 2013 was added in version 8.1.0 and then all the build steps in all the projects should be modified to use a proper MSBuild version accordingly. Screen shots below illustrate the build step settings before and after the change:

After the change:

PDF4NET: Setting Transparent Background for a Barcode

PDF4NET is my favourite PDF manipulation library for .NET. It is very versatile and has many useful and unique features but unfortunately lacks good documentation. So often in order to achieve something it requires some digging and researching to do.

Here is a small trick that allows setting a transparent background for a bar code inside a PDF document.

Cheat Sheet for Enabling Output Caching in ASP.NET MVC

What is Output Caching

ASP.NET output caching feature allows increasing web application performance by returning content that was built some time ago instead of rebuilding it on every request. Returning content from cache only takes a few milliseconds as opposed to executing a full request cycle that could take much longer.
The content is being cached on the ASP.NET level and the request would not reach the application code for the content that is still in cache. ASP.NET output caching can be used in both WebForms and MVC applications and using it in MVC has become even easier than before.

When to use Output Caching

Output caching is useful when content returned by a controller action method does not change frequently, requires more than few CPU cycles or database access and does not require a lot of memory to store. For example, I would not recommend to use output caching for a large binary object like an image or a file. Also there is no point to cache a short string that only takes a few milliseconds to build. The best use case would be an average size content that requires some calculation or database access to produce but does not change on every request.

How to enable Output Caching

The easiest way to enable output caching in MVC is using an OutputCache attribute on the controller or controller action method. Applying output caching on a controller action method is recommended as it gives much better granularity and control over output caching. The best way to control output caching behaviour is via caching profiles that allow defining all parameters in a web.config file and override them for each environment where the web application is deployed (i.e. Dev, QA, Stage, Live, etc.)

Limitations

ASP.NET MVC has some limitations for output caching, for example: caching profile is not supported for a partial/child controller method therefore caching parameters including Duration and VaryByParam must be set in the code.

Implementation and code review check points

1. Apply to individual action methods
2. Use Caching profiles
3. Partial/child action methods should not be cached for too long
4. Disable output caching in Dev/QA/Stage environment.
5. Do not forget to set Duration and VaryByParam values in web.config

Example

<system.web>
  <caching>
    <outputcachesettings>
      <outputcacheprofiles>
        <add name="cacheProfile" duration="60" varyByParam="*" />
      </outputCacheProfiles>
    </outputCacheSettings>
  </caching>
</system.web>

[OutputCache(CacheProfile = "cacheProfile")]
public ActionResult Test()
{
    ViewData["result"] = "This text was rendered at " + DateTime.Now;
    return View();
}

Setting Cache Control HTTP Headers in Web API Controller Method

Controlling caching behaviour of an HTTP response is an important task that cannot be ignored as it will have a big impact on a web application load and performance. In ASP.NET Web Forms and MVC applications it usually is done by using special directives or attributes that take care of adding corresponding headers to the response. ASP.NET Web API framework unfortunately does not come with an out of the box support for this functionality so concious developers need to deal with that themselves. Luckily it is not a difficult task to do as the Web API framework has a number of support objects that simplify the job.

Basically in order to control caching behaviour we need to make sure that the output response will have what is called "Cache control header" with proper values that determine caching behaviour. Example below demonstrates how to make a response publicly cacheable for a period of time:

   var response = new HttpResponseMessage();
   response.Headers.CacheControl = new CacheControlHeaderValue
          {Public = true, MaxAge = TimeSpan.FromSeconds(maxAge)};

To simplify the usage further we can even create a static extension method that can be easily applied in a controller method:

// somewhere in a static class
public static HttpResponseMessage PublicCache(this HttpResponseMessage response, int maxAge)
{
   response.Headers.CacheControl = new CacheControlHeaderValue
          {Public = true, MaxAge = TimeSpan.FromSeconds(maxAge)};
   return response;
}
...
// inside an ApiController class
public HttpResponseMessage MyApiMethod(long id)
{
   var response = new HttpResponseMessage();
   ..............
   return response.PublicCache(24 * 7 * 60);
}

ASP.NET and SSL Offloading

SSL offloading is quite a popular method of improving a web server performance by delegating a CPU-intensive SSL processing operations to an external device such as a load balancer. Because having a load balancer only makes sense when there are multiple web servers working in a farm it looks like an attractive solution for maximizing usage of a load balancer and at the same time improving performance of web servers by freeing them from doing an "unnecessary" SSL processing. What also makes SSL offloading a high profile feature is load balancing appliance vendors marketing SSL offloading as one of the selling points of their products.

With all above said being correct it is not always recommended to rush for turning SSL offloading on. One very specific use case is an ASP.NET application using  Web Forms authentication configured to serve protected content over secure connection only. What that means is when a request for a protected page sent over unsecured HTTP the application will redirect it to a secured version of the Url. But because the SSL request never reaches a web server (since it's being intercepted by a load balancer and converted into HTTP request) the application logic will be stuck in an infinite redirect loop. To resolve the problem there are only two options: allow protected content over an unsecured connection or disable SSL offloading. If there is more of an application logic that requires secured connection between client an a server it makes more sense to not use SSL offloading at all.

There are however scenarios when SSL offloading is very helpful. Examples include but no limited to many kinds of data processing service-like web applications that are not intended to return human-readable HTML output, TCP/IP based data processing applications that do not use HTTP at all and so forth. Most likely such applications will not base their logic on a condition of a secure connection with a client.

What are PROs and CONs of using SSL offloading? Obviously one positive point is web servers performance improvement by removing SSL processing from their plates. How important that could be? Most likely that depends on traffic. For high traffic high profile publicly accessible web servers that could be very important but in such cases all the efforts must be made to not couple application logic with a type of client-server connection. From another hand modern server CPUs implement specialized command sets that make SSL processing much less intensive and not so terrifying a task for web servers so it could be more convenient to allow secure traffic directly to web servers.

Another consideration is how secure traffic passing between an SSL offloading load balancer and a web server is. Obviously both devices are supposed to be behind a firewall but it needs to be take care of because traffic may contain sensitive information and in such a case load balancer could become a point of attack.

One more concern is IT maintenance. Secured infrastructure requires setting up and maintaining SSL certificates and other components. If there are a lot of web servers in a farm it could make sense to minimize maintenance by centralizing all SSL configurations on SSL offloading devices.

In conclusion, understanding specifics of a particular implementation is crucial for making a correct decision about using SSL offloading technique.

Why Some Web Sites Suck

I am an active Internet user. I visit and use a lot of different web sites including those for all kinds of e-commerce and utilities. Being at the same time an experienced professional web developer allows me to understand why some web sites are not as good as others and some of them simply suck. The first and foremost reason from my experience is that web site's developers do not follow web development best practices and most likely do not test their creations thoroughly.

In this article I am trying to analyse and describe most commonly occurring problems not visible to a non-professional eye but significantly affecting users' experience.

Base technology platform selection

Often a web site could be built on top of a platform that is familiar to a development team but not necessarily a good choice for a new project but sill would be selected. One particular example is a high traffic e-commerce web site built upon Microsoft Sharepoint Portal Server because the developer team used this technology for a corporate news portal. The platform choice always brings its limitations and must be validated against a particular web site goals and usages. How fast do you expect it to work? How cross-browser compatible do you expect it to be? Those and other similar questions should be asked to make sure the technological platform choice is proper.

Web development framework

Obviously the platform choice limits the options for a Web development framework but if you are a .NET developer you most likely have at least two choices: ASP.NET WebForms and ASP.NET MVC. Both frameworks are modern first-class web development tools and support all the recent technological advances of web development. However regardless of the framework choice a team needs to make sure it uses all those advances and follows known best practices for a selected framework.

Most frequently noticed ASP.NET development bad practices

When I come across a web site that I don't like but still have to use I usually take a pick at the source code and easily spot some from the known bad practices listed below.

• HTML 4.01 Transitional. Remind me what year is it?
• JavaScript and CSS references neither bundled nor minified. Page load time, anyone?
• Inline JavaScripts that are neither minified nor obfuscated. See the previous point.
• Inline JavaScripts do not use OOP: still the old school functions only.
• Inline handling of cross-browser differences like "<!--[if IE 7]>". Well, this is really old school. No need to do that any more especially with JS libraries like jQuery.
• Inline CSS using ASP.NET long IDs. Shame on you, development team. That was considered poor poor practice since I can remember. Basically it's just a hack.
• Still using long ASP.NET IDs. You really need to do your home work and read documentation.
• Viewstate embedded right on a page. No comments here. If I see that my only hope that it's properly encrypted. How fast have you expected it to load?
• Still using hidden input elements to manage state. Have you considered other methods like Session maybe?
• Still using server controls for simple HTML like 'a' or 'li'. I call this lazy and I want to blame your team's coding best practice policies for that.
• Multiple different JavaScript libraries are present for example jQuery and Microsoft Ajax, that functionally overlap with each other. You do need to make up your mind and pick up just one.

Recommendations

Try follow the suggestions below as they are proven best practice and help develop much better web sites.

• Use HTML 5.
• Minify and bundle CSS and JavaScript files. Not only doing that improves web site load times but also makes development and maintenance easier. Take a look at http://msdn.microsoft.com/en-us/library/system.web.optimization.aspx and also try to search for ASP.NET Bundling.
• When develop on JavaScript treat it as a first class programming language and apply all the same practices as you would for a server-side language like C#. Read this http://msdn.microsoft.com/en-us/magazine/dd453033.aspx and notice that it's from the year 2009.
• Use proven JavaScript libraries that handle cross-browser compatibility well and do not try to invent your own wheel.
• Implement well known solutions for hiding framework's specifics like client IDs, Viewstate, etc. Even though you need to use them it does not mean you cannot do it right.
• Always prefer server-side technique instead of client-side for handling application state. If nothing else Session is your best friend.
• Learn and use HTML directly for simple cases instead of server controls. It's not 2001 any more and you call yourself a Web developer.
• Do not use multiple client-side libraries that functionally are similar. Not only that makes development and maintenance a nightmare but also tells a lot about your software design skills. Make your choice and use the best tool for your job.

User Experience

It's enough of technical stuff. Let me become just a regular user and tell you that when a  web site is slow, confusing, inconsistent and not cross-browser compatible it sucks. Test it thoroughly and remember the web site is your professional resume. 

Conclusion

This post was not intended as a thorough review of web development best practices and more like a spur of a moment cry-out, but come on! It is 2013 and professional web developers are supposed to build Web 3.0 web sites today for tomorrow and not for a day before yesterday. It also touches me personally when I visit web sites with similar issues because they bring shame on all the web professionals not just those who worked on a web site. Colleagues, please listen up and make improvements. Everybody will like that.