Monday, March 27, 2017

web.config size limits

It turns out, microsoft has a size limit of 250KB on the web.config file of a web site by default.

I ran into this problem because a test website I maintain has a lot of things in its config file, and so the size went past the above threshold.

The workaround is this:
  1. If you create a DWORD MaxWebConfigFileSizeInKB under HKLM\SOFTWARE\Microsoft\InetStp\Configuration\ and set it to the number of kilobytes that the new max size limit must be, that will override the 250kb limit. 
  2. The registry key for IIS 7 / 7.5 running under 32 bit windows is HKLM\SOFTWARE\Wow6432Node\Microsoft\InetStp\Configuration\MaxWebConfigFileSizeInKB 
The above solves this for IIS. However, the issue will also cause a problem for you when developing your web site using IIS Express. The corresponding registry key for IIS express is:
  •  HKLM/HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\IISExpress\10.0\mimic\Configuration\MaxWebConfigFileSizeInKB 
And finally, this seems to not be a problem on Windows Azure - at least I did not hit the issue with a web.config of 260KB on Azure. However, I do not know if there is a size limit at all on Azure. Google did not help me shed any light on this.

References:
.

Sunday, March 26, 2017

Download a website using wget

Sometimes, you wish you had an offline copy of a web site, especially a reference web site, so that you can read it and refer to it at your leisure.

You can do this using the linux wget command:

$ wget \
     --continue \
     --recursive \
     --no-clobber \
     --page-requisites \
     --html-extension \
     --convert-links \
     --restrict-file-names=windows \
     --domains=website.com \
     --no-parent \
         [http://]www.website.com/path/to/folder/[file.html]
Reference: 
 . 

Monday, March 20, 2017

Remove Nuget from your solution

If you have installed a package in Microsoft Visual Studio using Manage NuGet Packages, back in the day you might have used "Enable NuGet Package Restore" to make Visual Studio automatically restore these packages.

However, "Enable NuGet Package Restore" is the old way of doing things (technically called MS-build integrated restore). If you don't do that step, you are automatically on "NuGet automatic restore" which is the recommended way of doing things.

Since this is incredibly counter intuitive - when we see the "Enable NuGet Package Restore," that almost sounds like what we need, even though we already have the better option without knowing it - almost everyone has at some point clicked on "Enable NuGet Package Restore." And then, if you're doing any major scale development, you probably have SVN or GIT - do you commit the nuget.exe and other files that the above option adds to your project? I'm guessing you probably did - I would not be surprised. Some of you may not even realize that there is another option.

So how do you undo "Enable NuGet Package Restore" and go back to "NuGet automatic restore." (In)conveniently, Visual Studio has no menu options to do this!

What changes does "Enable NuGet Package Restore" make?

To answer this question, I made a new project in Visual Studio 2013, added a NuGet package - JSON.NET - to the project, and committed this to a local SVN repository. Then I did "Enable NuGet Package Restore" and did a diff for changes. Here's my change-log:
  1. This adds the ".nuget" folder, with files "NuGet.exe," "NuGet.config," and "NuGet.targets" to the solution. So the .sln is updated, and the above folders and files are made.
  2. It also adds the following changes to the .csproj file:

If you you have multiple projects in your solution, such entries can exist in more than one .csproj files, depending on whether you added NuGet packages to them or not. 

Hence to remove MS-build integrated restore

And so, to remove "Enable NuGet package restore" from your solution, basically unwind the effects of the above changes.
  1. Delete the .nuget foldler and all its contents from your solution. 
  2. Remove the NuGet added XML for , and tabs. Note that in the above I also have the element attributed to NuGet. This seems to be the case - this element also gets inserted by NuGet. And while its removal may not be essential to migrating to new style package management, I have removed this at least a few times from various projects, without issues.

 References:


Thursday, March 16, 2017

Fiddler Reverse Proxy

I manage a web server. It is an internal web server used for testing traffic, but it is a web server all the same. And since it is not a production web server, I have a fair amount of carte blanche to try things on it and learn.

One of the things I wanted to do was to watch the incoming requests and their responses using fiddler from the web server itself. Turns out it is not obvious to do (unlike outbound traffic from your web apps, which you can watch by running fiddler and running your app pool as the logged in user). Here’s what I found out.

HTTP reverse proxy

Setting up HTTP proxy is pretty simple. Let’s say you have a web server that’s serving up content on HTTP (port 80, or whatever other port you have configured). You can make Fiddler listen on another port, let’s say port 8889 and redirect it to port 80, while also watching the traffic. Now if you open your web browser and browse to http://yourserver.yourdomain.ext:8889 fiddler will reroute that traffic to port 80 + watch that traffic for you. To do this:
  1. Open registry editor (run > regedit), go to “HKEY_CURRENT_USER\SOFTWARE\Microsoft\Fiddler2” and add a DWORD named “ReverseProxyForPort” with decimal value = 80 (or whatever port your http server is listening to). This will cause fiddler to redirect traffic it captures onto port 80.
  2. Within Fiddler, tools menu > Fiddler options > connections tab, specify port number for Fiddler to listen on (we’ll use 8889), and check “Allow remote computers to connect.” This requires restarting Fiddler, plus you may see the User Access Control warning for enabling firewall rules to allow the port you are listening on (8889) to be exposed to the network.
Now, with fiddler running on the web server, browse to your site from another computer, with port 8889 specified in the url, and you should be able to watch the traffic on fiddler.

HTTPS reverse proxy

Creating an HTTPS reverse proxy is slightly different, since the “Allow remote computers to connect” option in Fiddler above does not support the SSL handshake. So instead you have to use the Fiddler QuickExec box (The black box below the left pane – you can get to it with ALT+Q) and issue the listen command to make an ssl listener. Also, since the ReverseProxyForPort registry key above does not do SSL, you have to Customize Fiddler rules and modify the OnBeforeRequest to reroute the SSL request.
  1. Modifying the OnBeforeRequest method: To do this, within Fiddler, go to Rules menu > Customize rules. There, find the OnBeforeRequest method. At the very top of this method, add code to catch connections to yourserver:8888 (8888 is the port we’ll use for our proxy) and route that to port 443 (or whatever port your SSL end point listens on)
    if ((oSession.HostnameIs("yourserver")) && (oSession.oRequest.pipeClient.LocalPort == 8888)){
        oSession.Host = "yourserver:443";
    }
  2. Next, make fiddler listen with SSL on port 8888. To do this, go to the QuickExec box (either click in the black box below the left pane, or use ALT+Q), and enter this command:
    !listen 8888 yourserver
This causes Fiddler to start listening, and you get a confirmation dialog to the same effect. Now if you open your browser and navigate to https://yourserver:8888, you can see the traffic in fiddler, and it gets rerouted with SSL onto port 443.

Advanced stuff

For both HTTP and HTTPS, you could run your web server on a different port than 80 or 443 respectively. Then you can setup the reverse proxy on 80 or 443 respectively, and now you can watch traffic without having to explicitly specify port.

Nasty stuff

The observant among you have noticed that in the OnBeforeRequest, you are setting a new host endpoint. This actually does not have to be on the same machine. For example, in the above, I could instead do oSession.Host = "www.google.com:443"; and it works. Now you can navigate to https://yourserver:8888 and you will get google.com but you will also be watching the traffic.

Let’s say you are the network manager of an enterprise network. You could override the local dns server to point google.com at another machine (middleman), and within that machine explicitly set google.com back to the right IP address in the local hosts file. Next run the proxy on port 443 on your middleman machine and route it to back to google.com:443. Since the middleman itself has google at a different IP, this won’t cause infinite recursion. So now you’re watching your entire enterprise’s google traffic.

Please don’t use this to do harm. The above is a theoretical discussion of how bad you can get; but be warned that this could get you in jail. However, you may find need to put a middleman to debug why a website/webservice is misbehaving – make sure your users are aware that this is happening.

References:

.