I like building applications, and these days that means web applications. The challenge is enjoyable. During my vacation this year I decided to spend some time expanding some code that I’ve been working on over the years. In this case, a tool that runs on my PC that integrates with a database. Previously, I tended towards integrating with a local DB, be it via ODBC, a local text or Excel file, etc. This year I’m setting myself the challenge of integrating with a MySQL database over HTML.
The reason for this is because there is a substantial amount of information in my company’s online database (hosted by NetSuite), but getting access to it for any kind of automation can be tricky. However, it can be done if you’re willing to invest the development time to building an application (both sides of one). But, such a task is not something I’ve ever done before. So I’m combining a hobby and self improvement with the intention of building skills which may be useful at some point in the future at work.
As an example, an application which retrieves system information and helps organize and launch system connections (http, ssh, rdc) is a lot more portable if it retrieves the system information from a central database than something which requires all of the system information to be stored locally. If someone updates the system information and you don’t notice it, it means you could be attempting to connect to the wrong IP or server name. By automating the information retrieval you can save minutes (or over a calendar year, save hours or even days) worth of lookup time.
There are plenty of other uses, such as pulling information from a system, parsing it and then pushing it to a MySQL database directly, and then allowing that information to be displayed in an HTML format. Log parsing, etc., could be streamlined. Even, in one case, parsing the Microsoft hotfixes applied to servers and scrubbing it against a database for which hotfixes have been tested/approved by the manufacturer, would be a great application for my environment. Avaya already has something like that for the CS1000 that was created by the engineers back in the Nortel days. But they don’t have anything like that for the Avaya Aura Contact Center product line, even though they have the audit tool that would be necessary to implement the first half of that endeavor.
While tooling around with the XMLHTTP GET/POST integration for the client tool, I ran into an error on my website that was generated by Mod_Security. “An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security.”
Upon further investigation, I managed to capture an error log out of the shared error log on my web host “ModSecurity: Access denied with code 406 (phase 2). Match of “rx ^0$” against “REQUEST_HEADERS:Content-Length” required. [file “/etc/httpd/modsecurity.d/10_asl_rules.conf”] [line “101”] [id “392301”] [rev “5”] [msg “Request Containing Content, but Missing Content-Type header”] [severity “NOTICE”]”
The rest of the error is largely environment specific, so I’m omitting that info, but if you’ve run into this error yourself, you know what it looks like.
Here’s what I learned in my search (I’m effectively building a custom browser using Microsoft XMLHTTP— the mechanism isn’t too important, be it Power Shell, vbscript or jscript)
- Must declare RequestHeader User-Agent
- Must declare RequestHeader Content-Type
- For POST, must declare RequestHeader Content-Length
These are not mandatory for all HTML interactions, but some security configurations may require certain headers in order to process an XMLHTTP request. In the case of my web server (shared webhosting) and my custom browser application, for a GET request only the User-Agent and Content-Type were required. However, Mod_Security was configured on the shared webhost to mandate Content-Length.
What triggered this research and error was a typo in my code.
The typo came down to a bad choice in variable declaration. In a foreach ( item in array ) statement, I poorly chose the variables to be foreach ( item in items ) and, I’m sure you can see the typo risk already, I accidentally typed foreach ( item in item ). As such, the foreach loop did not properly iterate over the array… and since the foreach loop set the RequestHeaders, the request headers were not being set. Thus, the mod_security error.
I didn’t catch the typo initially, as the first error message was mostly meaningless and I couldn’t immediately determine the cause. Still, I saw a number of articles (including some wordpress blog support requests for this error, with some fixes involving changing the behavior of Mod_Security). While the Mod_Security error isn’t very meaningful, if you dig into the error_logs deep enough, you’ll find the error message which will lead you to the root cause of the problem. In my case, an HTTP 406 indicating that “Rquest Containing Content, but Missing Content-Type header.”
Once I found the typo in the code and fixed it, the XMLHTTP request worked perfectly (except that mod_security was also configured to require a content-length request header on POST requests, but once I’d fixed the one problem the other was easily to identify and fix.)