Hunter's blog

Fixing a Bug In Hayaku

Recently I ran into a bug in my web framework hayaku, and I learned some valuable lessons trying to fix it.

The bug occurred when trying to login to an application I was writing, but only on my server. Everything worked as expected locally. After a little bit of debugging, I found that the issue was with form parsing. Printing the body showed that everything was fine there, so it likely wasn't an issue with invalid data.

Looking at the form parsing code, the only other thing that it did was look for a Content-Type: application/x-www-form-urlencoded header. Looking at the network monitor in Firefox, the header was being sent exactly as expected. Checking for the header on the server got nothing.

On my server, the application was behind an NGINX reverse proxy, which seemed like it might be the issue. I searched for issues with NGINX not passing headers through and found nothing helpful.

After leaving the problem alone for a day, I added a debug function to hayaku that printed all of the headers. After building my application with the added debug function, I found that content-type was being passed rather than Content-Type. It seems that NGINX converts headers to lowercase, and because I was looking for an exact case, the header wasn't being matched.

I based my form parsing code on how Golang's net/http handled it. When I saw that the headers were lowercase, I immediately remembered that in net/http they converted the request header to lowercase and then compared it. When I implemented my code, I didn't see a reason for this as everything worked fine when I tested it. I've read most of the HTTP RFC, RFC 2616, but I didn't look at it when I wrote my header code. Looking at it again, it does specify that headers should be matched case insensitively. When trying to find why NGINX lowercased the headers, I read something that seems to suggest HTTP/2 specifies lowercase headers. This would explain why they are lowercase as I have NGINX configured to use HTTP/2.

So what lessons did I learn? The first is to test in a production-like environment. In this case, testing everything on a spare computer proxied by NGINX would work. The second is to think about why established libraries made the choices they made. Chances are they know what they're doing. If I had thought about this more, I might have checked the RFC. Finally, when I have a specification available, I should make use of it. Glancing at the relevant sections of RFC 2616 when writing hayaku-http might also have prevented this issue.

I'm planning to rewrite most of hayaku soon, and when I do I plan to apply these lessons. I'll update this post with the number of bugs this fixes.