When you implement a solution, you test it in different situation and depending on the results, of course, you correct the little mistakes that may be present. What I found very interesting so far in software development is the way to reproduce a behaviour following a given set of steps. For the same steps, you get the same result. That is also a way to qualify a bug or a defect, most of the times.
So, what is more frustrating when the problem or the bug happens randomly ?
However, that is what happened recently on a SharePoint 2007 web site during a roll-out.
The symptoms ?
Having activated the anonymous access, pages prompted the users for credentials and when no credentials were entered, a simple message "401 Unauthorized" was displayed. Now, on that same page, simply pressing F5 to refresh was loading and displaying the page correctly.
Checked that all the resources were accessible by anonymous users, being CSS, images, documents, etc. Some images were referenced using absolute URLs, but apart from that, nothing special.
Checked in the Windows Event Log, absolutely nothing related to this problem.
Checked the ULS, crawled megs of text files, several "excessive number of SPRequest" messages were present, but they were here before the roll-out as well. The only clue was some "Access denied" messages, but, no reference to a resource or no stack trace.
Checked also the web.config, everything was ok. And anyway, this is normally handled by SharePoint.
Checked the IIS Directory Security permissions, verifying the identity of the user used by the application pool, but here again, everything was normal.
Custom code doing a forbidden action ? After a complete removal of the three custom controls, the problem persisted, so, it was not that.
After hours and hours of investigation, I found one thing. The Master Page and Page Layouts gallery was not accessible anonymously, so I fixed this first, but still, the 401 issue was still here. Being in the library I decided to check the content of the master pages and the page layouts and found couple of "__designer" tags referencing resources. We indeed used the SharePoint Designer to apply some changes, but never had this problem before. But, to be sure and to definitely clean these files, every "__designer" tags have been removed and pages provisionned again.
The result ?
Everything was fine. So, for some reasons, the tags added by SharePoint Designer seem to cause access issues, but not all the times.
In this particular case, what was really annoying is that nothing can lead you to the solution or where the problem is located. A lot of people wrote about 401 random issues, caused by either a hotfix applied here or load-balancer problems there, but mine was still another one.
So, before spending hours or days looking for useless (in that specific situation) logs, check your master page or your page layouts and remove "__designer" tags. Check also that they are accessible anonymously.