I recently ran into a confusing stack trace occurring in several Sitefinity applications running on a single server. They all varied, but followed a similar pattern:
System.Web.HttpCompileException (0x80004005): External component has thrown an exception. at System.Web.Compilation.AssemblyBuilder.Compile() at System.Web.Compilation.BuildProvidersCompiler.PerformBuild() at System.Web.Compilation.BuildManager.CompileWebFile(VirtualPath virtualPath) at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal(VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate) at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile, Boolean throwIfNotFound, Boolean ensureIsUpToDate) at System.Web.Compilation.BuildManager.GetVirtualPathObjectFactory(VirtualPath virtualPath, HttpContext context, Boolean allowCrossApp, Boolean throwIfNotFound) at System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath(VirtualPath virtualPath, Type requiredBaseType, HttpContext context, Boolean allowCrossApp) at Telerik.Sitefinity.Web.Compilation.CompilationHelpers.<>c__DisplayClass0_1`1.b__0(Object p) at Telerik.Sitefinity.Services.SystemManager.RunWithElevatedPrivilege(RunWithElevatedPrivilegeDelegate delegateToRun, Object parameters, String urlRequest) at Telerik.Sitefinity.Web.Compilation.CompilationHelpers.LoadControl[T](String virtualPath, Boolean isCausedByUserInteraction, PageSiteNode pageSiteNode) at Telerik.Sitefinity.Web.PageRouteHandler.InstantiateHandler(RequestContext requestContext, PageSiteNode pageDataNode, String& theme) at Telerik.Sitefinity.Web.PageRouteHandler.BuildHttpHandler(RequestContext requestContext) at Telerik.Sitefinity.Frontend.Mvc.Infrastructure.Routing.MvcPageRouteHandler.BuildHttpHandler(RequestContext requestContext) at System.Web.Routing.UrlRoutingModule.PostResolveRequestCache(HttpContextBase context) at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step) at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
So there’s a few things I notice with this stack trace, which helped me narrow down what might be at fault:
- The top of the stack trace is in System.Web. That means it’s the .NET framework throwing this exception. Either there’s a bug in the .NET framework’s code (possible, but seems unlikely!) or something invalid was passed to it.
- The entire stack trace only contains System.Web and Telerik.Sitefinity. None of it is our own code! That leads me to believe it’s not our fault, via our own custom widget or similar. Not entirely out of the woods though.
Because of the above points, I’m leaning towards that there’s either a legitimate Sitefinity bug or there was a problem in the last Sitefinity upgrade. At this point I do what anyone does when these receive an error message: put it into Google and see what comes back out. At this point, the results are all over the place. It turns out there’s a log of different reasons you can receive an error within
Some results seem to indicate that seeing “External component has thrown an exception” hides the real error that may not be caught in the trace. So I fire up PerfView and reproduce the issue on the server. I see a few other exceptions being thrown and caught, but none of them appear to be relevant. The trail is cold.
Back to thinking this may be a Sitefinity bug, I look towards the Progress SupportLink Knowledge Base. I’m not sure if this search engines have trouble indexing this, but it always seems to be worth searching here directly. Definitely try to whittle it down to a single keyword. I went with “AssemblyBuilder.Compile(“, which only returned 9 results. None of them relevant.
I turn back to Google and land on this very under-rated StackOverflow post. The top two answers, tied in votes, recommending the same thing:
These answers suggest the ASP.NET temporary files are corrupt. There are three IIS cache locations:
- C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files
- C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files
The first two being the most important.
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files is the location for your 64-bit applications pools and
C:\Windows\Microsoft.NET\Framework\v4.0.30319\Temporary ASP.NET Files for your 32-bit. At this point I followed the following procedure:
- In an administrator command prompt, run
- Delete the contents of the above two folders.
Unfortunately with Sitefinity, we were still not out of the woods. Navigating to the pages having issues still threw the same stack trace as above. I now went into the Sitefinity backend page editor for each of the pages and re-published them. Now the problem was solved.
The problem was eerily similar though. The steps to solve this problem are very similar to what we’ve had to do in the past for an issue caused by IIS running out of memory. To my knowledge, the problem is only documented by this StackOverflow post. In this post we see that the bug was introduced in .NET Framework 4.7.x. The posts also believes that it is resolved in .NET 4.8. In the release notes we see “Fixed an issue introduced in ASP.NET 4.7, where the unexpected removal of a particular type of cache item can result in an orphaned *.delete file that prevents web applications from running. [750653, System.Web.dll, Bug, Build:3734]”. That release note is a bit cryptic for me, and may actually refer to something else.
I know from experience that simply installing .NET 4.7.x on a server introduces this bug, even if the application does not target .NET 4.7.x. This would lead me to believe that simply installing .NET 4.8 on a server resolves this bug. However, this server has 4.8 installed! This however, may be due to the application targeting .NET 4.7.x.
- Shut down IIS.
- Delete the IIS cache folders.
- Start up IIS.
- Republish pages if neccesary.