How to unit test a secured controller which uses thymeleaf (without getting TemplateProcessingException)?

Question

I am trying to run a unit test in spring-boot using spring security and a simple home (root) controller which uses thymeleaf for the template processing. I am trying to write some unit tests to verify that my security permissions are working right and that the right data is hidden or shown from my template (which uses the thymeleaf spring security integration). The app itself does work correctly when I run it. I just want to verify it is working with a set of integration tests. You can find all the code here but I will include relevant snippets below also:

https://github.com/azeckoski/lti_starter

The controller is really simple and does nothing but render the template (at the root - i.e. "/").

@Controller
public class HomeController extends BaseController {
    @RequestMapping(method = RequestMethod.GET)
    public String index(HttpServletRequest req, Principal principal, Model model) {
        log.info("HOME: " + req);
        model.addAttribute("name", "HOME");
        return "home"; // name of the template
    }
}

The template has a lot in it but the relevant bits for the test are:

<p>Hello Spring Boot User <span th:text="${username}"/>! (<span th:text="${name}"/>)</p>
<div sec:authorize="hasRole('ROLE_USER')">
    This content is only shown to users (ROLE_USER).
</div>
<div sec:authorize="isAnonymous()"><!-- only show this when user is NOT logged in -->
    <h2>Form Login endpoint</h2>
    ...
</div>

And finally the test:

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
@WebAppConfiguration
public class AppControllersTest extends BaseApplicationTest {

    @Autowired
    WebApplicationContext wac;

    @Autowired
    private FilterChainProxy springSecurityFilter;

    private MockMvc mockMvc;

    @Before
    public void setup() {
        // Process mock annotations
        MockitoAnnotations.initMocks(this);
        // Setup Spring test in webapp-mode (same config as spring-boot)
        this.mockMvc = MockMvcBuilders.webAppContextSetup(wac)
                .addFilter(springSecurityFilter, "/*")
                .build();
    }

    @Test
    public void testLoadRoot() throws Exception {
        // Test basic home controller request
        MvcResult result = this.mockMvc.perform(get("/"))
                .andExpect(status().isOk())
                .andExpect(content().contentTypeCompatibleWith(MediaType.TEXT_HTML))
                .andReturn();
        String content = result.getResponse().getContentAsString();
        assertNotNull(content);
        assertTrue(content.contains("Hello Spring Boot"));
        assertTrue(content.contains("Form Login endpoint"));
    }

    @Test
    public void testLoadRootWithAuth() throws Exception {
        Collection<GrantedAuthority> authorities = new HashSet<>();
        authorities.add(new SimpleGrantedAuthority("ROLE_USER"));
        Authentication authToken = new UsernamePasswordAuthenticationToken("azeckoski", "password", authorities);
        SecurityContextHolder.getContext().setAuthentication(authToken);
        // Test basic home controller request
        MvcResult result = this.mockMvc.perform(get("/"))
                .andExpect(status().isOk())
                .andExpect(content().contentTypeCompatibleWith(MediaType.TEXT_HTML))
                .andReturn();
        String content = result.getResponse().getContentAsString();
        assertNotNull(content);
        assertTrue(content.contains("Hello Spring Boot"));
        assertTrue(content.contains("only shown to users (ROLE_USER)"));
    }
}

The erorr I get on BOTH of the above tests is:

testLoadRoot(ltistarter.controllers.AppControllersTest) Time elapsed: 0.648 sec <<< ERROR! org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.thymeleaf.exceptions.TemplateProcessingException: Error during execution of processor 'org.thymeleaf.extras.springsecurity3.dialect.processor.AuthorizeAttrProcessor' (home:33) at org.springframework.web.context.support.WebApplicationContextUtils.getRequiredWebApplicationContext(WebApplicationContextUtils.java:84) at org.thymeleaf.extras.springsecurity3.auth.AuthUtils.getExpressionHandler(AuthUtils.java:260) at org.thymeleaf.extras.springsecurity3.auth.AuthUtils.authorizeUsingAccessExpression(AuthUtils.java:182) at org.thymeleaf.extras.springsecurity3.dialect.processor.AuthorizeAttrProcessor.isVisible(AuthorizeAttrProcessor.java:100) at org.thymeleaf.processor.attr.AbstractConditionalVisibilityAttrProcessor.processAttribute(AbstractConditionalVisibilityAttrProcessor.java:58) at org.thymeleaf.processor.attr.AbstractAttrProcessor.doProcess(AbstractAttrProcessor.java:87) at org.thymeleaf.processor.AbstractProcessor.process(AbstractProcessor.java:212) at org.thymeleaf.dom.Node.applyNextProcessor(Node.java:1016) at org.thymeleaf.dom.Node.processNode(Node.java:971) at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672) at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655) at org.thymeleaf.dom.Node.processNode(Node.java:990) at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672) at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655) at org.thymeleaf.dom.Node.processNode(Node.java:990) at org.thymeleaf.dom.NestableNode.computeNextChild(NestableNode.java:672) at org.thymeleaf.dom.NestableNode.doAdditionalProcess(NestableNode.java:655) at org.thymeleaf.dom.Node.processNode(Node.java:990) at org.thymeleaf.dom.Document.process(Document.java:93) at org.thymeleaf.TemplateEngine.process(TemplateEngine.java:1155) at org.thymeleaf.TemplateEngine.process(TemplateEngine.java:1060) at org.thymeleaf.TemplateEngine.process(TemplateEngine.java:1011) at org.thymeleaf.spring4.view.ThymeleafView.renderFragment(ThymeleafView.java:335) at org.thymeleaf.spring4.view.ThymeleafView.render(ThymeleafView.java:190) at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1221) at org.springframework.test.web.servlet.TestDispatcherServlet.render(TestDispatcherServlet.java:102) at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1005) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:952) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:870) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:852) at javax.servlet.http.HttpServlet.service(HttpServlet.java:735) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837) at org.springframework.test.web.servlet.TestDispatcherServlet.service(TestDispatcherServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.springframework.mock.web.MockFilterChain$ServletFilterProxy.doFilter(MockFilterChain.java:170) at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:137) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:154) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:110) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:85) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:57) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:50) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) at org.springframework.mock.web.MockFilterChain.doFilter(MockFilterChain.java:137) at org.springframework.test.web.servlet.MockMvc.perform(MockMvc.java:141) at ltistarter.controllers.AppControllersTest.testLoadRoot(AppControllersTest.java:70)

HOWEVER that only happens if both tests are enabled and the springSecurityFilter is included. If I disable one of the tests and remove the springSecurityFilter code (.addFilter(springSecurityFilter, "/*")) then I no longer get that error. I suspect something is maybe messing up the WebApplicationContext or leaving the security stuff in a failure state of some kind but I am not sure what I need to reset or change.

So if I take out the second testand remove the springSecurityFilter then my first test will still fail (this one in particular assertTrue(content.contains("Form Login endpoint"))) but I no longer get any error. When I look at the generated HTML I am not seeing ANY of the tags content which uses the sec:authorize attribute.

So I searched around and found a suggestion that I need to add in the springSecurityFilter (which I have done in the code sample above), however, as soon as I do that I get the failure immediately (it doesn't even get to the point where it fails without it). Any suggestions on what is causing that exception and how to fix it?

Solution

I have a workaround solution which seems to completely solve this problem for spring-boot:1.1.4, spring-security:3.2.4, and thymeleaf:2.1.3 (though it is a bit of a hack).

This is the modified unit test class:

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
@WebAppConfiguration
public class AppControllersTest {

    @Autowired
    public WebApplicationContext context;

    @Autowired
    private FilterChainProxy springSecurityFilter;

    private MockMvc mockMvc;

    @Before
    public void setup() {
        assertNotNull(context);
        assertNotNull(springSecurityFilter);
        // Process mock annotations
        MockitoAnnotations.initMocks(this);
        // Setup Spring test in webapp-mode (same config as spring-boot)
        this.mockMvc = MockMvcBuilders.webAppContextSetup(context)
                .addFilters(springSecurityFilter)
                .build();
        context.getServletContext().setAttribute(
            WebApplicationContext.ROOT_WEB_APPLICATION_CONTEXT_ATTRIBUTE, context);
    }
...

The magic here is forcing the WebApplicationContext.ROOT_WEB_APPLICATION_CONTEXT_ATTRIBUTE to be the actual web app context (which I injected). This allows the actual sec: attributes to work BUT my second test where I try to set the authority so the user is logged in does not pass (it looks like the user is still ANONYMOUS).

UPDATE

There was something missing (which I think is a gap in how spring security works) but it is lucky fairly easy to solve (though it's a bit of a hack). See this for more details on the issue: Spring Test & Security: How to mock authentication?

I needed to add a method which creates a mock session for the test. This method will set the security Principal/Authentication and force the SecurityContext into the HttpSession which can then be added to the test request (see test snippet below and NamedOAuthPrincipal class example).

public MockHttpSession makeAuthSession(String username, String... roles) {
    if (StringUtils.isEmpty(username)) {
        username = "azeckoski";
    }
    MockHttpSession session = new MockHttpSession();
    session.setAttribute(HttpSessionSecurityContextRepository.SPRING_SECURITY_CONTEXT_KEY, SecurityContextHolder.getContext());
    Collection<GrantedAuthority> authorities = new HashSet<>();
    if (roles != null && roles.length > 0) {
        for (String role : roles) {
            authorities.add(new SimpleGrantedAuthority(role));
        }
    }
    //Authentication authToken = new UsernamePasswordAuthenticationToken("azeckoski", "password", authorities); // causes a NPE when it tries to access the Principal
    Principal principal = new NamedOAuthPrincipal(username, authorities,
            "key", "signature", "HMAC-SHA-1", "signaturebase", "token");
    Authentication authToken = new UsernamePasswordAuthenticationToken(principal, null, authorities);
    SecurityContextHolder.getContext().setAuthentication(authToken);
    return session;
}

Class to create the Principal (with OAuth support via ConsumerCredentials). If you are not using OAuth then you can skip the ConsumerCredentials part just implement the Principal (but you should return the collection of GrantedAuthority).

public static class NamedOAuthPrincipal extends ConsumerCredentials implements Principal {
    public String name;
    public Collection<GrantedAuthority> authorities;
    public NamedOAuthPrincipal(String name, Collection<GrantedAuthority> authorities, String consumerKey, String signature, String signatureMethod, String signatureBaseString, String token) {
        super(consumerKey, signature, signatureMethod, signatureBaseString, token);
        this.name = name;
        this.authorities = authorities;
    }
    @Override
    public String getName() {
        return name;
    }
    public Collection<? extends GrantedAuthority> getAuthorities() {
        return authorities;
    }
}

And then modify the test like so (to create the session and then set it on the mock request):

@Test
public void testLoadRootWithAuth() throws Exception {
    // Test basic home controller request with a session and logged in user
    MockHttpSession session = makeAuthSession("azeckoski", "ROLE_USER");
    MvcResult result = this.mockMvc.perform(get("/").session(session))
            .andExpect(status().isOk())
            .andExpect(content().contentTypeCompatibleWith(MediaType.TEXT_HTML))
            .andReturn();
    String content = result.getResponse().getContentAsString();
    assertNotNull(content);
    assertTrue(content.contains("Hello Spring Boot"));
}

Comments

[MarcF]

Many thanks for such an excellent answer. It's odd that the security filters are not loaded alongside the context!