Spring Boot and Amazon AWS - how to connect to S3 using Spring Cloud AWS?

Question

I'm pretty new to Spring in general, as well as Amazon AWS, so I apologize in advance if I'm doing something horribly wrong.

I can not seem to get my Spring Boot application to communicate with AWS services in any way when deployed onto Elastic Beanstalk. I've been following the guide here to try to set my project up correctly.

I'm essentially taking the code they provide and attempting to run from my project:

@Component
public class Storage {

    @Autowired
    private ResourceLoader resourceLoader;

    Logger logger = LoggerFactory.getLogger(this.getClass());

    public Storage() {

    }

    public void writeResource() throws IOException {
        Resource resource = this.resourceLoader.getResource("s3://server-images/rootFile.log");
        WritableResource writableResource = (WritableResource) resource;
        try (OutputStream outputStream = writableResource.getOutputStream()) {
            outputStream.write("test".getBytes());
        }
    }
}

Super, seems simple enough. I also set my application.properties file:

cloud.aws.credentials.accessKey = A******A
cloud.aws.credentials.secretKey = i******K

cloud.aws.credentials.instanceProfile = true
cloud.aws.region.auto = true

so I set the key and secret in the profile, and then lastly, I add the stuff to the build.gradle file to get the Spring Cloud stuff:

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:1.2.4.RELEASE")
        classpath("io.spring.gradle:dependency-management-plugin:0.5.2.RELEASE")
    }
}

apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'spring-boot'
apply plugin: 'war'
apply plugin: "io.spring.dependency-management"

war {
    baseName = 'server'
    // version = '0.0.1'
}

dependencyManagement {
    imports {
        mavenBom 'org.springframework.cloud:spring-cloud-aws:1.0.2.RELEASE'
    }
}

//jar {
//    baseName = 'gs-spring-boot'
//    version =  '0.1.0'
//}

repositories {
    mavenCentral()
    jcenter()
}

sourceCompatibility = 1.8
targetCompatibility = 1.8

configurations {
    providedRuntime
}

dependencies {
    // tag::jetty[]
    // compile("org.springframework.boot:spring-boot-starter-web") {
    //    exclude module: "spring-boot-starter-tomcat"
    //}
    compile("org.springframework.boot:spring-boot-starter-web")
    providedRuntime("org.springframework.boot:spring-boot-starter-tomcat")

    // Spring Cloud AWS
    compile 'org.springframework.cloud:spring-cloud-starter-aws'

    //compile("org.springframework.boot:spring-boot-starter-jetty")
    // end::jetty[]
    // tag::actuator[]
    compile("org.springframework.boot:spring-boot-starter-actuator")
    compile("org.springframework.boot:spring-boot-starter-thymeleaf")
    // end::actuator[]
    testCompile("junit:junit")
}

task wrapper(type: Wrapper) {
    gradleVersion = '2.4'
}

Lastly, I don't know how much this matters, but I gave "AmazonS3FullAccess" and "AmazonEC2FullAccess" to the elasticbeanstalk role in AWS. I also set permissions for "Everyone" to list/upload/delete.

So far, All I get is:

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.NullPointerException
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:978)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:644)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:295)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:102)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:85)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:68)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.springframework.boot.context.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:113)
    at org.springframework.boot.context.web.ErrorPageFilter.access$000(ErrorPageFilter.java:59)
    at org.springframework.boot.context.web.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:88)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.springframework.boot.context.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:106)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
    at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:673)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:516)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1086)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:659)
    at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:223)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1558)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1515)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException: null
    at app.data.Storage.readTerminator(Storage.java:44)
    at app.controllers.CodeController.getCode(CodeController.java:54)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:137)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:776)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:705)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:959)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:893)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
    ... 54 common frames omitted

As I can see, the resourceLoader isn't getting anything

Maybe there is some permission problems? Maybe I have misconfigured it? I figure this shouldn't be this hard, so I'm definitely doing something wrong! If anyone has any ideas, they are more than welcome.

Solution

Brian.

If the cause is wrong permissions you should check if your user(you created in IAM) has corresponding permissions(Policies):

AmazonS3FullAccess, AWSConnector

Hope this will help.

Comments

[Brian]

I ended up ditching Spring Cloud and just running with the AWS client library. Thanks for the response though!

[fgonzalez]

Got the same problem, +1 Vladimir :)