Automatically generate Java from .proto with maven/m2e in Eclipse IDE

Question

For my team, I'd like to configure maven/eclipse build to automatically generate Java code from *.proto files (in a project that uses gRPC). Currently one needs to run mvn generate-source or mvn protobuf:compile (as in plugin usage page). Or what is the same add Run configuration to invoke maven goal compile.

Whenever Eclipse Maven project is refreshed (Alt+F5) or IDE is restarted, project is rebuilt but without what should appear in target/generated, thus turning project into red. So one need to generate and refresh project (F5). UPDATE Eclipse has needed source folders configured in .clathpath file.

As I know that should be m2e connector, but I could only find one https://github.com/masterzen/m2e-protoc-connector for the oldest Googles plugin com.google.protobuf.tools:maven-protoc-plugin, that is even not mentioned currently at https://github.com/grpc/grpc-java

We use exactly referenced/recommended

  <groupId>org.xolstice.maven.plugins</groupId>
  <artifactId>protobuf-maven-plugin</artifactId>

that is:

<build>
  <extensions>
    <extension>
      <groupId>kr.motd.maven</groupId>
      <artifactId>os-maven-plugin</artifactId>
      <version>1.4.1.Final</version>
    </extension>
  </extensions>
  <plugins>
    <plugin>
      <groupId>org.xolstice.maven.plugins</groupId>
      <artifactId>protobuf-maven-plugin</artifactId>
      <version>0.5.0</version>
      <configuration>
        <protocArtifact>com.google.protobuf:protoc:3.1.0:exe:${os.detected.classifier}</protocArtifact>
        <pluginId>grpc-java</pluginId>
        <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.0.1:exe:${os.detected.classifier}</pluginArtifact>
      </configuration>
      <executions>
        <execution>
          <goals>
            <goal>compile</goal>
            <goal>compile-custom</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

Related:

• Linking to generated Java protobuf code in Eclipse
• looked at this but that author was using other older, not supported now plugin : Eclipse build loop caused by protobuf generated code (related to Maven Project Builder)
  P.P.S That plugin https://github.com/igor-petruk/protobuf-maven-plugin however has continuation as https://github.com/os72/protoc-jar-maven-plugin

Solution 1

for protobuf-maven-plugin

Thanks to sergei-ivanov answer in https://github.com/xolstice/protobuf-maven-plugin/issues/16, that gave link https://github.com/trustin/os-maven-plugin#issues-with-eclipse-m2e-or-other-ides :

One need to download os-maven-plugin-x.x.x.Final.jar (the version as in your pomx.ml) and put it into the <ECLIPSE_HOME>/plugins directory.

After that Eclipse will generate source on project clean, including after Maven -update project... (Alt+F5), but not after Project -> Build (or with default Build Automatically). Also on IDE start it will not compile.

Yes, that is illogical:

Project - Clean will generate and compile Java source
but
Project - Build will not.

P.S. Raised Bug 507412

Solution 2

Instead of using org.xolstice.maven.plugins:protobuf-maven-plugin my team has used com.github.os72:protoc-jar-maven-plugin to generate the message classes. I believe they are the same since under the hood they all seem to be using the tools from Google.

I am not using any m2e connectors for this plugin (Edit: protoc-jar-maven-plugin's m2e connector is bundled with it so no extra installation is needed, which is why it seemed like I wasn't using one, but technically I was, but this doesn't really matter). Unfortunately the changes in the .proto file are not "automatically" propagated to the generated .java files, you need to manually run Maven or trigger the project to be built in Eclipse (instructions below), but fortunately the target/generated-sources file is not vanishing or emptying or anything strange like what you describe.

If you want to rebuild the .java files from the .proto classes without using mvn clean compile from the command line you can clean the Eclipse project . Project → Clean... → select your project → Select build option (only shows if you have "Build Automatically" from the Project menu is unchecked).

I was able to do this in the latest Eclipse Neon (it will probably work in later ones too, but I don't know for certain).

Below is the POM I am using. I don't think it requires any special explanation, my solution is to simply use a different plugin than the one you are using. (If some explanation is needed I'll be happy to provide it though.)

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>io.github.jacksonbailey</groupId>
    <artifactId>protobuf-m2e-sample</artifactId>
    <version>0.1.0-SNAPSHOT</version>
    <dependencies>
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>3.1.0</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>com.github.os72</groupId>
                <artifactId>protoc-jar-maven-plugin</artifactId>
                <version>3.1.0.1</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <protocVersion>3.1.0</protocVersion>
                            <inputDirectories>
                                <include>src/main/resources</include>
                            </inputDirectories>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Solution 3

Both eclipse and vscode can automatically compile proto when changed.

            <plugin>
                <groupId>kr.motd.maven</groupId>
                <artifactId>os-maven-plugin</artifactId>
                <version>1.6.2</version>
                <executions>
                    <execution>
                        <phase>initialize</phase>
                        <goals>
                            <goal>detect</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>0.6.1</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:3.12.0:exe:${os.detected.classifier}</protocArtifact>
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.32.1:exe:${os.detected.classifier}</pluginArtifact>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>compile-custom</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <source>8</source>
                    <target>8</target>
                    <encoding>UTF-8</encoding>
                </configuration>
            </plugin>

Comments

[Paul Verest]

Voted up. Thanks. However this does not answer "how to Automatically"

[Paul Verest]

If you do mvn clean and then start Eclipse, will it get generated Java file during initial workspace build? The goal is that new developers can get code into Eclipse, and be able to run it.

[Captain Man]

@PaulVerest I told you it was only a partial answer but you told me to answer anyways :) -- As for the second comment, I thought it wouldn't but it actually does. I am updating the answer (it's still not as "automatic" as you probably would like)

[Captain Man]

@PaulVerest Updated. However if I run mvn clean then open Eclipse the .java files aren't made with the initial workspace build, but still if I clean the project through Eclipse it works.

[Edwin Buck]

Eclipse's Maven integration is just ...bizarre... it always was, because instead of treating the pom.xml as the authoritative source for builds, it basically scans it to create an authoritative eclipse build configuration, which always seems to be "just a bit off" in the corner cases. So, in the clean builds, Eclipse will deep scan all folders to "find out" the new stuff, but in a "build" it tends to trust its alternate universe of the Maven config, which often misses generated items. The m2e plugin is not much more than a "signal Eclipse to rescan" plugin.

[Paul Verest]

I have found answer for protobuf-maven-plugin. It would be super great to have it for protoc-jar-maven-plugin. The sample is github.com/paulvi/com.example.grpc.protoc-jar-maven-plugin

[Captain Man]

My answer builds at the same time in the same way with the same limitations, am I missing something?

[Paul Verest]

Different output comparing to protobuf-maven-plugin github.com/os72/protoc-jar-maven-plugin/issues/20

[Paul Verest]

One issues added for the answer as comment. The bounty is yours anyways.

[Captain Man]

@PaulVerest, popped back because someone upvoted this. I know this is an ancient answer. I haven't used gRPC/Protobuf in past year or so but since this answer I feel pretty confident I've use a Protobuf plugin that was able to properly build the target/generated-sources as you edit the proto files as well as having target/generated-sources be a source folder in Eclipse. Paul, if you have figured this out I think it would be good to add since it doesn't seem like a large amount of people use this stuff and this may still be a top result in search engines. :) Either way is fine though