How to clean old dependencies from maven repositories?

Question

I have too many files in .m2 folder where maven stores downloaded dependencies. Is there a way to clean all old dependencies? For example, if there is a dependency with 3 different versions: 1, 2 and 3, after cleaning there must be only 3rd. How I can do it for all dependencies in .m2 folder?

Solution 1

If you are on Unix, you could use the access time of the files in there. Just enable access time for your filesystem, then run a clean build of all your projects you would like to keep dependencies for and then do something like this (UNTESTED!):

find ~/.m2 -amin +5 -iname '*.pom' | while read pom; do parent=`dirname "$pom"`; rm -Rf "$parent"; done

This will find all *.pom files which have last been accessed more than 5 minutes ago (assuming you started your builds max 5 minutes ago) and delete their directories.

Add "echo " before the rm to do a 'dry-run'.

Solution 2

Short answer - Deleted .m2 folder in {user.home}. E.g. in windows 10 user home is C:\Users\user1. Re-build your project using mvn clean package. Only those dependencies would remain, which are required by the projects.

Long Answer - .m2 folder is just like a normal folder and the content of the folder is built from different projects. I think there is no way to figure out automatically that which library is "old". In fact old is a vague word. There could be so many reasons when a previous version of a library is used in a project, hence determining which one is unused is not possible.

All you could do, is to delete the .m2 folder and re-build all of your projects and then the folder would automatically build with all the required library.

If you are concern about only a particular version of a library to be used in all the projects; it is important that the project's pom should also update to latest version. i.e. if different POMs refer different versions of the library, all will get downloaded in .m2.

Solution 3

Given a POM file for a maven project you can remove all its dependencies in the local repository (by default ~/.m2/respository) using the Apache Maven Dependency Plugin.

It includes the dependency:purge-local-repository functionality that removes the project dependencies from the local repository, and optionally re-resolve them.

To clean the local dependencies you just have to used the optional parameter reResolve and set it to false since it is set to true by default.

This command line call should work:

mvn dependency:purge-local-repository -DreResolve=false

Solution 4

1. Download all actual dependencies of your projects

   find your-projects-dir -name pom.xml -exec mvn -f '{}' dependency:resolve
   

2. Move your local maven repository to temporary location

   mv ~/.m2 ~/saved-m2
   

3. Rename all files maven-metadata-central.xml* from saved repository into maven-metadata.xml*

   find . -type f -name "maven-metadata-central.xml*" -exec rename -v -- 's/-central//' '{}' \;
   

4. To setup the modified copy of the local repository as a mirror, create the directory ~/.m2 and the file ~/.m2/settings.xml with the following content (replacing user with your username):

   <settings>
    <mirrors>
     <mirror>
      <id>mycentral</id>
      <name>My Central</name>
      <url>file:/home/user/saved-m2/</url>
      <mirrorOf>central</mirrorOf>
     </mirror>
    </mirrors>
   </settings>
   

5. Resolve your projects dependencies again:

   find your-projects-dir -name pom.xml -exec mvn -f '{}' dependency:resolve
   

6. Now you have local maven repository with minimal of necessary artifacts. Remove local mirror from config file and from file system.

Solution 5

It's been more than 6 years since this question was asked, but I still didn't find any tool to satisfactorily clean up my repository. So I wrote one myself in Python to get rid of old local artefacts. Maybe it will be useful for someone else also:

repo-cleaner.py:

from os.path import isdir
from os import listdir
import shutil
import semver

import Constants

# Change to True to get a log of what will be removed
dry_run = False


def check_and_clean(path):
    files = listdir(path)
    only_files = True
    for index, file in enumerate(files):
        if isdir('/'.join([path, file])):
            only_files = False
        else:
            files[index] = None
    if only_files:
        return

    directories = [d for d in files if d is not None]
    latest_version = check_if_versions(directories)
    if latest_version is None:
        for directory in directories:
            check_and_clean('/'.join([path, directory]))
    elif len(directories) == 1:
        return
    else:
        print('Update ' + path.split(Constants.m2_path)[1])
        for directory in directories:
            if directory == latest_version:
                continue
            print(directory + ' (Has newer version: ' + latest_version + ')')
            if not dry_run:
                shutil.rmtree('/'.join([path, directory]))


def check_if_versions(directories):
    if len(directories) == 0:
        return None
    latest_version = ''
    for directory in directories:
        try:
            current_version = semver.VersionInfo.parse(directory)
        except ValueError:
            return None
        if latest_version == '':
            latest_version = directory
        if current_version.compare(latest_version) > 0:
            latest_version = directory
    return latest_version


if __name__ == '__main__':
    check_and_clean(Constants.m2_path)

Constants.py (edit to point to your own local Maven repo):

# Paths
m2_path = '/home/jb/.m2/repository/'

Make sure that you have Python 3.6+ installed and that the semver package has been installed into your global environment or venv (use pip install semver if missing).

Run the script with python repo-cleaner.py.

It recursively searches within the local Maven repository you configured (normally ~/.m2/repository) and if it finds a catalog where different versions reside it removes all of them but the newest.

Say you have the following tree somewhere in your local Maven repo:

.
└── antlr
    ├── 2.7.2
    │   ├── antlr-2.7.2.jar
    │   ├── antlr-2.7.2.jar.sha1
    │   ├── antlr-2.7.2.pom
    │   ├── antlr-2.7.2.pom.sha1
    │   └── _remote.repositories
    └── 2.7.7
        ├── antlr-2.7.7.jar
        ├── antlr-2.7.7.jar.sha1
        ├── antlr-2.7.7.pom
        ├── antlr-2.7.7.pom.sha1
        └── _remote.repositories

Then the script removes version 2.7.2 of antlr and what is left is:

.
└── antlr
    └── 2.7.7
        ├── antlr-2.7.7.jar
        ├── antlr-2.7.7.jar.sha1
        ├── antlr-2.7.7.pom
        ├── antlr-2.7.7.pom.sha1
        └── _remote.repositories

Any old versions, even ones that you actively use, will be removed. It can easily be restored with Maven (or other tools that manage dependencies).

You can get a log of what is going to be removed without actually removing it by setting dry_run = True. The output will look like this:

    update /org/projectlombok/lombok
    1.18.2 (newer version: 1.18.6)
    1.16.20 (newer version: 1.18.6)

This means that versions 1.16.20 and 1.18.2 of lombok will be removed and 1.18.6 will be left untouched.

The latest version of the above files can be found on my github.

View more solutions

Comments

[user2339071]

Simply delete the .m2repository folder. It will get created automatically once you compile the project.

[smajlo]

or buy bigger hard drive and dont care :)

[Cherry]

May be there is more elegant solution than wait for compilation and spent money to hard drive? :) But seriously, I work remotely on virtual machine, so disk space (small) and compilation time (long) are significant. That's why I can not simply change HDD or processor. So I need a way to use it more effectively.

[digital illusion]

If you have the IDE opened as well as ALL your recent projects, the filesystem locks will prevent you from deleting the jars in use

[Tushar Banne]

How can cleaning of dependencies be achieved through pom file?

[S Gaber]

where does .m2repository folder located?

[Cherry]

"hence determining which one is unused is not possible'. I need not this determination I need to leave only new versions.

[Gyanendra Dwivedi]

Then delete .m2 folder and then make sure that all the projects are having only new versions of jar entry in pom.xml. Re-build the project. The .m2 folder will remain with only latest version.

[Cherry]

Good :) But this cleanup dependecies only for current project, not all repository.

[Juanjo Marron]

That's true! To cleanup the whole repository I would go manually and remove directories from ./m2/repository as was commented before or in newer versions of Nexus (after 2.6.4-02.) they provide the Scheduled task link Remove Releases From Repository capability out of the box. It could be also useful

[Cherry]

delete .m2 it leads to delete all dependencies and to download new ones from repository which is extremelly slow.

[Gyanendra Dwivedi]

For the stated problem - one solution fit for all - we could go with above way. For the problem of slowness, I would recommend that the project should consult to a local repo (kind of a prod repository setup in organization); if not available download from public repository. At some point of time, it is recommended that the artifact is eventually uploaded to local repository - if it is so widely used in all the projects.

[Brice]

On OSX (could work on GNU tools too) find ~/.m2/repository/ -atime +30 -iname '*.pom' -print0 | while read -d '' -r pom; do echo rm -rf "$(dirname $pom)"; done where atime is in days (compared to amin in minutes)

[Geoffrey Wiseman]

Glad to hear it. Guess I should go to the trouble of making it into a gem.

[catholicon]

it seems I spoke too soon. I don't know the pattern - but some of the jars got deleted although I know I had used them recently. But, at the same time, some remained.

[Geoffrey Wiseman]

Hm. Well, if you're able to give me more specifics, I'd be happy to look into it. If you get that far, file an issue on GitHub for me to track. Seems to be working fairly consistently for me, but there could be something specific for Cygwin I need to look into, for instance. Did you look at the list it outputted before agreeing, or was it too long to be worth the review? If you do get around to trying it again, would it help if I added the last-used date to the summary? (github.com/geoffreywiseman/mvnclean/issues/10)

[catholicon]

yeah, sure you can add some option to dump debug output (jar:last access date)... I've subscribed the issue you've opened.

[Geoffrey Wiseman]

Updated; take a look at the issue, describes some of your options.

[Geoffrey Wiseman]

Out of curiosity, did you ever try this again?

[catholicon]

no sorry I didn't get around to try it again. I have it at the back of my mind though :).

[Jose Manuel Gomez Alvarez]

Unless you had legacy jars added manually to the repository, or the dependencies are no longer available in the internet. This answer is a bit dangerous... at least back up first!

[Robert Mikes]

Tried this, still got: "No plugin found for prefix 'dependency' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/home/user/.m2/repository), mycentral (file:/home/user/saved-m2/)]"

[Meeh]

I use find ~/.m2 -atime +1w -iname '*.pom' | while read pom; do parent=$(dirname "$pom"); rm -rf "$parent"; done which works for me on OSX. Should work fine on other unixes as well :)

[Jonas Eicher]

I just fell in love with "find"

[Andronicus]

@ᴠɪɴᴄᴇɴᴛ yes, the script is very simplified, as it only served me in a single scenario. Thank you for improving it in pull request, since it's your work would you like to edit the answer as well?