In order to store and share your java artifacts, they are usually stored in Maven repositories, the most prominent one being maven-central.
For hosting non-public code artifacts, organizations need to have their own Maven repository. The best-known ones are artifactory and sonatype nexus. There are SaaS offerings as well as self-hosted solutions. One of the perks of these repositories is the proxy-feature: Every artifact is resolved from the private repository and if it is not present there already, it is downloaded from the public repositories. This way, you build a cache of the artifacts you need and protect against a downtime of the public repositories.
While the proxy-feature might be important for bigger companies to protect their productivity, you might not need the overhead for smaller teams/organizations. So setting up and hosting your own repository or paying for a SaaS solution might be an overkill that one would like to avoid.
So already some time ago, people came up with the idea of using an AWS S3 Bucket as a maven repository. S3 has some major benefits: It is cheap, highly-available, easy-to-use and supports lifecycle management. Also, AWS is widely used nowadays.
A guide how to do this using maven can be found here.
There are various Maven plugins supporting s3 as a storage protocol, so called wagons(Examples: 1(abandoned), 2(we used this one) and 3) as well as gradle plugins for that purpose. The promise sounds great: You set up an s3 bucket, give access to it using AWS’ Default Credential Provider Chain, and the respective plugin takes care of resolving and deploying artifacts to it.
So, we decided to go for it. We set up the settings.xml and adjusted our pom.xml ‘s to deploy the artifacts to s3 according to the guide above.
The first problem we encountered was the fact that maven parents can not be resolved using s3 wagons extensions, since by default, maven loads parents before the extensions. The issue was described in 2015 here.
Nowadays, the issue can be mitigated using the core extensions mechanism introduced with maven 3.3.1. You add a .mvn/extensions.xml in the projects root, e.g. looking like this:
<extensions xmlns="http://maven.apache.org/EXTENSIONS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/EXTENSIONS/1.0.0 http://maven.apache.org/xsd/core-extensions-1.0.0.xsd">
<extension>
<groupId>com.gkatzioura.maven.cloud</groupId>
<artifactId>s3-storage-wagon</artifactId>
<version>2.3</version>
</extension>
</extensions>
One problem stays tough: You will need to add a .mvn/extensions.xml to every child project that uses a maven parent stored in s3. In our case, we decided that this effort was still worth the benefits. In case you do not have access to you child projects or there is no way to automate adding this file to the projects, this might already be a blocker for you.
If you are using Jenkins as a CI Server (many of us still do, right?), you might be using the Maven Integration Plugin for it to build your maven projects and not a standard freestyle project.
There is an issue with the Maven integration though: It does not pick up the .mvn/extensions.xml in the project, resulting again in the problem that parent projects can not be resolved from s3. The issue has been created in 2015 and is still not resolved as we speak in 2020.
We had to work around this issue by migrating all the maven builds to freestyle builds. The feature of triggering a downstream project if its parent has been updated is lost.
In case you are not using Maven Parents, you seemingly do not have a problem. Just follow a guide like this one and you should be good. For projects with maven parents coming from s3, the workaround with the .mvn/extensions.xml should work just fine.
In case you are building these projects using the Jenkins Maven Integration, you might indeed have a problem and there might be blockers for you to migrate to freestyle projects (e.g. the feature to build a project if its maven snapshot dependencies have been updated).
Without these issues, using a S3 Bucket as Maven Repository might have been the perfect solution for us. This way, we had to put a little effort into making it work. But in the end, the benefits still justified the compromises we had to make.
Written By