As I made several module of my company’s code to be more modular, I have several problems with publishing the package. One of them is publishing certain tag as version. Composer package depends on Git’s tag for determining the version number. GitHub and GitLab API actually provides composer a way to fetch the package as a zip that they can extract and install to the project vendor folder. So I definitely need a way to automatically tag the commit, then trigger the GitLab’s Private Packagist API to register the new version.

Few months back, while checking upon Gitbeaker (JS API Wrapper for GitLab) , I see that the author made an integration that automatically tags merged MR with version label and it automatically trigger the build system to ship them to NPM. I WANT THAT.

To me, this is very helpful because I just need to check, test, and publish with single button: MERGE. After checking how it works and see the dependencies, Gitbeaker actually uses Auto by Intuit (I’ll call it Intuit Auto from now on). Few weeks later, I’ve successfully integrated that with some of my repos (Daily Watori, node-myanimelist-url-to-id, and node-mal-api). The only drawbacks are… it only works on GitHub, and to run it, you need NodeJS Runtime to run that tool. So i have to find another alternatives to that: Autotag by Pantheon Systems.

Autotag by Pantheon Systems

Autotag is a tool that automatically tags your git repository with SemVer versioning standards. The tool were made by Pantheon Systems and published under Apache2.0 License, hence I’m all-in for this tool to help me publishing my composer packages… and other packages as well.

Unlike Intuit Auto, autotag works with the terminal version of git, so it will work with all git hosting provider that supports git tags. Since it is coded in Golang, the compiled version of that app would merely a few megabytes of dependency-less elf binary that you can run on Linux machine. Very ideal to run it on most CI/CD systems.

So this blog would mainly talks about my pipeline approach and problems i encounter while implementing this. Spoiler alert, I have new Container Image to simplify the solution, so you can test yours too.

Pipeline Approach

The trigger to release new versions are:

• Push to default branch.
  This will be our mainline, all new features, next improvement bug fixing, all will be in this branch. Usually this branch will be our latest release.
• Push on Protected Branch.
  Usually there’s patch for certain major version that need to be back-ported to older version of the that project. With protected branch, we can just:
  • checkout to that version
  • git checkout -b
  • cherry-pick the patch, or adjust accordingly; then
  • push the commit and your newly made branch
  • and let the autotag publish the tag for you.

Then tag will run into build, test, then the actual finalized build that everyone use right there.

Incompatible Docker Image

So now that I got my CI/CD flow sorted out, I test it on my own personal sample project: chez14’s PHP Codestyle.

Pantheon Systems provide dockered version of Autotag, but somehow, autotag’s default image is not compatible with GitLab Runner. It throws “unknown flag `c’” error.

After checking in the GitLab Runner’s source code, it seems like GitLab Runner tries to run some commands via bash/sh with -c parameter (spoiler alert, it is not, lol). So I move back to php:8 image, because it works on previous build.

After checking here and there, apparently its because the Docker’s entry point for autotag that are incompatible to start a shell sessions.

Scaling-it-up!

I don’t want to copy my currently long-and-unreadable CI/CD script for every project I want to use. Not to mention, if the dependencies suddenly upgraded, doesn’t that mean I have to update the CI/CD script for all of my projects as well?

So this section will be my rant for scaling up the solution I get while testing this.

Scaling it up means I need to make it somehow quick to set up and easy to use. This means I have to somehow take whatever I find tedious to do on every project from my experiment to package them as a script that can be downloaded and used within the pipeline.

Git, SSH, and PGP.

One of the essential part of autotagging are pushing the tagged commit back to the server.

Based on how Git authenticate, there’s two way to do it:

• via HTTP(S)
• via SSH

Doing those requires us to request some secrets from the environment variables, that we need to set them up so that git can just use it when we execute git push --tags.

Authenticate Git with HTTP(S)

Most git hosting sites usually accept personal token as the password. Then, this means you can just set the remote to https://(username):(password)@(server)/(repo).git. — Recalled from early days of me accessing FTP servers on Firefox in 2010s.

This are relatively straightforward to set up.

Authenticate Git with SSH

This one is quite tricky. In theory we can just add the private key and set the git remote to use the ssh address instead of whatever being used now. In reality though, we need to accept the server’s fingerprint, set the key permission with a proper mode, and start an SSH agent so that git can communicate with it to start a ssh session to push things.

So actually one of the main reasons of why i call it as “tedious” are because the script to do this are… very long and boring.

One of the project are I manage on the office uses the SSH key because we can just add a deploy key that are specific for that project and are writable. To me this are MUCH more secure because deploy token on GitLab cannot write into projects.

In short, all we need to do are:

• Copy ssh key to proper place, chmod them to 600.
• Start an ssh agent
• Import the ssh key
• Update the git remote to use ssh instead of http.
• Make sure the server’s key has been inserted to authorized_keys.

Then that’s all what we need to do before we can git push.

Extras: PGP

I see that several companies have requirements of their things to be signed with PGP keys. This means I have extra justification to ship this feature to this solution beside me, liking the “verified” badge next to the commit and tag link ????.

Setting up OpenPGP are very much like SSH: they’re very tedious to set up.

Here’s what we need to do:

• Copy PGP key to a proper place, chmod them to 600.
• Import the key.
• Configure the Git so that it will sign things with that key, automatically.
  This means we need to:
  • Get the key ID
  • Set user.signingkey to that key id
  • Set tag.gpgSign and commit.gpgSign to true so that the signing part are automatically done without us have to pass the extra param.

Packing Up the Solution

To pack those solution, I made a docker image just to do that.

In GitLab’s CI/CD Realm, we can just add extra job to pull and do stuffs. Because most of my projects and my office work are done in GitLab, I prioritize the solution for this first.

This is the back story of I made Publisher repo.

Publisher is an image that specifically made to support the publishing duty of packages. This includes autotagging and ssh-ing to things and execute commands with extra helpers of setting up parts (the tedious parts).

Conclusion

Autotagging is one of perks I always wanted because it helps me to do the publishing stuffs. Luckily, there’s autotag tools by Pantheon Systems that we can use to automatically tag our git commit with semver convention.

To publish the tag, we need to setup the git so that it can just authenticate with the server. To do that we need to setup either the HTTPS auth or via SSH. SSH setup script are tedious to do and hard to maintain if we have so many projects to manage, hence a better solution has to be made.

Publisher is a docker image that contains script to setup the git to authenticate via HTTPS and SSH, including setting up the SSH for the git to use. Publisher also have PGP support out of the box for companies that are highly regulated and required to sign their tag releases (and hobbyist like me).

Publisher repo: https://gitlab.com/chez14/publisher.