Automating publication of a Joomla Plugin using an update server

I decided to submit my little plugin to the Joomla Extensions Directory. That might have been rash, given theres quite a lot of moving parts here to get comprehension of. Theres the plugin and its structure itself, and theres the Joomla update system, and theres the submission to the Joomla Extensions Directory. Anyway, heres the story about how I published a plugin on the Joomla Extensions Directory in this article.

TL:DR – There was a lot to learn here but it was quite a fun thing to do. It will definitely be easier next time around. The plugin is now available see Automatic Meta Description in the Joomla! Extensions Directory.

DevOps approach

I adopted a devops mentality and approach toward this process.

DevOps defined

"a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality""

DevOps(set of software practices) - Wikipedia

I talk a lot about DevOps in another article here about Software Assurance and wanted to practice what I preach in as much as automating everything, securely, as much as possible. I had built the plugin for myself really and just manually installed it but now, it seemed, that it needed to be properly released to an update server so that Joomla would notice the update was available and allow it to be installed easily. So I needed a script to help. And a few other things.

I felt that once this was all working I would be able to document it and push it to the Joomla! Extensions Directory so that anyone else who wanted it could find it.

A DevOps script for releasing a Joomla plugin

Here’s the shell script that does this:

1. Zips up the Joomla plugin after tidying up the folders from macOS Finder detritus because i use a Mac
2. Generates SHA-256, SHA-384, and SHA-512 hashes
3. Creates the Joomla update XML file
4. Uploads the ZIP and XML file to the update server

Environment variables

In an attempt to make this a generic script, environment variables are used wherever possible, so as to keep the plugin and server details out of the script :

PLUGIN_NAME: The human readable name of the Joomla plugin

PLUGIN_ELEMENT: The internal element name of the Joomla plugin

PLUGIN_VERSION: The plugin version

PLUGIN_DESCRIPTION: The plugin description

PLUGIN_DIR: The directory containing the plugin files

OUTPUT_DIR: Where to save the ZIP and XML

UPDATE_SERVER: The update server’s URL

SSH_USER: The SSH username

SSH_HOST: The SSH server

REMOTE_PATH: The path on the remote server to upload files

The script

Environment variables setup

This list of exports sets up the envirronment. I'm using direnv - a tool which sets environment variables for a folder from the command line and this is a .envrc file. 

Place it in the folder, install direnv and then run direnv allow . and it sets your variables. Its lovely really and doesn't get in the way of anything.

export PLUGIN_NAME="My Awesome Plugin"
export PLUGIN_ELEMENT="myplugin"
export PLUGIN_VERSION="1.2.3"
export PLUGIN_DESCRIPTION="This is an awesome Joomla plugin that does amazing things"  
export PLUGIN_DIR="/path/to/plugin/files"
export OUTPUT_DIR="/path/to/output"
export UPDATE_SERVER="https://updates.example.com"
export SSH_USER="youruser"
export SSH_HOST="yourserver.com"
export REMOTE_PATH="/var/www/html/updates"

Now you have your environment variables you can crreate a script which uses them!

Script

Heres my script. Broken down with an explanation of what each bit does.

#!/bin/bash

# Check required environment variables
REQUIRED_VARS=("PLUGIN_NAME" "PLUGIN_ELEMENT" "PLUGIN_VERSION" "PLUGIN_DESCRIPTION" "PLUGIN_DIR" "OUTPUT_DIR" "UPDATE_SERVER" "SSH_USER" "SSH_HOST" "REMOTE_PATH")
for VAR in "${REQUIRED_VARS[@]}"; do
  if [ -z "${!VAR}" ]; then
    echo "Error: $VAR is not set."
    exit 1
  fi
done

This bit checks your environment varialbes are set as the script won't work unless they are.

# Remove .DS_Store files from current folder recursively
echo "Removing .DS_Store files from current folder recursively..."
find . -type f -name '*.DS_Store' -ls -delete

This bit removes all those annoying files that macOS Finder puts in place in folders if you visit them through the Finder app. It is not particularly necessary but it removes this detritus.

# Ensure output directory exists
mkdir -p "$OUTPUT_DIR"

This bit makes sure the folder exists for the output files. Again its a bit unnecessary as it really should exist already.

# Define filenames
ZIP_FILE="${OUTPUT_DIR}/${PLUGIN_ELEMENT}-${PLUGIN_VERSION}.zip"
ZIP_FILE2="${OUTPUT_DIR}/${PLUGIN_ELEMENT}-latest.zip"
XML_FILE="${OUTPUT_DIR}/${PLUGIN_ELEMENT}.xml"

# Create zip archive
echo "Zipping plugin..."
cd  "$PLUGIN_DIR"
zip -r "$ZIP_FILE" *

This bit figures out what the filenames should be from your environment variables. The -latest.zip is because I needed a single download file for my website that doesn't need updating. After a first install, Joomla update uses the versioned zip file.

# Generate hashes
SHA256=$(sha256sum "$ZIP_FILE" | awk '{print $1}')
SHA384=$(sha384sum "$ZIP_FILE" | awk '{print $1}')
SHA512=$(sha512sum "$ZIP_FILE" | awk '{print $1}')

This bit creates hashes that the Joomla update system uses to verify your update is what you intended it to be.

# Generate update XML
echo "Generating update XML..."
cat > "$XML_FILE" <<EOL
<?xml version="1.0" encoding="utf-8"?>
<updates>
  <update>
    <name>${PLUGIN_NAME}</name>
    <element>${PLUGIN_ELEMENT}</element>
    <type>plugin</type>
    <folder>content</folder>
    <version>${PLUGIN_VERSION}</version>
    <description>${PLUGIN_DESCRIPTION}</description>
    <client>site</client>
    <sha256>${SHA256}</sha256>
    <sha384>${SHA384}</sha384>
    <sha512>${SHA512}</sha512>
    <downloads>
      <downloadurl type="full">${UPDATE_SERVER}/${PLUGIN_ELEMENT}-${PLUGIN_VERSION}.zip</downloadurl>
    </downloads>
    <infourl>${UPDATE_SERVER}/updates/${PLUGIN_ELEMENT}-readme.html</infourl>
    <targetplatform name="joomla" version="((4\.4)|(5\.(0|1|2|3|4|5|6|7|8|9)))"/>
    <tags>
      <tag>stable</tag>
    </tags>
  </update>
</updates>
EOL

This bit generates the entire plugin XML file. I like that it is generated by code rather than hand edited. It makes for repeatability and reduces the opportunity for errors.

# Upload files to update server
echo "Uploading files to ${SSH_HOST}..."
scp "$ZIP_FILE" "$XML_FILE" "${SSH_USER}@${SSH_HOST}:${REMOTE_PATH}/"

echo "Build and deployment complete!"

 This bit uploads it to your Joomla Update Server into the right place. I've set up SSH keys so that no login is required which is nice and secure and only works on my development computer.

Output

It tells you a little about what it does. useful for debugging. Very happy with the result.

% direnv allow .
direnv: loading .envrc                                                                           
direnv: export +OUTPUT_DIR +PLUGIN_DESCRIPTION +PLUGIN_DIR +PLUGIN_ELEMENT +PLUGIN_NAME +PLUGIN_VERSION +REMOTE_PATH +SSH_HOST +SSH_USER +UPDATE_SERVER

That's direnv setting the environment.

% ./build_plg.sh
Removing .DS_Store files from current folder recursively...
Zipping plugin...
  adding: autometa.php (deflated 53%)
  adding: gpl-3.0.txt (deflated 66%)
  adding: language/ (stored 0%)
  adding: language/en-GB/ (stored 0%)
  adding: language/en-GB/en-GB.plg_content_autometa.sys.ini (deflated 34%)
  adding: plg-autometa.xml (deflated 47%)
Generating update XML...
Uploading files to multizone.co.uk...
autometa-1.1.28.zip                                                                               100%   14KB 250.9KB/s   00:00    
autometa-latest.zip                                                                               100%   14KB 504.8KB/s   00:00    
autometa.xml                                                                                      100% 1126    67.5KB/s   00:00    
Build and deployment complete!

That's the build and deploy to my Joomla update server.

Possible future enhancements

1. Some of the XML is hard coded for it specifically that it is a site content plugin.
2. Some of the environment variables should probably be somewhere else to avoid duplication.
3. Version number, Date etc is in three or four places in the code which is tedious.
4. Provide a changelog

It works

It took quite a bit of testing and debugging over a day or so, but most of that was my unfamiliarity with the joomla update server requirements. (Thanks Joomla Doc people for the signposts see links at the top of this article). 

End result

Now its working it is easy to reliably push an update to my Joomla Update Server which then becomes automatically available to any Joomla site which has the plugin.

Next Steps

Submit the plugin to the Joomla Extensions Directory. You can read about how to do that in From Start to Finish: How to Submit to Joomla Extension Directory Effectively