/ Jenkins

Jenkins Integration on Steroids

Parameterized jobs in Jenkins is nothing new. But integrating with services like GitHub is much more than just accepting some GET parameters. Generic Webhook Trigger aims at simplifying triggering of jobs from complex JSON/XML webhook structures.

I got the idea while working on the GitLab plugin. I just needed a simple integer value from the GitLab webhook. After I opened the PR, it took 4+ months before it was merged and later released. Such a simple thing should not require that level of effort. But it is, and it is like that in other trigger plugins as well!

The idea with Generic Webhook Trigger is very simple.

  1. Post content is received.
  2. Values are extracted from the received post content, using JSONPath or XPath. And are contributed to the build with the configured variable names.
  3. Regular expression is used to match the patterns that should trigger a build.

Simple use case

Let's say we have a Git repository. It has some files in it, structured something like this:

src/main/java/...
src/test/java/...
src/main/resources/sql/...

Requirements

  • If the files in src/main/resources/sql is changed, then we want to run the SQL in a Docker to verify them. If these files are not changed, then we don't want to verify them. Perhaps it is a time consuming task and we want to avoid it.
  • We are not allowed to use polling. We want instant builds and cannot have a large amount of Jenkins jobs polling our Git service every minute.

Implementation

First step would be to setup the webhook in GitHub and point it to the running Jenkins instance.

Next step, create a job in Jenkins and configure Generic Webhook Trigger. It supports Multibranch pipelines but I usually just create pipeline jobs using Job DSL Plugin. That is a bit off topic and covered in the documentation in GitHub.

  • Go to you Jenkins instance: http://localhost:8080/jenkins/
  • Create a new job, perhaps Pipeline or Free Style.
  • A new Trigger will appear, once Generic Webhook Trigger is installed, named "Generic Webhook Trigger". I'm adding variable named changed_files with expression $.commits[*].['modified','added','removed'][*].
    generic_trigger_expr_files You will probably want more variables here. Perhaps $.ref to get the branch name or $.after to get the commit that the ref points at after the change.
  • Configure a token, this is how the invoker will know which job to trigger. If a job has a token configured then that token has to be supplied. If the job has no token configured then anyone can trigger the job.
    configure-token-1
  • Add a filter with text like $changed_files and expression like "src/main/resources/sql/[^"]+?".
    generic_expr_folder I'm only using one variable here but you can resolve more variables. Add more variables to the text-field and have a more complex regular expression.

That is it. Save the job and it will trigger for changes in SQL-scripts and nothing else!

Explaination

So what happens in the example above? When a commit is pushed to GitHub, it will invoke the the configured URL with a webhook. In reality, there is much more in the webhook then this:

{
 "ref": "refs/heads/changes",
 "after": "0d1a26e67d8f5eaf1f6ba5c57fc3c7d91ac0fd1c",
 "commits": [
  {
   "added": [
    ".gitignore"
   ],
   "removed": [
   ],
   "modified": [
    "README.md",
    "src/main/resources/sql/some_script.sql"
   ]
  }
 ],
 repository {
  "git_url": "git://github.com/baxterthehacker/public-repo.git"
 }
}

The JSONPath used above will resolve to [".gitignore", "README.md", "src/main/resources/sql/some_script.sql"] and it will be available in a variable $changed_files (that can also be used later in the build).

The text field of the Optional filter will also resolve to this, because we used $changed_files in that field.

The expression field of the Optional filter will match src/main/resources/sql/some_script.sql which will trigger the build. Without that change, the build would not have been triggered.

If you also define variables like:

  • branch with expression $.ref
  • clone_url with expression $.repository.git_url

Then you can have a shell script build step like:

git clone $clone_url
git checkout $branch
...

Further reading

Always turn to the wiki for the most accurate documentation:
https://plugins.jenkins.io/generic-webhook-trigger

Any issues should be reported here:
https://github.com/jenkinsci/generic-webhook-trigger-plugin/issues

Jenkins Integration on Steroids
Share this