/ Jenkins

Building Atlassian Stash pull requests in Jenkins

Update 160912: There is a new post here.

We are just about to introduce pull requests at my current position. We are using Git with Atlassian Stash and Jenkins. We want to verify that the pull requests:

  • Compile
  • Does not break any test cases
  • Can be merged to target branch
  • Compiles after merge
  • Does not break test cases after merge

After some Googling around the issue I found no solution, so I tought I'd make a post about how I solved it.

Verifying source of the pull request

There is a really nice plugin for Jenkins Stash Notifier Plugin that can be used to notify Stash of the status of a build. Enable it on any Jenkins job that builds the branch you are merging from. It will add an icon and a link to Jenkins in the pull request view of Stash.

Discovering new pull requests

I initially solved this with a Jenkins job that is polling Stash for new pull requests. But polling is never good so I created a Stash plugin that will notify Jenkins about new pull requests.

Pull Request Notifier Plugin for Stash

The plugin is available in Atlassian Marketplace and at GitHub. When installed, you will have this configuration GUI.

pull-request-notifier-for-stash

The features include:

  • Trigger on one, or several, event(s) regarding pull requests.
  • Invoke one, or several, URL(s) when event(s) are triggered.
  • Optionally with basic authentication headers.
  • Completely custom URL supporting variable parameters
    • ${PULL_REQUEST_ID} Example: 1
    • ${PULL_REQUEST_ACTION} Example: OPENED
    • ${PULL_REQUEST_AUTHOR_DISPLAY_NAME} Example: Administrator
    • ${PULL_REQUEST_AUTHOR_EMAIL} Example: [email protected]
    • ${PULL_REQUEST_AUTHOR_ID} Example: 1
    • ${PULL_REQUEST_AUTHOR_NAME} Example: admin
    • ${PULL_REQUEST_AUTHOR_SLUG} Example: admin
    • ${PULL_REQUEST_FROM_HASH} Example: 6053a1eaa1c009dd11092d09a72f3c41af1b59ad
    • ${PULL_REQUEST_FROM_ID} Example: refs/heads/branch_mod_merge
    • ${PULL_REQUEST_FROM_REPO_ID} Example: 1
    • ${PULL_REQUEST_FROM_REPO_NAME} Example: rep_1
    • ${PULL_REQUEST_FROM_REPO_PROJECT_ID} Example: 1
    • ${PULL_REQUEST_FROM_REPO_PROJECT_KEY} Example: PROJECT_1
    • ${PULL_REQUEST_FROM_REPO_SLUG} Example: rep_1
    • And same variables for TO, like: ${PULL_REQUEST_TO_HASH}

You can have several notifications and have them trigger different URL:s. If you trigger Jenkins builds, you may want each repo to have its own build job in Jenkins. The filtering functionality is highly configurable. Create a string with the variables and then a regexp that should match that string.
pull-request-notifier-for-stash-filter

Polling Jenkins with Groovy script

Note that you should only do it this way if you cannot use the plugin described above! For example, uou may not have enaugh access to Stash to install plugins.

Stash has really nice REST API:s. I created a scheduled job in Jenkins that runs every 5 minutes. I implemented it in Groovy using the Groovy plugin.

String summary = ""
int newPullRequests = 0;
File previousPullRequests = new File("/ci/lib/jenkins/workspace/Pull Request Poller/previousPullRequests.txt")

String getJson(String addr) {
 manager.listener.logger.println("Getting URL: "+addr)
 def authString = "user:pass".getBytes().encodeBase64().toString()
 java.net.URLConnection conn = addr.toURL().openConnection()
 conn.setRequestProperty( "Authorization", "Basic ${authString}" )
 conn.connect()
 def reader = new BufferedReader(new InputStreamReader(conn.getInputStream()))
 def stringBuilder = new StringBuilder()
 String line = null
 while ((line = reader.readLine()) != null) {
  stringBuilder.append(line + "\n")
 }
 String json = groovy.json.JsonOutput.prettyPrint(stringBuilder.toString())
 manager.listener.logger.println("Got response:\n"+json)
 return json
}

new groovy.json.JsonSlurper().parseText(getJson("http://stash/rest/api/1.0/projects/PROJECTS/repos/")).values.each { repo ->
 manager.listener.logger.println("Repo: "+repo.slug)

 String prettyJSON = getJson("http://stash/rest/api/1.0/projects/PROJECTS/repos/"+repo.slug+"/pull-requests?base&details&filterText&orderBy")
 def jsonData = new groovy.json.JsonSlurper().parseText(prettyJSON);
 jsonData.values.each { value ->
  String title = value.title
  String from = value.fromRef.latestChangeset
  String fromRepo = value.fromRef.repository.links.clone.find { it.name == "ssh" }.href
  String to = value.toRef.latestChangeset
  String toRepo = value.toRef.repository.links.clone.find { it.name == "ssh" }.href
  String repositorySlug = repo.slug
  String pullRequestId = value.id
  String requestUrl =  "http://stash/projects/PROJECTS/repos/"+repositorySlug+"/pull-requests/"+pullRequestId+"/overview"

  //Remember that this request has been triggered, and avoid triggering it again
  String identifier = from+" "+to
  if (previousPullRequests.text.contains(identifier)) {
   manager.listener.logger.println("Ignoring: "+identifier)
   return;
  }
  previousPullRequests.append(identifier+"\n")

  //Trigger a jenkins job that will verify the pull request
  String invokeBuildUrl = "http://jenkins/job/Pull%20Request%20Builder/buildWithParameters?token=SECRET_CONFIGURED_IN_BUILD&FROM="+from+"&TO="+to+"&FROMREPO="+fromRepo+"&TOREPO="+toRepo+"&REPOSITORY_SLUG="+repositorySlug+"&PULL_REQUEST_ID="+pullRequestId
  manager.listener.logger.println(invokeBuildUrl)
  new URL(invokeBuildUrl).getText()

  summary += "<h1>"+title+"</h1><br><a href='"+requestUrl+"'>"+requestUrl+"</a><br>From: "+jsonData.values[0].fromRef.id+" ("+from+") in "+fromRepo+"<br>To: "+jsonData.values[0].toRef.id+" ("+to+") in "+toRepo+"<br><a href='"+invokeBuildUrl+"'>"+invokeBuildUrl+"</a><hr>"
  newPullRequests++;
 }
}

//Add some info to the build
if (newPullRequests == 0) {
 manager.createSummary("gear2.gif").appendText("<h1>No new pull requests found!</h1>" , false)
} else {
 manager.addShortText("+"+newPullRequests, "grey", "white", "0px", "white")
 manager.createSummary("gear2.gif").appendText(summary , false)
}

Merging and building the pull request

I created a parameterized job to merge the pull request from source branch to target branch. It takes FROM_HASH, FROM_REPO, TO_HASH, TO_REPO, REPOSITORY_SLUG and PULL_REQUEST_ID as parameters.

The job has a build step execute shell that does the actual verification.

git clone $TO_REPO
cd *
git reset --hard $TO_HASH
git status
git remote add from $FROM_REPO
git fetch from
git merge $FROM_HASH
git --no-pager log --max-count=10 --graph --abbrev-commit

#compile command here ...

The job uses the Stash Notifier Plugin to record result in the pull request in Stash. Use the ${FROM_HASH} variable to get the build status reported correctly in the pull request in Stash.

It adds a comment to the pull request, like this.

curl -D- -u user:pass -X POST -H "Content-Type: application/json"  --data "{ \"text\": \"Looking good :) http://jenkins/job/Pull%20Request%20Builder/${BUILD_NUMBER}/\" }" http://stash/rest/api/1.0/projects/PROJECT/repos/$REPOSITORY_SLUG/pull-requests/$PULL_REQUEST_ID/comments

Static code analyzers

If you are using static code analyzers you may want to have a look at Jenkins Violation Comments to Stash Plugin for Jenkins.

It is configured like this.

alt

And will comment the pull requests like this.

alt