Home » Tech blog

Website status monitor using Jenkins

Jenkins is a great tool for continuous integration and deployment. It can also be used for monitoring websites that are live. In this post I'll show you my approach using a simple shell script.

1 February, 2016 jenkins dsl http groovy shell 2102 words 10 min Wouter Bulten 13 comments

Table of Contents

Creating a test script
Failing a build
Creating the Jenkins job using DSL
Preventing multiple emails
Wrapping up
Update (February 21)

Jenkins is a great tool for continuous integration and deployment. In a sense, and greatly simplified, it is a very complex and feature rich task runner. I already use it for testing development applications (e.g. with Mocha, PHPUnit, etc.) but was wondering whether it could also keep track of applications in an production environment. This gave me the idea to build a Jenkins job for monitoring live websites.

In this post I’ll show you how to create a website monitoring tool in Jenkins using the DSL approach and a shell script.

Creating a test script

First we’ll need some method of testing a website’s status. This part has nothing to do with Jenkins itself. The most simple way is using a shell script and a cURl request. The following command tries to access a website defined in $url and returns the status code:

#!/bin/bash

url='http://website-to-test'
code=`curl -sL --connect-timeout 20 --max-time 30 -w "%{http_code}\\n" "$url" -o /dev/null`

--connect-timeout and --max-time are added to make sure that we eventually give up. The server can become unresponsive and we also want to detect these kinds of events. After running, $code contains the http status code or ‘0’ in case of a failure. Given the status code we can react on the result:

#!/bin/bash

...

if [ "$code" = "200" ]; then
  echo "Website $url is online."
else
  echo "Website $url seems to be offline."
fi

We can further extend on this approach by wrapping the full check within a loop. By testing the website a few times we can hopefully prevent false negatives. Adding a loop in Bash is fairly simple given you know the syntax:

#!/bin/bash
url='http://website-to-test'
attempts=5
timeout=5

echo "Checking status of $url."

for (( i=1; i<=$attempts; i++ ))
do
  code=`curl -sL --connect-timeout 20 --max-time 30 -w "%{http_code}\\n" "$url" -o /dev/null`

  echo "Found code $code for $url."

  if [ "$code" = "200" ]; then
    echo "Website $url is online."
    break
  else
    echo "Website $url seems to be offline. Waiting $timeout seconds."
    sleep $timeout
  fi
done

Note: Using variables for constants, like the timeout and url, makes it easier to populate this with data from our Jenkins job.

Failing a build

We’ve created our test script and the next step is the Jenkins integration. For this we need some way of communicating the results back to Jenkins. As we are using a shell script, the most simple approach is using return codes: We’ll let our script return 0 when everything is online (which will complete the build) or 1 in case of an error. Using the default settings, Jenkins will fail a build (and trigger a failure event) when a shell script returns a non-zero value.

To make this work a simple flag is added to the cURL loop:

#!/bin/bash
url='http://website-to-test'
attempts=5
timeout=5
online=false

echo "Checking status of $url."

for (( i=1; i<=$attempts; i++ ))
do
  code=`curl -sL --connect-timeout 20 --max-time 30 -w "%{http_code}\\n" "$url" -o /dev/null`

  echo "Found code $code for $url."

  if [ "$code" = "200" ]; then
    echo "Website $url is online."
    online=true
    break
  else
    echo "Website $url seems to be offline. Waiting $timeout seconds."
    sleep $timeout
  fi
done

if $online; then
  echo "Monitor finished, website is online."
  exit 0
else
  echo "Monitor failed, website seems to be down."
  exit 1
fi

Our script will now test the website multiple times, if all of these tests fail the monitor will fail the build. This is a conservative check: all cURL requests must fail before we report an error. You can also turn this around and fail the build if a single cURL request fails. This can however increase the number of false positives!

Creating the Jenkins job using DSL

We’ll create the Jenkins job using a DSL file. Not sure what a DSL is? It is a great way to programmatically define Jenkins jobs. To quote the Github project page:

The Jenkins “Job DSL / Plugin” is made up of two parts: The Domain Specific Language (DSL) itself that allows users to describe jobs using a Groovy-based language, and a Jenkins plugin which manages the scripts and the updating of the Jenkins jobs which are created and maintained as a result. DSL Plugin GitHub Page

See for more info the wiki page of the plugin. Of course, you also do this all manually but, in general, using DSL files makes managing Jenkins a lot easier.

For your convenience, the full DSL file first (explanation follows below):

def recipients = 'youremail@example.com someother@example.com'
def website = 'http://website-we-want-to-test'

//Optional pre-send script, see further in this article for more info.
//If removed, make sure to also remove the 'presendScript' variable
//in the publisher block below.
def localPreSendScript = readFileFromWorkspace('<path to script>/pre_send_script.groovy_script')

//Job identifier, also used for the directory
job('website-monitor') {

  //Name of the job in Jenkins
  displayName('Website status of ExampleWebsite')

  triggers {
      //Run every 5 minutes
      cron('H/5 * * * * ')
  }

  steps {
    environmentVariables {
      env('WEBSITE', websie)
      env('TIMEOUT', 5)
      env('ATTEMPTS', 5)
    }

    //Run a shell script from the workspace
    shell(readFileFromWorkspace('<path to your shell script>/check_status_code.sh'))
  }

  logRotator {
    //Remove logs after two days
    daysToKeep(2)
  }

  publishers {
    extendedEmail(recipients, 'Website is offline') {

      //Events on which a email is sent
      trigger(triggerName: 'Failure', subject: 'Website offline!', body: 'Website ' + website + ' is offline!')
      trigger(triggerName: 'Fixed', subject: 'Website online!', body: 'Website ' + website + ' is back online!')

      //Custom configuration
      configure { node ->
        node << {
          contentType 'text/html'
          //Make sure that recipients mail to each other and not back to the system
          replyTo recipients
          //Optional script for extra conditions
          presendScript localPreSendScript
        }
      }
    }
  }
}

Now let’s run through all the different parts, starting with the trigger. In my case I want to check the website every 5 minutes; this gives a guarantee that there is a maximum downtime of 5 minutes. To do this I define a cron trigger:

triggers {
    //Run every 5 minutes
    cron('H/5 * * * * ')
}

You can of course increase or decrease the interval according to available server resources and importance of the website.

Then we define the actual job steps. We already created a shell script to do the actual checking so we call that using the shell function. In this case, the Bash script is in the same repository as my DSL file so I can use the readFileFromWorkspace function to load the content. Our environment variables are defined before we run the shell script. By extracting these variables we can use the same shell script for multiple jobs:

steps {
  environmentVariables {
    env('WEBSITE', websie)
    env('TIMEOUT', 5)
    env('ATTEMPTS', 5)
  }

  //Run a shell script from the workspace
  shell(readFileFromWorkspace('<path to your shell script>/check_status_code.sh'))
}

Given that the job will run 288 times a day (12 jobs per hour) the Jenkins logs can become rather full. I advise you to use log rotation and automatically remove old logs. For this specific jobs the logs are not very interesting so there is no real harm in removing them. We usually set the time limit to two days.

logRotator {
  //Remove logs after two days
  daysToKeep(2)
}

Last part (of almost any Jenkins job) is the publisher section: How should Jenkins report the results? We opted for an email system. Every time a website goes offline (or back online) the job sends an email. You could easily extend this with an SMS service or something similar.

The example below makes use of the Email-ext plugin which adds usefull functionality for sending emails (more than used here).

publishers {
  extendedEmail(recipients, 'Website is offline') {

    //Events on which a email is sent
    trigger(triggerName: 'Failure', subject: 'Website offline!', body: 'Website ' + website + ' is offline!')
    trigger(triggerName: 'Fixed', subject: 'Website online!', body: 'Website ' + website + ' is back online!')

    //Custom configuration
    configure { node ->
      node << {
        contentType 'text/html'
        //Make sure that recipients mail to each other and not back to the system
        replyTo recipients
        //Optional script for extra conditions
        presendScript localPreSendScript
      }
    }
  }
}

The job sends an email every time a build fails or when it is fixed (i.e. moved from a failure to a success state). These are the most usefull events for a website monitoring system.

The body and subject can be customized based on your requirements. The plugin can inject additional information into the email using variables. Some useful variables are:

${BUILD_NUMBER}: Current build number, makes every email unique.
${BUILD_STATUS}: Status of the buidl (e.g. FIXED or FAILURE).
${PROJECT_URL}: Direct url to the Jenkins project.
${BUILD_URL}: Direct url to the build.
${JENKINS_URL}: Url of the Jenkins installation.
${BUILD_LOG}: Full build log (tip: wrap this in <pre> tags to preserve formatting).

Using these variables you could construct a more extended email like so:

def emailTemplate = '' +
    '<h1>[#${BUILD_NUMBER}] ' + website + ' is down (Status: ${BUILD_STATUS})</h1>' +
    '<br><hr><br>' +
    '<p><b>' + website + ' seems to be down after 5 attempts.</b></p>' +
    '<p>This email was automatically generated by the Jenkins Monitoring Job.</p>' +
    '<p>The following users have been notified: <ul><li>' + recipients.replace(' ', '</li><li>') + '</li></ul></p>' +
    '<p><i>Reply to this email to send an email to all recipients automatically.</i></p>' +
    '<a href="${PROJECT_URL}">View project</a> | ' +
    '<a href="${BUILD_URL}">View build</a> | ' +
    '<a href="${JENKINS_URL}">Go to Jenkins</a>' +
    '<br><hr><br>' +
    '<p>Full build log:</p>' +
    '<pre>${BUILD_LOG}</pre>'

Preventing multiple emails

When a website goes down the watcher will send an email every 5 minutes, this can easily overflow your inbox. To fix this we need to add an additional restriction on the email publisher.

This restriction is implemented using an pre-send script: a simple Java script that is executed just before an email is sent. This script can then determine whether the email should be sent or not. We already defined the pre-send script in the DSL above through the presendScript option in the publisher. I use an extended version of a script I found on Stack Overflow with some very important additions.

Every time the build fails the script will check a file in our workspace. If that file does not exist it will be created and the email will be sent. The newly created file acts as a flag. Next time the build fails (e.g. after 5 minutes) the script checks whether the flag file is at least an hour old before a new email will be send out. This limits the amount of emails sent per hour to one.

Additionally, if the website goes online within the hour the flag is removed to make sure any additional errors are reported.

The full script is shown below. Save this to a file in your workspace and load it using the readFileFromWorkspace function (see the DSL above for an example).

//Build failed, check whether we may send an email
if (build.result.toString().equals("FAILURE")) {
  try {
    long minEmailGap = 1000 * 60 * 60; // Send max 1 email per hour
    def env = build.getEnvironment();
    String ws = env['WORKSPACE']

    //Create a new flag file in the workspace
    File file = new File(ws + "/delay-email-flag.txt");

    if (file.exists() == false) {
      file.createNewFile();
    }
    else {
      long currentTime = (new Date()).getTime();

      if (file.lastModified() + minEmailGap > currentTime) {
        cancel = true; //This stops the Email-Ext plugin sending the email
      }
      else {
        //Send an email and reset the timer
        file.setLastModified(currentTime);
      }
    }
  }
  catch(IOException e) {
    // Something went wrong, send email anyway
  }
}
//Build is successful, re-enable the emails
else if(build.result.toString().equals("SUCCESS")) {
  try {

    def env = build.getEnvironment();
    String ws = env['WORKSPACE']

    File file = new File(ws + "/delay-email-flag.txt");

    if(file.exists()) {

      //Project is fixed, remove the flag file
      file.delete();
    }
  }
  catch(IOException e) {
    // Something went wrong, send email anyway
  }
}

Note that using this pre-send script is not required, it will just make the job a bit more practical!

Wrapping up

Using this simple approach you can implement a easy but effective Jenkins website monitor. While it doesn’t offer all the fancy features of paid website monitors it gives the most vital information: the status of your website. Using the email plugin you will be notified of any errors within minutes and gives you the possibility to address the issue quickly.

Not using the DSL plugin? Start using it! It makes managing your jobs a lot easier. Still not convinced? You can also manually add the Bash script to a job and work from there! Good luck :)

Update (February 21)

The DSL implementation for the Extended Email plugin has been updated and the old way of registering triggers has been deprecated. The latest version uses a trigger block like so:

triggers {

  firstFailure {
    recipientList('email@example.com')
    attachBuildLog(true)
    content('Website is offline')
  }
}

See the DSL API viewer for more info.

13 comments

Add a new comment

phulei on Saturday, August 13, 2016

Great article! if a jenkins website is down, we may need to restart the jenkins process as well, am planning to add that.

your thoughts ?

Reply to phulei ↵

Use the form below to reply to the comment of phulei:

Wouter Bulten on Monday, August 15, 2016

Thanks!

I would recommend running your Jenkins installation on a different machine (not on the machine that is also running your production website). By separating the two you have an independent watcher. Moreover, any errors in your build or other resource-intensive Jenkins tasks will not affect your website.

Reply to Wouter Bulten ↵

Use the form below to reply to the comment of Wouter Bulten:

Atul on Thursday, April 27, 2017

Brilliant article..im able to clear lots of concepts ..Thanks for taking out time and writing such a wonderful article..Cheers.

Reply to Atul ↵

Use the form below to reply to the comment of Atul:

dileep on Thursday, September 12, 2019

i want list of urls to check in jenkins, can you help me on this.

Reply to dileep ↵

Use the form below to reply to the comment of dileep:

Wouter Bulten on Tuesday, September 17, 2019

Hi! You could probably alter the shell script to check more than one url and then only return the exit code when all websites are online.

Reply to Wouter Bulten ↵

Use the form below to reply to the comment of Wouter Bulten:

Ashok on Wednesday, June 15, 2022

Hi Wouter Bulten could you please tell me which statements need to modify for multi urls checking .

Reply to Ashok ↵

Use the form below to reply to the comment of Ashok:

Wouter Bulten on Wednesday, June 15, 2022

Hi Ashok,

It’s been a long time since I worked on this so I don’t have something laying around. In general: I would either 1) implement it for each website you want to check (so you also check the status per website) or 2) do multiple curl requests in the shell script.

Reply to Wouter Bulten ↵

Use the form below to reply to the comment of Wouter Bulten:

Anand Kumar on Monday, March 23, 2020

When using the curl command, we get 403 forbidden status for most of the URLs we search, and then on using “–user name:password”, we get statua OK 200 for all the websites. Is there a reliable way to determine 200 OK status code for any website, or we need to try all the options with curl command?

Reply to Anand Kumar ↵

Use the form below to reply to the comment of Anand Kumar:

Wouter Bulten on Monday, March 23, 2020

Hi Anand,

Are the websites you want to check protected with http authentication? If so, yes you will need to supply curl with the credentials, otherwise it cannot access the url. You could always make a specific url accessible without a password and then check that url.

Regards, Wouter

Reply to Wouter Bulten ↵

Use the form below to reply to the comment of Wouter Bulten:

vmp on Thursday, August 12, 2021

Thanks for this article. However, after build I am getting errors on the line below: ‘shell(readFileFromWorkspace(’/check_status_code.sh’))’

I am testing this on the localhost Jenkins. Can you tell me where I have to put this shell script and what should be the path for this on local Jenkins as well as on the live Jenkins environment?

Reply to vmp ↵

Use the form below to reply to the comment of vmp:

Wouter Bulten on Friday, August 13, 2021

Hi! I haven’t worked with this for quite some time so I’m not sure if I can help you fix it. What was the error you received? From first glance, your path is probably incorrect, maybe something like this will work:

shell(readFileFromWorkspace(‘./check_status_code.sh'))

Note that the plugins and syntax could have changed since I created this blogpost.

Regards, Wouter

Reply to Wouter Bulten ↵

Use the form below to reply to the comment of Wouter Bulten:

Kadir on Thursday, November 3, 2022

Great article thanks for sharing. Can we apply this to a server check. I mean if we want to check if an ip address is up or down?

Reply to Kadir ↵

Use the form below to reply to the comment of Kadir:

Wouter Bulten on Thursday, November 3, 2022

Sure! Instead of curl I would something like ping in the script that checks it.

Reply to Wouter Bulten ↵

Use the form below to reply to the comment of Wouter Bulten:

Creating a test script¶

Failing a build¶

Creating the Jenkins job using DSL¶

Preventing multiple emails¶

Wrapping up¶

Update (February 21)¶

13 comments

phulei on Saturday, August 13, 2016

Wouter Bulten on Monday, August 15, 2016

Atul on Thursday, April 27, 2017

dileep on Thursday, September 12, 2019

Wouter Bulten on Tuesday, September 17, 2019

Ashok on Wednesday, June 15, 2022

Wouter Bulten on Wednesday, June 15, 2022

Anand Kumar on Monday, March 23, 2020

Wouter Bulten on Monday, March 23, 2020

vmp on Thursday, August 12, 2021

Wouter Bulten on Friday, August 13, 2021

Kadir on Thursday, November 3, 2022

Wouter Bulten on Thursday, November 3, 2022

Creating a test script

Failing a build

Creating the Jenkins job using DSL

Preventing multiple emails

Wrapping up

Update (February 21)