KB at your finger tips

This is one stop global knowledge base where you can learn about all the products, solutions and support features.

Getting started with Pipeline

Table of Contents

Prerequisites
Defining a Pipeline
- Through Blue Ocean
- Through the classic UI
- In SCM
Built-in Documentation
- Snippet Generator
- Global Variable Reference
- Declarative Directive Generator
Further Reading
- Additional Resources

As mentioned previously, Jenkins Pipeline is a suite of plugins that supports implementing and integrating continuous delivery pipelines into Jenkins. Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines "as code" via the Pipeline DSL. ^[1]

This section describes how to get started with creating your Pipeline project in Jenkins and introduces you to the various ways that a Jenkinsfile can be created and stored.

Prerequisites

To use Jenkins Pipeline, you will need:

Jenkins 2.x or later (older versions back to 1.642.3 may work but are not recommended)
Pipeline plugin, ^[2] which is installed as part of the "suggested plugins" (specified when running through the Post-installation setup wizard after installing Jenkins).

Read more about how to install and manage plugins in Managing Plugins.

Defining a Pipeline

Both Declarative and Scripted Pipeline are DSLs ^[1] to describe portions of your software delivery pipeline. Scripted Pipeline is written in a limited form of Groovy syntax.

Relevant components of Groovy syntax will be introduced as required throughout this documentation, so while an understanding of Groovy is helpful, it is not required to work with Pipeline.

A Pipeline can be created in one of the following ways:

Through Blue Ocean - after setting up a Pipeline project in Blue Ocean, the Blue Ocean UI helps you write your Pipeline’s Jenkinsfile and commit it to source control.
Through the classic UI - you can enter a basic Pipeline directly in Jenkins through the classic UI.
In SCM - you can write a Jenkinsfile manually, which you can commit to your project’s source control repository. ^[3]

The syntax for defining a Pipeline with either approach is the same, but while Jenkins supports entering Pipeline directly into the classic UI, it is generally considered best practice to define the Pipeline in a Jenkinsfile which Jenkins will then load directly from source control.

This video provides basic instructions on how to write both Declarative and Scripted Pipelines.

Writing a Pipeline script in Jenkins

Through Blue Ocean

If you are new to Jenkins Pipeline, the Blue Ocean UI helps you set up your Pipeline project, and automatically creates and writes your Pipeline (i.e. the Jenkinsfile ) for you through the graphical Pipeline editor.

As part of setting up your Pipeline project in Blue Ocean, Jenkins configures a secure and appropriately authenticated connection to your project’s source control repository. Therefore, any changes you make to the Jenkinsfile via Blue Ocean’s Pipeline editor are automatically saved and committed to source control.

Read more about Blue Ocean in the Blue Ocean chapter and Getting started with Blue Ocean page.

Blue Ocean status

Blue Ocean will not receive further functionality updates. Blue Ocean will continue to provide easy-to-use Pipeline visualization, but it will not be enhanced further. It will only receive selective updates for significant security issues or functional defects.

The Pipeline syntax snippet generator assists users as they define Pipeline steps with their arguments. It is the preferred tool for Jenkins Pipeline creation, as it provides online help for the Pipeline steps available in your Jenkins controller. It uses the plugins installed on your Jenkins controller to generate the Pipeline syntax. Refer to the Pipeline steps reference page for information on all available Pipeline steps.

Through the classic UI

A Jenkinsfile created using the classic UI is stored by Jenkins itself (within the Jenkins home directory).

To create a basic Pipeline through the Jenkins classic UI:

If required, ensure you are logged in to Jenkins.
From the Jenkins home page (i.e. the Dashboard of the Jenkins classic UI), click New Item at the top left.
In the Enter an item name field, specify the name for your new Pipeline project.
Caution: Jenkins uses this item name to create directories on disk. It is recommended to avoid using spaces in item names, since doing so may uncover bugs in scripts that do not properly handle spaces in directory paths.
Scroll down and click Pipeline , then click OK at the end of the page to open the Pipeline configuration page (whose General tab is selected).
Click the Pipeline tab at the top of the page to scroll down to the Pipeline section.
Note: If instead you are defining your Jenkinsfile in source control, follow the instructions in In SCM below.
In the Pipeline section, ensure that the Definition field indicates the Pipeline script option.

Enter your Pipeline code into the Script text area.
For instance, copy the following Declarative example Pipeline code (below the Jenkinsfile ( … ) heading) or its Scripted version equivalent and paste this into the Script text area. (The Declarative example below is used throughout the remainder of this procedure.)

Jenkinsfile (Declarative Pipeline)

pipeline {
    agent any (1)
    stages {
        stage('Stage 1') {
            steps {
                echo 'Hello world!' (2)
            }
        }
    }
}

Toggle Scripted Pipeline (Advanced)

1	`agent` instructs Jenkins to allocate an executor (on any available agent/node in the Jenkins environment) and workspace for the entire Pipeline.
2	`echo` writes simple string in the console output.
3	`node` effectively does the same as `agent` (above).

Example Pipeline in the classic UI

Note: You can also select from canned Scripted Pipeline examples from the try sample Pipeline option at the top right of the Script text area. Be aware that there are no canned Declarative Pipeline examples available from this field.

Click Save to open the Pipeline project/item view page.
On this page, click Build Now on the left to run the Pipeline.
Under Build History on the left, click #1 to access the details for this particular Pipeline run.
Click Console Output to see the full output from the Pipeline run. The following output shows a successful run of your Pipeline.

Notes:
- You can also access the console output directly from the Dashboard by clicking the colored globe to the left of the build number (e.g. #1 ).
- Defining a Pipeline through the classic UI is convenient for testing Pipeline code snippets, or for handling simple Pipelines or Pipelines that do not require source code to be checked out/cloned from a repository. As mentioned above, unlike Jenkinsfile s you define through Blue Ocean (above) or in source control (below), Jenkinsfile s entered into the Script text area area of Pipeline projects are stored by Jenkins itself, within the Jenkins home directory. Therefore, for greater control and flexibility over your Pipeline, particularly for projects in source control that are likely to gain complexity, it is recommended that you use Blue Ocean or source control to define your Jenkinsfile .

In SCM

Complex Pipelines are difficult to write and maintain within the classic UI’s Script text area of the Pipeline configuration page.

To make this easier, your Pipeline’s Jenkinsfile can be written in a text editor or integrated development environment (IDE) and committed to source control ^[3] (optionally with the application code that Jenkins will build). Jenkins can then check out your Jenkinsfile from source control as part of your Pipeline project’s build process and then proceed to execute your Pipeline.

To configure your Pipeline project to use a Jenkinsfile from source control:

Follow the procedure above for defining your Pipeline through the classic UI until you reach step 5 (accessing the Pipeline section on the Pipeline configuration page).
From the Definition field, choose the Pipeline script from SCM option.
From the SCM field, choose the type of source control system of the repository containing your Jenkinsfile .
Complete the fields specific to your repository’s source control system.
Tip: If you are uncertain of what value to specify for a given field, click its ? icon to the right for more information.
In the Script Path field, specify the location (and name) of your Jenkinsfile . This location is the one that Jenkins checks out/clones the repository containing your Jenkinsfile , which should match that of the repository’s file structure. The default value of this field assumes that your Jenkinsfile is named "Jenkinsfile" and is located at the root of the repository.

When you update the designated repository, a new build is triggered, as long as the Pipeline is configured with an SCM polling trigger.

Since Pipeline code (i.e. Scripted Pipeline in particular) is written in Groovy-like syntax, if your IDE is not correctly syntax highlighting your Jenkinsfile , try inserting the line #!/usr/bin/env groovy at the top of the Jenkinsfile , ^[4] footnotegroovy_shebang:[Shebang line (Groovy syntax)] which may rectify the issue.

Built-in Documentation

Pipeline ships with built-in documentation features to make it easier to create Pipelines of varying complexities. This built-in documentation is automatically generated and updated based on the plugins installed in the Jenkins instance.

The built-in documentation can be found globally at ${YOUR_JENKINS_URL}/pipeline-syntax . The same documentation is also linked as Pipeline Syntax in the side-bar for any configured Pipeline project.

Classic UI left column on an item

Snippet Generator

The built-in "Snippet Generator" utility is helpful for creating bits of code for individual steps, discovering new steps provided by plugins, or experimenting with different parameters for a particular step.

The Snippet Generator is dynamically populated with a list of the steps available to the Jenkins instance. The number of steps available is dependent on the plugins installed which explicitly expose steps for use in Pipeline.

To generate a step snippet with the Snippet Generator:

Navigate to the Pipeline Syntax link (referenced above) from a configured Pipeline, or at ${YOUR_JENKINS_URL}/pipeline-syntax .
Select the desired step in the Sample Step dropdown menu
Use the dynamically populated area below the Sample Step dropdown to configure the selected step.
Click Generate Pipeline Script to create a snippet of Pipeline which can be copied and pasted into a Pipeline.

Snippet Generator

To access additional information and/or documentation about the step selected, click on the help icon (indicated by the red arrow in the image above).

Global Variable Reference

In addition to the Snippet Generator, which only surfaces steps, Pipeline also provides a built-in " Global Variable Reference ." Like the Snippet Generator, it is also dynamically populated by plugins. Unlike the Snippet Generator however, the Global Variable Reference only contains documentation for variables provided by Pipeline or plugins, which are available for Pipelines.

The variables provided by default in Pipeline are:

env: Exposes environment variables, for example: env.PATH or env.BUILD_ID . Consult the built-in global variable reference at ${YOUR_JENKINS_URL}/pipeline-syntax/globals#env for a complete, and up to date, list of environment variables available in Pipeline.
params: Exposes all parameters defined for the Pipeline as a read-only Map, for example: params.MY_PARAM_NAME .
currentBuild: May be used to discover information about the currently executing Pipeline, with properties such as currentBuild.result , currentBuild.displayName , etc. Consult the built-in global variable reference at ${YOUR_JENKINS_URL}/pipeline-syntax/globals for a complete, and up to date, list of properties available on currentBuild .

This video reviews using the currentBuild variable in Jenkins Pipeline.

Declarative Directive Generator

While the Snippet Generator helps with generating steps for a Scripted Pipeline or for the steps block in a stage in a Declarative Pipeline, it does not cover the sections and directives used to define a Declarative Pipeline. The "Declarative Directive Generator" utility helps with that. Similar to the Snippet Generator, the Directive Generator allows you to choose a Declarative directive, configure it in a form, and generate the configuration for that directive, which you can then use in your Declarative Pipeline.

To generate a Declarative directive using the Declarative Directive Generator:

Navigate to the Pipeline Syntax link (referenced above) from a configured Pipeline, and then click on the Declarative Directive Generator link in the sidepanel, or go directly to ${YOUR_JENKINS_URL}/directive-generator .
Select the desired directive in the dropdown menu
Use the dynamically populated area below the dropdown to configure the selected directive.
Click Generate Directive to create the directive’s configuration to copy into your Pipeline.

The Directive Generator can generate configuration for nested directives, such as conditions inside a when directive, but it cannot generate Pipeline steps. For the contents of directives which contain steps, such as steps inside a stage or conditions like always or failure inside post , the Directive Generator adds a placeholder comment instead. You will still need to add steps to your Pipeline by hand.

Jenkinsfile (Declarative Pipeline)

stage('Stage 1') {
    steps {
        // One or more steps need to be included within the steps block.
    }
}

Pipeline

Chapter Sub-Sections

Table of Contents

What is Jenkins Pipeline?
- Declarative versus Scripted Pipeline syntax
Why Pipeline?
Pipeline concepts
- Pipeline
- Node
- Stage
- Step
Pipeline syntax overview
- Declarative Pipeline fundamentals
- Scripted Pipeline fundamentals
Pipeline example

This chapter covers all recommended aspects of Jenkins Pipeline functionality, including how to:

get started with Pipeline - covers how to define a Jenkins Pipeline (i.e. your Pipeline ) through Blue Ocean, through the classic UI or in SCM,
create and use a Jenkinsfile - covers use-case scenarios on how to craft and construct your Jenkinsfile ,
work with branches and pull requests,
use Docker with Pipeline - covers how Jenkins can invoke Docker containers on agents/nodes (from a Jenkinsfile ) to build your Pipeline projects,
extend Pipeline with shared libraries,
use different development tools to facilitate the creation of your Pipeline, and
work with Pipeline syntax - this page is a comprehensive reference of all Declarative Pipeline syntax.

For an overview of content in the Jenkins User Handbook, see User Handbook Overview.

What is Jenkins Pipeline?

Jenkins Pipeline (or simply "Pipeline" with a capital "P") is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.

A continuous delivery (CD) pipeline is an automated expression of your process for getting software from version control right through to your users and customers. Every change to your software (committed in source control) goes through a complex process on its way to being released. This process involves building the software in a reliable and repeatable manner, as well as progressing the built software (called a "build") through multiple stages of testing and deployment.

Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines "as code" via the Pipeline domain-specific language (DSL) syntax. ^[1]

The definition of a Jenkins Pipeline is written into a text file (called a Jenkinsfile ) which in turn can be committed to a project’s source control repository. ^[2] This is the foundation of "Pipeline-as-code"; treating the CD pipeline a part of the application to be versioned and reviewed like any other code.

Creating a Jenkinsfile and committing it to source control provides a number of immediate benefits:

Automatically creates a Pipeline build process for all branches and pull requests.
Code review/iteration on the Pipeline (along with the remaining source code).
Audit trail for the Pipeline.
Single source of truth ^[3] for the Pipeline, which can be viewed and edited by multiple members of the project.

While the syntax for defining a Pipeline, either in the web UI or with a Jenkinsfile is the same, it is generally considered best practice to define the Pipeline in a Jenkinsfile and check that in to source control.

Declarative versus Scripted Pipeline syntax

A Jenkinsfile can be written using two types of syntax - Declarative and Scripted.

Declarative and Scripted Pipelines are constructed fundamentally differently. Declarative Pipeline is a more recent feature of Jenkins Pipeline which:

provides richer syntactical features over Scripted Pipeline syntax, and
is designed to make writing and reading Pipeline code easier.

Many of the individual syntactical components (or "steps") written into a Jenkinsfile , however, are common to both Declarative and Scripted Pipeline. Read more about how these two types of syntax differ in Pipeline concepts and Pipeline syntax overview below.

Why Pipeline?

Jenkins is, fundamentally, an automation engine which supports a number of automation patterns. Pipeline adds a powerful set of automation tools onto Jenkins, supporting use cases that span from simple continuous integration to comprehensive CD pipelines. By modeling a series of related tasks, users can take advantage of the many features of Pipeline:

Code : Pipelines are implemented in code and typically checked into source control, giving teams the ability to edit, review, and iterate upon their delivery pipeline.
Durable : Pipelines can survive both planned and unplanned restarts of the Jenkins controller.
Pausable : Pipelines can optionally stop and wait for human input or approval before continuing the Pipeline run.
Versatile : Pipelines support complex real-world CD requirements, including the ability to fork/join, loop, and perform work in parallel.
Extensible : The Pipeline plugin supports custom extensions to its DSL ^[1] and multiple options for integration with other plugins.

While Jenkins has always allowed rudimentary forms of chaining Freestyle Jobs together to perform sequential tasks, ^[4] Pipeline makes this concept a first-class citizen in Jenkins.

What is the difference between Freestyle and Pipeline in Jenkins

Building on the core Jenkins value of extensibility, Pipeline is also extensible both by users with Pipeline Shared Libraries and by plugin developers. ^[5]

The flowchart below is an example of one CD scenario easily modeled in Jenkins Pipeline:

Pipeline Flow

Pipeline concepts

The following concepts are key aspects of Jenkins Pipeline, which tie in closely to Pipeline syntax (see the overview below).

Pipeline

A Pipeline is a user-defined model of a CD pipeline. A Pipeline’s code defines your entire build process, which typically includes stages for building an application, testing it and then delivering it.

Also, a pipeline block is a key part of Declarative Pipeline syntax.

Node

A node is a machine which is part of the Jenkins environment and is capable of executing a Pipeline.

Also, a node block is a key part of Scripted Pipeline syntax.

Stage

A stage block defines a conceptually distinct subset of tasks performed through the entire Pipeline (e.g. "Build", "Test" and "Deploy" stages), which is used by many plugins to visualize or present Jenkins Pipeline status/progress. ^[6]

Step

A single task. Fundamentally, a step tells Jenkins what to do at a particular point in time (or "step" in the process). For example, to execute the shell command make use the sh step: sh 'make' . When a plugin extends the Pipeline DSL, ^[1] that typically means the plugin has implemented a new step .

Pipeline syntax overview

The following Pipeline code skeletons illustrate the fundamental differences between Declarative Pipeline syntax and Scripted Pipeline syntax.

Be aware that both stages and steps (above) are common elements of both Declarative and Scripted Pipeline syntax.

Declarative Pipeline fundamentals

In Declarative Pipeline syntax, the pipeline block defines all the work done throughout your entire Pipeline.

Jenkinsfile (Declarative Pipeline)

pipeline {
    agent any (1)
    stages {
        stage('Build') { (2)
            steps {
                // (3)
            }
        }
        stage('Test') { (4)
            steps {
                // (5)
            }
        }
        stage('Deploy') { (6)
            steps {
                // (7)
            }
        }
    }
}

1	Execute this Pipeline or any of its stages, on any available agent.
2	Defines the "Build" stage.
3	Perform some steps related to the "Build" stage.
4	Defines the "Test" stage.
5	Perform some steps related to the "Test" stage.
6	Defines the "Deploy" stage.
7	Perform some steps related to the "Deploy" stage.

Scripted Pipeline fundamentals

In Scripted Pipeline syntax, one or more node blocks do the core work throughout the entire Pipeline. Although this is not a mandatory requirement of Scripted Pipeline syntax, confining your Pipeline’s work inside of a node block does two things:

Schedules the steps contained within the block to run by adding an item to the Jenkins queue. As soon as an executor is free on a node, the steps will run.
Creates a workspace (a directory specific to that particular Pipeline) where work can be done on files checked out from source control.
Caution: Depending on your Jenkins configuration, some workspaces may not get automatically cleaned up after a period of inactivity. See tickets and discussion linked from JENKINS-2111 for more information.

Jenkinsfile (Scripted Pipeline)

node {  (1)
    stage('Build') { (2)
        // (3)
    }
    stage('Test') { (4)
        // (5)
    }
    stage('Deploy') { (6)
        // (7)
    }
}

1	Execute this Pipeline or any of its stages, on any available agent.
2	Defines the "Build" stage. `stage` blocks are optional in Scripted Pipeline syntax. However, implementing `stage` blocks in a Scripted Pipeline provides clearer visualization of each `stage’s subset of tasks/steps in the Jenkins UI.
3	Perform some steps related to the "Build" stage.
4	Defines the "Test" stage.
5	Perform some steps related to the "Test" stage.
6	Defines the "Deploy" stage.
7	Perform some steps related to the "Deploy" stage.

Pipeline example

Here is an example of a Jenkinsfile using Declarative Pipeline syntax - its Scripted syntax equivalent can be accessed by clicking the Toggle Scripted Pipeline link below:

Jenkinsfile (Declarative Pipeline)

pipeline { (1)
    agent any (2)
    options {
        skipStagesAfterUnstable()
    }
    stages {
        stage('Build') { (3)
            steps { (4)
                sh 'make' (5)
            }
        }
        stage('Test'){
            steps {
                sh 'make check'
                junit 'reports/**/*.xml' (6)
            }
        }
        stage('Deploy') {
            steps {
                sh 'make publish'
            }
        }
    }
}

Toggle Scripted Pipeline (Advanced)

1	`pipeline` is Declarative Pipeline-specific syntax that defines a "block" containing all content and instructions for executing the entire Pipeline.
2	`agent` is Declarative Pipeline-specific syntax that instructs Jenkins to allocate an executor (on a node) and workspace for the entire Pipeline.
3	`stage` is a syntax block that describes a stage of this Pipeline. Read more about `stage` blocks in Declarative Pipeline syntax on the Pipeline syntax page. As mentioned above, `stage` blocks are optional in Scripted Pipeline syntax.
4	`steps` is Declarative Pipeline-specific syntax that describes the steps to be run in this `stage` .
5	`sh` is a Pipeline step (provided by the Pipeline: Nodes and Processes plugin) that executes the given shell command.
6	`junit` is another Pipeline step (provided by the JUnit plugin) for aggregating test reports.
7	`sh` is a Pipeline step (provided by the Pipeline: Nodes and Processes plugin) that executes the given shell command.

Read more about Pipeline syntax on the Pipeline Syntax page.

1. Domain-specific language

2. Source control management

3. Single source of truth

4. Additional plugins have been used to implement complex behaviors utilizing Freestyle Jobs such as the Copy Artifact, Parameterized Trigger, and Promoted Builds plugins

5. GitHub Organization Folder plugin

6. Blue Ocean, Pipeline: Stage View plugin

â Using Jenkins

Index

Getting started with Pipeline â

Was this page helpful?

Please submit your feedback about this page through this quick form.

Alternatively, if you don't wish to complete the quick form, you can simply indicate if you found this page helpful?

See existing feedback here.

Read article

Welcome to Knowledge Base!

KB at your finger tips

Getting started with Pipeline

Prerequisites

Defining a Pipeline

Through Blue Ocean

Through the classic UI

In SCM

Built-in Documentation

Snippet Generator

Global Variable Reference

Declarative Directive Generator

Further Reading

Additional Resources

Pipeline

What is Jenkins Pipeline?

Declarative versus Scripted Pipeline syntax

Why Pipeline?

Pipeline concepts

Pipeline

Node

Stage

Step

Pipeline syntax overview

Declarative Pipeline fundamentals

Scripted Pipeline fundamentals

Pipeline example

Thank you for subscribing to AiDOOS! Stay connected for updates!