Deploy and update a job

12min
|
Nomad

Now that you have deployed your Nomad cluster and configured the CLI to interact with it, the next step is deploying an application.

In this tutorial, you will deploy and update an example application. In the process, you will learn about the Nomad job specification.

The example application runs in Docker containers and consists of a database and a web frontend that reads from the database. You will set up the database with a parameterized batch job and then use a periodic batch job to start additional short-lived jobs that write data to the database.

Job types

Nomad supports several job types including service, parameterized and periodic batch jobs, and system. In this tutorial, you will be introduced to the service job and both parameterized and periodic batch jobs.

Service jobs are for long lived services that run until explicitly stopped.

Batch jobs are for short lived jobs that run until they exit successfully.

The parameterized block lets you configure a batch job to accept required or optional inputs. You trigger the job with the nomad job dispatch command.
The periodic block lets you schedule a Nomad job to run at set times. These are also known as a Nomad cron jobs.

Review the example application

The example application simulates employees working at a technology company. They come online, complete their tasks, and then log off.

Navigate to the jobs directory of the example repository on your local machine

Note

If you are following the getting started collection and are using a cloud-based cluster, navigate back to the root of the repository's directory first.

$ cd jobs

Each of the jobspec files below that make up the application sets the driver attribute to docker and specifies an image stored on the GitHub Container Registry in the config block with the image attribute. The Redis job is the exception as it uses an official Redis image hosted on Docker Hub. By default, Nomad looks for images on Docker Hub so the full path including https:// is not necessary for the Redis job.

task "redis-task" {
  driver = "docker"
  config {
    image = "redis:7.0.7-alpine"
  }
}

The parameterized and periodic configurations of the pytechco-setup and pytechco-employee jobs below are more advanced use cases of the jobspec system and are here to provide an example of what you can do with batch jobs. Many users only use service type jobs because the applications they deploy are long-living. Not all applications require batch jobs and it is okay if yours does not use them.

Diagram illustrating the relationship between the Pytechco application components

pytechco-redis.nomad.hcl - This service job runs and exposes a Redis database as a Nomad service for the other application components to connect to. The jobspec sets the type to service, configures a Nomad service block to use Nomad's native service discovery, and creates a service named redis-svc.

Click here to view the contents of pytecho-redis.nomad.hcl

job "pytechco-redis" {
  type = "service"

  group "ptc-redis" {
    count = 1
    network {
      port "redis" {
        to = 6379
      }
    }

    service {
      name     = "redis-svc"
      port     = "redis"
      provider = "nomad"
    }

    task "redis-task" {
      driver = "docker"

      config {
        image = "redis:7.0.7-alpine"
        ports = ["redis"]
      }
    }
  }
}

pytechco-web.nomad.hcl - This service jobs runs the web application frontend that displays the values stored in the database and the active employees. The jobspec sets the type to service and uses a static port of 5000 for the application. It also uses the nomadService built-in function to retrieve address and port information of the Redis database service.

Click here to view the contents of pytecho-web.nomad.hcl

job "pytechco-web" {
  type = "service"

  group "ptc-web" {
    count = 1
    network {
      port "web" {
        static = 5000
      }
    }

    service {
      name     = "ptc-web-svc"
      port     = "web"
      provider = "nomad"
    }

    task "ptc-web-task" {
      # Retrieves the .Address and .Port connection values for
      # redis-svc with nomadService and saves them to env vars
      # REFRESH_INTERVAL is how often the UI refreshes in milliseconds
      template {
        data        = <<EOH
{{ range nomadService "redis-svc" }}
REDIS_HOST={{ .Address }}
REDIS_PORT={{ .Port }}
FLASK_HOST=0.0.0.0
REFRESH_INTERVAL=500
{{ end }}
EOH
        destination = "local/env.txt"
        env         = true
      }

      driver = "docker"

      config {
        image = "ghcr.io/hashicorp-education/learn-nomad-getting-started/ptc-web:1.0"
        ports = ["web"]
      }
    }
  }
}

pytechco-setup.nomad.hcl - This parameterized batch job sets up the database with default values. You can dispatch it multiple times to reset the database. The jobspec sets the type to batch and has a parameterized block with a meta_required attribute that requires a value for budget when dispatching.

Click here to view the contents of pytecho-setup.nomad.hcl

job "pytechco-setup" {
  type = "batch"

  parameterized {
    meta_required = ["budget"]
  }

  group "ptc-setup" {
    count = 1

    task "ptc-setup-task" {

      # Retrieves the .Address and .Port connection values for
      # redis-svc with nomadService and saves them to env vars
      # NOMAD_META_budget is read from the job's meta vars
      template {
        data        = <<EOH
{{ range nomadService "redis-svc" }}
REDIS_HOST={{ .Address }}
REDIS_PORT={{ .Port }}
{{ end }}
PTC_BUDGET={{ env "NOMAD_META_budget" }}
EOH
        destination = "local/env.txt"
        env         = true
      }
      driver = "docker"

      config {
        image = "ghcr.io/hashicorp-education/learn-nomad-getting-started/ptc-setup:1.0"
      }
    }
  }
}

pytechco-employee.nomad.hcl - This periodic batch job brings an employee online. It randomizes the employee's job type and other variables such as how long they work for and the rate at which they complete their tasks. The jobspec sets the type to batch and has a periodic block that sets the cron attribute to a value that will allow it to start a new job every 3 seconds.

Click here to view the contents of pytecho-employee.nomad.hcl

job "pytechco-employee" {
  type = "batch"

  periodic {
    crons = [
      "0/3 * * * * * *"
    ]
    prohibit_overlap = false
  }

  group "ptc-employee" {
    count = 1

    task "ptc-employee-task" {

      restart {
        attempts = 2
        delay    = "15s"
        interval = "10s"
        mode     = "fail"
      }

      # Retrieves the .Address and .Port connection values for
      # redis-svc with nomadService and saves them to env vars
      # NOMAD_SHORT_ALLOC_ID is read from the job's runtime vars
      template {
        data        = <<EOH
{{ range nomadService "redis-svc" }}
REDIS_HOST={{ .Address }}
REDIS_PORT={{ .Port }}
PTC_EMPLOYEE_ID={{ env "NOMAD_SHORT_ALLOC_ID"}}
{{ end }}
EOH
        destination = "local/env.txt"
        env         = true
      }
      driver = "docker"

      config {
        image = "ghcr.io/hashicorp-education/learn-nomad-getting-started/ptc-employee:1.0"
        // args = [
        //     "--employee-type", "sales_engineer"
        // ]
      }
    }
  }
}

As of Nomad 1.5.0, the datacenter attribute defaults to all available datacenters in the cluster ("*"). As a result, the jobspecs for this tutorial omit the attribute because the default value is sufficient. If you are running earlier versions of the Nomad CLI and cluster binary, you will need to include and set the attribute.

Deploy the application

The Docker images in the jobspec files are pre-built but you can checkout and modify the source code and rebuild them. If you do, be sure that you publish them to an accessible image repository like Docker Hub or GitHub Container Registry and update each of the corresponding jobspecs with the new image locations.

Submit the database job.

$ nomad job run pytechco-redis.nomad.hcl
==> 2023-03-10T12:16:09-05:00: Monitoring evaluation "d44af37b"
    2023-03-10T12:16:09-05:00: Evaluation triggered by job "pytechco-redis"
    2023-03-10T12:16:10-05:00: Evaluation within deployment: "0ea05651"
    2023-03-10T12:16:10-05:00: Allocation "be0bda79" created: node "2c9f4b7e", group "ptc-redis"
    2023-03-10T12:16:10-05:00: Evaluation status changed: "pending" -> "complete"
==> 2023-03-10T12:16:10-05:00: Evaluation "d44af37b" finished with status "complete"
==> 2023-03-10T12:16:10-05:00: Monitoring deployment "0ea05651"
  ✓ Deployment "0ea05651" successful
    
    2023-03-10T12:16:24-05:00
    ID          = 0ea05651
    Job ID      = pytechco-redis
    Job Version = 0
    Status      = successful
    Description = Deployment completed successfully
    
    Deployed
    Task Group  Desired  Placed  Healthy  Unhealthy  Progress Deadline
    ptc-redis   1        1       1        0          2023-03-10T17:26:23Z

Submit the webapp frontend job.

$ nomad job run pytechco-web.nomad.hcl
==> 2023-03-10T12:16:42-05:00: Monitoring evaluation "62615d0e"
    2023-03-10T12:16:42-05:00: Evaluation triggered by job "pytechco-web"
    2023-03-10T12:16:43-05:00: Evaluation within deployment: "cdb7d282"
    2023-03-10T12:16:43-05:00: Allocation "b5dc39d5" created: node "2c9f4b7e", group "ptc-web"
    2023-03-10T12:16:43-05:00: Evaluation status changed: "pending" -> "complete"
==> 2023-03-10T12:16:43-05:00: Evaluation "62615d0e" finished with status "complete"
==> 2023-03-10T12:16:43-05:00: Monitoring deployment "cdb7d282"
  ✓ Deployment "cdb7d282" successful
    
    2023-03-10T12:16:59-05:00
    ID          = cdb7d282
    Job ID      = pytechco-web
    Job Version = 0
    Status      = successful
    Description = Deployment completed successfully
    
    Deployed
    Task Group  Desired  Placed  Healthy  Unhealthy  Progress Deadline
    ptc-web     1        1       1        0          2023-03-10T17:26:57Z

Get the IP address of the webapp. The following command gets the allocation ID of the web job and uses that ID to get the status of the allocation. It then searches for the IP address in the allocation status output and formats the IP address into a link with the webapp's port. Open the URL from the output in your browser to see the webapp frontend.

$ nomad node status -verbose \
    $(nomad job allocs pytechco-web | grep -i running | awk '{print $2}') | \
    grep -i ip-address | awk -F "=" '{print $2}' | xargs | \
    awk '{print "http://"$1":5000"}'

$ nomad node status -verbose \
    $(nomad job allocs pytechco-web | grep -i running | awk '{print $2}') | \
    grep -i public-ipv4 | awk -F "=" '{print $2}' | xargs | \
    awk '{print "http://"$1":5000"}'

$ nomad node status -verbose \
    $(nomad job allocs pytechco-web | grep -i running | awk '{print $2}') | \
    grep -i external-ip | awk -F "=" '{print $2}' | xargs | \
    awk '{print "http://"$1":5000"}'

$ nomad node status -verbose \
    $(nomad job allocs pytechco-web | grep -i running | awk '{print $2}') | \
    grep -i public-ipv4 | awk -F "=" '{print $2}' | xargs | \
    awk '{print "http://"$1":5000"}'

Submit the setup job.

$ nomad job run pytechco-setup.nomad.hcl
Job registration successful

Dispatch the setup job by providing a value for budget.

$ nomad job dispatch -meta budget="200" pytechco-setup
Dispatched Job ID = pytechco-setup/dispatch-1678468734-396cfa83
Evaluation ID     = 53a77034

==> 2023-03-10T12:18:54-05:00: Monitoring evaluation "53a77034"
    2023-03-10T12:18:54-05:00: Evaluation triggered by job "pytechco-setup/dispatch-1678468734-396cfa83"
    2023-03-10T12:18:54-05:00: Allocation "d6c60ffd" created: node "2c9f4b7e", group "ptc-setup"
    2023-03-10T12:18:54-05:00: Evaluation status changed: "pending" -> "complete"
==> 2023-03-10T12:18:54-05:00: Evaluation "53a77034" finished with status "complete"

Submit the employee job.

$ nomad job run pytechco-employee.nomad.hcl
Job registration successful
Approximate next launch time: 2023-03-10T17:19:18Z (3s from now)

Navigate to the Nomad UI, click on the Jobs page, and then click on the pytechco-employee job. Since this is a cron batch job, you can see that it creates a new job every three seconds.

Employee job generating new jobs

Navigate back to the webapp URL. Note how new employees come online under the Online Employees section as the pytechco-employee batch job creates new jobs.

New employees coming online

Update the application

By default, the example application randomizes the type of employee it brings online. However, you can configure it to only bring on employees of a specific type.

Update the employee jobspec to bring only sales engineers online and increase the budget by dispatching the setup job again.

Stop the employee job. The -purge flag removes the job history and any children jobs from the Nomad database, and also removes them from the Nomad UI. If they still show up in the UI, you can force Nomad to do a garbage collection to remove them from the UI with nomad system gc.

$ nomad job stop -purge pytechco-employee

Reset the database by dispatching the setup job again with a new budget of 500.

$ nomad job dispatch -meta budget="500" pytechco-setup
Dispatched Job ID = pytechco-setup/dispatch-1678469008-6ffe1c0c
Evaluation ID     = e8e420f0

==> 2023-03-10T12:23:28-05:00: Monitoring evaluation "e8e420f0"
    2023-03-10T12:23:28-05:00: Evaluation triggered by job "pytechco-setup/dispatch-1678469008-6ffe1c0c"
    2023-03-10T12:23:28-05:00: Allocation "74acf63e" created: node "2c9f4b7e", group "ptc-setup"
    2023-03-10T12:23:28-05:00: Evaluation status changed: "pending" -> "complete"
==> 2023-03-10T12:23:28-05:00: Evaluation "e8e420f0" finished with status "complete"

Open the pytechco-employee.nomad.hcl file and uncomment the args block of the task config to have the job create only employees of the type sales_engineer. Then update the cron to 0/1 * * * * * * to have the job run every second. Save the file.

job "pytechco-employee" {
  datacenters = ["dc1"]

    type = "batch"
    periodic {
        cron = "0/1 * * * * * *"
        prohibit_overlap = false
    }
  group "ptc-employee" {
    count = 1

    task "ptc-employee-task" {

        # ...

      config {
        image = "ghcr.io/hashicorp-education/learn-nomad-getting-started/ptc-employee:1.0"
        args = [
            "--employee-type", "sales_engineer"
        ]
      }
    }
  }
}

Submit the employee job again and navigate to the webapp. Note how all of the online employees are now only Sales Engineers and more of them are online at any given time.

$ nomad job run pytechco-employee.nomad.hcl
Job registration successful
Approximate next launch time: 2023-03-10T17:24:18Z (1s from now)

Only sales engineers coming online

The simulation will continue running until the amount of money reaches 0.

Next steps

In this tutorial, you submitted jobs to Nomad, updated a jobspec, and resubmitted it to Nomad. Continue on to the next tutorial by clicking on the Next button below and learn how to clean up the jobs and your cluster.

Create a cluster

Stop the cluster