Jobs & Steps

Understanding Jobs and Steps in AllSpice Actions

AllSpice Actions workflows are a powerful way to automate tasks in your repositories. They are composed of jobs and steps, which define what actions are taken and how they are organized. In this article, we'll delve into how jobs and steps work, how to sequence jobs, pass information between them, and control the flow of your workflows using logic operations.

Jobs and Steps: The Building Blocks

What are Jobs?

Jobs are units of work that execute on runners (virtual machines or containers).

• Each job consists of a series of steps.
• By default, jobs run in parallel, allowing for efficient execution.
• Jobs can be configured to run in a specific order using dependencies.

Basic Job Definition:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - run: echo "Building the project"

Job Keywords

• runs-on: Specifies the type of runner (virtual environment) to execute the job.
• needs: Defines job dependencies. This job will wait for the specified jobs to complete successfully before running.
• steps: A sequence of tasks to be executed in the job.

Job with Dependencies:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - run: make build
  test:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - run: make test

What are Steps?

Steps are individual tasks within a job. Each step can run commands, use actions, or call reusable workflows.Steps run sequentially within a job, one after another.

Example of a Workflow with Jobs and Steps:

name: Check PCBA
on: [push, pull_request]
jobs:
  Check-Schematic-job:
    runs-on: ubuntu-latest
    steps:
      - name: Check out code
        uses: actions/checkout@v4
      - name: Check Schematic
        run: check-schematic.py
  Check-PCB-Job:
    runs-on: ubuntu-latest
    steps:
      - name: Check out code
        uses: actions/checkout@v4
      - name: Check PCB
        run: check-PCB.py

Parallel Execution of Jobs

In the example above, Check-Schematic-Job and Check-PCB-Job run in parallel by default. This means they start at the same time and execute independently, as long as there are available runners.

Benefits of Parallel Jobs

• Speed: Parallel execution reduces total workflow runtime.
• Efficiency: Utilizes available resources effectively.
• Isolation: Jobs run in separate environments, preventing side effects.

Sequencing Jobs with Dependencies

Sometimes, you need jobs to run in a specific order. You can achieve this using the needs keyword, which creates dependencies between jobs.

Using the needs Keyword

The needs keyword specifies that a job depends on the successful completion of one or more other jobs.

Example: Sequencing Jobs

jobs:
  Check-Schematic-Job:
    runs-on: ubuntu-latest
    steps:
      - name: Check Schematic
        run: Check-Schematic.py
  Check-BOM-Job:
    runs-on: ubuntu-latest
    needs: Check-Schematic-Job
    steps:
      - name: Check Bill of Material (BOM)
        run: check-BOM.py
  Check-PCB:
    runs-on: ubuntu-latest
    needs: [Check-Schematic, check-BOM]
    steps:
      - name: Check-PCB
        run: check-PCB.py

In this example:

• Check-BOM-Job runs after Check-Schematic-Jobcompletes successfully.
• Check-PCB-Job runs after both Check-Schematic-Job and Check-BOM-Job have both completed successfully.

Benefits of Job Sequencing

• Control: Ensures that jobs run in the desired order.
• Reliability: Subsequent jobs depend on the success of previous jobs.
• Error Handling: If a job fails, dependent jobs are skipped, preventing unnecessary work.

Passing Data Between Jobs

Jobs run in separate environments, so they don't share state by default. To pass information or files between jobs, you can use artifacts.

Using Artifacts

Artifacts are files created during a workflow run that can be downloaded after the workflow completes. Artifacts can be shared between jobs and runs.

Steps to Use Artifacts:

1. Upload an Artifact in one job.
2. Download the Artifact in another job.

Example: Passing Files Between Jobs

jobs:
  Generate-BOM-Job:
    runs-on: ubuntu-latest
    steps:
      - name: Generate Bill of Materials (BOM)
        run: generate-BOM.py --output_file BOM.csv
      - name: Upload BOM Artifact
        uses: actions/upload-artifact@v3
        with:
          name: BOM-Artifact
          path: BOM.csv
  Generate-COGS-Job:
    runs-on: ubuntu-latest
    needs: Generate-BOM-Job
    steps:
      - name: Download BOM Artifact
        uses: actions/download-artifact@v3
        with:
          name: BOM-Artifact
      - name: Generate Cost-of-Goods-Sold (COGS)
        run: generate-COGS.py --input_BOM BOM.csv --output_file COGS.csv
      - name: Upload COGS Artifact
        uses: actions/upload-artifact@v3
        with:
          name: Cogs-Artifact
          path: COGS.csv

In this example:

• Generate-BOM-Job creates a BOM.csv file and uploads it as an artifact named BOM-Artifact.
• Generate-COGS-Job downloads the BOM-Artifact and uses it as input to a python script that generates COGS --input_BOM BOM.csv.

Control Flow and Logic Operations

Control flow in workflows allows you to execute jobs or steps conditionally based on certain criteria. This is achieved using the if keyword and expressions.

Conditional Execution of Jobs and Steps

You can use expressions to conditionally run jobs or steps.

Syntax:

if: <expression>

Using Expressions

Expressions are written using the ${{ }} syntax and can include:

• Contexts: Access information about the workflow, job, steps, etc.
• Operators: &&, ||, ==, !=, <, >, <=, >=
• Functions: success(), failure(), always(), cancelled(), contains(), etc.

Example: Conditional Step Execution

steps:
  - name: Check Netlist
    run: check-netlist.py
  - name: Notify on Failure
    if: ${{ failure() }}
    run: echo "Tests failed. Sending notification..."

Common Control Flow Examples

Running a Job Only on a Specific Branch

jobs:
  deploy:
    if: ${{ allspice.ref == 'refs/heads/main' }}
    runs-on: ubuntu-latest
    steps:
      - name: Generate Fab Files
        run: generate-fab-files.py

Skipping a Job Based on an Input

jobs:
  optional-job:
    if: ${{ inputs.run_optional_job == 'true' }}
    runs-on: ubuntu-latest
    steps:
      - name: Run Optional Task
        run: echo "Running optional task..."

Advanced Control Flow Techniques

Using continue-on-error

By default, if a step fails, the job fails. You can allow a step to fail without failing the job using continue-on-error.

Example: Allowing a Step to Fail

steps:
  - name: Run Linter
    run: npm run lint
    continue-on-error: true
  - name: Run Tests
    run: npm test

Best Practices for Jobs and Steps

• Name Your Jobs and Steps: Use the name keyword to make your workflow more readable.
• Use needs Wisely: Only create dependencies when necessary to optimize workflow execution time.
• Limit the Scope of Secrets: Pass secrets only to jobs or steps that require them.
• Clean Up After Jobs: Remove temporary files or artifacts that are no longer needed.
• Monitor Workflow Duration: Be mindful of execution time, especially in CI/CD pipelines.

Conclusion

Understanding how jobs and steps work in AllSpice Actions workflows allows you to create efficient, reliable, and maintainable automation pipelines. By leveraging parallel execution, job sequencing, data sharing through artifacts, and control flow with expressions, you can tailor your workflows to meet complex requirements.

Key Takeaways

• Jobs run in parallel by default but can be sequenced using the needs keyword.
• Artifacts are used to pass files between jobs.
• Expressions and control flow allow for conditional execution of jobs and steps.
• Best practices include naming conventions, minimal permissions, and efficient use of resources.

By applying these concepts, you can optimize your CI/CD processes and ensure that your projects build, test, and deploy smoothly.