First place to go to check how to use this and get a list of supported models is definitely on the docs. It lists only three models as of today and a single example that does not tell you much about what are all the models that support thinking budget and what are the reasoning start and end string that we can use while serving the model. With that I started digging into the source code of vLLM and got a gist of which models in vLLM supports thinking_token_budget and how to find them.

Finding all the reasoning parsers

All of the reasoning parsers supported by vLLM can be found conveniently inside of the vllm/reasoning/__init__.py file. It shows, what's the name of the parser (required while running vLLM), filename (inside of the same directory) and classname (inside the file) to load it.

---

Classes to look for

If we look into the source code, then we can see two Base Classes used by all the parsers. One is ReasoningParser and another one is BaseThinkingReasoningParser which inherits from ReasoningParser.

Looking at the code, any Reasoning class that directly inherits from ReasoningParser is an older parser for models that do not support thinking_token_budget or do not contain simple tokens for determining start and stop (take granite as an example). Also a big giveaway is their reasoning_start_str and reasoning_end_str property are null.

So if we are looking for a model that supports thinking budget, then we have to look for reasoning parser classes that inherit from BaseThinkingReasoningParser. And to be able to use it, we have to find reasoning_start_str and reasoning_end_str within that class. Some of the classes use start_token and end_token instead of the aforementioned ones, but those are the values you are looking for to pass to your --reasoning-config flag.

Putting it together

As a hard requirement vLLM specifically requires us to pass --reasoning-parser along with --reasoning-config where we need to pass reasoning_start_str and reasoning_end_str token to use thinking budget control. You can find all the reasoning tokens tokens from the parsers using this command:

rg -l 'BaseThinkingReasoningParser' vllm/reasoning/ | xargs rg -t py -A 2 'def (start_token|end_token|reasoning_start_str|reasoning_end_str)\b' 

First command: finds all Python files under vllm/reasoning/ that reference BaseThinkingReasoningParser.

Second command: prints each definition of start_token, end_token, reasoning_start_str, or reasoning_end_str (plus the 2 lines following it) from those files.

Finally

Now if we want to use thinking_token_budget for gemma4 suppose; I need to get:

• name of the parser (from vllm/reasoning/__init__.py)

• reasoning config (reasoning_start_str / start_token and reasoning_end_str / stop_token) from vllm/reasoning/gemma4_reasoning_parser.py

vllm serve google/gemma-4-31B \
    --reasoning-parser gemma4 \
    --reasoning-config '{"reasoning_start_str": "<|channel>", "reasoning_end_str": "<channel|>"}'

When the server is up and running, I can send my request with thinking_budget_token:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-4-31B",
    "messages": [
      { "role": "user", "content": "How many r's in strawberry?" }
    ],
    "thinking_token_budget": 10
  }'

Here's how it goes, all the virtual environments go inside of the central ~/.venvs directory. If I am inside of a project called project-xyz, I create a virtual environment inside of the central directory like ~/.venvs/project-xyz. I have one-to-one mapping of my project directory name with my virtual environment name and it comes with some benefits and shortcuts.

Creating Virtual Environments

I have a zsh function that helps me build a virtual environment using the python version of my choice. And yes I use uv.

This is what I have in my ~/.zshrc

vm() {
    base_name=\((basename "\)PWD")
    uv venv --python 3.\({1:-11} ~/.venvs/"\)base_name"
}

• vm: name of the function, will also automatically become a SHELL alias

  For me everything relating to virtual environment starts with v so it's "virtual environment make", aka vm

• base_name=\((basename "\)PWD"): we are taking the base name of the "present working directory (pwd)" which will be the name of the project directory.

  example:

  if pwd is "/home/yankee/projects/tldr-generator"

  then basename is tldr-generator

• uv venv --python 3.${1:-11}: create a virtual environment using uv. Set the python version that defaults to 11 and if I want to use any other version it will take the first argument passed to the alias.

  example:

  if I run vm 13, it will create a virtual environment with python version 3.13. At times I feel lazy to write the whole 3.13 so I just pass the minor version I want to use.

• ~/.venvs/"$base_name": The later part is just the location where I want to create my virtual environment.

Now with this command, I can to go to any project directory, then run:

vm

# OR
# for python 3.13
vm 13

# for python 3.12
vm 12

# and so on...

Activating Virtual Environments

Adding on to the idea of having the base_name as the virtual environment on the centralized location. Any time I am in the project directory I can easily activate the virtual environment from ~/.venvs/<base_name>/bin/activate. For that I have a function called va (virtual environment activate).

va() {
    base_name=\((basename "\)PWD")
    source ~/.venvs/"$base_name"/bin/activate
}

Now if I am in any of my project directory, I can simply run va and it will activate the correct virtual environment for the project. And if it does not exist, I can do vm followed by va.

Deactivating

To deactivate a virtual environment, one can simply write deactivate but it ‘s a long word to type and mistakes have happened. So following the pattern, I have an alias for this one called vd (more about aliases in this video)

alias vd="deactivate"

Deletion

One common action I have to do sometimes is delete a virtual environment and build a new one again. For deletion I have an alias called vrm that deletes the existing virtual environment.

vrm(){
    base_name=\((basename "\)PWD")
    venv_path="\(HOME/.venvs/\)base_name"
    rm -rf "$venv_path"
    echo "Deleted $venv_path"
}

Same idea with basename to find the virtual environment and delete it. Now with creation and activating the new venv, the whole flow looks like this:

vrm 
vm 
va
uv sync # or whatever package installer action

Putting it all together

Why do this?

• Birds-eye view of all virtual environments and deletion: most of the time there are projects with venv that takes huge amounts of storage. They are just there sitting idle and it’s hard to find them. With everything under ~/.venvs I have an birds-eye view of which project is consuming what amount of storage. And if it’s taking too much storage or I no longer need it, I can delete it from there. We can do this with a simple built-in du command or you can point your storage manager app here.

  du -sh */ | sort -hr
  

  • -sh: shows size of each directory in human-readable form

  • sort -h → sorts by size (handles KB, MB, GB correctly)

    

• Good with git worktrees, kinda*: When I am working with multiple features (mostly two at a time) for a project, I usually work on the main worktree and create a new worktree to hack on a different feature. So the way I create worktrees is similar to virtual environments, where things are kept central. The pattern I follow is:

  ~/.worktrees/<feature-name>/<project-name>
  

  Here the project-name is same as the main project directory name (basename). For va to work, we explicitly need to match the name of the project directory, hence I create a nested directory structure where feature branch is a parent directory that contains the main project directory.

  This way I can switch to my worktree, run va again, and by nature of finding virtual environment and activating based on the basename, it will automatically activate the existing virtual environment that I created from my main worktree.

  

  I have a dedicated tool created to switch between worktrees with breeze called git-worktree-switcher. And to see how you can work with git worktrees and make use of this tool to be more effective you can check my dedicated video:

https://www.youtube.com/watch?v=fRPpM5kvlls

So the `kinda*` is, sometimes the feature that I am building requires a version upgrade, addition or removal of a package. In such cases using the same virtual environment across two different feature branch creates chaos. When this happens I don’t bother creating a project directory nested inside the feature directory. I simply create a worktree that does not match the main worktree project directory name and create a new virtual environment for it.

```shell
git worktree add -b new-feature-x ~/.worktrees/new-feature-x
```

• Faaaast and error free: with all this zsh/bash function and aliases I can go about my day to day dealings with virtual environment and worktrees super fast. I have been using this workflow for couple of years now and these aliases has helped me immensely to type less and make less mistakes. You are very less likely to mistake a two letter alias compared to writing a whole command or a tab autocompletion suggestion which might have had a typo.

In this tutorial, I'll walk you through the process of creating a web-searching LLM from scratch, covering web crawling, vector databases, and semantic search. No API keys or cloud services needed!

In this video, we will be building our application that can use both local and hosted large language models (Gemini, ChatGPT, and so on) for generating code, and then we'll build our own sandbox to run that code and preview the output. This is similar to what you can experience with ChatGPT Canvas and Claude Artifact; but in our case, we have complete end-to-end control over the environment, dependencies, and the entire workflow. Go beyond generation – execute and iterate with confidence! 🚀

If your company is invested in AWS then it makes perfect sense to use AWS Code Artifact to host private packages, be it for Python, Node, Java or others.

Benefits of Code Artifact includes:

• a private PyPI instance, where pip or other tools work the same as with the public instance. Just mention the vanilla package name and that's it. No more pip installgit+ssh , git+https or ./path-to-package.

• secure infrastructure that you can fully control and define who can access what.

• packages encrypted by default with AWS KMS key.

🔎 Overview of elements in AWS Code Artifact

TLDR; you get a top-level construct called domain, under a domain you can have different repositories for same or different programming languages, each repository can store packages and their versions.

🌐 Create your domain

In AWS Code Artifact, domains are the namespace where you can host package repositories (PyPI, npm, maven). This is similar to creating domain name for website. Whatever name you put for your domain will be part of the URL that pip or other related tools will call for managing the packages.

1. Go to Code Artifact and under Artifacts select Domains.

2. 

   Then add a domain name and select Create Domain.

   

3. Apply Domain policy
   Now that we have the domain, let's set the domain policy which allows us to get temporary authorization token for accessing repositories in the domain.

   

4. Paste the following resource-based policy and save it.
   Here we are allowing a particular IAM principal, in this case a user called 'test-user' to be able to perform GetAuthorizationToken action against the resource.
   Any valid IAM Principal is allowed here.

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": [
                        "arn:aws:iam::xyz:user/test-user"
                    ]
                },
                "Action": "codeartifact:GetAuthorizationToken",
                "Resource": "*"
            }
        ]
    }
   

Note: AWS Code Artifact requires both resource-based policy and identity-based policy to work.

📦 Create your repository

You can host multiple repositories under a single domain. We'll just create one for PyPI. Each repository is an independent entity and comes with its own configurations, and resource-based policies.

1. Click on create repository

   

2. Name your repository, add description to identify its usage properly.

   

3. Setting Public upstream repositories
   When you authorize pip or related tools to fetch packages from private repository, if the requested package is not in the private repo it will fetch from the selected public repository. The requested package will then be stored on the private repository as well.

   We can select PyPI repository as upstream or leave it as blank if we intend to keep just the private packages. For me, I set the upstream as public PyPI as most of the private packages depend on other third-party packages which needs to be pulled while installing them.

4. Select Create Repository and we are done.

   

5. Similar to domain policy we need to set repository-level policy as well.

   

6. Apply the following Policy.
   Policy has two statements, "PublishPackages" for publishing packages, here the principal will likely be an IAM Role, the pipeline assumes this role and uploads the built package to the private repository. For this demo, we are providing access to an IAM user.

   The "InstallPackages" statement is a read-only policy which allows the IAM Principal to read and install packages from the private repository.

   IAM principals that create the package will most likely be different from the ones that consume it, so it makes sense to have them separate. This is also a good approach in-terms of security.

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "PublishPackages",
                "Effect": "Allow",
                "Principal": {
                    "AWS": "arn:aws:iam::xyz:user/test-user"
                },
                "Action": "codeartifact:PublishPackageVersion",
                "Resource": "*"
            },
            {
                "Sid": "InstallPackages",
                "Effect": "Allow",
                "Principal": {
                    "AWS": [
                        "arn:aws:iam::xyz:user/test-user"
                    ]
                },
                "Action": [
                    "codeartifact:DescribePackageVersion",
                    "codeartifact:DescribeRepository",
                    "codeartifact:GetPackageVersionReadme",
                    "codeartifact:GetRepositoryEndpoint",
                    "codeartifact:ListPackageVersionAssets",
                    "codeartifact:ListPackageVersionDependencies",
                    "codeartifact:ListPackageVersions",
                    "codeartifact:ListPackages",
                    "codeartifact:ReadFromRepository",
                    "codeartifact:GetRepositoryEndpoint"
                ],
                "Resource": "*"
            }
        ]
    }
   

🚀 Push your package

I will be pushing my CLI-tool called timezones-cli for this demo. You can use any Python package of your choice.

🌟Note: Make sure you have AWS CLI v2 installed on your machine.

Before we can push anything, we need to configure the policy for our user to be able to authorize to the Code Artifact domain and get the packages.

Set the following policy for the IAM user:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "codeartifact:GetAuthorizationToken",
            "Resource": "arn:aws:codeartifact:ap-south-1:xyz:domain/blog-pypi"
        },
        {
            "Effect": "Allow",
            "Action": "sts:GetServiceBearerToken",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "sts:AWSServiceName": "codeartifact.amazonaws.com"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "codeartifact:DescribePackageVersion",
                "codeartifact:DescribeRepository",
                "codeartifact:GetPackageVersionReadme",
                "codeartifact:GetRepositoryEndpoint",
                "codeartifact:ListPackageVersionAssets",
                "codeartifact:ListPackageVersionDependencies",
                "codeartifact:ListPackageVersions",
                "codeartifact:ListPackages",
                "codeartifact:ReadFromRepository"
            ],
            "Resource": "arn:aws:codeartifact:ap-south-1:xyz:repository/blog-pypi/private-pypi"
        }
    ]
}

That's it for the policy, we are all set now! Let's get back to publishing the package.

AWS Code Artifact provides you a list of tools you can authorize using AWS CLI.

Let's configure twine to upload packages

1. Inside of the repository, select on View connection instructions

   

2. Select twine and copy the aws codeartifact CLI command.
   You should have twine installed on your virtual environment or global scope.

   

3. Let's create a source distribution (.tar.gz) and a wheel distribution (.whl) for our package.
   From the root of the project run the following command.

   🌟 Note: If you are creating wheel distribution then make sure wheel package is installed on your machine.

    python setup.py bdist_wheel sdist
   

   This command will create a dist directory which will contain our package ready to be pushed.

   

4. On your terminal, paste the command copied for Code Artifact, if you need to use any particular AWS CLI profile then use the --profile flag to pass the profile name like in the example.

   

   After successful login, twine configurations will be in ~/.pypirc, if you want to logout then delete this file.

5. Let's upload our package 🎉
   We need to specify the repository we are pushing to, which is done using the -r flag.

    twine upload -r codeartifact dist/*
   

   Package is uploaded successfully! 🙌

   

   We can confirm this checking our repository.

   

⚡️ Pull your package

Now that we've pushed our package, it's time to download them.

Go to the repository page and select "View connection instructions" again. This time select the pip tool and copy the login command.

We should be out of our package's virtual environment, in a completely new virtual environment, or installing the package in the global context. I will be installing it in the global context.

Paste the command for pip login, once logged in, pip generates configuration on ~/.config/pip/pip.conf. To logout, we can delete this file or run pip config unset global.index-url.

Now every pip install command we put out will try to communicate with our private repository. We can install the package with the regular pip install:

pip install timezones-cli

We have successfully uploaded and downloaded package from our private PyPI repository hosted on AWS Code Artifact. ⚡️

🌟 Conclusion

Using AWS Code Artifact is one of many ways of installing private python packages. If you or your company is already invested in AWS then it makes perfect sense to use AWS Code Artifact. If not then you can evaluate the pros and cons for using this solution. But all AWS Code Artifact provides a complete and secure solution to get you up and running with your private package infrastructure in no time.

One of the benefits of GitHub Actions is that it allows you to write DRY (Don't Repeat Yourself) code for your pipeline, using reusable steps. Composite actions let you combine multiple run steps into a single reusable action that you can share across repos, organizations or publicly.

🧠 Mental Model for Composite Actions

🧩 As extensions

When you think of composite action you should think of it as an extension of your existing workflow. If certain steps are repeated in multiple workflows, then it makes sense to put them as a composite action.

Note that: composite action is one of the three ways you can create actions that can be shared in public.

For example, there is a process of building, tagging and uploading a Docker image to the registry for many of your projects, then you can create a composite action that can be referenced in all of your required projects.

🧬 As Functions

Now as developers, we are pretty familiar with what functions are and how they work. Just for a refresher, you wrap a piece of code as a function when it is supposed to be used in multiple places. You can pass arguments to it which it processes and produces outputs.

Similarly, we can think of Composite Actions as functions to which we can pass arguments, which process and produce certain output.

📚 How to structure your actions?

There are two ways of structuring Composite Actions:

1. Single repo single action (root action)

2. Single repo multi-action (path-based)

Single repo action

This is the most common pattern that you see in publicly shared GitHub Actions. You have a repository that houses action.yaml file on the root which defines a set of steps that can be reused.

Example of a simple hello-world composite action:

name: Greeter
description: Composite Action to greet someone

inputs:
  who-to-greet:
    required: false
    default: World
    description: who to greet

runs:
  using: composite
  steps:
    - name: greet
      run: echo "Hello ${{ inputs.who-to-greet }}"
      shell: bash

You can browse the above sample action here:

Single repo multiple actions

In this pattern, you have a single git repository that houses multiple actions that you can reuse across your organizations. This pattern is mostly popular in organizations where you house multiple composite actions in the same place which is referenced across different projects, teams or domains.

When using this pattern you have to specify exactly where your action is located so the path of the action is a key here.

An example of this pattern is as follows:

You can notice that each composite action has its directory and the workflow file is named action.yaml .

When we name the files action.yaml, we don't have to mention the file name when calling the action. Meaning we can do this:

uses: yankeexe/actions-combined/actions/ping@main

If your filename is something different then you have to mention the filename in the calling workflow as well. For example:

uses: yankeexe/actions-combined/actions/ping/ping-action.yaml@main

👻 Actions Gotchas

🔎 Is your Action Accessible?

For both single repo actions and single repo multi-actions, we have to note repository visibility. If the composite action is in a public repository then it's accessible for everyone. If the action is in a private repository then it can be accessed within the organization or users repositories by going to: settings > Actions > General and enabling the following option.

🫵 Referencing your action

You can reference composite actions in multiple ways:

1. 🪵 Branch name

   Takes the steps from the workflow file present in a branch.

    uses: yankeexe/actions-hello-world@main
   

2. #️⃣ Commit hash

   Takes the steps from the workflow file present in a particular commit.

    uses: yankeexe/actions-hello-world@a37bf8bbc2b75c505b41a0741ce589bf403ed9f9
   

3. 🏷️ Tag

   Takes the steps from the workflow file present in a particular tag.

    uses: yankeexe/actions-hello-world@0.0.1
   

🌌 Action Context

The context of the composite action is always set to the repo of the calling workflow if they have checkout action used in them. Otherwise, we can explicitly use the checkout action on the composite action as well.

To use checkout action in the calling workflow it needs the permissions for reading the content of the repository.

# calling workflow
permissions: 
    contents: read

demo_job:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: call_composite_action
        uses: demo_user/demo_actions/.github/actions/do_something@main
        ...

To use the checkout action in the composite action itself, we need the GITHUB_TOKEN passed from the calling workflow to the composite action.

# Calling workflow
- name: call_composite_action
  uses: demo_user/demo_actions/.github/actions/do_something@main
  with:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

# composite action 
inputs: 
   GITHUB_TOKEN:
       type: string
       description: Github token passed from calling workflow
       required: true

runs:
  using: composite
  steps:
    - uses: actions/checkout@v4
      shell: bash
      with:
          token: ${{ inputs.GITHUB_TOKEN }}
          repository: demo_user/demo_project
          path: main

🐚 Each step requires a shell

Whenever we are writing our composite action we'll likely forget to add the shell key for each of the steps. Shell defines where we want to execute the commands defined in the run field. Possible values for this field are:

• sh

• bash

• pwsh

• python

• nodejs

• cmd

• powershell

You can read more about the shell field here.

🔐 OIDC in reusable workflow

One of the benefits of composite actions over other reusable workflows is its ability to use OIDC to authenticate with cloud providers without having to store any credentials for it. You can use OIDC with:

• Amazon Web Services

• Azure

• Google Cloud Platform

• HashiCorp Vault

Example: If your reusable action has some step that involves using AWS resources like S3 to upload an object, then using OIDC you can authenticate with AWS, assume a role you want to use and access the services. This is not possible with workflow_run or workflow_call.

But a gotcha when using OIDC with composite action is that the calling workflow or the main workflow that calls/consumes composite action should have permission set to:

permissions: 
    id_token: write
    contents: read

# Calling workflow
demo_job:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - name: call_composite_action
      uses: demo_user/demo_actions/.github/actions/do_something@main
      with:
          role_to_assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}

Then your composite action can have steps as:

inputs:
  aws_role_to_assume:
    required: true
    description: AWS Role ARN to use

runs:
  using: composite
  steps:
    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v2
      with:
        role-to-assume: ${{ inputs.aws_role_to_assume }}

    - name: List S3 buckets
      run: aws s3 ls
      shell: bash

👋 Conclusion

As we navigate through the ever-evolving landscape of technology, tools like Composite Actions become indispensable and can be a game-changer propelling you toward more streamlined, effective, and enjoyable deployment experiences.

Thank you for reading!

If by any means, we cannot verify that the artifact built is not the artifact we are going to deploy then it’s a RED flag and we might be likely compromised. In this walkthrough, we are going to look at how we can use components of Project Sigstore: Cosign, Flucio, and Rekor - to sign and verify our artifacts.

This guide is mostly focused on Binary Large Objects (Blob) and does not cover workflows related to Containers.

Primer 🐳

We produce different types of artifacts on our build system, which can range from container images, zip files, log records, SBOMs and many more depending on the use case. These artifacts can be business essential and when tampered with will disrupt our businesses.

The process of verifying the artifacts that happen in 3 main high-level steps:

• Signing the artifact ✍️

• Storing the artifact and signature 📦

• Verifying the artifact ✅

Signing Artifacts ✍️

Sigstore offers us two ways of signing any artifact:

1. using a public-private key (asymmetric encryption)

2. using keyless mode (recommended) ✨

Using Public-Private Keys

This mechanism is something we have been using for ages. We sign our data using Private Key and verify the signature using the Public Key.

With cosign we can generate public-private key pair secured with a password.

Download cosign ✨

cosign generate-key-pair

This will generate a public key: cosign.pub and a private key cosign.key.

Now using this private key, let’s sign our artifact. For this demo, I have created a zip file called artifact.zip where I have kept an arbitrary README file. You can do the same with a zip file or any file of your preference.

cosign sign-blob --key cosign.key

# OR
# we can also store our public key and signature as a bundle using: 
cosign sign-blob --key cosign.key --bundle cosign.bundle

What just happened here?

We just finished signing our artifact but a few things have happened in the background.

• Cosign signed generated a SHA256 hash of our artifact and created a signature by signing it

• then it uploaded the public key and the signature to the transparency log Rekor and returned its index

• Rekor is a public infrastructure that logs the public key and signature of all the data signed using cosign.

Understanding the log record from Rekor 🧐

We get the index of the log entry on the Rekor after signing the artifact which is provided as: tlog entry created with index. We can view the Rekor log for our artifact using the rekor-cli.

rekor-cli get --log-index <rekor-log-index>

Let's look at this result in depth (numbers co-relate to the image above ):

1. The SHA256 sum of our artifact.zip file, we can verify this using:

    shasum -a 256 artifact.zip
   

   

2. Inside signature > content we have the same signature generated after signing the SHA of our artifact.

    rekor-cli get --log-index <log-index> --format json | jq -r .Body.HashedRekordObj.data.hash.value
   

3. Our public key that was generated when we ran cosign generate-key-pair. It is base64 encoded right now, if we decode it, we’ll get the contents of our public key aka cosign.pub

    rekor-cli get --log-index <rekor-log-id> --format json | jq -r .Body.HashedRekordObj.signature.publicKey.content | base64 -D
   

   

Verifying the artifact ✅

We’ll need the signature to verify our artifact. We can get it from the Rekor log and store it on a file called cosign.sig.

rekor-cli get --log-index <rekor-log-id> --format json | jq -r .Body.HashedRekordObj.signature.content > cosign.sig

Now let’s verify our artifact! ⚡

We will be using our public key and signature for this.

cosign verify-blob --key cosign.pub artifact.zip --signature cosign.sig  artifact.zip

# If you generated cosign.bundle when signing the artifact, you can use that bundle to verify the artifact

cosign-verify-blob --bundle cosign.bundle artifact.zip

We’ll get the message of “Verified OK”. If our public key didn’t match or the signature didn’t match, then we’ll get an error.

Congratulations, we have successfully signed an artifact and verified it 🎉.

In real world scenario, we’ll be signing the artifact and storing it somewhere, and at the time of dispatching/deployment/usage, we’ll make sure it is verified and only then make the intended use of it. This ensures that our artifact has not been tampered and it’s safe for our workflow/workload.

Keyless Mode ✨

Keyless mode is a convenient feature that hugely reduces the friction in artifact signing. When we use Public-Private keys, there’s this hassle of managing the keys, and keeping them safe. With keyless mode, we don’t have the overhead of managing any Private keys at all. It is all managed for us!

But the catch of using keyless mode is, it requires OIDC (Open ID Connect) to associate our identity to the X.509 certificate it generates; this is to attest the signer of the artifact which we can later verify. And the certificate is pushed to a public append-only log Rekor, where our identity will be available. Identity can be an email or pipeline ID which we and our team can verify is authentic and trust. For OIDC, it currently supports:

• Google

• Microsoft

• GitHub

Preparing Artifact 🔨

Let’s create a new zip from the existing zip and call it artifact-keyless.zip (or choose any file to your liking). For this demo, SHA is different for both of the zips created.

zip -r artifact-keyless.zip artifact.zip

shasum -a 256 artifact.zip | awk '{print $1}'

shasum -a 256 artifact-keyless.zip | awk '{print $1}'

Sign the artifact

To sign in the keyless mode we just have to run the following command:

cosign sign-blob artifact-keyless.zip

# OR 
# sign-blob and generate a bundle containing X.509 ceritifcate and signature. 
cosign sign-blob artifact-keyless.zip --bundle cosign.bundle

Press y on the prompt. Then a browser tab will open for the OIDC verification. This is the identity verification step for the Certificate Authority (CA) before creating our digital certificate.

We need to select one of the OIDC providers, and once the authentication is successful, we’ll see a similar message as shown below.

You can notice that it has generated an ephemeral certificate which is only valid for 10 mins.

What just happened here?

• Cosign generated a private key in memory and then a public key out of that. The private key never touches the physical storage.

• It generated the SHA of our artifact and signed it generating a signature.

• It then communicated with the Certificate Authority (CA) Flucio to bind its identity with the public key on an X.509 certificate.

• For identity proofing, we were prompted for OIDC.

• Once we verify our identity, it receives the OIDC token.

• It took the public key, signature and OIDC token to the Fulcio Certificate Authority.

• the CA created an X.509 certificate which binds our public key with our identity, along with the signature of the signed artifact.

• CA signed that certificate with its Private Key and pushed the certificate details to the Rekor Transparency log.

See a lot of things happened in the background, but when we were using it; it was frictionless! It is one of the benefits of keyless signing with cosign. 🤩

Like before, on the output we have received the index of our Rekor log. Let’s take that log index and see the details of our artifact.

rekor-cli get --log-index <your-log-index>

The output is of the same format as before. But we should notice that the size of publicKey > content is bigger than before. In the private-public key mode, we were using our public key and its size was small, but now it holds the X.509 certificate generated by Flucio.

Let’s see what our certificate looks like; the content is Base64 encoded, we should first decode it and save it to a file called artifact-keyless.crt:

rekor-cli get --log-index <rekor-log-index> --format json | jq -r .Body.HashedRekordObj.signature.publicKey.content | base64 -D > artifact-keyless.crt

Now, let’s use a tool called step to view our certificate. You can download it here.

step certificate inspect artifact-keyless.crt --format json

It will generate a huge JSON, if you search for your email address you can find it in the subject_alt_name section.

step certificate inspect artifact-keyless.crt --format json  | jq .extensions.subject_alt_name

Chain of Trust ⛓️

Now we have the X.509 certificate, how do we make sure that the Fulcio Certificate Authority has issued it?

Well, our X.509 certificate has been signed with the private key from Fulcio and we can use its public key to verify them. The Root Certificate and Intermediate Certificate for Fulcio is available here. You can download them and save them to a file called chain.crt or any name you prefer.

You can also find a copy of it here, but it's best if you download it from the official sources 😉

•     -----BEGIN CERTIFICATE-----
      MIICGjCCAaGgAwIBAgIUALnViVfnU0brJasmRkHrn/UnfaQwCgYIKoZIzj0EAwMw
      KjEVMBMGA1UEChMMc2lnc3RvcmUuZGV2MREwDwYDVQQDEwhzaWdzdG9yZTAeFw0y
      MjA0MTMyMDA2MTVaFw0zMTEwMDUxMzU2NThaMDcxFTATBgNVBAoTDHNpZ3N0b3Jl
      LmRldjEeMBwGA1UEAxMVc2lnc3RvcmUtaW50ZXJtZWRpYXRlMHYwEAYHKoZIzj0C
      AQYFK4EEACIDYgAE8RVS/ysH+NOvuDZyPIZtilgUF9NlarYpAd9HP1vBBH1U5CV7
      7LSS7s0ZiH4nE7Hv7ptS6LvvR/STk798LVgMzLlJ4HeIfF3tHSaexLcYpSASr1kS
      0N/RgBJz/9jWCiXno3sweTAOBgNVHQ8BAf8EBAMCAQYwEwYDVR0lBAwwCgYIKwYB
      BQUHAwMwEgYDVR0TAQH/BAgwBgEB/wIBADAdBgNVHQ4EFgQU39Ppz1YkEZb5qNjp
      KFWixi4YZD8wHwYDVR0jBBgwFoAUWMAeX5FFpWapesyQoZMi0CrFxfowCgYIKoZI
      zj0EAwMDZwAwZAIwPCsQK4DYiZYDPIaDi5HFKnfxXx6ASSVmERfsynYBiX2X6SJR
      nZU84/9DZdnFvvxmAjBOt6QpBlc4J/0DxvkTCqpclvziL6BCCPnjdlIB3Pu3BxsP
      mygUY7Ii2zbdCdliiow=
      -----END CERTIFICATE-----
      -----BEGIN CERTIFICATE-----
      MIIB9zCCAXygAwIBAgIUALZNAPFdxHPwjeDloDwyYChAO/4wCgYIKoZIzj0EAwMw
      KjEVMBMGA1UEChMMc2lnc3RvcmUuZGV2MREwDwYDVQQDEwhzaWdzdG9yZTAeFw0y
      MTEwMDcxMzU2NTlaFw0zMTEwMDUxMzU2NThaMCoxFTATBgNVBAoTDHNpZ3N0b3Jl
      LmRldjERMA8GA1UEAxMIc2lnc3RvcmUwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAAT7
      XeFT4rb3PQGwS4IajtLk3/OlnpgangaBclYpsYBr5i+4ynB07ceb3LP0OIOZdxex
      X69c5iVuyJRQ+Hz05yi+UF3uBWAlHpiS5sh0+H2GHE7SXrk1EC5m1Tr19L9gg92j
      YzBhMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRY
      wB5fkUWlZql6zJChkyLQKsXF+jAfBgNVHSMEGDAWgBRYwB5fkUWlZql6zJChkyLQ
      KsXF+jAKBggqhkjOPQQDAwNpADBmAjEAj1nHeXZp+13NWBNa+EDsDP8G1WWg1tCM
      WP/WHPqpaVo0jhsweNFZgSs0eE7wYI4qAjEA2WB9ot98sIkoF3vZYdd3/VtWB5b9
      TNMea7Ix/stJ5TfcLLeABLE4BNJOsQ4vnBHJ
      -----END CERTIFICATE-----
  

Let’s verify our X.509 certificate.

If we are validating the certificate within 10 mins of signing then we can use the following command:

openssl verify -CAfile chain.crt artifact-keyless.crt

If we are trying to validate the certificate, beyond 10 mins then we should use the following command, to skip checking time in order to validate the overall certificate:

openssl verify  -no_check_time -CAfile chain.crt artifact-keyless.crt

Verify the artifact ✅

Now that we have verified that the certificate is signed by the root CA. Let’s move into verifying our artifact. For this, we will also require the signature of our artifact. We can grab it from the signature > content section from the output of the rekor cli. And we can store it in a file called artifact-keyless.sig

rekor-cli get --log-index <index-number> --format json | jq -r .Body.HashedRekordObj.signature.content > artifact-keyless.sig

With that in place, let’s use the following command where we pass the certificate, signature, certificate identity and certificate OIDC issuer to verify our artifact.

cosign verify-blob artifact-keyless.zip \
 --certificate artifact-keyless.crt \
 --signature artifact-keyless.sig \
 --certificate-identity=<your-email-address> \
 --certificate-oidc-issuer=https://accounts.google.com

# OR 
# If you have generated the cosign.bundle when signing the artifact, you can you that bundle to replace certificate and signature, as the bundle contains both of these.

cosign verify-blob artifact-keyless.zip \
 --bundle cosign.bundle \
 --certificate-identity=<your-email-address> \
 --certificate-oidc-issuer=https://accounts.google.com

If we have got all the things right then we should get the “Verified OK” message, else we’ll get an error.

Conclusion

Artifact signing and verification have become one of crucial steps in securing our infrastructure against supply chain attacks. Project Sigstore has provided us with the tools and infrastructure to do this with minimal effort. If you are not signing your artifacts then it’s high time you start doing so!

One of the powerful feature it provides is the ability to re-use Actions, which is available on Marketplace and we can use it from any GH repos. Most common Actions that I see being used almost everywhere are related to:

• creating a Release

• generating Release Notes for the release

• and uploading any Artifacts for the release

Well, I was doing the same until recently. Playing around with the GitHub CLI I figured everything I mentioned above could be done with it effortlessly! ✨ Not to mention it is flexible and provides options to move things our way. ++ It's already installed on the GitHub Actions runner.

Enough talk, now let's see it in Action 🥁

Creating a Release

GitHub CLI provides a built-in sub-command to create a release. 🌟

For a basic example: we want to create a release whenever we push a tag 🏷️

steps:
  - uses: actions/checkout@v3
  - name: Create Release
    run: gh release create ${GITHUB_REF#refs/*/} -t ${GITHUB_REF#refs/*/}

• gh release create <tag> : create a new release with the latest tag pushed

• -t <tag>: Set title of the release; same as the pushed tag

Apart from general releases, we can also create draft releases and prerelease with the following flags.

• -d : draft release

• -p: prerelease

Generate Release Notes

Now this one is pretty easy. We just have to pass the --generate_notes flag to the above command and it will be generated for us.

steps:
  - uses: actions/checkout@v3
  - name: Create Release
    run: gh release create ${GITHUB_REF#refs/*/} -t ${GITHUB_REF#refs/*/} --generate-notes

If we want more flexibility then it also provides options to generate release notes from a file or from STDIN with the -F flag.

Upload artifacts with the Release

Now this gets more interesting. You won't believe me if I say it.

To upload artifacts to our release all we have to do is pass the file or directory names to the same command!

I am taking an example of my repository where I build browser extension artifacts for Chrome and Firefox:

permissions: write-all
name: Create GH Release
steps:
  - uses: actions/checkout@v3
  - name: Build Artifacts
    run: ./release.sh

  - name: Create Release
    run: gh release create ${GITHUB_REF#refs/*/} -t ${GITHUB_REF#refs/*/} 12_ft_chrome.zip 12_ft_firefox.zip --generate-notes

• permissions: write-all: For uploading artifacts, we will need to assign additional permissions to our GITHUB_TOKEN.

• ./release.sh: Builds Release Artifacts

• 12_ft_<browser>.zip: are the artifacts generated by the above script

If we want a separate step to upload our artifacts, we can do so with the following change:

permissions: write-all
name: "Create GH Release"
steps:
  - uses: actions/checkout@v3
  - name: "Build Artifacts"
    run: ./release.sh

  - name: Create Release
    run: gh release create ${GITHUB_REF#refs/*/} -t ${GITHUB_REF#refs/*/} --generate-notes

  - name: Upload Artifact to Release
    run: gh release upload  ${GITHUB_REF#refs/*/} 12_ft_chrome.zip 12_ft_firefox.zip

Conclusion

We don't have to dabble around with multiple third-party actions, all of it can be done from a single command using GitHub CLI. This is more manageable, and effortless.

Hope this has been helpful! ✨

Full reference to the above Action YAML 🚀

One of the basic tasks, we do with EC2 is SSH into it, to “do stuff”. This article revolves around the very topic of accessing the EC2 instance via SSH.

Conventional Way

A private Key is required whenever you want to establish a connection between your machine and the EC2 instance. It’s an accepted standard practice that has been used for many years to connect to a remote machine. But using a private key comes with its baggage.

What if:

• you lose your keys

• delete your keys

• someone steals your keys

• you don’t know who has access to your keys (this bothers me)

Every one of us is susceptible to the above scenarios. And it brings chaos, confusion, and vulnerability when you just want to perform your task.

Access to your keys means access to your machine. If you are distributing the keys to multiple team members or using multiple keys so that they can connect and perform some action, then it immediately becomes a problem. You are entrusting the access and security of your workload that spreads across different machines, regions and god knows where. Also, logging becomes a hassle to get visibility into what users are doing on the system.

Also, SSH-ing in itself is a tedious task:

• you have to assign an elastic IP address so you are always connecting to the same public IP address.

• add a port for SSH to the Security Group and expose it to the world or a part of it.

• enter a long command or configure ssh config

• make sure private keys are in a secure location on your machine

Example of a command for ssh-ing into an EC2 instance.

ssh -i "private_key.pem" username@<ip-address>.<region>.compute.amazonaws.com

AWS Session Manager: The modern secure way

Well, I am not the only one talking about all of the issues above. It’s faced by the development, DevOps, and security teams across different organizations. When it comes to security, AWS made sure it is providing the right set of tools, mostly an intuitive one to make all of the aforementioned issues go away.

AWS Session Manager is a one-stop service that should always be used with your EC2 instance. It immediately shows its benefits upfront:

• no need for private keys 🙌

• provide access directly to IAM users with fine-grained policies 🔐

• no need to expose SSH ports on Security Groups ✨

• no need for an elastic IP address 🎉

• centralized logging with direct access to CloudWatch, S3 🪵

• and most importantly, an intuitive command to connect to an instance 🌟

Example of a command for connecting to an EC2 instance.

aws ssm start-session --target <instance-id>

Setting up Session Manager with EC2

Setting up Session Manager with EC2 is an effortless process. There is one extra step before we launch any new instances.

Create a new Role that allows access from Session Manager to the EC2 instance.

1. Go to IAM → Roles

   

2. Select the AmazonSSMManagedInstanceCore policy

   

3. Name it anything that makes sense, I will name it as SSMInstanceCore.

Now that we have a Role, let’s create a new EC2 instance and attach it to it.

1. Click on Launch Instances

   

2. Give your instance a relatable name.

3. I will choose the default Amazon Linux 2 AMI and Instance type (it selects the free tier by default).

   

4. Finally, in the Key Pair section, choose Proceed without a key pair ⚡️

   

5. On Network Settings you can name your security group and keep it empty.

   

6. Make sure Auto-assign public IP is set to enabled inside of Network settings, for Session Manager to connect to it.

   

7. I will leave the storage as default.

8. On Advanced Settings → IAM Instance Profile: Select the Role we created in the earlier section. I named it SSMInstanceCore.

   

9. Select Launch Instance 🚀

Setting up permissions to access via Session Manager

The minimum policy configuration we’ll for an IAM user to be able to connect to an EC2 instance is as follows:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ConnectToSSMInstance",
            "Effect": "Allow",
            "Action": [
                "ssm:TerminateSession",
                "ssm:StartSession"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:instance/i-010a4c6d9cb6223e3",
                "arn:aws:ssm:ap-southeast-1:*:session/*"
            ]
        }
    ]
}

You can make it more secure by tightening up this policy. But we'll leave it like above.

Once you attach this policy to a user, they’ll be able to connect to the defined EC2 instance using Session Manager.

NOTE: Installing SSM Agent on your Instance if it's not pre-installed

Before we move further, we need to make sure our EC2 instance has the SSM agent installed. This list provides a list of AMIs that has the agent pre-installed. If you are using an instance that does not have an SSM agent, follow the docs and follow the instructions on manually installing the agent for Linux, Windows or Mac.

Accessing EC2 Instance using Session Manager

You can access EC2 Instance using Session Manager, in 3 different ways:

1. From the EC2 instance connect option

   This will redirect the user to the Session Manager console and start the session from there, but we’ll see how to connect from the Session Manager console after this.

   1. Select your EC2 instance and click on connect.

      

   2. Select Session Manager and Click on Connect

      

   3. You can start using your instance

      

2. Connect from Session Manager

   1. Go to Systems Manager and Select Session Manager on the left-hand side menu.

   2. Click on Start Session

      

   3. Select the name of the instance you want to connect to and select Start Session

      

   4. We’ll get the same terminal window, we got from the first option.

      

Connect from Terminal

Probably my favorite way of connecting to the EC2 instance with the Session Manager is via terminal. For this, you need to have AWS CLI installed on our machine and configured. After that, you need to install the Session Manager plugin for the CLI.

After that, anytime we want to connect to our instance, we can run the following command:

aws ssm start-session --target <instance-id>

aws ssm start-session --target i-010a4c6d9cb6223e3

You’ll get a similar screen as below:

Setting Up Logging

One of the benefits of Session Manager is centralized logging. You can have multiple users doing different things on the instance, and you can track all of that in a single log group in CloudWatch or you can upload it to an S3 bucket. In this blog, we'll be sending it to CloudWatch.

Before we dive into logging, we need to update our Role SSMInstanceCore and attach the CloudWatchAgentServerPolicy. This will allow the EC2 instance to send logs to the CloudWatch log group.

Setup Permission to send logs to Cloudwatch

1. Go to IAM -> Roles, Search for the role we created earlier(if you are following the blog: it is SSMInstanceCore) and select Attach policies

   

2. On the search bar, paste CloudWatchAgentServerPolicy and select it and click on Attach Policies.

   

   Create a new Log Group

   Before we can send any logs, we need to create a new log group to hold our logs. Run the command below to create a new log group:

    aws logs create-log-group --log-group-name SSMInstanceLogs
   

   

   Note: that all the session logs for any EC2 instance will be forwarded to this log group. You can't assign a log group per instance. The best thing is the logs will be JSON formatted so they'll be easy to read.

   Send Session Logs to CloudWatch

   Now that our EC2 instance has permission to write to CloudWatch, we can go to Session Manager under Systems Manager.

   1. Select the "Preferences" tab and Click on "Edit"

   2. Enable CloudWatch logging and select "Stream session logs"

      

   3. Then add the name of the CloudWatch Log group we created earlier.

   4. Our log group is not encrypted with Customer Managed Keys (CMK) so we do not enable the "Enforce encryption" option.
      But a side note, CloudWatch logs are always encrypted using AWS-managed keys.

   5. Click on Save

      Viewing Logs

      Now that we have logs enabled for Session Manager, let's jump into our EC2 instance and run some commands.

      

      Now let's had to our Log Group and check our log stream:

      

Port Forwarding

Managing Access from EC2 to private RDS

Another common use case of an EC2 instance is to use it as a bastion host (jump server) to connect to resources on a private subnet securely.

Let's say there's an RDS Postgres instance on a private subnet running on port 5432 that we'd like to connect to. RDS has no access to the internet and is accessible only by the services within that VPC.

To connect to this database service we need to:

1. create an EC2 instance in that VPC

2. go to the Security Group attached to the RDS instance and, select "Edit inbound rules" and add a new inbound rule with:

   • type as PostgreSQL

   • Source as the Security Group name of our EC2 instance.

     

3. Then Click on Save.

That's it! Now our EC2 instance has access to RDS. Now we can use our EC2 instance as a bridge to connect our local machine to the RDS instance.

Connecting to private RDS

Session manager makes it easy to forward port from our EC2 instance with the use of Documents. Documents are blueprints of action that the Session manager performs on our EC2 instances. AWS manages hundreds of these documents for common actions and they have one for Port Forwarding too.

To view what documents are available. Go to System Managers, and on the left-hand menu you can find "Documents" under "Shared Resources".

The document we are going to use is titled AWS-StartPortForwardingSessionToRemoteHost. You can search for it in the Documents section to get its details. Click on the result.

Go to the Details tab and under Parameters section you can find what parameters we can pass to this document. Parameters are passed as a key-value pair.

To use this document, we need to update the policy we created above for users:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ConnectToSSMInstance",
            "Effect": "Allow",
            "Action": [
                "ssm:TerminateSession",
                "ssm:StartSession"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:instance/i-010a4c6d9cb6223e3",
                "arn:aws:ssm:ap-southeast-1:*:session/*",
                "arn:aws:ssm:*:*:document/AWS-StartPortForwardingSessionToRemoteHost",
            ]
        }
    ]
}

Now the only thing we need to grab is the database host URL.

You can go to the RDS instance and on the "Connectivity & Security" tab find "Endpoint & port". Now we have the parameters gathered for port forwarding let's see it in action.

aws ssm start-session --target i-010a4c6d9cb6223e3\
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{"portNumber":["5432"],"localPortNumber":["5432"],"host":["testdatabase.csrc3be2awyi.ap-southeast-1.rds.amazonaws.com"]}'

I am port forwarding from RDS on a private subnet to my local machine on port 5432. Since I am using Postgres I can connect to it using the following command:

psql -U postgres -h localhost -p 5432

That's it! Now you have a connection to a private RDS instance from your EC2 instance using session manager. ✨

Conclusion

Session Manager is a powerful tool from a security and usability perspective. There are many actions you can do with it on an EC2 instance and having the power of Documents makes it more resourceful.

If you have any suggestions or queries about this article, you can reach out to me on Twitter or add a comment below. 🎉

Python is the first language many developers pick if they need to hack together something quickly. But what we scrap together is not a CLI in its entirety most of the time, you need to manage flags, parse the arguments, chain sub-commands, and many more which is a hassle, thus results in multiple small and unmanaged scripts.

In todays article we are going to put an end to this and see how we can build reasonably feature rich CLI in mere minutes without any fancy decorators or anything.

Create and activate a virtual environment

python -m venv venv

source venv/bin/activate

# Install python-fire 🔥
pip install fire

Your first sub-command

Our CLI application is going to be an aggregation of bunch of tools, so we will just call it tools CLI.

With python-fire you can use either function or class to create your subcommands. But I find working with classes more intuitive and manageable. Our first command is going to be a sub-command that shows us the UTC time.

We will create a new method utc() which will be our sub-command and we have an argument called pretty which will be the flag for our sub-command which prints UTC date time in more human readable format. This argument already has a default value so this is not a required flag.

# tools.py

from datetime import datetime
import fire

class Tools:
    def utc(self, pretty: bool = False):
        """
        Get UTC date time
        """
        utc_time = datetime.utcnow()

        if pretty:
            ## strftime format codes:
            # https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
            print(utc_time.strftime("%B %d :: %H:%M %p"))
        else:
            print(utc_time)

Next we need to run this file as a script so at the end of our file we need to add the following:

if __name__ == "__main__":
    fire.Fire(Tools)

Now we have our CLI ready! Let’s run it!!

python tools.py utc
python tools.py utc --pretty

# For help message
python tools.py

# For sub-command help message
python tools.py utc --help

It’s a bit overwhelming to type through all of these overheads python tools.py <command>; well it’s not even like a CLI I was hoping for 🤷‍♂️. You might have questions like:

• How can I invoke it from any location I want?
• I want to name it to something I find intuitive, how do I do that?

Well for that you would want to create a distribution for it.

Package for Command Line 📦

First we need to revamp our program a little to accommodate packaging:

# REMOVE THIS CHUNK
if __name__ == "__main__":
    fire.Fire(Tools)

# ADD THIS CHUNK
def run():
    fire.Fire(Tools)

Now let’s create a setup.py file to manage our packaging/distribution. You can use this file as a reference to create your own CLI:

# setup.py
"""Package setup"""
import setuptools

# Development Requirements
requirements_dev = ["pytest", "black", "mypy", "flake8", "isort"]

setuptools.setup(
    name="tools_cli",
    version="0.0.1",
    author="Yankee Maharjan",
    url="https://yankee.dev/build-cli-blazingly-fast-with-python-fire",
    description="Collection of handy tools using CLI",
    license="MIT",
    packages=setuptools.find_packages(exclude=["dist", "build", "*.egg-info", "tests"]),
    install_requires=["fire"],
    extras_require={"dev": requirements_dev},
    entry_points={"console_scripts": ["to = tools:run"]},
)

The line you need to focus here is the entry_points, which describes the entry point to our program as a console script.

entry_points={"console_scripts": ["to = tools:run"]},

Here to is the name of our CLI, you can name it to anything you like. If you want to name it to brr it will go like this:

entry_points={"console_scripts": ["brr = tools:run"]},

tools:run represent the name of our module followed by the function it needs to run. Console Scripts always require a function to run hence the modification we did earlier.

Feels like a CLI 💆‍♂️

Now let’s install our CLI in an editable mode within our virtual environment. This is like hot reloading for your CLI, whatever changes you make is reflected instantly.

Inside your project directory run the following command.

pip install -e .

Now you can use your CLI using the command to or whatever you put on the console_scripts

to utc
to utc --pretty
to utc --help

This is pretty neat!

Now how do I make sure I can run it from any location I want?

• Deactivate your virtual environment:

    deactivate
  

• Install the project in editable mode again on your global site-packages:

    pip install -e .
  

Now this is done, you will have your CLI accessible throughout the system. But note that if you make any changes to the main CLI logic, it will be reflected instantly.

Bonus: Nested Commands ➿

If you have made it this far, then you are set to make your own CLI and get done with most of the use cases. But if you want to see some more then stick around for a bit.

Let’s add other commands to our tool, first a sub-command called leap() that validates if the given year is leap or not, lastly a sub-command called pw() to generate a strong password.

...
import calendar
import string
import secrets

class Tools:
    def utc(self, ...):
        ...

    def leap(self, year:int): # required: since no default value here
        """
        Check if given year is leap or not
        """
        print(calendar.isleap(year))

    def pw(self, len: int = 16):
        """
        Generate strong password
        """
        alphabet = string.ascii_letters + string.digits + string.punctuation
        pwd_length = len

        pwd = ""
        for i in range(pwd_length):
            pwd += "".join(secrets.choice(alphabet))

        print(pwd)

def run():
    fire.Fire(Tools)

Now run the commands

to leap 2022
to pw
to pw --len 22
to pw 25

Sometimes there are related commands that you want to group together, like in our case we can group utc and leap under something like datetime or dt for short. Basically what we want to do here is nested commands.

Let’s group our commands. We will move our leap() and utc() method inside of a new class called DateTime.

...

class DateTime:
    def utc(self, pretty: bool = False):
        """
        Get UTC time
        """

        from datetime import datetime

        utc_time = datetime.utcnow()

        if pretty:
            print(utc_time.strftime("%B %d :: %H:%M %p"))
        else:
            print(utc_time)

    def leap(self, year:int): # required: since no default value here
        """
        Check if given year is leap or not
        """
        print(calendar.isleap(year))

class Tools:
    def __init__(self):
        self.dt = DateTime()

    def pw(self, ...):
        ...

Sub-command is determined by whatever we put as variable name when instantiating the new DateTime class. Here we have named it as dt, but you can name it as datetime, dtt or whatever you want.

Now, we have more organized sub-commands for our CLI. If you want to run commands that are related to date time you can do so using to dt <command-name> ; for example:

to dt utc
to dt leap 2025

Password command will be normal:

to pw
to pw --len 30

Conclusion

Using python-fire makes the process of creating CLIs really easy and intuitive because you are using nothing but Python functions and classes. I hope this mini rundown of the tool and how to package it for your daily use has been helpful.

Last Updated: Oct 20, 2023

Overwriting Entrypoint

Whenever you use the docker run command the first process that runs inside of your container is determined by the ENTRYPOINT command.

The most common thing you want to do with a container is to attach a pseudo-terminal to it. We do it using:

docker run -it <image-name>  <shell-name> 
docker run -it ubuntu:latest bash

Now if you are using some images like the official AWS python image, you can’t actually do it by default.

docker run -it amazon/aws-lambda-python sh

You get some generic info related to lambdas like your current working directory and handler but no sign of a terminal 🤔. If you dig into the image layers we can see that at the end of the Dockerfile there’s an ENTRYPOINT command that runs a shell script.

Docker is pretty flexible in how we want to make it work. So we can overwrite the default ENTRYPOINT which can be any valid Linux command and attach a pseudo-terminal.

docker run --rm -it --entrypoint sh amazon/aws-lambda-python

Create Images from running containers

Docker offers the ability to export a new Image from a running container. This can be handy when you quickly need whatever you're running in a container as an image that can be distributed for debugging or playing around. Note that, this will not export data from any volume attached to the container.

Example :

Run a base Ubuntu container:

docker run --rm -it --name ubuntu-base ubuntu:latest bash

Install a few tools that you always require, perhaps Vim, cURL, iputils-ping, to name a few.

Inside the container:

apt update && apt install -y vim curl iputils-ping

Now on your host terminal, you can commit the contents of the running container and create a new image:

docker commit ubuntu-base ubuntu:tools

Now if we run the image ubuntu:tools, we'll have the tools like vim, cURL already installed to work with.

docker run --rm -it ubuntu:tools bash

With the --change or -c flag we can also overwrite Dockerfile instructions.

Let's say we want to overwrite ENTRYPOINT and CMD of ubuntu:tools and create a new image that pings a certain URL.

# Create a container 
 docker create --name ubuntu-tools ubuntu:tools

# Commit a new image overwriting ENTRYPOINT and CMD
 docker commit -c 'ENTRYPOINT ["ping"]' -c 'CMD ["google.com"]' ubuntu-tools ubuntu:ping

Now if we run our new image, it'll be running ping command by default.

The best part about it is, that you won't require a Dockerfile to do any of this. But keep note that, this is intended to be used for quick debugging and not for production usage.

One-off containers

Sometimes we just want to run the container for a single purpose, and don’t need it in our system. These can be cases where we want to attach a terminal inside an image and check for contents or debug something, run a container for testing purposes, and all the other things that might float your boat.

For that kind of need, docker ships with the --rm command, when you run your container with it, it immediately removes the container instead of putting it in a stop state.

By default, when you stop a container, it is in a dormant state but occupies space on your hard drive. You can use -a flag to view all your stopped containers.

docker ps -a

To remove them from your system you’d have to use:

docker rm <container-id>/<container-name>

For example: I use these one-off containers to test out the new features from the latest release of Python by opening a Python Shell from the container.

docker run --rm -it python:3.11.0b4-alpine3.16 python

Command Substitution

Docker has some filters and flags that you can use to perform an iterative task.

Common tasks like creating new versions of the image with the same name and tag can result in dangling images. The command below shows us a list of all the dangling images in the system eating our disk space.

docker images --filter dangling=true

Now we want to remove all of them to clear up some storage! It’s not productive to remove them one by one using

docker rmi <image-name or id>

With -q flag we can list only the ids of the images and pass its output to the remove command using command substitution (supported by most SHELL)

docker rmi $(docker images -q --filter dangling=true)

NOTE: If your SHELL supports it, you can replace $(...) with back-ticks:

docker rmi `docker images -q --filter dangling=true`

Software Bill of Materials (SBOM)

If you are shipping images that are used by multiple teams or are public for use then it is essential that you publish the Software Bill of Materials (SBOM) along with your images. Software Bill of Materials provides a transparent overview of the packages that make up your final image. These can be your OS system packages, or language-specific packages, along with their version. It makes it easy for teams and end-users to identify packages on our image that are vulnerable with simple search or vulnerability scanners like grype.

There are two ways you can go about generating SBOMs for your Docker image:

1. Docker introduced native support for generating SBOM in Docker Desktop 4.7.0. You can also install it as a plugin.

    curl -sSfL https://raw.githubusercontent.com/docker/sbom-cli-plugin/main/install.sh | sh -s --
   

   Note: that the plugin is in the Experimental stage and things are subject to change rapidly.

2. You can use the syft command-line tool that Docker uses internally.

Generating SBOM

You can generate SBOM in two widely used standard formats SPDX and CycloneDX or take the plain JSON route if you aren’t sure of the mentioned formats. Plain JSON has a dump of all the information syft can find in your image.

# Docker CLI
docker sbom <image-name> -o <image-name.sbom.json>
docker sbom ubuntu:latest -o ubuntu.sbom.json

# Syft CLI
syft <image-name> -o json=<image-name.sbom.json>
syft ubuntu:latest -o json=ubuntu.sbom.json

Vulnerability Scanning

Once you have the SBOM, the next step is to scan for vulnerabilities. One of the profound tools for this is grype. You can pass the SBOM JSON generated into grype to list the vulnerabilities. It will include the installed version, fixed version, and severity of the packages scanned.

grype ubuntu.sbom.json

If you want to go further you can add an SBOM to an image as an attestation using Syft and Cosign.

There are multiple tools to scan for vulnerabilities in Docker images with/without the need for SBOM, you can find these tools listed in this article by geekflare.

Always be scanning

With the introduction of Docker extensions on Docker Desktop scanning for vulnerability in your images has never been easier.

Open your Docker Desktop and go to extensions:

From there, you can download Snyk and Trivy. You can download their CLI's if you are not using Docker Desktop.

Go to one of its dashboards, select the image to scan and you’ll be provided with a list of vulnerabilities in your image.

Cross-platform Images

When you want to distribute images among your team; you have to make sure that it runs on all of the machines. That’s why we are using Docker in the first place, right?

Although Docker makes it quite easy to package your application and move around in any environment; we do have to make sure that the underlying CPU architecture supports the image we have packaged. Images are tied to the CPU architecture where it is built and it specified on the OCI Image Index properties. To view this, you can inspect your any image on your machine:

# Search for Architecture
docker image inspect busybox:latest | less

To view all the available CPU architecture for a particular image, you can run the following command:

docker buildx imagetools inspect busybox:latest

Best practice is to build these multi-platform images on your CI pipelines; for this tutorial, we will be doing it on our local machine. After we build the multi-arch images, we need to push them to the Docker Hub since the Docker engine registry only supports single CPU architecture. For this you need to authenticate your Docker CLI for Docker Hub before you proceed to the next section.

**Create a new builder: **

docker buildx create --name multi --use

Create a new test Dockerfile

FROM alpine:latest
CMD echo “Running on $(uname -a)”

Build and push to Docker Hub

To create multi-arch images we use the --platforms flag and specify which architecture we want to build against.

docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t yankexe/host-info --push .

View multi-arch images in Docker Hub

In DockerHub, we have images that can be used against multiple platforms. Now, if anyone pulls our image from the Hub, Docker CLI will fetch the one specific to their CPU architecture.

Connecting to the host machine

Publishing port using -p or —port flags to make services available outside of Docker (host machine) is common in everyone's development workflow. But what if you want to connect to a service running on your host machine from inside of the container?

For this, Docker has a built-in DNS name: host.docker.internal which resolves to the host machine IP address. Note that, this is only available on Docker for Mac and Windows.

If you have a database instance running on your host machine at port :5432 then you can access it from the application inside your container using host.docker.internal:5432

Demo

Create a simple flask server: app.py on your host machine.

from flask import Flask

app = Flask(__name__)

@app.route("/")
def index():
    return "Hello world"

Run the server:

flask --app app --debug run

Get inside an one-off container:

docker run --rm -it ubuntu bash

Make a request from inside of the container to the host machine.

apt update; apt install curl -y
curl host.docker.internal:5000

Thank you for reading. If you have any suggestions or feedback feel free to reach out to me on Twitter.

There are multiple tools for running Kubernetes on your local machine, but it basically boils down to two approaches on how it is done:

1. running it from a single binary package
2. running it as a container using Docker in Docker (DinD)

Kubernetes Marketplace

Before we move on to talk about all the tools, it will be beneficial if you installed arkade on your machine. It will help you get these tools with a single command.

curl -sLS https://get.arkade.dev | sudo sh

Features?

All of the tools listed here more or less offer the same feature, including but not limited to:

1. Multi-Node cluster
2. Persistent volumes
3. Networking
4. Certificates
5. Bare-metal support
6. Dashboard
7. Kubernetes Versions
8. Add-ons
9. Cross-platform
10. Tracks upstream Kubernetes

Single package binary

• k3s

  k3s is a lightweight Kubernetes distribution from Rancher Labs. It is specifically targeted for running on IoT and Edge devices, meaning it is a perfect candidate for your Raspberry Pi or a virtual machine.

  It comes with a single binary of mere <40 MB and takes as low as 500 MB of RAM.

  You can bootstrap k3s quickly using k3sup !

  https://yankee.dev/multi-node-kubernetes-k3s-locally

  arkade get k3sup
  

• k0s k0s is the latest entry on the block. By name, you might think it's a more stripped version of k3s, but it's an entirely different distribution from an entirely different company, called Mirantis. Contrary to the name, it comes in a larger binary of 150 MB+.

  It can be run as a binary or in DinD mode. k0s takes security seriously and out of the box, it meets the FIPS compliance. Although, a new distribution, k0s has reached the production-ready status, so there wouldn't be an issue for development usage.

  arkade get k0s
  

• Microk8s MicroK8s is a Kubernetes distribution by Canonical, the company behind Ubuntu. You already saw this coming; it can only be installed using snap. It comes with loads of add-ons baked in like Fluentd, Grafana and Prometheus.

  If you are on Ubuntu or its derivatives that uses snap you'll feel right at home using MicroK8s.

  sudo snap install microk8s --classic
  

Docker in Docker (DinD)

Running Docker inside of Docker (Inception anyone?) is a popular way of bootstrapping Kubernetes. The isolating nature of Docker makes running a multi-node cluster a breeze on a single machine, and also ensures the running instance does not affect the machine itself.

As the title suggests, you need to have Docker installed on your machine to go this route.

• minikube Despite running on top of Docker and similar container technologies, minikube is really flexible in how it operates and supports multiple virtualization drivers making it adaptable to different computing environments. These include KVM2, Virtualbox, Podman, Hyperkit, Hyper-V and many more.

  arkade get minikube
  

• KinD Kubernetes in Docker (KinD) is similar to minikube but it does not spawn VM's to run clusters and works only with Docker. KinD for the most part has the least bells and whistles and offers an intuitive developer experience in getting started with Kubernetes in no time.

  arkade get kind
  

• k3d k3d is basically running k3s inside of Docker. It provides an instant benefit over using k3s on a local machine, that is, multi-node clusters. Running inside Docker, we can easily spawn multiple instances of our k3s Nodes.

  arkade get k3d
  

Conclusion

Either you choose a single binary package or DinD approach, Kubernetes has made itself pretty accessible. For new learners, the barrier to entry is low, and the feedback loop is instantaneous.

I hope this article has been helpful in deciding which tool to use for running your local Kubernetes instance.

What is Shell's history?

Think of it like your browser history, where all the sites you have visited are stored, but when you don't want to keep that history, you switch to Incognito Mode. Like the browser, we can switch to Incognito Mode when working with secrets and other sensitive data that we don't want to be leaked on the history.

How many times have you done this to test out something, and the next thing you know, this is logged in your history.

export AWS_PASSWORD="xyz#31"

All of our histories are saved on a file; depending on your shell environment, HISTFILE can be in a certain location with a certain name.

echo $HISTFILE

# View the contents of HISTFILE
cat ${HISTFILE}

And how much history is saved is determined by the HISTSIZE environment variable.

echo $HISTSIZE

You can change the HISTSIZE to let's say store only the last 100 commands you have written.

export HISTSIZE=100

Incognito Mode on Bash shell

To use the Bash shell on Incognito mode all we have to do is unset the HISTFILE.

unset HISTFILE

This will unset the history file for the current terminal session, meaning, if you restart the terminal session or open a new instance of the Bash shell, you won't find any history.

To see that in action, write some random unique commands you can identify on the terminal, and restart the terminal.

# Do random things 
# ... 

# Refresh bash session
exec bash

# Check history
history

Incognito mode on ZSH

ZSH has one of the easiest ways of disabling history, all you have to do is add a space in front of the command and that won't be listed on the HISTFILE.

# space in front of the command 
 echo "Goodbye History"

And just like that, you won't have any history. If this is not working out for you, go to the zsh configuration file, usually .zshrc, and follow the instructions below.

# open the config file and write
setopt HIST_IGNORE_SPACE
# close and save the config file

# execute the contents of config file 
source <path-to-zsh_config_file>

Incognito mode on Fish Shell

Fish shell has a built-in private mode that can be enabled with ease using:

fish --private

Once you are done with dealing with sensitive data, you can just exit from the private mode and there will be no single trace of history.

# exit from private mode
exit

# check history
history

Conclusion

Hope this has been a helpful guide on working with sensitive data on your terminal. If you want to read more articles like this make sure to subscribe to my newsletter and YouTube channel.

Problem Statement

Imagine this; you are working on a feature where you have made bunch of changes to files that are not yet commited, and suddenly you need to work on a hot fix or a more priority feature. There are two ways you can tackle this:

1. Using Git Stash workflow
2. Using Git Worktree (You are here! 📍)

Git worktree to the rescue 🌳

Git worktree helps you manage multiple working trees attached to the same repository.

In short, you can check out multiple branches at the same time by maintaining multiple clones of the same repository.

OK back to our problem! Update changes? New Feature? Hot Fix? Whatever it is, you need to change to a different branch and work on it without any changes to your current work directory.

Let’s say it’s a new feature, your workflow would look like this:

1. create an replica of your project and switch to a new branch
2. create a new feature
3. push it
4. back to previous working directory

Create worktree

Let’s say the name of your feature is feature-x and you want the branch with the same name. You can create additional worktree on the same directory or move it to a desired path, I prefer the later.

git worktree add command creates a worktree along with a branch that is named after the final word in your path.

git worktree add <PATH>

# Create feature-x directory and branch with the same name.
git worktree add ../feature-x

Named Branch

If you want to give you branch a unique name then you can use the -b flag with the add command.

git worktree add -b feature-xyz ../feature-xyz

Track remote branch

Let’s say you want to switch to a new branch that is tracking the branch at remote, where you want to push changes to.

git worktree add -b <branch-name> <PATH> <remote>/<branch-name>

git worktree add -b feature-zzz ../feature-x origin/feature-zzz

View the list of worktrees with git worktree list

Switching Worktrees

As much easy it is to create a worktree, it is equally difficult to navigate back and forth between them if they are spread across. You have to git worktree list and then copy the path navigate to the worktree of your choice. To minimize this friction, I have built a small tool that let’s you switch between worktrees just with their partial or complete directory name.

Download wt CLI tool for faster switching between worktrees.

With this I can simply switch between my worktrees. You can wt list which is equivalent to git worktree list to see the list of your worktrees. Now to move to feature-x worktree directory, I can just use wt feature-x to cd into that directory to continue with the work. To go back to my main worktree directory I can just wt -.

Remove Worktrees

Now that you have created a new worktree, switched to it and made your changes and pushed it. To remove the worktree, we can run:

git worktree remove <name-of-worktree>

git worktree remove feature-x

Conclusion

Git worktree is a handy feature that let's you context switch in your project to try out things on a completely different environment, without modifying your main work directory. This might come handy occasionally but it's pretty neat being able to do so without breaking a sweat.

I hope this guide has been helpful, if you have any queries or corrections, feel free to reach out to me.

With a set of rules in Makefile, you can get up and running in no time, keeping the process sane and saving time and effort for everyone in the team. We'll be going through the basics to some interesting stuffs we can do with Makefile.

There are two pieces to this equation, one is the make CLI tool and the next is the Makefile . The basics is make reads the rules from the Makefile and executes them. What I will be showing today is just a small part of what make is capable of.

Writing Makefile

If you have worked with YAML files before then you will feel right at home writing Makefiles.

Anatomy of Rules

Every Makefile consists of rules with the anatomy of:

target: dependencies
    recipe

• target: target can be an executable, object or just a name for an action that we want to carry out. We will be using targets purely with the placeholder name for the rule. Be mindful about the name, as it should resonate with the action we want to perform with no confusion whatsoever.
• dependencies: Dependencies are the rules that needs to be executed, in order for the current rule to work.
• recipe: recipe is the meat of the Makefile, it is the action that we want to perform with our target name. Make sure to put a tab character at the start of every recipe line (just like YAML). You can also replace the tab character with anything you want using the .RECIPEPREFIX variable.

Next we will be looking into some examples on how to make use of Makefile. These examples will be based on setting up development environments.

Basic Rules

A basic rule where you just want to put some alias is straight forward.

Let’s say you have a python project and you want to hand it over to a new team member. How do you streamline the setup process. Maybe it can look something like this.

Note: # is for comments.

@ symbol is to disable printing the recipe to stdout. Test without the @ symbol at the beginning of the recipe.

:= is the expansion operator which prevents using subsequent value with the same variable name.

SHELL variable determines the default shell to execute the recipe.

SHELL :=/bin/bash

.PHONY: format check

venv: # setup a virtual environment
    @python3 -m venv venv

setup: # install dev dependencies
    @pip install -e .[dev]
    @echo -e "\nInstalling pre-commit hook..."
    @pre-commit install

format: # format code using black
    @black .

check: # check for formatting using black
    @black --check --diff -v .

test: # run pytest
    @pytest -vvv

You can do something similar with your existing project.

Now to get up and running, all you have to do is:

$ make venv 

$ . venv/bin/activate 

$ make setup

$ make format 

# and so on

Rules with Dependencies

Taking the reference from the example above, suppose we want to print out the output of check target every time we run the format target. So how do we create that dependency? It’s plain simple, we just have to update the format target to look something like this:

format: check # run the formatter on files.
 @black .

We have added the dependency of check to the right of the target, just like showcased on the anatomy Anatomy of Rules section.

Variables

We can also define variables if we have some piece of command for repeated use. For this example we will be taking the reference of the Django management command.

Variables are normally written with all caps and uses := to assign variable name to a value. Variables can be accessed using either $() or ${} syntax.

DJANGO_MANAGE := python manage.py
run: 
 @${DJANGO_MANAGE} runserver

show: 
 @${DJANGO_MANAGE} showmigrations

migrate: 
 @${DJANGO_MANAGE} migrate

Also your SHELL environment variables are converted in to Makefile environment variables, so you can directly make use of them while creating your rules.

Example:

In our shell we can export an environment variable called INFO.

$ export INFO="Run make help to show all the available rules."

And now in the Makefile we can refer to it as any variable.

info: # show project info
    @echo ${INFO}

Default target

If you just run make on your command line and it will run the first target on your Makefile. But we can change that by using the .DEFAULTGOAL special variable and assigning the target we want to run by default.

.DEFAULT_GOAL := run

Now, next time you run make it is going to run the Django server by default.

Self documenting

Now we have bunch of targets on our Makefile and we also called this combo as a custom mini CLI app. Wouldn’t it be great, if we could have a help command similar to a real CLI app? Say no more, thanks to the blog from Victoria Drake we have the script to do so.

Just create a help target and assign it as a .DEFAULT_GOAL. With this, all the comments we have been writing on our target gets converted into a nice help message.

.DEFAULT_GOAL := help 
help: # Show this help
 @egrep -h '\s#\s' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?# "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

Include other Makefiles

We can separate out Makefiles based on the tasks they perform and include them into the main Makefile. We usually have separate Makefile managed for environment variables, Docker and Kubernetes. This offloads all the tasks from project set up to Deployment to the Makefile.

I will show a brief example of each of the file just to give an example:

Note: Since make runs each recipe on a new instance of the shell, we can lazy evaluate the variables using ?= meaning, they are initialized only when referenced for a single shell instance.

Makefile Root makefile composed of other Makefiles.

SHELL :=/bin/bash
APP_ROOT := $(PWD)
TMP_PATH := $(APP_ROOT)/.tmp
VENV_PATH := $(APP_ROOT)/.venv

export ENVIRONMENT_OVERRIDE_PATH ?= $(APP_ROOT)/env/Makefile.override

-include $(ENVIRONMENT_OVERRIDE_PATH)
include $(APP_ROOT)/targets/Makefile.docker
include $(APP_ROOT)/targets/Makefile.k8s

Environment Variables Makefile.override Makefile containing just the essential environment variables.

STAGE ?= <stage>
SERVICE_NAME ?= <service-name>
AKS_RESOURCE_GROUP ?= <resource-group>
AKS_CLUSTER_NAME ?= <cluster-name>
REGISTRY_URL ?= <registry-url>
AZ_ACR_REPO_NAME ?= <repo-name>

Docker Makefile.docker Makefile containing docker rules.

export GIT_COMMIT ?= $(shell cut -c-8 <<< `git rev-parse HEAD`)
export BRANCH ?= $(shell git rev-parse --abbrev-ref HEAD)

export DOCKER_BUILD_FLAGS ?= --no-cache
export DOCKER_BUILD_PATH ?= $(APP_ROOT)
export DOCKER_FILE ?= $(APP_ROOT)/Dockerfile

export TARGET_IMAGE ?= $(REGISTRY_URL)/$(AZ_ACR_REPO_NAME)/$(SERVICE_NAME)
export TARGET_IMAGE_LATEST ?= $(TARGET_IMAGE):$(BRANCH)-$(GIT_COMMIT)

acr-docker-login:
    az acr login --name $(AZ_ACR_REPO_NAME)

docker-build:
    docker build $(DOCKER_BUILD_FLAGS) -t $(SERVICE_NAME) -f $(DOCKER_FILE) $(DOCKER_BUILD_PATH)

docker-tag:
    docker tag $(SERVICE_NAME) $(TARGET_IMAGE_LATEST)

docker-push: acr-docker-login
    docker push $(TARGET_IMAGE_LATEST)

Kubernetes

Makefile.k8s Makefile containing rules for Kubernetes.

export OVERLAY_PATH ?= $(APP_ROOT)/k8s/overlays/$(STAGE)/

define kustomize-image-edit
    cd $(OVERLAY_PATH) && kustomize edit set image api=$(1) && \
    cd $(APP_ROOT)
endef

kubectl-apply:
    kustomize build $(OVERLAY_PATH)
    kustomize build $(OVERLAY_PATH) | kubectl apply -f -

update-kubeconfig:
    az aks get-credentials --resource-group $(AKS_RESOURCE_GROUP) --name $(AKS_CLUSTER_NAME)

aks-deploy: update-kubeconfig
    $(call kustomize-image-edit,$(TARGET_IMAGE_LATEST))
    make kubectl-apply

aks-delete: update-kubeconfig
    kubectl delete namespace $(STAGE)-api

kustomize-edit:
    $(call kustomize-image-edit,$(TARGET_IMAGE_LATEST))

Now we have orchestrated all these Makefiles, it is easier to keep track of all the rules and makes working with Makefiles sane, if you are doing a lot with it.

Conclusion

So with the use of Makefile we can streamline a lot of redundant tasks in our projects without having to remember overwhelmingly long and varying commands.

It increases the productivity of the whole team; with easier project setup and redundant tasks outsourced to the Makefile with intuitive target names, leaving the devs to focus on more serious tasks at hand.

Primer

Deployments

Deployment is one of the mechanisms for handling workloads (applications) in Kubernetes. It is managed by Kubernetes Deployment Controller.

In Kubernetes, controllers are control loops that watch the state of your cluster, then make or request changes where needed. Each controller tries to move the current cluster state closer to the desired state.

In case of deployment here, the desired state we want to achieve is for the pods. Everything is declarative in K8s, so the desired state is written as a spec in Deployment manifest file.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80

If any of our pod instances should fail or update(a status change), the Kubernetes system responds to the difference between manifest spec and status by making a correction, i.e. matching the state of Deployment as defined on the spec.

Deployment under the hood

Deployment is an abstraction over ReplicaSet. Under the hood, Deployment creates a ReplicaSet which in turn creates pods on our cluster. As per the name, ReplicaSet is used for managing the replicas of our pods.

In summary, Controller reads the Deployment spec, forwards the pod configuration to ReplicaSet and then it creates the pods with proper replicas.

Deployment > ReplicaSet > Pods

Rolling Deployment

Kubernetes promises zero down time and one of the reasons behind it is Rolling Deployments. With Rolling Deployments, Kubernetes makes sure that the traffic to the pods are not interrupted when updated pods are being deployed.

Rollback Deployment

Rollback Deployment means, going back to the previous instance of the deployment if there is some issue with the current deployment.

Hands on 🙌

Let’s get our hands-on with Kubernetes and see how these deployment strategies are carried out.

Setup KinD

We will setup a single node Kubernetes cluster on our local machine using KinD (Kubernetes in Docker). Make sure you have Docker running.

Install KinD

We will create a K8s cluster using:

$ kind create cluster

With our cluster ready we are ready for Deployments.

For the purpose of this tutorial, we won’t be touching any YAML files and will be fully utilizing the power of kubectl CLI tool.

Create Deployment

Let’s create our first Deployment using the nginx image.

$ kubectl create deployment test-nginx --image=nginx:1.18-alpine

Like mentioned earlier, a Deployment creates a ReplicaSet followed by Pods. You can check these newly created resources using:

$ kubectl get deploy,rs,po -l app=test-nginx

List newly created Deployment, ReplicaSet and Pod

You can check to make sure the Replicaset was created by our Deployment using:

$ kubectl describe rs <replica-set-name>

Check ReplicaSet created by Deployment

You can do the same with Pods:

$ kubectl describe po <pod-name>

Scale Deployment

Let’s scale the deployment to have 3 instances of nginx pods.

$ kubectl scale deploy test-nginx --replicas=3

Creating 3 replicas of nginx pods

Now that we have significant number of pods on our cluster. Let’s try out the Deployment strategies.

Hands-on: Rolling Update Deployment 🍥⏩

Let’s say you were having some issues with v18 of nginx and the v19 fixes it for you. You need to rollout a new update of nginx image to your pod.

By updating the image of the current pods (state change), Kubernetes will rollout a new Deployment.

$ kubectl set image deploy test-nginx nginx=nginx:1.19-alpine

After we set the new image, we can see the old pods getting terminated and new pods getting created.

Rolling update, new pod replacing old ones

We can see Kubernetes at work, making sure the pods are maintained properly. The last of the old pod doesn’t get terminated until the complete replicas for the new pods are created. The old pods also have a grace period which makes sure the traffic it is serving isn’t disconnected for certain time until the requests can be safely routed to the newly created pods.

We successfully updated all our pods to use nginx v19.

$ kubectl describe deploy test-nginx

Pods updated to nginx v19

Hands-on: Rollback Deployment 🍥⏪

Let’s assume the new nginx update has even more problems than the last one and now you realized how blissful life was with the old version. Time for a rollback to the previous version of nginx.

But how do we do that? You might have noticed that there are now two ReplicaSets. It’s due to the same Deployment pattern we discussed earlier, we update our Deployment, it creates a new ReplicaSet which creates new Pods. Kubernetes holds history of up to 10 ReplicaSet by default, we can update that figure by using revisionHistoryLimit on our Deployment spec.

These history are tracked as rollouts. Only the latest rollout is active.

By now, we have made two changes to our Deployment test-nginx so the rollout history should be two.

$ kubectl get rs 

$ kubectl rollout history deploy test-nginx

$ kubectl rollout history deploy test-nginx --revision=1

Rollout history for test-nginx Deployment

Alright let’s get to rolling back our update to the previous rollout. We want to rollback to the stage where we were using nginx v18 which is rollout Revision 1.

$ kubectl rollout undo deploy test-nginx --to-revision=1

Rollout undo pod states

Like the Rolling Update Deployment, the Rollback Deployment terminates the current pods and replaces them with the pods containing the spec from Revision 1.

If you check the rollout history once again, you can see that the Revision 1 has been used to create the latest pods tagging it with Revision 3. There is no point in maintaining the same spec repeated for multiple revisions, so Kubernetes removes the Revision 1 since we have the latest Revision 3 of the same spec.

Revision history after rollout undo

Now we are back on the nginx v18. with the Rollback Deployment. You can check that out by:

$ kubectl describe deploy test-nginx

Check rollback nginx version

Conclusion 🚀

Kubernetes makes it easy to control the Deployment of our applications with these strategies. This was just a rundown of how Rolling and Rollback Update Deployments work from a ground level. In real-life, we rarely do all these steps manually, since we hand it down to our CI/CD pipeline like ArgoCD.

Thank you for reading. Hope this guide has been helpful for you to understand how Rolling and Rollback Deployments work in Kubernetes. If you have any corrections, suggestions, or feedback feel free to DM me on Twitter or comment down below.

For tackling this very issue, we will be looking into how we can setup a lightweight Kubernetes cluster using K3s and Multipass for our VMs.

If you are interested in what makes K3s so light, you can watch the talk on K3s under the hood.

Prerequisites

Installing Multipass

Multipass is a command line tool to orchestrate virtual Ubuntu instances on your local machine. With the CLI, you can spawn VMs within minutes allocating optimal CPU, memory and disk space.

Download Multipass

Installing K3sup (ketchup)

We talked about K3s but it can be daunting to set up if you just want a quick hands on with Kubernetes. To abstract all that we will be installing a handy CLI called k3sup. It will bootstrap Kubernetes cluster on our VMs using K3s within a minute! 🚀

k3sup uses SSH under the hood, so make sure you have SSH service on your machine.

Install k3sup

Creating Virtual machines

Nodes are just virtual machines on Kubernetes working in sync. For our cluster we will be creating 3 virtual machines, each with 1CPU, 1GB RAM, 2GB Storage.

Generating keys

If you have SSH installed and already have ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub , you can skip this part and move to creating config file section.

Create Private/Public key:

$ ssh-keygen

This will create the aforementioned files on your system. Copy the contents of ~/.ssh/id_rsa.pub

$ cat ~/.ssh/id_rsa.pub

Creating config file

Create a file called multipass.yaml and place your public key in ssh-rsa.

# multipass.yaml
ssh_authorized_keys:

  - ssh-rsa <add-your-public-key>

This config file makes sure the public key is stored on the virtual machine once it’s created. Let’s create our VMs with proper names based on their roles we will be assigning (master/worker).

$ multipass launch --cpus 1 --mem 1G --disk 2G --name master-node --cloud-init multipass.yaml

$ multipass launch --cpus 1 --mem 1G --disk 2G --name agent-master --cloud-init multipass.yaml

$ multipass launch --cpus 1 --mem 1G --disk 2G --name agent-worker --cloud-init multipass.yaml

$ multipass ls

In K3s terms, a master node is called the server and the rest of the nodes are called agents. Agents are simply the nodes that gets added to the master node; they can be another master node or a worker node.

Adding K8s sauce with k3sup

Now that we have our VMs ready, let’s install Kubernetes on them. First we will be creating a master node to setup a control plane.

Adding a master node

We will need the IP and username of our virtual machine to SSH into and install Kubernetes. Run multipass ls and take note of IP of master-node . All the usernames for VMs are ubuntu by default.

$ k3sup install --ip <IP> --user ubuntu --k3s-extra-args "--cluster-init"

We are passing the --k3s-extra-args "--cluster-init" to make sure this node is prepared to connect with another master node, else it might cause errors.

Once installed it downloads the kubeconfig file on the directory where you invoked your command. You can set environment variable KUBECONFIG with path to recently downloaded kubeconfig file.

# mac / linux
$ export KUBECONFIG=<path-to-kubeconfig>

# windows
$ setx KUBECONFIG <path-to-kubeconfig>

Now you can kubectl get nodes to view the nodes on your cluster.

Adding a second master node (HA)

Multi master setup on local machine is an overkill for most use cases but for the purpose of this tutorial we are going to add it anyway. To join a new master node in the cluster we use the join command.

Here, all we have to do is introduce the server IP and username (previously created) that this node should connect to . And additionally pass the --server flag specifying this will be a server node (master).

$ k3sup install --ip <IP-of-agent-master> --user ubuntu --server-ip <IP-of-master-node> --server-user ubuntu --server

And that’s about it. Now you can kubectl get nodes again, and you’ll see two master nodes that is Highly Available (HA).

Adding a worker node

Finally, we have to setup our worker node where we will be deploying our applications and services.

For this grab the IP of agent-worker and pass on the same command as before minus the --server flag.

$ k3sup join --ip <IP-of-agent-worker> --user ubuntu --server-ip <IP-of-master-node> --server-user ubuntu

Now if you kubectl get nodes , you will find two master nodes and a single worker node composing your multi node cluster setup. You can create new VMs and add more master or worker nodes depending on how you plan to utilize your cluster.

Conclusion

Thank you for reading. Hope this guide has been helpful for you to setup and play with multi-node Kubernetes cluster. If you have any corrections, suggestions, or feedback feel free to DM me on Twitter or comment below.

Happy hacking! 🚀

In this blog, we will be exploring the development to deployment of our Serverless function locally.

The beauty of OpenFaas and KinD is that you don’t need to deploy your functions to cloud to test them, you can run them locally!

Prerequisite:

You just need to have Docker installed on your machine. The rest we will setup as we go along. We will be installing a few tools so bear with me. 🐻

OpenFaas? 🐬

OpenFaas is an awesome tool/framework developed by Alex Ellis with the Open Source community that helps to deploy burstable functions and microservices to Kubernetes without repetitive, boiler-plate coding.

It was built with developer experience in mind creating a low barrier of entry to the Kubernetes ecosystem. It comes with it’s own cli-tool called the faas-cli which makes a clever use of templates and makes pushing docker images and deployment of containers a breeze. And once deployed it takes care of auto-scaling and auto-provisioning based on the network traffic.

With all the overhead being handled by OpenFaas, all you have to do is focus on writing your function. But in our case we have some setting up to do 👨‍🏭

Properties of functions:

Here are few things to keep in mind when you are creating a function.

• A rule of thumb is to make your function stateless.

• Function should be atomic (single responsibility).

• It should be idempotent.

• Trigger to the function could be TCP or event, however effect must be an event.

Let’s get started! 🚀

Install arkade

arkade will be our one stop tool to install and deploy apps and services to Kubernetes. Think of this tool as a package manager like apt or pacman. We will be needing a bunch of CLI utilities to get up and running.

Make sure you install this if you have never set up a Kubernetes tooling in your system.

$ curl -sLS https://dl.get-arkade.dev | sudo sh

Just to make things clear:

• use arkade get to download cli tools and applications related to Kubernetes.

• use arkade install to install Kubernetes applications using helm charts or vanilla yaml files.

Install kubectl

kubectl is a command line tool that talks to the Kubernetes Master Controller API for performing actions on our cluster.

$ arkade get kubectl

# follow the instructions on the screen.

Creating a local Kubernetes cluster 🧱

We will be deploying our function on a local Kubernetes cluster so let’s set that up. We will be using a tool called KinD(Kubernetes in Docker) for this.

Unlike Minikube which requires VM; KinD utilizes Docker container to deploy your cluster. It is comparatively faster.

Install KinD

Our goal is to keep everything local so we will be creating a local Docker registry (Dockerhub for your local machine). KinD provides a shell script to create a Kubernetes cluster along with local Docker registry enabled.

#!/bin/sh
set -o errexit

# create registry container unless it already exists
reg_name='kind-registry'
reg_port='5000'
running="$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true)"
if [ "${running}" != 'true' ]; then
  docker run \
    -d --restart=always -p "${reg_port}:5000" --name "${reg_name}" \
    registry:2
fi

# create a cluster with the local registry enabled in containerd
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:${reg_port}"]
    endpoint = ["http://${reg_name}:${reg_port}"]
EOF

# connect the registry to the cluster network
docker network connect "kind" "${reg_name}"

# tell https://tilt.dev to use the registry
# https://docs.tilt.dev/choosing_clusters.html#discovering-the-registry
for node in $(kind get nodes); do
  kubectl annotate node "${node}" "kind.x-k8s.io/registry=localhost:${reg_port}";
done

kind-with-registry.sh from KinD website

Once you have the file on your local machine. Run the following command to make it executable.

$ chmod +x kind-with-registry.sh

Now you can run the script to create your local Kubernetes cluster with local Docker registry.

$ ./kind-with-registry.sh

To make sure the kubectl context is set to the newly created cluster; run:

$ kubectl config current-context

If the result is not kind-kind; then run:

$ kubectl config use kind-kind

Make sure your cluster is running:

$ kubectl cluster-info

Make sure Docker registry is running.

$ docker logs -f kind-registry
OR 
$ docker ps -l

Deploying OpenFaas to KinD cluster

Now that we have our local Kubernetes cluster up and running lets deploy our OpenFaas services to support our functions.

$ arkade install openfaas

Make sure OpenFaas is deployed. It might take a minute or two.

$ kubectl get pods -n openfaas

Templates📂

Before we move to creating our function, it is essential to understand the concept of templates in OpenFaas.

What are templates?

Templates are basically a wrapper for your functions. Same template can be used for multiple functions.

OpenFaas already provides a variety of templates for different programming languages to start with.

There are two type of templates based on the webserver :

1. classic watchdog which uses template format from here.

2. of-watchdog which uses template format from here.

Watchdogs are basically webservers to proxy request to our functions.

We will be using the templates that supports the latest of-watchdog. You can read more about OpenFaas watchdog.

The most important piece in the templates directory is the Dockerfile, it drives everything from your watchdog to how and where your function gets placed in the template. You can also create your own template based on the need of your function.

Pull the template from store:

# list all the templates 
$ faas-cli template store list

# pull python3-flask template from store
$ faas-cli template store pull python3-flask

This will pull all the templates for python from OpenFaas store to your template directory of your project folder. We will be using the python3-flask-debian template. You can ignore rest of the templates or delete them.

Openfaas template structure.

Shown above is the basic template structure for all languages.

Template structure explanation:

1. index.py is an entrypoint to our Flask application.

2. requirements.txt on the root project directory contains dependencies for our Flask application. It contains Flask and waitress (WSGI).

3. template.yml contains the deployment specific configurations.

4. requirements.txt inside of the function directory is for the function specific dependencies.

5. Dockerfile contains the instructions to build image for our function.

Building our function 🔨

Now it’s time to create our Serverless function.

We’ll keep the scope really small and build a simple dictionary function that returns the meaning of words you query. For this we will be using the PyDictionary module.

Since we have our template in place, we can utilize it to scaffold our function. You may have noticed the function directory and template.yml file inside our python3-flask-debian template folder earlier. We can leverage that to create our new function using the faas-cli.

$ faas-cli new <function-name> --lang <template-name>

$ faas-cli new pydict --lang python3-flask-debian

This will create a folder with the function name(pydict) along with the contents we saw earlier inside the templates function folder, plus a yaml file with the function name(pydict).

Let’s add our dependency to the requirements.txt file of function folder.

PyDictionary==2.0.1

Update the handler.py with the following code.

from PyDictionary import PyDictionary

dictionary = PyDictionary()

def handle(word):     
    return dictionary.meaning(word)

Our function is complete. ✔

Openfaas YAML configuration 📄

A Kubernetes guide won’t be complete without talking about YAML files. They are necessary to describe the state of our deployments. faas-cli has already created a yaml file for our function, namely, pydict.yml

This file will tell the faas-cli about the functions and images we want to deploy, language templates we want to use, scaling factor, and other configurations.

contents of pydict.yml

YAML file breakdown

1. gateway: determines the IP address to connect to the OpenFaas service. Since we are testing it locally the gateway provided by default is fine. Note: if we were deploying on managed cloud: we will take the IP address of gateway-external

2. Under functions: we can register different Serverless functions we want to deploy along with their configuration.

3. lang: name of the language template the function uses.

4. handler: location of our Serverless function.

5. image: fully qualified docker registry URL along with image name and tag.

Since we are using Local registry we have to change the image tag to incorporate it. Change value of image to the following:

Change image tag to use local container registry

Port-forward to Localhost ⏩

Now, we need to port-forward the OpenFaas gateway service to our localhost port. Remember the gateway on our yaml file?

To check the gateway of OpenFaas service; run:

$ kubectl get service -n openfaas

$ kubectl port-forward -n openfaas svc/gateway 8080:8080

Although the the cluster is deployed locally, internally Kubernetes manages it’s own IP addresses. We are port forwarding to access the service inside the Kubernetes cluster from our local machine. Once that is done, open a new terminal window, and prepare for take off!

Build ➡Push ➡Deploy 🚀

We have most of the setup ready; now we can build our image, push it to the registry, and deploy it to Kubernetes.

Build our image

$ faas-cli build -f pydict.yml

Push our image to the local registry

$ faas-cli push -f pydict.yml

Now to deploy the function to the cluster; we need to login to faas-cli Let’s generate credentials for authentication.

Generate password:

$ PASSWORD=$(kubectl get secret -n openfaas basic-auth -o jsonpath="{.data.basic-auth-password}" | base64 --decode; echo)

Check if password exists:

$ env | grep PASSWORD

Now, login to faas-cli :

$ echo -n $PASSWORD | faas-cli login --username admin --password-stdin

Time for deployment:

$ faas-cli deploy -f pydict.yml -g http://127.0.0.1:8080

Check if our deployment has been successful.

$ kubectl get pods -n openfaas-fn

Testing our function 👨‍🔬

Test using CLI

We can invoke our function from CL I to get the result.

$ echo "brevity" | faas-cli invoke pydict

Test using Openfaas Portal

OpenFaas also comes with a neat UI portal to invoke our function.

Navigate to localhost:8080 in your browser. A prompt will ask you for username and password. Use admin for username and for password check the password we generated earlier. In case you forgot, you can always repeat the earlier command, i.e.

PASSWORD=$(kubectl get secret -n openfaas basic-auth -o jsonpath="{.data.basic-auth-password}" | base64 --decode; echo) && echo $PASSWORD

OpenFaas UI

Build more functions 🔨

Now that you have your cluster deployed with OpenFaas, you can create function with ease and deploy them. If you want to create a new function, all you have to do is:

1. scaffold a new function using faas-cli new

2. write your function

3. faas-cli up -f <config-yaml-file>

faas-cli up is a shortcut for build, push and deploy command.

Conclusion

Hope this has been a helpful guide in learning how to use OpenFaas for Serverless deployment. If you have any queries or suggestions, let’s talk in the comment section or you can DM me on Twitter.

Check out example functions on GitHub