# Streamline your projects using Makefile `make` is one of the tools that we use heavily for streamlining tasks on our projects. It has proven to be helpful specifically for streamlining the development process, repeating mundane tasks with custom CLI like subcommands and mainly **onboarding new team members**. %[https://youtu.be/vybdPTfNLo4] With a set of rules in **Makefile**, you can get up and running in no time, keeping the process sane and saving time and effort for everyone in the team. We'll be going through the basics to some interesting stuffs we can do with Makefile. There are two pieces to this equation, one is the `make` CLI tool and the next is the Makefile . The basics is `make` reads the **rules** from the Makefile and executes them. What I will be showing today is just a small part of what `make` is capable of. ## Writing Makefile If you have worked with `YAML` files before then you will feel right at home writing Makefiles. ### Anatomy of Rules Every `Makefile` consists of rules with the anatomy of: ```makefile target: dependencies recipe ``` - **target**: target can be an executable, object or just a name for an action that we want to carry out. We will be using targets purely with the placeholder name for the rule. Be mindful about the name, as it should resonate with the action we want to perform with no confusion whatsoever. - **dependencies**: Dependencies are the rules that needs to be executed, in order for the current rule to work. - **recipe**: recipe is the meat of the `Makefile`, it is the action that we want to perform with our `target` name. Make sure to put a `tab` character at the start of every recipe line (just like YAML). You can also replace the `tab` character with anything you want using the `.RECIPEPREFIX` variable. Next we will be looking into some examples on how to make use of `Makefile`. These examples will be based on setting up development environments. ### Basic Rules A basic rule where you just want to put some alias is straight forward. Let’s say you have a **python** project and you want to hand it over to a new team member. How do you streamline the setup process. Maybe it can look something like this. > **Note**: > `#` is for comments. > > `@` symbol is to disable printing the recipe to stdout. > Test without the `@` symbol at the beginning of the recipe. > > `:=` is the expansion operator which prevents using subsequent value with the same variable name. > > `SHELL` variable determines the default shell to execute the recipe. ```makefile SHELL :=/bin/bash .PHONY: format check venv: # setup a virtual environment @python3 -m venv venv setup: # install dev dependencies @pip install -e .[dev] @echo -e "\nInstalling pre-commit hook..." @pre-commit install format: # format code using black @black . check: # check for formatting using black @black --check --diff -v . test: # run pytest @pytest -vvv ``` You can do something similar with your existing project. Now to get up and running, all you have to do is: ```bash $ make venv $ . venv/bin/activate $ make setup $ make format # and so on ``` ### Rules with Dependencies Taking the reference from the example above, suppose we want to print out the output of `check` target every time we run the `format` target. So how do we create that dependency? It’s plain simple, we just have to update the `format` target to look something like this: ```makefile format: check # run the formatter on files. @black . ``` We have added the dependency of `check` to the right of the target, just like showcased on the anatomy **Anatomy of Rules** section. ## Variables We can also define variables if we have some piece of command for repeated use. For this example we will be taking the reference of the `Django` management command. Variables are normally written with all caps and uses `:=` to assign variable name to a value. Variables can be accessed using either `$()` or `${}` syntax. ```makefile DJANGO_MANAGE := python manage.py run: @${DJANGO_MANAGE} runserver show: @${DJANGO_MANAGE} showmigrations migrate: @${DJANGO_MANAGE} migrate ``` Also your **SHELL environment variables** are converted in to Makefile environment variables, so you can directly make use of them while creating your rules. **Example:** In our shell we can export an environment variable called `INFO`. ```bash $ export INFO="Run make help to show all the available rules." ``` And now in the Makefile we can refer to it as any variable. ```makefile info: # show project info @echo ${INFO} ``` ## Default target If you just run `make` on your command line and it will run the first target on your Makefile. But we can change that by using the `.DEFAULTGOAL` special variable and assigning the target we want to run by default. ```makefile .DEFAULT_GOAL := run ``` Now, next time you run `make` it is going to run the `Django` server by default. ## Self documenting Now we have bunch of targets on our `Makefile` and we also called this combo as a custom mini CLI app. Wouldn’t it be great, if we could have a help command similar to a real CLI app? Say no more, thanks to the [blog](https://dev.to/victoria/how-to-create-a-self-documenting-makefile-2b0e) from [Victoria Drake](https://dev.to/victoria) we have the script to do so. Just create a `help` target and assign it as a `.DEFAULT_GOAL`. With this, all the comments we have been writing on our target gets converted into a nice help message. ```makefile .DEFAULT_GOAL := help help: # Show this help @egrep -h '\s#\s' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?# "}; {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}' ``` ## Include other Makefiles We can separate out Makefiles based on the tasks they perform and `include` them into the main `Makefile`. We usually have separate `Makefile` managed for **environment variables**, **Docker** and **Kubernetes**. This offloads all the tasks from project set up to Deployment to the Makefile. I will show a brief example of each of the file just to give an example: > Note: Since make runs each recipe on a new instance of the shell, we can lazy evaluate the variables using `?=` meaning, they are initialized only when referenced for a single shell instance. **Makefile** Root makefile composed of other Makefiles. ```makefile SHELL :=/bin/bash APP_ROOT := $(PWD) TMP_PATH := $(APP_ROOT)/.tmp VENV_PATH := $(APP_ROOT)/.venv export ENVIRONMENT_OVERRIDE_PATH ?= $(APP_ROOT)/env/Makefile.override -include $(ENVIRONMENT_OVERRIDE_PATH) include $(APP_ROOT)/targets/Makefile.docker include $(APP_ROOT)/targets/Makefile.k8s ``` **Environment Variables** *Makefile.override* Makefile containing just the essential environment variables. ```makefile STAGE ?= SERVICE_NAME ?= AKS_RESOURCE_GROUP ?= AKS_CLUSTER_NAME ?= REGISTRY_URL ?= AZ_ACR_REPO_NAME ?= ``` **Docker** *Makefile.docker* Makefile containing docker rules. ```makefile export GIT_COMMIT ?= $(shell cut -c-8 <<< `git rev-parse HEAD`) export BRANCH ?= $(shell git rev-parse --abbrev-ref HEAD) export DOCKER_BUILD_FLAGS ?= --no-cache export DOCKER_BUILD_PATH ?= $(APP_ROOT) export DOCKER_FILE ?= $(APP_ROOT)/Dockerfile export TARGET_IMAGE ?= $(REGISTRY_URL)/$(AZ_ACR_REPO_NAME)/$(SERVICE_NAME) export TARGET_IMAGE_LATEST ?= $(TARGET_IMAGE):$(BRANCH)-$(GIT_COMMIT) acr-docker-login: az acr login --name $(AZ_ACR_REPO_NAME) docker-build: docker build $(DOCKER_BUILD_FLAGS) -t $(SERVICE_NAME) -f $(DOCKER_FILE) $(DOCKER_BUILD_PATH) docker-tag: docker tag $(SERVICE_NAME) $(TARGET_IMAGE_LATEST) docker-push: acr-docker-login docker push $(TARGET_IMAGE_LATEST) ``` **Kubernetes** *Makefile.k8s* Makefile containing rules for Kubernetes. ```makefile export OVERLAY_PATH ?= $(APP_ROOT)/k8s/overlays/$(STAGE)/ define kustomize-image-edit cd $(OVERLAY_PATH) && kustomize edit set image api=$(1) && \ cd $(APP_ROOT) endef kubectl-apply: kustomize build $(OVERLAY_PATH) kustomize build $(OVERLAY_PATH) | kubectl apply -f - update-kubeconfig: az aks get-credentials --resource-group $(AKS_RESOURCE_GROUP) --name $(AKS_CLUSTER_NAME) aks-deploy: update-kubeconfig $(call kustomize-image-edit,$(TARGET_IMAGE_LATEST)) make kubectl-apply aks-delete: update-kubeconfig kubectl delete namespace $(STAGE)-api kustomize-edit: $(call kustomize-image-edit,$(TARGET_IMAGE_LATEST)) ``` Now we have orchestrated all these Makefiles, it is easier to keep track of all the rules and makes working with Makefiles sane, if you are doing a lot with it. ## Conclusion So with the use of `Makefile` we can streamline a lot of redundant tasks in our projects without having to remember overwhelmingly long and varying commands. It increases the productivity of the whole team; with easier project setup and redundant tasks outsourced to the `Makefile` with intuitive target names, leaving the devs to focus on more serious tasks at hand.