# AWS Code Artifact for Private PyPI repository

Having a private PyPI repository can be beneficial in many ways. We can host our internal private packages; from security perspective, it can be a controlled environment for public packages from where all the dependencies are installed in our project/product.

If your company is invested in AWS then it makes perfect sense to use AWS Code Artifact to host private packages, be it for Python, Node, Java or others.

Benefits of Code Artifact includes:

* a private PyPI instance, where `pip` or other tools work the same as with the public instance. Just mention the vanilla package name and that's it. No more `pip installgit+ssh` , `git+https` or `./path-to-package`.
    
* secure infrastructure that you can fully control and define who can access what.
    
* packages encrypted by default with AWS KMS key.
    

## 🔎 Overview of elements in AWS Code Artifact

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1704219590435/d346a5a5-5206-47f4-b1c2-970bf192806a.png align="center")

**TLDR;** you get a top-level construct called domain, under a domain you can have different repositories for same or different programming languages, each repository can store packages and their versions.

## 🌐 Create your domain

In AWS Code Artifact, domains are the namespace where you can host package repositories (PyPI, npm, maven). This is similar to creating domain name for website. Whatever name you put for your domain will be part of the URL that `pip` or other related tools will call for managing the packages.

1. Go to Code Artifact and under Artifacts select Domains.
    
2. ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706807366692/ccad4d19-91c6-46b9-ab0b-4ac90b85a6ff.png align="center")
    
    Then add a domain name and select Create Domain.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706812594886/4e4faa78-4d1f-476f-a5c0-4df3bfce5940.png align="center")
    
3. Apply Domain policy  
    Now that we have the domain, let's set the domain policy which allows us to get temporary authorization token for accessing repositories in the domain.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706807734712/6ae64101-03e8-4035-bbc4-e5df792dd69f.png align="center")
    
4. Paste the following resource-based policy and save it.  
    Here we are allowing a particular IAM principal, in this case a user called 'test-user' to be able to perform `GetAuthorizationToken` action against the resource.  
    Any valid IAM Principal is allowed here.
    
    ```yaml
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": [
                        "arn:aws:iam::xyz:user/test-user"
                    ]
                },
                "Action": "codeartifact:GetAuthorizationToken",
                "Resource": "*"
            }
        ]
    }
    ```
    

> Note: AWS Code Artifact requires both resource-based policy and identity-based policy to work.

## 📦 Create your repository

You can host multiple repositories under a single domain. We'll just create one for PyPI. Each repository is an independent entity and comes with its own configurations, and resource-based policies.

1. Click on create repository
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706808592524/3891edf5-52fc-4e12-bc7f-469003d14a91.png align="center")
    
2. Name your repository, add description to identify its usage properly.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706812463348/d9be9c67-ee2c-4626-b5fb-950068e357b2.png align="center")
    
3. Setting Public upstream repositories  
    When you authorize `pip` or related tools to fetch packages from private repository, if the requested package is not in the private repo it will fetch from the selected public repository. The requested package will then be stored on the private repository as well.
    
    We can select PyPI repository as upstream or leave it as blank if we intend to keep just the private packages. For me, I set the upstream as public PyPI as most of the private packages depend on other third-party packages which needs to be pulled while installing them.
    
4. Select Create Repository and we are done.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706812523553/a5e8f06b-1983-4f8d-b535-90f15c7d57c1.png align="center")
    
5. Similar to domain policy we need to set repository-level policy as well.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706812702305/f38a31d3-0700-4186-95b9-843651b867e0.png align="center")
    
6. Apply the following Policy.  
    Policy has two statements, "PublishPackages" for publishing packages, here the principal will likely be an IAM Role, the pipeline assumes this role and uploads the built package to the private repository. For this demo, we are providing access to an IAM user.
    
    The "InstallPackages" statement is a read-only policy which allows the IAM Principal to read and install packages from the private repository.
    
    IAM principals that create the package will most likely be different from the ones that consume it, so it makes sense to have them separate. This is also a good approach in-terms of security.
    
    ```yaml
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "PublishPackages",
                "Effect": "Allow",
                "Principal": {
                    "AWS": "arn:aws:iam::xyz:user/test-user"
                },
                "Action": "codeartifact:PublishPackageVersion",
                "Resource": "*"
            },
            {
                "Sid": "InstallPackages",
                "Effect": "Allow",
                "Principal": {
                    "AWS": [
                        "arn:aws:iam::xyz:user/test-user"
                    ]
                },
                "Action": [
                    "codeartifact:DescribePackageVersion",
                    "codeartifact:DescribeRepository",
                    "codeartifact:GetPackageVersionReadme",
                    "codeartifact:GetRepositoryEndpoint",
                    "codeartifact:ListPackageVersionAssets",
                    "codeartifact:ListPackageVersionDependencies",
                    "codeartifact:ListPackageVersions",
                    "codeartifact:ListPackages",
                    "codeartifact:ReadFromRepository",
                    "codeartifact:GetRepositoryEndpoint"
                ],
                "Resource": "*"
            }
        ]
    }
    ```
    

## 🚀 Push your package

I will be pushing my CLI-tool called [timezones-cli](https://github.com/yankeexe/timezones-cli) for this demo. You can use any Python package of your choice.

> 🌟Note: Make sure you have [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) installed on your machine.

Before we can push anything, we need to configure the policy for our user to be able to authorize to the Code Artifact domain and get the packages.

Set the following policy for the IAM user:

```yaml
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "codeartifact:GetAuthorizationToken",
            "Resource": "arn:aws:codeartifact:ap-south-1:xyz:domain/blog-pypi"
        },
        {
            "Effect": "Allow",
            "Action": "sts:GetServiceBearerToken",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "sts:AWSServiceName": "codeartifact.amazonaws.com"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "codeartifact:DescribePackageVersion",
                "codeartifact:DescribeRepository",
                "codeartifact:GetPackageVersionReadme",
                "codeartifact:GetRepositoryEndpoint",
                "codeartifact:ListPackageVersionAssets",
                "codeartifact:ListPackageVersionDependencies",
                "codeartifact:ListPackageVersions",
                "codeartifact:ListPackages",
                "codeartifact:ReadFromRepository"
            ],
            "Resource": "arn:aws:codeartifact:ap-south-1:xyz:repository/blog-pypi/private-pypi"
        }
    ]
}
```

That's it for the policy, we are all set now! Let's get back to publishing the package.

AWS Code Artifact provides you a list of tools you can authorize using AWS CLI.

### **Let's configure twine to upload packages**

1. Inside of the repository, select on View connection instructions
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706810789452/c81e6028-d094-4536-874a-5eed92c5b31a.png align="center")
    
2. Select twine and copy the `aws codeartifact` CLI command.  
    You should have [twine](https://pypi.org/project/twine/) installed on your virtual environment or global scope.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706809914380/427d1ab6-08c9-417b-befa-1e8ca7a69f50.png align="center")
    
3. Let's create a source distribution (.tar.gz) and a wheel distribution (.whl) for our package.  
    From the root of the project run the following command.
    
    > 🌟 Note: If you are creating wheel distribution then make sure `wheel` package is installed on your machine.
    
    ```bash
    python setup.py bdist_wheel sdist
    ```
    
    This command will create a `dist` directory which will contain our package ready to be pushed.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706810315228/85a2b8f1-2a85-4c75-8d0f-2bcd6d2e56e0.png align="center")
    
4. On your terminal, paste the command copied for Code Artifact, if you need to use any particular AWS CLI profile then use the `--profile` flag to pass the profile name like in the example.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706810922898/c4a160d9-6845-4dfb-ae6f-74088defcd6d.png align="center")
    
    After successful login, twine configurations will be in `~/.pypirc`, if you want to logout then delete this file.
    
5. Let's upload our package 🎉  
    We need to specify the repository we are pushing to, which is done using the `-r` flag.
    
    ```bash
    twine upload -r codeartifact dist/*
    ```
    
    Package is uploaded successfully! 🙌
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706811604629/b9a3d06c-00a6-425c-be41-4417837125f4.png align="center")
    
    We can confirm this checking our repository.
    
    ![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706811666668/adde2dc8-7388-4eb9-8d9f-c2d1b387e0fc.png align="center")
    

## ⚡️ Pull your package

Now that we've pushed our package, it's time to download them.

Go to the repository page and select "View connection instructions" again. This time select the `pip` tool and copy the login command.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706811795598/7c5ea57c-3d65-46f7-884b-ec06aef04b6e.png align="center")

We should be out of our package's virtual environment, in a completely new virtual environment, or installing the package in the global context. I will be installing it in the global context.

Paste the command for pip login, once logged in, pip generates configuration on `~/.config/pip/pip.conf`. To logout, we can delete this file or run `pip config unset global.index-url`.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706812031409/1a332ae6-05f1-4e0f-9fd4-971cf3030bf3.png align="center")

Now every `pip install` command we put out will try to communicate with our private repository. We can install the package with the regular pip install:

```yaml
pip install timezones-cli
```

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1706812811827/2692da36-8993-4ca3-8b92-95f23f2201c2.png align="center")

We have successfully uploaded and downloaded package from our private PyPI repository hosted on AWS Code Artifact. ⚡️

## 🌟 Conclusion

Using AWS Code Artifact is one of many ways of installing private python packages. If you or your company is already invested in AWS then it makes perfect sense to use AWS Code Artifact. If not then you can evaluate the pros and cons for using this solution. But all AWS Code Artifact provides a complete and secure solution to get you up and running with your private package infrastructure in no time.