Hands-On vLLM Thinking Token Budget
vLLM is a workhorse to run inference for any LLM under the sun. One of the recent developments in the project is the ability to define thinking_token_budget, basically a request level argument that ca
Yankee Maharjan
Software Engineer, Open Source Enthusiast, Cloud-Native fanatic
Latest articles
How I manage my Python Virtual Environments
In a common workflow we create a virtual environment using different tools inside of the project directory. You might have venv, .venv, env and so on. I find it a bit cluttered. My approach is a bit d
Build LLM App with Web Search: Step-by-Step Guide
Learn how to build your own powerful Large Language Model (LLM) application that can search the web, extract information, and answer your questions – all LOCALLY! In this tutorial, I'll walk you through the process of creating a web-searching LLM fro...
Build your own Artifact Previewer using Local LLM
Large Language Models are great at generating code, but we need to go an extra mile to run them; well not any more. In this video, we will be building our application that can use both local and hosted large language models (Gemini, ChatGPT, and so o...
AWS Code Artifact for Private PyPI repository
Having a private PyPI repository can be beneficial in many ways. We can host our internal private packages; from security perspective, it can be a controlled environment for public packages from where all the dependencies are installed in our project...
Effortless CI/CD with GitHub Composite Actions
GitHub Actions is a powerful and versatile CI/CD solution that works for different teams and projects. It's a built-in feature of GitHub, the platform that many developers use daily, so it's easy to set up and use. One of the benefits of GitHub Actio...
Securing Your Supply Chain: A Guide to Signing and Verifying Blobs
Signing artifacts should be a crucial part of our workflow. As a part of Supply Chain Security, we must ensure that the artifact built on our CI platform is the artifact we are deploying on our production environments. If by any means, we cannot veri...
Streamline Github Actions Releases with Github CLI
GitHub Actions is the default choice for CI/CD in many Open Source and Enterprise projects. It gets regular feature updates and is flexible enough to get most of the things done. One of the powerful feature it provides is the ability to re-use Action...
Streamlining EC2 Access with AWS Session Manager and Port Forwarding: An In-Depth Guide
EC2 is one of the fundamental services provided by AWS and used by many who come across the platform. This compute service has applications that extend far and wide depending on the use cases. One of the basic tasks, we do with EC2 is SSH into it, to...