Karambir Singh Nain

My Python Programming Workflow

Published

Note: This is a live document with pointers, not a blog post

How do I do python programming.

Basic Setup

Setup asdf to manage multiple python, node and golang versions

Install python versions using asdf:

asdf install python 3.9.17
asdf install python 3.10.12
asdf install python 3.11.4

Install pipx using system package manager like pacman, apt, brew. It creates separate virtualenvs for each python cli tool you install. Tell pipx to use one of the python version installed by asdf by adding following to .bashrc or .zshrc:

PIPX_DEFAULT_PYTHON=/home/user/.asdf/installs/python/3.10.12/bin/python

Use pipx to manage python cli software

  • cookiecutter
  • pgcli
  • poetry
  • pre-commit

Project Setup

Activate correct python version using asdf asdf local python 3.11.4. This creates a .tool-versions file in local directory so in-future it automatically picks correct version. This file can be committed to git as well.

Create and use virtualenv using built-in venv package via bash aliases

alias cv='python -m venv .venv'
alias cv='python -m venv .venv'

Poetry can also be used to create and activate virtualenv automatically using poetry run commands. Configure poetry with:

poetry config virtualenvs.create true
poetry config virtualenvs.in-project true

Why not using mkvirtualenvwrapper?

Use Poetry to install and manage dependencies.

For linting:

  • black
  • isort
  • mypy
  • ~~flake8~~ Ruff(install separately)

For documentation and versioning:

  • mkdocs
  • bump2version

For testing:

  • pytest with mock, cov extensions
  • faker
  • ipdb
  • tox if multiple python env and package versions are required

Keep most configs for these packages in pyproject.toml. If not, use setup.cfg. Avoid tool specific config file.

Use Makefile with frequently used commands.

Editor Setup

Use Vim or Vim Mode in Pycharm

Have plugins like EditorConfig

root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.{py,rst,ini}]
indent_style = space
indent_size = 4

[*.{html,css,scss,json,yml}]
line_length=120
indent_style = space
indent_size = 2

[*.md]
trim_trailing_whitespace = false

[Makefile]
indent_style = tab

For pycharm, setup configurations to run tests and tool being developed.

Major packages I find awesome

  • Django
  • Django Rest Framework
  • FastAPI and Starlette
  • SQLAlchemy
  • Encode databases and httpx
  • Celery
  • dotenv
  • click
  • loguru(structlog is equally awesome)
  • django-mptt
  • open telemetry
  • psycopg2
  • uwsgi or gunicorn

While Programming

Use extensive logging with packages like loguru. Enable debug logging locally for external calls like File IO, db or network requests. Try using structured logging(json?) to attach extra context to logs

CI

Use GitHub actions or Gitlab CI for testing, building and releasing.

Topic: Software