Karambir Nain

My Python Programming Workflow

Last updated: August 19, 2023

How do I do python programming.

Basic Setup

Set up asdf to manage multiple Python, Node.js, and Go versions.

Install Python versions using asdf:

asdf install python 3.9.21
asdf install python 3.10.16
asdf install python 3.11.11

Install pipx using a system package manager like pacman, apt, or brew. It creates separate virtual environments for each Python CLI tool you install. Tell pipx to use one of the Python versions installed by asdf by adding the following to .bashrc or .zshrc:

PIPX_DEFAULT_PYTHON=/home/user/.asdf/installs/python/3.11.11/bin/python

Use pipx to manage Python CLI software:

  • cookiecutter
  • pgcli
  • poetry
  • pre-commit

Project Setup

Activate the correct Python version using asdf: asdf local python 3.11.4. This creates a .tool-versions file in the local directory so in the future it automatically picks the correct version. This file can be committed to git as well.

Create and use virtual environments using the built-in venv package via bash aliases:

alias cv='python -m venv .venv'
alias av='source .venv/bin/activate'

Poetry can also be used to create and activate virtual environments automatically using poetry run commands. Configure Poetry with:

poetry config virtualenvs.create true
poetry config virtualenvs.in-project true

Why not use mkvirtualenvwrapper? (Personal preference for simpler tools)

Use Poetry to install and manage dependencies.

For linting:

  • black
  • isort
  • mypy
  • flake8 Ruff (install separately)

For documentation and versioning:

  • mkdocs
  • bump2version

For testing:

  • pytest with mock, cov extensions
  • faker
  • ipdb
  • tox if multiple Python environments and package versions are required

Keep most configurations for these packages in pyproject.toml. If not possible, use setup.cfg. Avoid tool-specific config files.

Use a Makefile with frequently used commands.

Editor Setup

Use Vim or Vim mode in PyCharm.

Have plugins like EditorConfig:

root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.{py,rst,ini}]
indent_style = space
indent_size = 4

[*.{html,css,scss,json,yml}]
line_length=120
indent_style = space
indent_size = 2

[*.md]
trim_trailing_whitespace = false

[Makefile]
indent_style = tab

For PyCharm, set up configurations to run tests and the tool being developed.

Major Packages I Find Awesome

  • Django
  • Django Rest Framework
  • FastAPI and Starlette
  • SQLAlchemy
  • Encode databases and httpx
  • Celery
  • dotenv
  • click
  • loguru (structlog is equally awesome)
  • django-mptt
  • OpenTelemetry
  • psycopg2
  • uwsgi or gunicorn

While Programming

Use extensive logging with packages like loguru. Enable debug logging locally for external calls like file I/O, database, or network requests. Try using structured logging (JSON?) to attach extra context to logs.

CI

Use GitHub Actions or GitLab CI for testing, building, and releasing.