Karambir Singh Nain

My Python Programming Workflow

Published

Note: This is a live document with pointers, not a blog post

How do I do python programming.

Basic Setup

Setup asdf to manage multiple python, node and golang versions

Install python versions using asdf:

bash asdf install python 3.9.17 asdf install python 3.10.12 asdf install python 3.11.4

Install pipx using system package manager like pacman, apt, brew. It creates separate virtualenvs for each python cli tool you install. Tell pipx to use one of the python version installed by asdf by adding following to .bashrc or .zshrc:

shell PIPX_DEFAULT_PYTHON=/home/user/.asdf/installs/python/3.10.12/bin/python

Use pipx to manage python cli software

  • cookiecutter
  • pgcli
  • poetry
  • pre-commit

Project Setup

Activate correct python version using asdf asdf local python 3.11.4. This creates a .tool-versions file in local directory so in-future it automatically picks correct version. This file can be committed to git as well.

Create and use virtualenv using built-in venv package via bash aliases

bash alias cv='python -m venv .venv' alias cv='python -m venv .venv'

Poetry can also be used to create and activate virtualenv automatically using poetry run commands. Configure poetry with:

bash poetry config virtualenvs.create true poetry config virtualenvs.in-project true

Why not using mkvirtualenvwrapper?

Use Poetry to install and manage dependencies.

For linting:

  • black
  • isort
  • mypy
  • ~~flake8~~ Ruff(install separately)

For documentation and versioning:

  • mkdocs
  • bump2version

For testing:

  • pytest with mock, cov extensions
  • faker
  • ipdb
  • tox if multiple python env and package versions are required

Keep most configs for these packages in pyproject.toml. If not, use setup.cfg. Avoid tool specific config file.

Use Makefile with frequently used commands.

Editor Setup

Use Vim or Vim Mode in Pycharm

Have plugins like EditorConfig

``` root = true

[*] charset = utf-8 end_of_line = lf insert_final_newline = true trim_trailing_whitespace = true

[*.{py,rst,ini}] indent_style = space indent_size = 4

[*.{html,css,scss,json,yml}] line_length=120 indent_style = space indent_size = 2

[*.md] trim_trailing_whitespace = false

[Makefile] indent_style = tab ```

For pycharm, setup configurations to run tests and tool being developed.

Major packages I find awesome

  • Django
  • Django Rest Framework
  • FastAPI and Starlette
  • SQLAlchemy
  • Encode databases and httpx
  • Celery
  • dotenv
  • click
  • loguru(structlog is equally awesome)
  • django-mptt
  • open telemetry
  • psycopg2
  • uwsgi or gunicorn

While Programming

Use extensive logging with packages like loguru. Enable debug logging locally for external calls like File IO, db or network requests. Try using structured logging(json?) to attach extra context to logs

CI

Use GitHub actions or Gitlab CI for testing, building and releasing.

Topic: Software