incolumepy.gwa#

Prospection GitHub Workflows Actions

Contains: + poetry + pytest + coverage + Python multiversions (pyenv) + tox + black + isort + flake8 + pylint + pydocstyle + mypy + semver

Start Project#

Create a github project on github.com;
Clone the project, on local host;

Create the branch dev and branches of control when necessary:

$ git br dev
$ git co -b feature#1 dev
$ git co -b feature#2 dev
$ git co -b bug#2 dev

Configure project structure:

Set Editor configuration:

$ curl https://pastebin.com/raw/TrDhuvFZ -o .editorconfig # or
$ curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.editorconfig -o .editorconfig

Update gitignore:

$ curl https://pastebin.com/raw/C910Dqze -o .gitignore   #or
$ curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.gitignore -o .gitignore

Commit changes

Configure poetry:
Init poetry
```
$ poetry init    # response that need
```

Adding package path in configuration:

[tool.poetry]
  ...
packages=[
    {include = 'incolume/', from=''},
]

Define Python version (3.10+): ```bash Change.. [tool.poetry.dependencies] python = "~3.10"

To .. [tool.poetry.dependencies] python = ~3.10 ``` 4. Commit changes 6. Configure unit tests with pytest: 1. Install pytest

$ poetry add -D pytest     ## adding pytest
$ poetry add -D pytest-cov     ## adding pytest-cov

2. Create tests structure

$ cd incolumepy.gwa;    # home_project
$ mkdir tests       # create directory
$ > conftest.py     # create file
$ > __init__.py     # create file

3. Add test environment

$  git add -f poetry.lock pyproject.toml  tests/

4. Commit changes

Linters and Checkers#

Adding lint/control tools
bash $ poetry add -D black flake8 isort mypy pydocstyle pylint pytest-cov tox[tomli] $ git ci -m "Adding tox and lint tools" poetry.lock pyproject.toml
Configure code coverage

Edit pyproject.toml, add on end file.

$ ...
[tool.pytest.ini_options]
addopts = "--cov=incolumepy.gwa"
testpaths = [
   "tests",
]

Configure pylint tool

Edit pyproject.toml, add on end file.

...
[tool.pylint.format]
# Maximum number of characters on a single line.
max-line-length = 160

[tool.pylint.basic]
# Allow shorter and longer variable names than the default.
argument-rgx = "[a-z_][a-z0-9_]*$"
attr-rgx = "[a-z_][a-z0-9_]*$"
variable-rgx = "[a-z_][a-z0-9_]*$"

# Ensure that orjson is analysed as a C extension by pylint.
extension-pkg-whitelist = "orjson"

[tool.pylint.messages_control]
disable = [
    # Disable too many and too few checks.
    "too-many-ancestors",
    "too-many-arguments",
    "too-many-boolean-expressions",
    "too-many-branches",
    "too-many-function-args",
    "too-many-instance-attributes",
    "too-many-lines",
    "too-many-locals",
    "too-many-nested-blocks",
    "too-many-public-methods",
    "too-many-return-statements",
    "too-many-statements",
    "too-few-public-methods",

    # Similar lines in files (often the case in tests).
    "duplicate-code",

    # Many functions (e.g. callbacks) will naturally have unused arguments.
    "unused-argument",

    # Disable checking that method could be a function in classes (often used for organisation).
    "no-self-use",

    # Disable failure for TODO items in the codebase (code will always have TODOs).
    "fixme",

    # Disable docstrings checks as we don't require excessive documentation.
    "missing-docstring",

    "no-member",
    "unspecified-encoding",
]

Commit

$ git ci -m "Configure pylint tool" pyproject.toml

Configure mypy tool

Edit pyproject.toml, add on end file.

...
   [tool.mypy]
   mypy_path = "incolumepy"
   check_untyped_defs = true
   disallow_any_generics = true
   ignore_missing_imports = true
   no_implicit_optional = true
   show_error_codes = true
   strict_equality = true
   warn_redundant_casts = true
   warn_return_any = true
   warn_unreachable = true
   warn_unused_configs = true
   no_implicit_reexport = true

Commit

$ git ci -m "Configure mypy tool" pyproject.toml

Configure isort tool

Edit pyproject.toml, add on end file.

...
[tool.isort]
multi_line_output = 3
line_length = 160
include_trailing_comma = true

Commit

$ git ci -m "Configure isort tool" pyproject.toml

Configure flake8 tool

Edit pyproject.toml, add on end file.
```
...
[flake8]
max-line-length = 160
```

Commit

$ git ci -m "Configure flake8 tool" pyproject.toml

Configure black tool:

Edit pyproject.toml, add on end file.
```
...
[tool.black]
line_length = 160
```

Commit

$ git ci -m "Configure black tool" pyproject.toml

Configure tox tool:

Edit pyproject.toml, add on end file.

[tool.tox]
legacy_tox_ini = """
[tox]
minversion = 3.3.0
isolated_build = True
envlist =
black
flake8
isort
mypy
pydocstyle
pylint
py{36,37,38,39,310}

;[tox:.package]
;basepython = python3

[testenv]
whitelist_externals = poetry
skip_install = true
commands =
poetry env use {envpython}
poetry install -vv --no-root
poetry run pytest {posargs} tests/

[testenv:flake8]
deps = flake8
commands = flake8 --config pyproject.toml incolumepy/ tests/

[testenv:mypy]
deps =
mypy
types-toml
commands = mypy incolumepy/

[testenv:pydocstyle]
commands = poetry run pydocstyle incolumepy/ tests/

[testenv:isort]
commands = poetry run isort --check --atomic --py all incolumepy/ tests/

[testenv:pylint]
commands = poetry run pylint incolumepy/ tests/

[testenv:black]
commands = poetry run black --check incolumepy/ tests/
"""

Commit

$ git ci -m "Configure tox tool" pyproject.toml

Testting lint configurate#

Test pydocstyle:
```
$ pydocstyle incolumepy/ tests/
```
Test pylint
```
$ pylint incolumepy/ tests/
```
Test mypy
```
$ mypy incolumepy/ tests/
```
Test isort
```
$ isort --check incolumepy/ tests/
```

Test flake8

$ flake8 --config pyproject.toml incolumepy/ tests/

Test black
```
$ black --check incolumepy/ tests/
```

Testting tox configurate#

Text tox
```
$ tox
```

Control version with poetry#

Unlike bumpversion, poetry changes the version of the main file only. To work around this situation and update the version of pyproject.toml, init.py and version.txt, we will do the following procedure.

$ cat > incolumepy/gwa/__init__.py << eof
import toml
from pathlib import Path
__root__ = Path(__file__).parents[0]
version_file = __root__.joinpath('version.txt')
version_file.write_text(f"{toml.load(Path(__file__).parents[2].joinpath('pyproject.toml'))['tool']['poetry']['version']}\n")
__version__ = version_file.read_text().strip()
eof

$ v=`poetry version prerelease`; pytest tests/ && git ci -m "$v" pyproject.toml $(find -name "version.txt")  #sem tag
$ s=`poetry version patch`; pytest tests/ && git ci -m "`echo $s`" pyproject.toml `find -name "version.txt"`; git tag -f `poetry version -s` -m "$(echo $s)"  #com tag

Facility with make#

Use make to easy various commands.

$ curl https://pastebin.com/raw/eTHpL70G -o Makefile    # or
$ curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/Makefile -o Makefile    # last version

Makefile help#

$ make help

New prerelease with Makefile#

$ git co dev
$ git merge --no-ff [branch] --autostash
$ make prerelease

New release with Makefile#

$ git co dev
$ make release

Checking lint#

$ make lint

Run tox over make#

$ make tox

Github Workflows Actions (GWA)#

First create the directory for GWA:

mkdir -pv .github/workflows

This Models:#

GitHub WorkFlow with tags Running tox; Build pack; Create release; publish TestPyPI and PyPI

curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.github/workflows/gwa-flow.yml -o .github/workflows/gwa-flow.yml

Run tox and generate coverage report:

curl https://raw.githubuserconcurl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.github/workflows/tests_ci_cd.yaml -o .github/workflows/tests_ci_cd.yml

Publish manualy releases/prereleases into test.pypi.org with poetry:

curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.github/workflows/poetry-testpypi-manual-publish.yml -o .github/workflows/poetry-testpypi-manual-publish.yml

Publish automatically releases into pypi.org and test.pypi.org with poetry, after pass on tests from tests_ci_cd:

curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.github/workflows/poetry-mine-publish-automatically.yml -o .github/workflows/poetry-mine-publish-automatically.yml

Publish automatically releases into pypi.org only, after pass on tests from tests_ci_cd:

curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.github/workflows/poetry-mine-pypi-autopublish.yml -o  .github/workflows/poetry-mine-pypi-autopublish.yml

Publish automatically releases into pypi.org and test.pypi.org without poetry, after pass on tests from tests_ci_cd:

curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.github/workflows/publish-automatically.yml -o .github/workflows/publish-automatically.yml

Publish automatically releases into pypi.org and test.pypi.org with multiprocess poetry, after pass on tests from tests_ci_cd:

curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.github/workflows/poetry-multiprocess.yml -o .github/workflows/poetry-multiprocess.yml

Git Hooks#

create .git-hooks folder into your project root directory, at the same level you already have .git folder;
```
mkdir .git-hooks
```
create hook files into this folder: pre-commit, prepare-commit-msg and all hooks (these files don't have an extension);

put the correct code into each file (I will add them below these steps);

curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.git-hooks/pre-commit -o .git-hooks/pre-commit
curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.git-hooks/prepare-commit-msg -o .git-hooks/prepare-commit-msg
curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.git-hooks/commit-msg -o .git-hooks/commit-msg
curl https://raw.githubusercontent.com/incolumepy-prospections/incolumepy.gwa/main/.git-hooks/pre-push -o .git-hooks/pre-push

change to execute mode;
```
chmod +x .git-hooks/*
```
run this command in your command line, into your main folder of your project (one level up from .git-hooks):
```
git config core.hooksPath .git-hooks;
```
READY;