About

How to write clean, maintainable and scalable code on Python:

• https://bigdatateam.org/python-course

Environment Configuration

1. Download requirements.txt
2. Create environment:

export env_name="bdt-python-course"
conda create -n $env_name python=3.7
conda activate $env_name
conda install --file requirements.txt

See available conda environments with the help of:

conda info --envs

If you need to remove environment use the following command:

conda remove --name $env_name --all

HowTos

How to use pylint:

pylint --output-format=colorized -v inverted_index.py
# in case you would like to ignore some warnings:
pylint --output-format=colorized -d C0111,C0103 -v inverted_index.py
pylint --output-format=colorized -d invalid-name,missing-docstring -v inverted_index.py

How to use pytest:

pytest -v .
pytest --cov -v .
pytest --cov -vv --durations=0 .

Study Datasets

1. Wikipedia sample - do not forget to unzip after download
2. Stop words