Python

Basic Resources

JetBrain’s Python Developers Survey 2018 Results

You can read the full results on JetBrain’s site. I simply want to highlight a few of the “most used” technologies…

  • Isolate Python Environments: virtualenv/pipenv (64%), Docker (31%), Conda (22%).
  • Web Frameworks: Flask (47%), Django (45%).
  • Data Science: NumPy (62%), Pandas (51%), Matplotlib (46%), SciPy (38%), SciKit-Learn (31%), TensorFlow (25%).
  • Other Frameworks/Libraries: Requests (53%), Pillow (29%), Scrapy (19%), Asyncio (18%), Tkinter (18%), PyQT (16%), Six (14%), aiohttp (11%), Pygame (11%).
  • Testing: pytest (46%), unittest (32%), mock (15%).
  • Databases: PostgresSQL (44%), MySQL (41%), SQLite (41%).
  • ORMs: SQLAlchemy (34%), Django ORM (33%).
  • CI: Jenkins/Hudson (25%), Gitlab (18%), Travis (18%).
  • Configuration Management: Ansible (20%).
  • Editors/IDE’s: PyCharm (35%), VSCode (16%), Vim (10%).

Specific Topics

Web Frameworks

Web Scraping / Crawling

Some Relevant Portions of the Python Standard Library

Take a look at PEP 594 which recommends removing some standard libraries from Python due to fundamental issues or maintenance burden.

  • array
  • bisect – manipulate sorted lists
  • collections – additional data types
  • configparser
  • csv
  • datetime
  • difflib
  • doctest – for running tests embedded in docstrings
  • email
  • heapq
  • html
  • itertools – efficient iterators
  • json
  • logging
  • math – tools for random, statistics, floating point math
  • os
    • os.path
  • pathlib
  • pdb – Python Debugger
  • pickle – object serialization
  • pprint – for pretty printing
  • pydoc – documentation generation
  • pytest
  • random – generate pseudo-random numbers
  • re – for regular expressions
  • sched – event scheduler
  • sqlite3
  • statistics
  • string
  • sys
  • sysconfig
  • typing – Allows type hinting
  • unittest – Framework for unit testing.
  • urllib
  • venv – Virtual Environments

Stuff I Use

Stuff I May Use But Haven’t Yet

Note that I try to avoid repeating items which are mentioned elsewhere. So for example, Django and Flask are not on this list because they can be found under Web Frameworks.

  • Airflow – Apache, workflow automation.
  • Arrow – Dates and times.
  • Ansible – IT automation platform.
  • attrs
  • BeeWare – Cross-platform (including mobile) UI framework. Fairly young, recommended by Dan Bader (11/24/19).
  • Bokeh – Interactive Web plotting.
  • buildbot – Continuous Integration testing framework.
  • CNTK – By Microsoft, deep learning.
  • elasticsearch-dsl-py – High-level official client for Elasticsearch.
  • elasticsearch-py – Low-level official client for Elasticsearch.
  • Eve – REST API framework.
  • Falcon – For web APIs.
  • Flake8
  • fuzzywuzzy – Fuzzy string matching.
  • gspread – For connecting with Google Sheets API.
  • Gooey – Create GUI app automatically for CLI apps.
  • graphene – GraphQL framework.
  • gspread – Google Sheets Python API.
  • httpie – HTTP CLI client.
  • Hypothesis – Automated testing.
  • Jinja2 – Template Engine.
  • Keras – Deep Learning Library, can run on top of TensorFlow, CNTK, Theano.
  • Kivy – Cross-platform (including mobile) UI framework. Recommended by Dan Bader (11/24/19).
  • loguru – Simplified logging.
  • MonkeyType – By Instagram.
  • MyPy – Optional static type checker.
  • NLTK – Natural Language Toolkit.
  • Nose2 – Unit Testing.
  • pandas – Data analysis / manipulation.
  • pickleDB – Key-value store.
  • Poetry
  • Prophet – Time series data from Facebook.
  • pyinstaller – Create stand-alone executables.
  • Python Fire – Auto generates CLIs for Python objects.
  • PyTest
  • PyTorch – Deep Learning.
  • PyYAML
  • q – Run SQL against CSV/TSV files.
  • Redash – Connect and query data.
  • Sanic – Web Framework.
  • SciPy – Data Science and Analysis toolset. Includes NumPy, SciPy, Matplotlib, IPython, pandas, Sympy, nose.
  • Scikit-Learn – Machine Learning, built on top of SciPy.
  • Serverless
  • schedule – Job scheduling.
  • SpaCy – NLP
  • Spyder – Scientific Python Development Environment.
  • SQLAlchemy – SQL Toolkit and ORM.
  • SQL for Humans by Kenneth Reitz.
  • Superset – Apache, BI.
  • TensorFlow – machine learning.
  • TextBlob – Text Processing, including sentiment analysis.
  • xmltodict – Allows working with XML more like it is JSON.

Software Made in Python

For a much more comprehensive list see Awesome Python Applications.

Python Community