Hello, I have summarized Style guide of python and programming device for coding rules in this entry. I described 5 different tools which are pep8, pyflakes, flake8, haking, Pylint.
Since this entry is longer than usual, if you don’t have much time I recommend you to read the chapter of pep8 and flake8.
Table of Contents
- How to use pep8
- What is PEP 8
- Concept of PEP 8
- Running pep8
- Output of the summary
- Examining this
- pep8 Error code meaning and option
- Customization of pep8
- Configuration file by users
- Configuration file by projects
- How to use pyflakes
- What is pyflakes
- Installing pyflakes
- Running pyflakes
- Design concept of pyflakes
- How to use flake8
- What is flake8
- Running flake8
- Running flake8
- flake8 Error code meaning and option
- How to use hacking
- What is hacking
- Installing hacking
- Error detection of hacking
- Error overview of hacking
- About customizing of hacking
- How to use Pylint
- What is Pylint
- Installing Pylint
- Running Pylint
- Pylint Error code meaning and option
How to use pep8
What is PEP 8
Before I explain about how to use pep8, I explain what is PEP 8.
python has document group called PEP
This is an aggregate of documents which explains about information, new features, process and environment settings for python community.
Document №8 is PEP 8 — Style Guide for Python Code.
As the title “Style Guide”indicates, it describes about code style of python.
For your information, “Code style of C” is PEP 7
(PEP 0007 — Style Guide for C Code _ Python.org) and “Conventions of Docstring” is PEP 257
(PEP 0257 — Docstring Conventions _ Python.org).
Concept of PEP 8
A Foolish Consistency is the Hobgoblin of Little Minds in PEP 8 is describing how to think about code consistency. This may be useful to understand about PEP 8. This is an extract of that.
- Codes are read much longer time than being written.
*As being said in PEP 20 — The Zen of Python, “Legibility” is important.
- This style guide exists for consistency.
- The most important thing is to know when you should break consistency. For instance, to keep this PEP you should not break backward compatibility.
A tool which verify the code style of PEP 8 is pep8.
This tool is implemented with python.
Running pep8
So I have already talked long, here we shall run pep8
.
As long as there isn’t any special note, each python tool are executed below environment.
- Mac OSX Yosemite(10.10.3)
- Python 2.7.9
- pip 7.1.2
First we shall install pep8.
You can install with below command as per README.
$ pip install pep8
I have prepared below file (parse.py
) for a trial.
- I have taken this from scrapy/scrapy and modified so that it will have warnings. *
from __future__ import print_function
import logging
class Command(ScrapyCommand):
@property
def max_evel(self):
levels = self.items.keys() + self.requests.keys()
if levels: return max(levels)
else: return 0
def add_items(self, lvl, new_items):
old_items = self.items.get(lvl, [])
self.items[lvl] = old_items + new_items
Execute pep8 for this file.
Basically it is $ pep8 [file name or directory name]
$ pep8 parse.py
scrapy/commands/parse.py:3:1: E302 expected 2 blank lines, found 0
scrapy/commands/parse.py:6:5: E303 too many blank lines (2)
scrapy/commands/parse.py:7:24: W291 trailing whitespace
scrapy/commands/parse.py:8:15: E221 multiple spaces before operator
scrapy/commands/parse.py:9:18: E701 multiple statements on one line (colon)
scrapy/commands/parse.py:10:13: E701 multiple statements on one line (colon)
scrapy/commands/parse.py:15:1: W391 blank line at end of file
In this case we have 7 violations.
As you see above, it outputs the file name which has violations, location, error code and that content.
This is displayed in simple English.
Output of the summary
When you would like to display the summary, use --statistics -qq
.
$ pep8 --statistics -qq ./parse.py
1 E221 multiple spaces before operator
1 E302 expected 2 blank lines, found 0
1 E303 too many blank lines (2)
2 E701 multiple statements on one line (colon)
1 W291 trailing whitespace
1 W391 blank line at end of file
If you execute $ pep8--statistics -qq.
in the root directory of the project, it will output the summary of PEP 8 violations.
Examining this
It can display the violations more clearly by using ^
This is $ --show-source
option.
$ pep8 --show-source ./parse.py
./parse.py:3:1: E302 expected 2 blank lines, found 0
class Command(ScrapyCommand):
^
./parse.py:6:5: E303 too many blank lines (2)
@property
^
./parse.py:7:24: W291 trailing whitespace
def max_evel(self):
^
./parse.py:8:15: E221 multiple spaces before operator
levels = self.items.keys() + self.requests.keys()
^
./parse.py:9:18: E701 multiple statements on one line (colon)
if levels: return max(levels)
^
./parse.py:10:13: E701 multiple statements on one line (colon)
else: return 0
^
./parse.py:15:1: W391 blank line at end of file
^
Moreover you can see the description of how to fix.
This is -- show-pep8
option.
$ pep8 --show-pep8 ./parse.py
./parse.py:3:1: E302 expected 2 blank lines, found 0
Separate top-level function and class definitions with two blank lines.
Method definitions inside a class are separated by a single blank line.
Extra blank lines may be used (sparingly) to separate groups of related
functions. Blank lines may be omitted between a bunch of related
one-liners (e.g. a set of dummy implementations).
Use blank lines in functions, sparingly, to indicate logical sections.
Okay: def a():n passnnndef b():n pass
Okay: def a():n passnnn# Foon# Barnndef b():n pass
E301: class Foo:n b = 0n def bar():n pass
E302: def a():n passnndef b(n):n pass
E303: def a():n passnnnndef b(n):n pass
E303: def a():nnnn pass
E304: @decoratornndef a():n pass
....(below continues the description for the same number of violations)
Although there’s quite many, since you can fix without checking the document one by one, this option seems to be also convenient.
pep8 Error code meaning and option
Below is the rough classification (extracted from source codes)
- Error and warning
- Starting with E … errors
- Starting with W … warnings
- 100 type … indentation
- 200 type … whitespace
- 300 type … blank lines
- 400 type … imports
- 500 type … line length
- 600 type … deprecation
- 700 type … statements
- 900 type … syntax errors
You can refer to this Document and this from the source codes for the details.
Excludes specific error code from the verification target
Specific errors can be ignored by giving a error code as a value in ignore
option.
For example, list up the error code which you would like to ignore by comma separation and then. (if you enter E41, E410- E419 will be ignored).
$ pep8 — ignore=E226,E302,E41 [file name or directory name]
Customization of pep8
We have seen how to run pep8 and the classification of errors and warnings.
Next we are going to see how to write the contents passed to pep8 by option in configuration file.
Configuration file by users
Default user configuration file is in ‘~/.config/pep8’.
You can write the configuration file as below, this is same as option.
[pep8]
ignore = E226,E302,E41
max-line-length = 160
List up the error code which you would like to ignore for the verification by comma separation for ignore(if you enter E41, E410- E419 will be ignored). This is same as command line option.
max-line-length
will specify the maximum number of the characters used in one line (default is 79, it outputs E501 once it exceeds more than 80 characters)
You can specify the configuration file location by --config=configration file location
.
Configuration file by projects
By storing above configuration file in a certain location in respective projects, you can share it in the projects.
If setup.cfg
file or tox.ini
file exist, the contents described in the file will be loaded.
By the way, .pep8
file is also loaded but it’s not recommended to use this file name(deprecated).
How to use pyflakes
What is pyflakes
Now we know how to use pep8, we shall learn about pyflakes.
Like pep8, pyflakes is also a verification tool for python source codes.
Unlikely pep8, pyflakes doesn’t verify the style at all but verify only logistic errors.
Installing pyflakes
You can install by below command.
$ pip install pyflakes
Running pyflakes
Basic usage is as below.
$ pyflakes [ file name or directory name]
We shall run this for parse.py
$ pyflakes ./parse.py
./parse.py:2: 'logging' imported but unused
./parse.py:3: undefined name 'ScrapyCommand'
It detected newly “library imported but unused” and “Undefined name”.
It’s different from what pep8 verified. You can see that “it doesn’t verify the style but verify only logistic error”.
Design concept of pyflakes
For the design concept of pyflakes, we have pyflakes described in README. This is distinctive, I explain here simply.
- It doesn’t refer to the style.
- Not to have false positive, it does extraordinary effort.
- Since it verifies each file individually, it’s faster than Pylint and PyChecker. Also it is comparatively limited what could be detected.
- To verify the style or to configure by each project, enable
flake8
.
It has few option (it has only — help and — version), and it doesn’t have “a scheme which controls and customizes warnings” commonly provided by such tool.
flake8 which we’ll talk about next will solve those. We can think that pyflakes is a tool which performs “to detect only logistic error” optimistic.
How to use flake8
What is flake8
Simply speaking flake8 is “the wrapper which verifies pep8, pyflakes and circular complexity “.
For other functions, it can control the warnings for specific line (impossible by a simple pyflakes)by # flake8: noqa
or it can customize warnings happening by configuration file such as pep8.
Installing flake8
You can install by below command.
$ pip install flake8
Running flake8
Usage and option are same as pep8
. it is $ flake8 [file name or directory name]
$ flake8 ./parse.py
./parse.py:2:1: F401 'logging' imported but unused
./parse.py:3:1: E302 expected 2 blank lines, found 0
./parse.py:3:15: F821 undefined name 'ScrapyCommand'
./parse.py:6:5: E303 too many blank lines (2)
./parse.py:7:24: W291 trailing whitespace
./parse.py:8:15: E221 multiple spaces before operator
./parse.py:9:18: E701 multiple statements on one line (colon)
./parse.py:10:13: E701 multiple statements on one line (colon)
./parse.py:15:1: W391 blank line at end of file
You can see the result of pep8 (error code is Exxx and Wxxx) and pyflakes (error code is Fxxx) are output together.
This means the error of pyflakes (Fxxx) can be arranged in the configuration file by ignore and select like pep8. It’s easy to customize.
flake8 Error code meaning and option
The error code of flake8 are E***
, W***
used in pep8
and F***
and C9**
.
E***
/W***
: Error and warning of pep8F***
: Detection of PyFlakesC9**
: Detection of circulate complexity by McCabe
You can see the description of error code in this document.
How to use hacking
flake8
is excellent for expandability. Next we talk about hacking
which has project own rule in flake8.
What is hacking
hacking
is flake8
plug-in which is made based on OpenStack OpenStack Style Guidlines.
Source code is on github and License is Apache License 2.0.
The origin of OpenStack Style Guidelines
is according to hacking page,
based on Google Python Style Guide and after own rule of OpenStack is added.
According to OpenStack Style Guidelines
, hacking has “Some purpose” which are as below.
- Not having too many un-practical reviews (such as docstring guidelines etc).
- Remove difficulty of read caused by environment of the people (unix vs windows newlines etc)
- Remove dangerous pattern (shadowing built-in or reserved words etc)
Installing hacking
$ pip install hacking
If you execute without having flake8, it will install with flake8.
Error detection of hacking
hacking can also detect such. Here we shall make H101
happen.
Adding comment # TODO we will fix
in the sample code.
@property
+ # TODO we will fix
def max_evel(self):
levels = self.items.keys() + self.requests.keys()
if levels: return max(levels)
If executing flake8 when hacking is installed
$ flake8 scrapy/commands/parse.py
scrapy/commands/parse.py:2:1: F401 'logging' imported but unused
scrapy/commands/parse.py:3:1: E302 expected 2 blank lines, found 0
scrapy/commands/parse.py:3:15: F821 undefined name 'ScrapyCommand'
scrapy/commands/parse.py:6:5: E303 too many blank lines (2)
scrapy/commands/parse.py:7:7: H101 Use TODO(NAME)
scrapy/commands/parse.py:8:24: W291 trailing whitespace
scrapy/commands/parse.py:9:15: E221 multiple spaces before operator
scrapy/commands/parse.py:10:18: E701 multiple statements on one line (colon)
scrapy/commands/parse.py:11:13: E701 multiple statements on one line (colon)
scrapy/commands/parse.py:17:1: W391 blank line at end of file
scrapy/commands/parse.py:7:7: H101 Use TODO(NAME)
has happened newly.
This rule is written in here of the guideline, due to
[H101] Include your name with TODOs as in # TODO(yourname). This makes it easier to find out who the author of the comment was.
As you see above, if you install hacking, it adds the hacking own rule (error code:Hxxx) when executing flake8.
Error overview of hacking
I have searched the document for hacking error code, I couldn’t find any (if anyone knows about error code, please let us know).さい)。
Based on the source code, we can classify as below.
H1xx
type
About license and comment
[H101] Include your name with TODOs as in ``# TODO(yourname)``. This makes
it easier to find out who the author of the comment was.
[H102 H103] Newly contributed Source Code should be licensed under the
Apache 2.0 license. All source files should have the following header::
[H104] Files with no code shouldn't contain any license header nor comments,
and must be left completely empty.
[H105] Don't use author tags. We use version control instead.
[H106] Don't put vim configuration in source files (off by default).
H2xx
type
About test and so on
[H201] Do not write ``except:``, use ``except Exception:`` at the very least.
[H202] Testing for ``Exception`` being raised is almost always a ...
H23x
type
Compatibility with Python3.x
[H231] ``except``. Instead of::
[H232] Python 3.x has become more strict regarding octal string
[H233] The ``print`` operator can be avoided by using::
[H234] ``assertEquals()`` logs a DeprecationWarning in Python 3.x, use
[H235] ``assert_()`` is deprecated in Python 3.x, use ``assertTrue()`` instead.
[H236] Use ``six.add_metaclass`` instead of ``__metaclass__``.
[H237] Don't use modules that were removed in Python 3. Removed module list:
[H238] Old style classes are deprecated and no longer available in Python 3
H3xx
type
Style of import
[H301] Do not import more than one module per line (*)
[H303] Do not use wildcard ``*`` import (*)
[H304] Do not make relative imports
[H306] Alphabetically order your imports by the full module path.
H4xx
type
About Docstrings
[H401] Docstrings should not start with a space.
[H403] Multi line docstrings should end on a new line.
[H404] Multi line docstrings should start without a leading new line.
[H405] Multi line docstrings should start with a one line summary followed
H5xx
type
About unification of the style
[H501] Do not use ``locals()`` or ``self.__dict__`` for formatting strings,
[H702] If you have a variable to place within the string, first
[H703] If you have multiple variables to place in the string, use keyword
[H903] Use only UNIX style newlines (``n``), not Windows style (``rn``)
About customizing of hacking
Since this is a plug-in of flake8, you can make such as “ignore” by specifying the error code similar to pep8.
For instance, as we talked above, the error type H1** are different by projects or companies such as “Explicit of Apache License2.0” and “How to write TODO comment”, we recommend you to have “ignore” for this.
How to use Pylint
As of now we have seen pep8 which can verify the each file individually and pyflake type tool,
We introduce you Pylint which is a tool to verify modules and packages used for multiple files to finish.
What is Pylint
Pylint is also one of the code verification of python.
A simple verification tool pep8
which we explain in the beginning has a history, but it seems Pylint has a long history too (initial release 0.0 version is 2003).
Comparing to pep8, Pylint has more verification items and options.
It has frequent commit even now (November, 2015) Source code is here (https://bitbucket.org/logilab/pylint).
Installing Pylint
Like other tool, you can install by pip.
$ pip install pylint
Running Pylint
You can use this as $ pylint [option] [module or package]
We shall run parse.py
used in previous hacking
as a sample. The result is as below.
$ pylint scrapy/commands/parse.py
No config file found, using default configuration
************* Module scrapy.commands.parse
C: 7, 0: Trailing whitespace (trailing-whitespace)
C: 8, 0: Exactly one space required before assignment
levels = self.items.keys() + self.requests.keys()
^ (bad-whitespace)
C: 1, 0: Missing module docstring (missing-docstring)
C: 3, 0: Missing class docstring (missing-docstring)
W: 3, 0: Class has no __init__ method (no-init)
E: 3,14: Undefined variable 'ScrapyCommand' (undefined-variable)
C: 7, 4: Missing method docstring (missing-docstring)
E: 8,18: Instance of 'Command' has no 'items' member (no-member)
E: 8,38: Instance of 'Command' has no 'requests' member (no-member)
C: 9,19: More than one statement on a single line (multiple-statements)
C: 12, 4: Missing method docstring (missing-docstring)
E: 13,20: Instance of 'Command' has no 'items' member (no-member)
E: 14, 8: Instance of 'Command' has no 'items' member (no-member)
W: 2, 0: Unused import logging (unused-import)
...(although the report continues, we omit here)
hacking
had 9 violations, this time we have a bit of fluctuation and total we have 12 violations.
Characteristically there are
E: 8,18: Instance of ‘Command’ has no ‘items’ member (no-member)
This is a new violation detected.
This violation (error code is E1101) means that the member named items
wasn’t found at the time of verification.
But in official wiki E1101 description, it is described as when adding a member dynamically. it may show false positive.
Hence if you write the code intentionally so, you have to pay attention.
Pylint Error code meaning and option
Since Pylint has a number of error code and options, I can’t explain all here.
It has a list this page in an official document, you can refer to it when you customize it.
This time, we introduce you an option to make the verification parallel and how to generate the configuration file.
Acceleration by paralleling the verification (-j option)
As a characteristic of Pylint, not verifying the each files individually but sine it verifies by package or module unit it takes more time to verify comparing to pyflakes and pep8.
You can parallel it by -j(--jobs)
option.
-j <n-processes>, --jobs=<n-processes>
Use multiple processes to speed up Pylint. [current:1]
How to generate the configuration file ( — generate-rcfile option)
You can output the template of configuration file(pylintrc) in standard output by — generate-rcfile option.
As long as you don’t specify the configuration file location by the option, pylintrc of current directory will be loaded in priority.
If you would like to decide the common rule for each projects, you can place pylintrc in the root directory of the project.
Summary
Hello, I have summarized Style guide of python and programming device for coding rules in this entry.
We have seen the tools related for pep8 and flake8, by customizing it for your project you can develop easier.
It’s a popular fact that flake8 is wrapping pep8 or hacking is made as the plug-in of flake8. Why don’t you try flake8 to start??
Also you can use flake8 from Sider.
More articles about Python(pep8, pyflakes, flake8, haking, Pylint)
For more information about Sider, please go to our website.