type annotations in python: whats, whys and wows!
TRANSCRIPT
![Page 1: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/1.jpg)
Type Annotations in Python: Whats, Whys & Wows
Andreas Dewes (@japh44)
Europython 2017 - Rimini
![Page 2: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/2.jpg)
Outline
Explain why type annotations areinteresting and how they cameto be.
Show you howyou can usethem in yourown code base.
Analyze howpeople actuallyuse them and show you whatelse you can do with them.
![Page 3: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/3.jpg)
Motivation: When We Discover Bugs
Formal Proof ofCorrectness
Design Review
Unit / Integration / … Testing
Compiler Errors via Type Checker
Code Reviews
Static Analysis
We Don‘t, OurCustomer Does
Position in Software Life Cycle
Enter: Type Hinting
![Page 4: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/4.jpg)
The Python Way: Gradual Typing
Unannotated code
Annotated code
external code
our code
external type information
![Page 5: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/5.jpg)
History of Type Hints in Python
https://www.python.org/dev/peps/****
PEP 482 – LiteratureOverview
PEP 483 – Theory ofType Hints
PEP 484 – Type Hints
2014
3.5
PEP 3107 – FunctionAnnotations
2006
3.0
DraftInformal Accepted / Final
PEP 544 – Protocols: Structural Subtyping
PEP 526 – Variable Annotations
2016
3.6
2017
![Page 6: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/6.jpg)
Annotation Syntax in Python
Return type annotation
Argument annotation
Variable annotation(Python 3.6 only)
![Page 7: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/7.jpg)
Architecture of Type Hinting in Python
Python Interpreter
Just stores type annotations in a special__annotations__ variable.No runtime effectsotherwise!
The „typing“ module
Allows us to specify the types that wewant to annotate our code with (requiredfor non-standard types and advanced usecases)
External tools(e.g. mypy)
Uses annotations toperform type checking(or other functionality)
![Page 8: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/8.jpg)
Getting StartedWith Type Hints: MyPy
• Originally Written by Jukka Lehtosalo, now also stronglypushed by Dropbox and Guido van Rossum
Easy to install & use
> pip install mypy> mypy [file to check] …
http://mypy-lang.org/
![Page 9: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/9.jpg)
Example Code Base: Flor
![Page 10: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/10.jpg)
Our initial code
• Small but functionalcodebase: Less than200 lines of code
• No external dependencies: Goodas an example
• Not many „exotic“ types: Easy toannotate
• Compatible withPython 2+3: Good fortesting different approaches
![Page 11: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/11.jpg)
A Test Script
![Page 12: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/12.jpg)
Adding Type Hints
• Go through the code functionby function, adding hints to all arguments and return types
• Possibly add hints toambiguous variable initializations (if needed)
• Import and use necessarytypes from the typing module
• Try to make mypy happy
![Page 13: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/13.jpg)
I brokemypy!
Quick fix:bytes → _bytes
![Page 14: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/14.jpg)
Running MyPy: It (Finally) Works!
Argument 1 to „BloomFilter“ has incompatible type „float“; expected „int“ (…)
Argument 1 to „add“ of„BloomFilter“ has incompatibletype „str“; expected „bytes“
Unsupported operandtypes for + (List[int] and „str“)
![Page 15: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/15.jpg)
But … now we lost Python 2 compatibility
![Page 16: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/16.jpg)
Second Approach:Type Comments
• Instead of writinghints as code, we addthem as comments
• A special syntax tellsmypy to treat themas type hints
• We still need toimport the „typing“ module (there is a workaround though)
![Page 17: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/17.jpg)
Nice! But what about code we can‘t change?
![Page 18: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/18.jpg)
Stub files (.pyi)
Mypy will look for Stubfiles in several places(search path, currentdirectory, typeshed, …). If it finds a .pyi file and a .py file, it will onlyload the .pyi!
Third Approach: Stub Files
![Page 19: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/19.jpg)
Building a Stub File
• As before, start withthe code
• Remove all actualvalues and functionbodies, just leavingthe signatures
• Use ellipsis (…) toindicate missing parts
• Add type hints
• Think „header files“ for Python
![Page 20: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/20.jpg)
ComparingOur Approaches
https://travis-ci.org/DCSO/flor
commentedinline
https://stackoverflow.com/questions/43516780/adding-type-information-without-dependency-on-typing-module
![Page 21: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/21.jpg)
Pros & Cons
Inline+ The „canonical“ way
of using type hints
+ Easy to read
+ Code and hints arekept in one place
- Only compatible withPython ≥ 3.3 (or ≥ 3.6 if using variable annotations)
Type Comments+ Keeps code
compatible withPython ≥ 2.7
+ Allows variable annotations regardlessof Python version
- Ugly (?)
- Still requires importingthe „typing“ module(but there is a workaround)
Stub Files+ Does not modify
original source at all
+ Allows use of latestfeatures regardless ofPython version
- Duplicatesmaintenance effort
- Does not (yet) allowchecking of the actualcode against the stubs
![Page 22: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/22.jpg)
There‘s(Much) More ToType Hints!
• Generics• Type variables• Classes• Generators• …
![Page 23: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/23.jpg)
Are People Actually Using This? Let‘s Check!
We download code from the top 1000 Python repositories on Github and check it for any kindof type hinting: Inline annotations, type comments and stubs.
https://github.com/adewes/type-annotations-in-the-wild
![Page 24: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/24.jpg)
103 projects with at least 1 type annotation
53 projects with at least 10
24 projects with at least 100
inline
comments
pyi files
![Page 25: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/25.jpg)
Top Python Repositories with Type Hints
![Page 26: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/26.jpg)
Function & Variable Annotations:Not Only For Type Hinting
> foo.__annotations__{‘x‘ : int,‘return‘ : bool}
![Page 27: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/27.jpg)
Example: Contracts in Python
(not saying this is a good idea, just that it works…)
https://gist.github.com/adewes/b87c8adc95e768ebf6366130ad6d85a7
![Page 28: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/28.jpg)
Summary
Type hintingworks & makesyour code morerobust (for youand for others)
You can use itnow already(regardless ifyou use Python 2 or 3)
Annotationscan do morethan type hinting (but think about ittwice)
![Page 29: Type Annotations in Python: Whats, Whys and Wows!](https://reader034.vdocument.in/reader034/viewer/2022050613/5a6e947b7f8b9a38568b4cc5/html5/thumbnails/29.jpg)
Thanks!
Slides:
https://slideshare.com/japh44
Me:
@japh44
License:
CC BY-NC 3.0: https://creativecommons.org/licenses/by-nc/3.0/en/