Here is the collection of information on python and howTos.
How to switch between python versions
To manage different versions of python, we will use update-alternatives. (man page: https://documentation.suse.com/es-es/sles/15-SP1/html/SLES-all/cha-update-alternative.html) to configure and be able to easily switch between version.
Step 1: Let’s take example of Python2 and Python3. First, lets find out python versions. Execute following command
Step 2: find out if the alternatives is configured in your system. To find out execute following command
sudo update-alternatives --list python
If you are getting “error: no alternatives for python” , then you will need to configure the alternatives.
Step 3: Let’s configure Python alternatives for python2 and python3. Execute the following commands
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2 1
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 2
Step 4: Check python version. Execute following command
#this will show your python version.
Step 4: If you want to switch and see what is configured, execute following command
sudo update-alternatives --configure python
The above command will show your options and what is currently configured
Generator expressions return an iterator that computes the values as necessary, not needing to materialize all the values at once. This means that list comprehensions aren’t useful if you’re working with iterators that return an infinite stream or a very large amount of data. Generator expressions are preferable in these situations.
With a list comprehension, you get back a Python list, a list containing the resulting lines but not an iterator.
Generators are a special class of functions that simplify the task of writing iterators. Regular functions compute a value and return it, but generators return an iterator that returns a stream of values.
Python and Luigi
Python logging has evolved over the years and the logging module is very rich and comparable to log4j logging. However, the documentation is great but examples are very skinny. Having said that the logging module is well implemented.
The logging module does not support YAML configuration file. I found out painfully. However, the configuration file default layout is very straight forward and works well. Had I known that the YAML is not supported, I would not have waisted my time.
If you are interested in YAML, you will need to write your own YAML parser and then provide the Dict to the logging configurator.
Understanding Python Path
Python path defines the search path for python file and classes imported in each script/code.
PYTHONPATHenvironment variable is often used to add directories to the search path. If this environment variable is found then the contents are added to the module search path.
Testing Python Code
There are many python tests now one is the feature that is included in the python Unittest, other PYTest that I have included.
I have used Unittest exclusively and it is out of the language. It is very powerful and if you are Java programer, this is pretty much looks like java tests.
Python Style Guide
Python Style Guide is similar to Java Coding Standards and other language coding Standards. This one every python programmer shall follow for better code readability.
Single and Double Underscore
In a class, names with a leading underscore indicate to other programmers that the attribute or method is intended to be be used inside that class. However, privacy is not enforced in any way. Using leading underscores for functions in a module indicates it should not be imported from somewhere else.
_single_leading_underscore: weak “internal use” indicator. E.g.
from M import *does not import objects whose name starts with an underscore.
Any identifier of the form
__spam(at least two leading underscores, at most one trailing underscore) is textually replaced with
classnameis the current class name with leading underscore(s) stripped. This mangling is done without regard to the syntactic position of the identifier, so it can be used to define class-private instance and class variables, methods, variables stored in globals, and even variables stored in instances. private to this class on instances of other classes.
Name mangling is intended to give classes an easy way to define “private” instance variables and methods, without having to worry about instance variables defined by derived classes, or mucking with instance variables by code outside the class. Note that the mangling rules are designed mostly to avoid accidents; it still is possible for a determined soul to access or modify a variable that is considered private.Python doc
_foo: Only a convention. A way for the programmer to indicate that the variable is private (whatever that means in Python).
__foo: This has real meaning. The interpreter replaces this name with
_classname__fooas a way to ensure that the name will not overlap with a similar name in another class.
__foo__: Only a convention. A way for the Python system to use names that won’t conflict with user names.
No other form of underscores have meaning in the Python world. Also, there’s no difference between class, variable, global, etc in these conventions.
Object Oriented Programming:
Python has provided a way to do an object oriented programming. However, it require more code to write OO than scripting. The data encapsulation can be done but with a work around. By design, python does not do the data encapsulation for the ease of programming. However, if you require an encapsulation, you can implement in python.
Python has inheritance and multiple inheritance; however, type casting for a class is not easily supported. The best way to type cast in python is to use the following:
object.__class__ = Class
For example: Class A, B, C inherits from I and you want to type cast an instanceA to specific class such as A, then following is required.
instanceA.__class__ = A
Java support type casting by design and thus there are a lot of libraries that use type casting features to do the dependency injection.
Class Attributes vs Instance Variables
In Python, you can declare attributes at Class level or at an instance level. The behavior is hugely different. The class attribute bit behaves like Static Attribute in JAVA. A class attribute is visible across all the instances is considered a global in scope.
After a lot of experimentation, following are few caveats in class attributes operation.
- Class attributes may be access using the ClassName.attribute.
- Best way to operate on a class attribute is to define a class method with @classmethod annotation and use ‘cls’ to access the attribute, e.g. cls.attribute = ?
- Above two method guarantees the change is global and not as instance variable
- ‘self’ may be used to access the global as python first search for instance variable and then to class attribute. However, ‘self’ shall not be used for update or assignment operation on the class attribute. When using ‘self,’ python will create a local instance variable, thus, you have a class attribute and instance attribute having two different values.
How to use Class Attributes
Case 1, loggers shall be shared across class instances. In Java, it is the best practice to define loggers as static variables.
Case 2, where you need to share attributes across instances, such as database connection pool. Instead of creating a singleton class, here you can use the class attribute to use the connection pool across all the data access instances.
Object toString equivalent in Python
In Python, the toString on object is as follows
string = ???
Python Language Documentation Pointers
- Logging module: https://docs.python.org/3/library/logging.html
- Logging config: https://docs.python.org/3/library/logging.config.html#configuration-file-format
- Thread-based parallelism in python: https://docs.python.org/3/library/threading.html#module-threading
- Python Tuple Data Structure: https://docs.python.org/3/tutorial/datastructures.html?highlight=tuple
- Index of Python Enhancement Proposals (PEPs): https://peps.python.org
- The Zen of Python: https://peps.python.org/pep-0020
- Python Path: https://docs.python.org/3/library/sys_path_init.html
- Python Subprocess : https://docs.python.org/3/library/subprocess.html
- Pytest how tos: https://docs.pytest.org/en/7.2.x/how-to/index.html#how-to
- Python Style Guide: https://peps.python.org/pep-0008/
- Python Built-in Types: https://docs.python.org/3/library/stdtypes.html#str
- Python Generators: https://wiki.python.org/moin/Generators
- Python Classes: https://docs.python.org/3/tutorial/classes.html
- Python Built-in functions: https://docs.python.org/3/library/functions.html?highlight=property
Python Projects that are worth taking a look
- Python pallets project: https://palletsprojects.com
- JWST Calibration Library: https://github.com/spacetelescope/jwst
- Euro python: https://ep2022.europython.eu
- Pytest: https://docs.pytest.org/en/7.2.x/index.html
- PyYaml: https://pyyaml.org/wiki/PyYAMLDocumentation
- Luigi Scaling and Containerizing, https://engineering.giphy.com/luigi-the-10x-plumber-containerizing-scaling-luigi-in-kubernetes/
- Github Luigi, https://github.com/spotify/luigi
- high-performance serialization utilities for Python – https://pypi.org/project/srsly/
Python Online course worth taking a look
- Python Course.edu : https://python-course.eu/oop/
- This is a Linode library of documentation in python – https://www.linode.com/docs/guides/development/python/