Multiprocessing vs. Multithreading

A process runs in its own memory space and it doesn’t share memory with other processes, but threads run inside of a process in the same memory space.

Race Condition: The OS will allocate memory to the process to be used at the runtime and to be accessed by all threads. It can happen that a thread would like to read data from a specific memory space, but the other thread is not yet loaded data into that memory space, which will lead to an error. We cannot tell the order in which the threads will try to reach the shared data.

Solution: Global Interpreter Lock (GIL). GIL do not allows for multiple threads to execute python code at the same time.

Multithreading

• It uses only one CPU for all of the threads.
• The code will not run faster if it’s already using 100% CPU.
• Python threads are best used in cases where the execution of a task involves some waiting (ie.: I/O operations)
• The threading module is used to start multiple threads inside a single process.

Pros

• Lightweight, lower memory consumption
• Shared memory, makes access to state from another context easier
• Allows you to easily make responsive UIs
• Great option for I/O applications

Cons

• CPython, subject to the GIL
• Threads are not killable
• If not following a command queue / message model, then manual use of synchronization becomes necessary
• Code is usually harder to understand and to get right due to the potential of race conditions increases dramatically

 

Multiprocessing

• Low risk of data-corruption when using multiprocessing
• Each process has its own allocated memory
• Each process has it’s own GIL so there is no resource conflict or race condition

Pros

• Separate memory space
• Code is usually straight forward
• Takes advantage on multiple CPU cores
• Avoids GIL limitations
• Child processes are interruptable

Cons

• Larger memory consumption
• Inter Process Communication (IPC) a little more complicated with more overhead

https://docs.python.org/2.7/library/queue.html

 

Tkinter

 

from Tkinter import *

root = Tk()

topFrame = Frame(root)
topFrame.pack()

bottomFrame = Frame(root)
bottomFrame.pack(side=BOTTOM)

button1 = Button(topFrame, text='Button 1')
button2 = Button(topFrame, text='Button 2')
button3 = Button(topFrame, text='Button 3')
button4 = Button(bottomFrame, text='Button 4')

button1.pack(side=LEFT)
button2.pack(side=LEFT)
button3.pack(side=LEFT)
button4.pack()

root.mainloop()

 

Pexpect

Pexpect can be used for automating interactive applications
such as ssh, ftp, passwd, telnet, etc.. Pexpect is in the spirit of Don
Libes’ Expect, but Pexpect is pure Python. Other Expect-like modules for Python
require TCL and Expect or require C extensions to be compiled. Pexpect does not
use C, Expect, or TCL extensions.

 

Debugging

Functions

• before() –
• after() –
• spawn(‘COMMAND’, logfile=sys.stdout)
  This way all input/output will be printed that happens within the pexpect object.

 

 

d

d

d

d

d

d

File handling

Opening and reading

Simple example:

myFile = open("input.txt","r")

• r – default setting, read-only
• w – write, also creates the file if not exist yet, else it overwrites it
• a – appending
• b – binary format
• w+ – Opens a file for writing and also reading

Functions:

• read() – Returns the entire content of  a file in a string. You can specify how many characters should it read: read(num)
• mode() – Shows in what mode was the file opened (ie.: w)
• seek(num) – Positions the cursor at the specified byte
• tell() – Returns the actual position of the cursor
• readlines() – Returns a list where each line is a list element.

 

Cursor:

myFile = open("input.txt","r")

print(myFile.read(10))
print(myFile.read(10))

--- Output ---
alma
korte

csereszny

As you can see the same command had 2 different output. This is caused because the “cursor” have been moved after read. It remembers the last index of reading. You need to adjust it to get the same result:

myFile = open("input.txt","r")

print(myFile.read(10))
myFile.seek(0)
print(myFile.read(10))

--- Output ---
alma
korte
alma
korte

 

File writing and appending

myFile = open("input.txt","w")
myFile.write("I like python\nDo you?\n")
myFile.close()

myFile = open("input.txt","r")
print(myFile.read())

--- Output ---
I like python
Do you?

You cannot read the file when it’s open for writing! Use w+ for this (and don’t forget to seek!! )

Useful functions:

• write(str) – Write a string into the file
• writelines() – Write list or tuples into the file
• close() – Closes the file.
• closed() – Returns True when a file is closed.

 

There is another solution for closing a file in Python. This will close automatically:

with open("input.txt","w+") as myFile:
  myFile.write("Hello python\n")
  myFile.seek(0)
  print(myFile.read())

--- Output ---
Hello python

When you leave the “level” of the with the file will be closed.

with open("input.txt","w+") as myFile:
  myFile.write("Hello python\n")
  myFile.seek(0)
  print(myFile.read())

print(myFile.read())

--- Output ---
Hello python

Traceback (most recent call last):
 File "C:\cygwin64\home\tamvarga\python-study\file_handling.py", line 6, in <module>
 print(myFile.read())
ValueError: I/O operation on closed file

d

d

d

Advanced Python Modules

Creating and importing an own module.

vi test_module.py

print "This is my own module"

vi main.py
import test_module

print "This is the main file"

--- Output ---
This is my own module
This is the main file

Note that every time you import a module it will be executed. Only once!
Also you cannot use a variable that already have been used in the imported module (at least it is not recommended).

You can use a trick to bypass the “run at import” rule:

vi test_module.py

if __name__ == "__main__":
   print "This is my own module"

vi main.py
import test_module

print "This is the main file"

--- Output ---
This is the main file

However when you run the module itself:

python test_module.py
This is my own module

Modules and Packages

There are modules (sometimes called packages) that are not included in your basic Python installation.
There are some build in modules, you can check them here.

Simple example:

import math

print(math.sqrt(4))

--- Output ---
2.0

 

We don’t have to import the whole module. You can import only the one method you need. Notice that in this example, you don’t have to specify the math method after you used “from math import …”

from math import sqrt

print(sqrt(4))

--- Output ---
2.0

 

How to install new packages

If you use windows you can go to cmd and use the:

pip install MODULE-NAME

pip will install the required modul for you.

You can find many modules on the internet ie.:

https://github.com
https://pexpect.readthedocs.io

Creating your own modules

A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py

create hello.py then write the following function:

def helloworld():
print "hello"
>>> import hello
>>> hello.helloworld()
'hello'
>>>

To group many .py files put them in a folder. Any folder with an __init__.py is considered a module by python and you can call them a package

|-HelloModule

|_ __init__.py

|_ hellomodule.py

You can go about with the import statement on your module the usual way.

For more information: http://docs.python.org/2/tutorial/modules.html#packages

 

 

Errors and Exceptions Handling

Errors detected during execution are called exceptions and are not unconditionally fatal.Even if a statement or expression is syntactically correct, it may cause an error when an attempt is made to execute it.

It is useful when you are expecting possible errors in your code and you can receive notifications but still, keep running the code.

The code which can cause an exception to occur is put in the try block and the handling of the exception is implemented in the except block.

Syntax:

try:
   You do your operations here...
except ExceptionI:
   If there is ExceptionI, then execute this block.
except ExceptionII:
   If there is ExceptionII, then execute this block
else:
   If there is no exception then execute this block

A simple example:

try:
 2 + 's'
except TypeError:
 print 'There was a type error'
else:
 print 'Else block'
finally:
 print 'Final block'

--- Output ---
There was a type error
Final block

In case the “try” block is executed without error the else + final block will be executed. If there is an error the except + final block will be executed.

You can find the  Python error types here.

If you don’t specify the error type, the except block will be run to any kind of errors:

try:
  f = open('testfile12','r')
  f.write('Test write this')
except:
  print 'Error in writing to the file'
else: 
  print 'Write successfully'

--- Output ---
Error in writing to the file

def askInt():
    while True:
        try:
            val = int(raw_input('Please enter an integer: '))
        except:
            print 'Looks like you did not enter an integer'
            continue
        else:
            break
        finally:
            print 'Finished'
        print(val)
askInt()

--- Output ---
Please enter an integer: five
Looks like you did not enter an integer
Finished
Please enter an integer: I mean 5
Looks like you did not enter an integer
Finished
Please enter an integer: 5
Finished

 

 

Object Oriented Programming

Objects

In Python, everything is an object.

print type(1)
print type([])
print type({})
print type(())

--- Output ---
<type 'int'>
<type 'list'>
<type 'dict'>
<type 'tuple'>

As objects they have specific functions ie.:

#The List object has a method called count()
myList = [1,2,2,2,3,4,5]
print(myList.count(2))

--- Output ---
3

 

 

---

Classes

The user defined objects are created using the class keyword. The class is a blueprint that defines a nature of a future object. From classes we can construct instances.

By convention class names are capitalised.
Classes can have attributes and methods. An attribute is a characteristic of an object while a method is an operation we can perform with the object.

Attributes

Syntax:

self.name_of_attribute = value

We can initialise the attribute of an object with the __init__() method. This method always run first when a new object is created.

class Dog(object):

 #Class Object Attribute
 species = 'mammal'
 
 def __init__(self,breed,name):
    self.breed = breed
    self.name = name
 
 
satsu = Dog(breed='Shiba Inu', name='Satsu')

print(satsu.breed)
print(satsu.species)
print(satsu.name)

--- Output ---
Shiba Inu

You can set a default value for an attribute. This way you don’t have to pass a value for that attribute when creating an object, only if you want to.

...
def __init__(self,breed='Shiba',name)
   self.breed = breed
   self.name = name

satsu = Dog(name='Satsu')

 

Methods

Methods are functions defined inside the body of a class. They are used to perform operations with the attributes of our objects.

class Circle(object):

 #Class object attribute
 pi = 3.14

 def __init__(self,radius=1):
   self.radius = radius
 
 def area(self):
   #radius**2 * pi
   return self.radius**2 *Circle.pi

 def setRadius(self,radius):
   self.radius=radius

 def getRadius(self):
   return self.radius

kor = Circle(5)
print(kor.area())

 

---

Inheritance

Inheritance is a way to form new classes using classes that have already been defined. The newly formed classes are called derived classes, the classes that we derive from are called base classes.
Important benefits of inheritance are code reuse and reduction of complexity of a program. The derived classes (descendants) override or extend the functionality of base classes (ancestors).

class Animal(object):
  def __init__(self):
     print "Animal created"
 
  def whoAmI(self):
     print "Animal"
 
  def eat(self):
     print "Eating"
 
 
class Dog(Animal):
 
  def __init__(self):
     Animal.__init__(self)
     print "Dog created"
 
  #This overwrote the inherrited whoAmI method 
  def whoAmI(self):
     print "Dog"
 
  def bark(self):
     print "Wuf wuf"


d = Dog()
print "----"
d.whoAmI()
print "----"
d.eat()

--- Output ---
Animal created
Dog created
----
Dog
----
Eating

 

---

 

Special Methods

There are some special methods for classes. These can be casted with the following commands:

• __str__ – print(object-name)
• __len__ – len(object-name)
• __del__ – del(object-name)

class Book(object):
 
  def __init__(self,title,author,pages):
     print "A book has been created"
     self.title = title
     self.author = author
     self.pages = pages

 def __str__(self):
     return "Title: %s, Author: %s, Pages: %s" %(self.title, self.author, self.pages)
 
 def __len__(self):
     return self.pages

 def __del__(self):
     print "A book is gone"


b = Book('Python','Jose',100)
print "----------"
print(b)
print "----------"
print(len(b))
print "----------"
del(b)

--- Output ---
A book has been created
----------
Title: Python, Author: Jose, Pages: 100
----------
100
----------
A book is gone

 

 

 

 

Useful functions

• startswith() – Returns true if strings starts with the specified prefix. False otherwise.

Methods and Functions

Methods

Methods are essentially functions built into objects.
Methods will perform specific actions on the object and can also take arguments, just like a function.

object.method(arg1,arg2,etc…)

Functions

A function is a useful device that groups together a set of statements so they can be run more than once. They let us specify parameters that can serve as inputs to the functions.

def name_of_function():
   code

 

A simple example:

def BGP_AD():
  print('eBGP: 20\niBGP: 200')
 
BGP_AD()

--- Output ---
eBGP: 20
iBGP: 200

Passing arguments to function:

def calc(arg1,arg2):
 print(arg1 + arg2)

calc(5, 5)

--- Output --- 
10

Another solution with return():

def add_num(num1,num2):
 return (num1 + num2)

print(add_num(5, 5))

--- Output --- 
10

We can pass strings for arguments also as python do not care about the variable type.

---

Lambda expression

The lambda expression allows us to create “anonymous” functions.
The lambda’s body is a single expression, not a block of statements!

Lambda vs simple function

#Simple function
def square(num):
 result = num**2
 return result

print(square(2))

--- Output --- 
4
----------------------------
#A function that is a bit similar to lambda
def square(num): return num**2

print(square(2))

--- Output --- 
4
----------------------------
# Lambda
square = lambda num: num**2

print(square(2))

--- Output --- 
4

 

The syntax is something like this:
<name of lambda> lambda <argument1>,<argument2> : <commands>

 

---

Name assignments and Scopes

 

When you create a variable name in Python the name is stored in a namespace. Variable names also have a scope, the scope determines the visibility of that variable name to other parts of your code.

#global name assignment
x = 25

def printer():
  #local name assignment
  x = 50
  return(x)

print(x)
print(printer())

--- Output ---
25
50

The name assignments have 4 scopes:

• local – Names assigned in any way within a function (def or lambda), and not declared global in that function.
• enclosing functions – Name in the local scope of any and all enclosing functions (def or lambda), from inner to outer.
• global – Names assigned at the top-level of a module file, or declared global in a def within the file.
• built-in – Names preassigned in the built-in names module: len, open, range, SyntaxError,…

 

name = 'This is global'

def greet():
 #enclosing local name assignment
 name = 'Sammy'
 
 def hello():
    print 'Hello ' + name
 hello()
 
greet()

--- Output ---
Hello Sammy