Python

Python is compiled as well as interpreted programming language invented by

Guido van Rossum , a Dutch scientist in 1989.
We can never download Python in our computers. We all download Python
compilers and interpreters which is available in Python implementations.
Python implementation
Python implementation means tools which you need to work in Python and the
tools are
a) Python Compiler
b) Python Interpreter
That means
Python implementation = "Python compiler + Python interpreter"

Cpython
The default implementation of the Python programming language is Cpython.
As the name suggests Cpython is written in C language. Cpython compiles the
python source code into intermediate bytecode, which is executed by the
Cpython virtual machine.

IronPython
A Python implementation written in C# targeting Microsoft’s .NET framework.
Similar to Jython, it uses .Net Virtual Machine i.e Common Language Runtime.
PyPy
PyPy is an implementation of the Python programming language written in
Python.
“If you want your code to run faster, you should probably just use PyPy.”
Interpreted or compiled?
Python is considered as both compiled as well as interpreted language, as it
involves a compilation step that converts the python code into bytecode which
is stored with a .pyc extension that gets deleted once the program is
executed. Bytecode is also binary representation executed by virtual machine
(not by CPU directly). The virtual machine (which is written different for
different machines) converts binary instruction into a specific machine
instruction.

[Link][source code] compile -> [Link][byte code] run -> PVM interpreter
and memory manager

Compile once run anywhere or any platform

Various applications of Python

 Artificial Intelligence and Machine Learning
It is heavily used in Face recognition, music recommendation on
youtobe,medical data,Driverless Cars,Google's Speech Recognition etc.
 Some of the popular web sites are designed in Python are:
NASA,Instagram,Udemy,Spotify,Mozilla,Dropbox
Youtube
The major part of youtube has been developed in Python.
  Hacking
"Hackers" generally develop "small scripts" and Python provides
"amazing performance" for small programs.

 Data Science
you collect data from the business
with the help of that data you decide how will you grow your business in
future. Numpy and Pandas are leading libraries of Data Science in Python.
 Internet Of Things [IOT]

Features of Python
1. simple
2. dynamically typed language
3. supports both POP and OOP
4. robust - exceptions
5. compiled as well as interpreted
Compilation part is hidden from the programmer , so mostly people say
that it is an "interpreted langauage" which is not correct.
6. cross platform language
7. Extensible
Python allows us to call C/C++/Java/Dot Net code from a Python code
and thus we say it is an "Extensible" language.
8. Huge library
it can help you do various things like "Database Programming","E-
mailing","GUI Programming" etc.
How does Python work?

Deleting a file in python:

import os, shutil

[Link]("[Link]")

[Link]("empty-dir")

[Link]("nonempty-dir")

Is Python Compiled or interpreted?

The compilation part is done first when we

execute our code and this will generate byte code
and internally this byte code gets converted by the
python virtual machine(PVM) according to the
underlying platform(machine+operating system).
Now the question is – if there is any proof that
python first compiles the program internally and
then run the code via interpreter?
The answer is yes! and note this compiled part is
get deleted by the python(as soon as you execute
your code) just it does not want programmers to
get into complexity.

What is PIP?
PIP [Preferred Installer Program] is a package manager
for Python packages, or modules.

Why don’t we need a amin function in python?

In python code is executed from top to bottom from
top of the file unlike other languages java ,C++ etc
If you want to organize your execution of code use
‘def ‘ function to do so
main function is not required because the Python
interpreter executes from the top of the file unless a
specific function is defined with the keyword "def".
# Define a function to calculate the area of a rectangle
def calculate_rectangle_area(length, width):
area = length * width
return area

# Define a function to calculate the area of a circle

def calculate_circle_area(radius):
pi = 3.14159
area = pi * radius**2
return area

# Main code starts here

print("Welcome to the Area Calculator!")

# Calculate the area of a rectangle

rectangle_length = 5
rectangle_width = 3
rectangle_area = calculate_rectangle_area(rectangle_length,
rectangle_width)
print("The area of the rectangle is:", rectangle_area)

# Calculate the area of a circle

circle_radius = 2.5
circle_area = calculate_circle_area(circle_radius)
print("The area of the circle is:", circle_area)

Output:

vbnet
Welcome to the Area Calculator!
The area of the rectangle is: 15
The area of the circle is: 19.6349375

Inputs
name=input("Enter your name")
print("Name entered is\t",name)
age=input("Enter your age")
print("Age entered is\t",age)
# age+=10 # TypeError: can only concatenate str (not "int") to str
age= int(age)
age+=10
print("Age after 10 years will be\t",age)

this give type error because if we add 10 to the age

which is a text means different datatype so we cannot
concatenate them

Datatypes in python?
 Numeric
o Int
o Float
o Bool
o Complex
  Sequence
o List
o Tupple
o Set
o String
o Range
o Dictionary
Types of data conversion in python:
Implicit Type Conversion in Python

In Implicit type conversion of data types in Python, the Python interpreter automatically
converts one data type to another without any user involvement.

x = 10

print("x is of type:",type(x))

y = 10.6
print("y is of type:",type(y))

z =x +y

print(z)
print("z is of type:",type(z))

Output

x is of type: <class 'int'>

y is of type: <class 'float'>
20.6
z is of type: <class 'float'>
Explicit Type Conversion in Python

In Explicit Type Conversion in Python, the data type is manually changed by the user as per
their requirement. With explicit type conversion, there is a risk of data loss since we are
forcing an expression to be changed in some specific data type.

# initializing string
s = "10010"

# printing string converting to int base 2

c = int(s,2)
print ("After converting to integer base 2 : ", end="")
print (c)

# printing string converting to float

e = float(s)
print ("After converting to float : ", end="")
print (e)

Output:

After converting to integer base 2 : 18

After converting to float : 10010.0

Python Type conversion using ord(), hex(), oct()

ord(): This function is used to convert a character to an integer.

hex(): This function is to convert an integer to a hexadecimal string.
oct(): This function is to convert an integer to an octal string.

# initializing integer
s = '4'

# printing character converting to integer

c = ord(s)
print ("After converting character to integer : ",end="")
print (c)

# printing integer converting to hexadecimal string

c = hex(56)
print ("After converting 56 to hexadecimal string : ",end="")
print (c)

# printing integer converting to octal string

c = oct(56)
print ("After converting 56 to octal string : ",end="")
print (c)

Output:
After converting character to integer : 52
After converting 56 to hexadecimal string : 0x38
After converting 56 to octal string : 0o70

Python Type conversion using tuple(), set(), list()

tuple(): This function is used to convert to a tuple.

set(): This function returns the type after converting to set.
list(): This function is used to convert any data type to a list type.

# initializing string
s = 'geeks'

# printing string converting to tuple

c = tuple(s)
print ("After converting string to tuple : ",end="")
print (c)

# printing string converting to set

c = set(s)
print ("After converting string to set : ",end="")
print (c)

# printing string converting to list

c = list(s)
print ("After converting string to list : ",end="")
print (c)

Output:

After converting string to tuple : ('g', 'e', 'e', 'k', 's')

After converting string to set : {'k', 'e', 's', 'g'}
After converting string to list : ['g', 'e', 'e', 'k', 's']

What is debugging and breakpoint in python?

Debugging is a process of finding errors (a.k.a. bugs) in
code or software programs.
Breakpoint is something that allows you to debug your
program by stopping execution at certain point in your
code.
Conditional breakpoints allow you to break inside a
code block when a defined expression evaluates to
true. Conditional breakpoints highlight as orange
instead of blue. Add a conditional breakpoint by right
clicking a line number, selecting Add Conditional
Breakpoint , and entering an expression.

Time Complexity:

Time complexity measures the amount of time an algorithm takes to complete as a function
of the size of its input. It is expressed using Big O notation. Common time complexities
include:

O(1) - Constant Time:

o The algorithm's runtime does not depend on the size of the input.
o Examples: Accessing an element in an array, inserting at the beginning of a linked
list.

O(log n) - Logarithmic Time:

o The runtime grows logarithmically with the size of the input.

o Examples: Binary Search, finding an element in a sorted list.

O(n) - Linear Time:

o The runtime is directly proportional to the size of the input.

o Examples: Linear Search, traversing an array or linked list.

O(n log n) - Linearithmic Time:

o Common for efficient sorting algorithms like Merge Sort and Heap Sort.
o Examples: Merge Sort, Heap Sort.
 O(n^2), O(n^3), ... - Polynomial Time:

o The runtime grows with the square, cube, etc., of the size of the input.
o Examples: Bubble Sort, Insertion Sort (inefficient for large datasets).

O(2^n), O(n!) - Exponential Time:

o Algorithms with this complexity are generally inefficient and should be

avoided for large inputs.
o Examples: Recursive algorithms without memoization, such as naive recursive
Fibonacci.

Space Complexity:

Space complexity measures the amount of memory an algorithm uses as a function of the size
of its input. Similar to time complexity, it is expressed using Big O notation.

O(1) - Constant Space:

o The algorithm uses a constant amount of memory regardless of the input size.
o Examples: Variables, a fixed-size array.

O(n) - Linear Space:

o The amount of memory used grows linearly with the size of the input.
o Examples: Arrays, linked lists.

O(n^2), O(n^3), ... - Polynomial Space:

o The algorithm uses memory proportional to the square, cube, etc., of the input
size.
o Examples: 2D arrays.

Other Complexity Measures:

I/O Complexity:

 For algorithms heavily dependent on input/output operations, the I/O complexity can
be considered.
 File reading/writing algorithms, network communication algorithms.

Cache Complexity:

 Algorithms that efficiently utilize CPU caches can have better performance. Cache
complexity considers cache hits and misses.
 Examples: Matrix multiplication optimized for cache usage.
Iterator in Python:

In Python, an iterator is an object that enables iteration (looping) over a sequence of elements. It
implements the iterator protocol, which consists of two methods: __iter__() and __next__().
Here's a simple definition:

The iterator protocol consists of two methods:

1. __iter__(): This method returns the iterator object itself. It is required for an object
to be considered an iterator.
2. __next__(): This method returns the next value from the iterator. If there are no
more items to return, it should raise the StopIteration exception.
Size Comparison The length of a tuple is fixed, whereas the length of a list is variable. Therefore,
lists can have a different sizes, but tuples cannot. Tuples are allocated large blocks of memory with
lower overhead than lists because they are immutable; whereas for lists, small memory blocks are
allocated.

Tuples are allocated large blocks of memory with lower overhead than lists because they are
immutable; whereas for lists, small memory blocks are allocated. Thus, tuples tend to be faster than
lists when there are a large number of elements.

*args and **kwargs.

The args stands for arguments that are passed to the function whereas kwargs stands for keyword
arguments which are passed along with the values into the function.

1. *args (Non-keyword Arguments/Positional Arguments) –

*args allows a function to accept any number of positional arguments i.e. arguments that are
non-keyword arguments, variable-length argument list.

Positional Arguments are those arguments that do not contain any keyword related to the data
value. For example, age = 38. Age is a keyword associated with data value 38. In Python
*args, we do not define any keywords to the arguments while passing values to the function.

 We use the single-asterisk (*) symbol for passing it as a parameter to the functions.
 args is just a word, we need to use the (*) symbol before it. We can use any
alphabet/word instead of args.
 *args accepts all the parameters passed to the function and computes the function
operations for each of them.
 2. **kwargs (Keyword Arguments) –

**kwargs is a special syntax that allows us to pass a variable length of keyword arguments
to the function.

One limitation of Python *args is that it does not accept any keyword arguments. Consider
the above example, the program will print all the values passed to the function. But how will
we identify what London is? Is it the city he lives in? Is it the work location? Is it a state?
What is 38? Is it age, salary, person_id? For such an issue, Python has a special symbol called
as **kwargs.

**kwargs allows us to pass a variable number of keyworded arguments to the function.

Python *kwargs allows only Keyword Arguments.

 We use a double-asterisk (**) before the parameter name in the function argument.
 Just like args, kwargs is just another idiom, we can use any other name but we need
to use the (**) symbol before it.
 Keyword arguments are like a dictionary, which maps the value to its associated key.

[Link] between *args and **kwargs.

 *args in function definitions are used to pass a variable number of arguments to a

function when calling the function. By using the *, a variable associated with it
becomes iterable.
 **kwargs in function definitions are used to pass a variable number of keyworded
arguments to a function while calling the function. The double star allows passing any
number of keyworded arguments.

[Link] is the difference between “is” and “==”?

Python's “is” operator checks whether two variables point to the same object. “==” is used to
check whether the values of two variables are the same.

E.g. consider the following code:

a = [1,2,3]

b = [1,2,3]

c=b

a == b

evaluates to true since the values contained in the list a and list b are the same but

a is b

evaluates to false since a and b refers to two different objects.

c is b

Evaluates to true since c and b point to the same object.

[Link] is memory managed in Python?

Memory in Python exists in the following way:

 The objects and data structures initialized in a Python program are present in a private
heap, and programmers do not have permission to access the private heap space.
 You can allocate heap space for Python objects using the Python memory manager.
The core API of the memory manager gives the programmer access to some of the
tools for coding purposes.
 Python has a built-in garbage collector that recycles unused memory and frees up
memory for heap space.

[Link] is a decorator?

A decorator is a tool in Python which allows programmers to wrap another function around a
function or a class to extend the behavior of the wrapped function without making any
permanent modifications to it. Functions in Python are first-class objects, meaning functions
can be passed or used as arguments. A function works as the argument for another function in
a decorator, which you can call inside the wrapper function.

[Link] lookups faster with dictionaries or lists in Python?

The time complexity to look up a value in a list in Python is O(n) since the whole list iterates
through to find the value. Since a dictionary is a hash table, the time complexity to find the
value associated with a key is O(1). Hence, a lookup is generally faster with a dictionary, but
a limitation is that dictionaries require unique keys to store the values.

[Link] can you return the binary of an integer?

The bin() function works on a variable to return its binary equivalent.

[Link] can you remove duplicates from a list in Python?

A list can be converted into a set and then back into a list to remove the duplicates. Sets do
not contain duplicate data in Python.

E.g.

list1 = [5,9,4,8,5,3,7,3,9]

list2 = list(set(list1))

list2 will contain [5,9,4,8,3,7]

Set() may not maintain the order of items within the list.
 [Link] is the difference between append and extend in Python?

The argument passed to append() is added as a single element to a list in Python. The list
length increases by one, and the time complexity for append is O(1).

The argument passed to extend() is iterated over, and each element of the argument adds to
the list. The length of the list increases by the number of elements in the argument passed to
extend(). The time complexity for extend is O(n), where n is the number of elements in the
argument passed to extend.

Consider:

list1 = [“Python”, “data”, “engineering”]

list2 = [“projectpro”, “interview”, “questions”]

[Link](list2)

List1 will now be : [“projectpro”, “interview”, “questions”, [“Python”, “data”,

“engineering”]]

The length of list1 is 4.

Instead of append, use extend

[Link](list2)

List1 will now be : [“projectpro”, “interview”, “questions”, “Python”, “data”, “engineering”]

The length of list1, in this case, becomes 6.

[Link] do you use pass, continue and break?

The break statement in Python terminates a loop or another statement containing the break
statement. If a break statement is present in a nested loop, it will terminate only the loop in
which it is present. Control will pass the statements after the break statement if they are
present.

The continue statement forces control to stop the current iteration of the loop and execute the
next iteration rather than terminating the loop completely. If a continue statement is present
within a loop, it leads to skipping the code following it for that iteration, and the next
iteration gets executed.

Pass statement in Python does nothing when it executes, and it is useful when a statement is
syntactically required but has no command or code execution. The pass statement can write
empty loops and empty control statements, functions, and classes.

[Link] can you check if a given string contains only letters and numbers?
[Link]() can be used to check whether a string ‘str’ contains only letters and numbers.

[Link] some advantages of using NumPy arrays over Python lists.

 NumPy arrays take up less space in memory than lists.

 NumPy arrays are faster than lists.
 NumPy arrays have built-in functions optimized for various techniques such as linear
algebra, vector, and matrix operations.
 Lists in Python do not allow element-wise operations, but NumPy arrays can perform
element-wise operations.

[Link] Pandas, how can you create a dataframe from a list?

import pandas as pd

days = [‘Sunday’, ‘Monday’, ‘Tuesday’, ‘Wednesday’]

# Calling DataFrame constructor on list

df = [Link](days)

df is the data frame created from the list ‘days’.

df = [Link](days, index =[‘1’,’2’,’3’,’4’], columns=[‘Days’])

Can be used to create the data frame and the values for the index and columns.

[Link] Pandas, how can you find the median value in a column “Age” from
a dataframe “employees”?

The median() function can be used to find the median value in a column. E.g.-
employees[“age”].median()

[Link] Pandas, how can you rename a column?

The rename() function can be used to rename columns of a data frame.

To rename address_line_1 to ‘region’ and address_line_2 to ‘city’

[Link](columns=dict(address_line_1=’region’, address_line_2=’city’))

[Link] can you identify missing values in a data frame?

The isnull() function help to identify missing values in a given data frame.

The syntax is [Link]()

It returns a dataframe of boolean values of the same size as the data frame in which missing
values are present. The missing values in the original data frame are mapped to true, and non-
missing values are mapped to False.

[Link] is SciPy?

SciPy is an open-source Python library that is useful for scientific computations. SciPy is
short for Scientific Python and is used to solve complex mathematical and scientific
problems. SciPy is built on top of NumPy and provides effective, user-friendly functions for
numerical optimization. The SciPy library comes equipped with functions to support
integration, ordinary differential equation solvers, special functions, and support for several
other technical computing functions.

[Link] a 5x5 matrix in NumPy, how will you inverse the matrix?

The function [Link]() can help you inverse a matrix. It takes a matrix as the input
and returns its inverse. You can calculate the inverse of a matrix M as:

if det(M) != 0

M-1 = adjoint(M)/determinant(M)

else

"Inverse does not exist

[Link] is an ndarray in NumPy?

In NumPy, an array is a table of elements, and the elements are all of the same types and you
can index them by a tuple of positive integers. To create an array in NumPy, you must create
an n-dimensional array object. An ndarray is the n-dimensional array object defined in
NumPy to store a collection of elements of the same data type.

[Link] NumPy, create a 2-D array of random integers between 0 and 500
with 4 rows and 7 columns.

from numpy import random

x = [Link](500, size=(4, 7))

[Link] all the indices in an array of NumPy where the value is greater
than 5.

import NumPy as np

array = [Link]([5,9,6,3,2,1,9])

To find the indices of values greater than 5

print([Link](array>5))

Gives the output (array([0,1,2,6])

Data Engineer Interview Questions on Azure

Most businesses are switching to cloud infrastructure these days. Organizations employ a
variety of providers including AWS, Google Cloud, and Azure for their BI and Machine
Learning applications. Microsoft Azure allows data engineers to build and deploy
applications using various solutions. Check out these common data engineer interview
questions on various Microsft Azure concepts, tools, and frameworks.

76. Explain the features of Azure Storage Explorer.

 It's a robust stand-alone application that lets you manage Azure Storage from any
platform, including Windows, Mac OS, and Linux.
 An easy-to-use interface gives you access to many Azure data stores, including ADLS
Gen2, Cosmos DB, Blobs, Queues, Tables, etc.
 One of the most significant aspects of Azure Storage Explorer is that it enables users
to work despite being disconnected from the Azure cloud service using local
emulators.

77. What are the various types of storage available in Azure?

In Microsoft Azure, there are five storage types classified into two categories.

 The first group comprises Queue Storage, Table Storage, and Blob Storage. It is
built with data storage, scalability, and connectivity and is accessible through a REST
API.
 The second group comprises File Storage and Disk Storage, which boosts the
functionalities of the Microsoft Azure Virtual Machine environment and is only
accessible through Virtual Machines.
 Queue Storage enables you to create versatile applications that comprise independent
components depending on asynchronous message queuing. Azure Queue storage
stores massive volumes of messages accessible by authenticated HTTP or HTTPS
queries anywhere.
 Table Storage in Microsoft Azure holds structured NoSQL data. The storage is
highly extensible while also being efficient in storing data. However, if you access
temporary files frequently, it becomes more expensive. This storage can be helpful to
those who find Microsoft Azure SQL too costly and don't require the SQL structure
and architecture.
 Blob Storage supports unstructured data/huge data files such as text documents,
images, audio, video files, etc. In Microsoft Azure, you can store blobs in three ways:
Block Blobs, Append Blobs, and Page Blobs.
  File Storage serves the needs of the Azure VM environment. You can use it to store
huge data files accessible from multiple Virtual Machines. File Storage allows users
to share any data file via the SMB (Server Message Block) protocol.
 Disk Storage serves as a storage option for Azure virtual machines. It enables you to
construct virtual machine disks. Only one virtual machine can access a disk in Disk
Storage.

78. What data security solutions does Azure SQL DB provide?

In Azure SQL DB, there are several data security options:

 Azure SQL Firewall Rules: There are two levels of security available in Azure.

 The first are server-level firewall rules, which are present in the SQL Master database
and specify which Azure database servers are accessible.
 The second type of firewall rule is database-level firewall rules, which monitor
database access.

 Azure SQL Database Auditing: The SQL Database service in Azure offers auditing
features. It allows you to define the audit policy at the database server or database
level.
 Azure SQL Transparent Data Encryption: TDE encrypts and decrypts databases
and performs backups and transactions on log files in real-time.
 Azure SQL Always Encrypted: This feature safeguards sensitive data in the Azure
SQL database, such as credit card details.
Exception handling
Errors interrupt the flow of the program at the point where they appear, so any further code stops
executing. This error is called an exception.

Exception handling allows you to separate error-handling code from normal code. An exception is a
Python object which represents an error. As with code comments, exceptions helps you to remind
yourself of what the program expects. It clarifies the code and enhances readability.

The try block lets you test a block of code for errors.

try:
print(x)
except:
print("An exception occurred")

The except block lets you handle the error.

try:
print(x)
except NameError:
print("Variable x is not defined")
except:
print("Something else went wrong")

You can use the else keyword to define a block of code to be executed if no errors were raised:

try:
print("Hello")
except:
print("Something went wrong")
else:
print("Nothing went wrong")

The finally block, if specified, will be executed regardless if the try block raises an error or not.

try:
print(x)
except:
print("Something went wrong")
finally:
print("The 'try except' is finished")
The raise keyword is used to raise an exception.

As a Python developer you can choose to throw an exception if a condition occurs.

To throw (or raise) an exception, use the raise keyword.

Example

Raise an error and stop the program if x is lower than 0:

x = -1

if x < 0:
raise Exception("Sorry, no numbers below zero")