Here's a breakdown of the differences between "Python," "Anaconda," "Module,"
"Package," and "Library":
1. Python:
- Definition: Python is a high-level, general-purpose programming language known for its
readability and versatility. It is widely used for various purposes such as web development, data
analysis, artificial intelligence, automation, and more.
2. Anaconda:
- Definition: Anaconda is a popular distribution of Python that includes many commonly used
packages for data science, machine learning, and scientific computing. It also comes with the
`conda` package manager for managing environments and packages.
3. Module:
- Definition: In Python, a module is a single file containing Python code. It can define functions,
classes, and variables. Modules are used for code organization and reusability, and they can be
imported and used in other Python files.
4. Package:
- Definition: A package in Python is a way to organize related modules into a directory hierarchy.
Packages include an `__init__.py` file that signifies the directory as a package. They allow you to
group modules under a common namespace.
5. Library:
- Definition: In Python, a library is a collection of packages and modules that provide related
functionality. Libraries are designed to be used together to solve specific problems or provide
specific features. They can consist of multiple packages and modules.
In summary:
- Python is the programming language itself.
- Anaconda is a distribution of Python that includes many pre-installed packages for data science
and scientific computing.
- Module is a single file containing Python code.
- Package is a collection of related modules organized in a directory structure.
- Library is a collection of related packages and modules designed to provide specific
functionality.
Lab 1 preprocess example
1. basic text cleaning by converting text to lowercase and removing special characters and numbers:
import re
def clean_text(text):
# Convert text to lowercase
text = [Link]()
# Remove special characters and numbers
text = [Link](r'[^a-zA-Z\s]', '', text)
return text
# Example text
text = "This is an Example TEXT with special characters like @#$ and numbers 1234."
cleaned_text = clean_text(text)
print(cleaned_text)
1.1. Lowercasing text:
text = "THIS IS AN EXAMPLE TEXT fOR LOWERCASING."
lowercased_text = [Link]()
print(lowercased_text)."
lowercased_text = [Link]()
print(lowercased_text)
2. Removing special characters:
import re
text = "This is an example text with special characters like @#$ and numbers 1234."
cleaned_text = [Link](r'[^a-zA-Z\s]', '', text)
print(cleaned_text)
3. Tokenization:
from [Link] import word_tokenize
def tokenize_text(text):
tokens = word_tokenize(text)
return tokens
# Example long text
long_text = """
Natural language processing (NLP) is a subfield of artificial intelligence that focuses on the
interaction between computers and humans using natural language.
It involves the development of algorithms and models that enable computers to understand and
generate human language. NLP has applications in various fields such as machine translation,
sentiment analysis, and chatbots.
"""
tokens = tokenize_text(long_text)
print(tokens)
4. Removing stopwords:
from [Link] import stopwords
text = "This is an example text with some stopwords."
stop_words = set([Link]('english'))
filtered_text = ' '.join(word for word in [Link]() if [Link]() not in stop_words)
print(filtered_text)
5. Normalization using NLTK:
import nltk
text = "The quick brown fox jumps over the lazy dog."
print(text)
6. Stemming using NLTK:
from [Link] import PorterStemmer
stemmer = PorterStemmer()
word = "running"
stemmed_word = [Link](word)
print(stemmed_word)
Remember to install NLTK (`pip install nltk`) and download the necessary resources using
`[Link]()` for stopwords and the Porter Stemmer if you haven't already.