Research

Programming
Software: An
Overview
Research programming software assists in data processing and analysis. It
spans disciplines from biology to social sciences. Around 67% of researchers
use programming to analyze data effectively and accurately.

Dr. Moutushi Singh

Professor & Head
Department of Computer Science & Engg. & Information Technology
Institute of Engineering & Management,
University of Engineering & Management, Kolkata

[Link]
 Research Software
Research software is a collection of tools, codes, libraries that allows a researcher to generate new data or analyze and make meaning of existing
data. It is a broad term that includes software programs, languages, libraries, scripts, and tools developed to support research. Research software helps
researchers make new discoveries and share their methods with others. Research software is often domain-specific.
Software Types:
✓ Open Source - software in which source code is released under a license in which the copyright holder grants users the rights to study, change, and
distribute the software to anyone and for any purpose. Typically, open-source software is developed in a collaborative public manner.
✓ Commercial - any software or program that is designed and developed for licensing or sale to end users or that serves a commercial purpose.
Commercial software was once considered to be proprietary software, but now a number of free and open-source software applications are
licensed or sold to end users. Off-the-shelf software programs, to be used with little or no modification, are examples of commercial software.
✓ Custom - software solutions that are specially developed for a specific user or organization and may include solutions that have been designed to
run on specific hardware (embedded software). Such software is used to address user needs that cannot be met by more traditional and
widespread off-the-shelf software.

Licensing Types:
✓ Perpetual license - authorizes an individual to use an application indefinitely. Generally, outside of termination, a perpetual software license allows
the holder to use a specific version of a given software program continually with payment of a single fee. Along with the license, the vendor
typically provides a technical support period of one to three years. During this initial period, the vendor also provides software updates often.
However, updates may or may not be provided for free in perpetuity.
✓ Subscription License - authorizes an individual to use an application for a defined period. This might be 30 days (a monthly subscription) or 365 days
(an annual subscription.) Subscriptions typically have no defined end or termination date and they automatically renew after the initial term.
✓ Individually Licensed – Individual Licenses can be based on per machine or per user and typically reflect the use of an application by an individual.
✓ Floating License - here a limited number of licenses for a software application are shared among a larger number of users over time. When an
authorized user wishes to run the application, they request a license from a central license server.
[Link]
Key Functionalities: Data Analysis & Visualization
Here’s a concise list of key functionalities related to Data Analysis & Visualization in common research software:

1. Data Import & Management 2. Data Analysis

•Support for various data formats (CSV, Excel, SPSS, Stata, JSON, etc.) •Descriptive Statistics (mean, median, standard deviation)
•Data cleaning tools (handling missing values, duplicates) •Inferential Statistics (t-tests, ANOVA, chi-square)
•Metadata handling and variable labeling •Regression Analysis (linear, logistic, multivariate)
•Merging and transforming datasets •Time Series Analysis
•Factor & Cluster Analysis
•Hypothesis Testing

3. Data Visualization 4. Advanced Statistical Modelling & Machine Learning

•Customizable charts: bar, line, scatter, pie, histograms, boxplots Predictive modelling (decision trees, random forests, SVMs)
•Heatmaps and correlation matrices Classification and clustering
•Interactive dashboards (in tools like Tableau or Power BI) Neural networks (in Python, R, MATLAB)
•Geographic mapping (GIS tools, R’s ggmap, QGIS) Model validation and performance metrics
•3D plots and animations (in MATLAB, Python’s Plotly)

6. Reporting and Export

5. Scripting and Automation
Generate publication-quality graphs and tables
•Use of programming languages like R, Python, or MATLAB for custom
Export results to PDF, Word, LaTeX, HTML
workflows
Integration with reference managers (Zotero,
•Reproducible code scripts (RMarkdown, Jupyter Notebooks)
EndNote)
•Batch processing

[Link]
Data Analysis & Visualization contd.
7. Software Examples
Software Strengths
R Statistical analysis, data visualization (ggplot2)
Python Versatile; great for ML & visualization (pandas, seaborn, matplotlib, Plotly)
SPSS User-friendly interface for statistical analysis
Stata Econometrics and social sciences
MATLAB Engineering, numeric computation, signal processing
Tableau/Power BI Drag-and-drop data visualization
NVivo Qualitative data analysis

Notes:
R and Python offer the most flexibility and customization, especially for complex
research projects.
SPSS and Stata are excellent for users with less coding experience.
Tableau and Power BI are top choices for creating interactive, shareable dashboards.
NVivo stands out for qualitative data (interviews, open-ended responses).

[Link]
 Comparison Table: Research Software for
Data Analysis & Visualization
Software Type Strengths in Data Analysis Strengths in Visualization Ideal For
Academic research,
R Programming Advanced statistics, modeling, custom analysis Powerful plotting (ggplot2, lattice, plotly)
statistics-heavy work
Machine learning, data manipulation (pandas, Interactive visualizations (matplotlib, Data science, ML, custom
Python Programming
scikit-learn) seaborn, plotly) automation
Easy-to-use interface, basic to advanced Social sciences, survey
SPSS GUI-based Basic charts, limited customization
statistics research
Stata GUI + Command Econometric modeling, time-series analysis Functional but limited visual output Economics, policy research
High-quality plots, 3D graphs, signal/data
MATLAB Programming Numerical computing, matrix operations Engineering, simulations
plots
Interactive, dynamic dashboards and Business intelligence, visual
Tableau GUI-based Some analysis (filters, aggregations)
storytelling reports
Real-time dashboards, good integration Business, organizational
Power BI GUI-based Aggregations, time-based analysis
with MS tools reporting
Qualitative/social science
NVivo GUI-based Qualitative coding, theme identification Word clouds, cluster maps, charts
research
General-purpose, quick
Excel GUI-based Basic statistics, pivot tables Simple charts, conditional formatting
analysis
Basic graphs, better with SAS Visual
SAS Programming + GUI Large-scale statistical analysis, data warehousing Healthcare, large datasets
Analytics
Geographical/Environmenta
QGIS GUI-based Spatial data manipulation Geographic maps, spatial visualizations
l research
[Link]
Key Functionalities: Simulation & Modeling
1. Model Development 2. Simulation Execution
Graphical modelling tools (drag-and-drop system blocks) •Real-time and accelerated-time simulation
Equation-based modelling (differential equations, algebraic systems) •Stochastic (randomized) vs deterministic (fixed
Agent-based modelling (individual-based behaviour simulation) outcome) runs
Discrete-event simulation (event-driven system representation) •Sensitivity analysis (test different inputs)
Continuous simulation (flow-based systems over time) •Multi-run batch processing for scenario testing

4. Result Analysis & Visualization

3. Parameter Control & Input Customization Time series plots (outputs over time)
User-defined variables, constraints, and parameters 2D/3D visualizations (plots, graphs, animation)
Sliders, input fields for real-time adjustments State diagrams and process flows
Import of parameter values from files or databases Heatmaps and distribution charts
Playback of simulation for step-by-step analysis
5. Optimization & Calibration
Goal-seeking algorithms (e.g., genetic algorithms, gradient 6. Integration with Other Tools
descent) Support for MATLAB, Python, R, Excel, or C/C++
Calibration against real-world data Co-simulation (running multiple models/software in
Automated model tuning for best fit sync)
Constraint-based optimization Data export for further statistical analysis

7. Documentation & Reproducibility

Model versioning
Auto-generated reports
Model sharing with metadata
Scripting for reproducibility (e.g., in Simulink or AnyLogic) [Link]
Popular Modeling & Simulation Software
Software Simulation Type(s) Special Strengths Use Case Examples
Block-based modeling, control Engineering, robotics, signal
MATLAB/Simulink Continuous, hybrid
systems processing
Multi-method simulation, interactive Supply chain, logistics,
AnyLogic Agent-based, discrete-event
3D modeling epidemiology
Easy interface for complex behavior
NetLogo Agent-based Ecology, social science
modeling

Arena Discrete-event Process and operations simulation Manufacturing, healthcare

Open-source, equation-based, multi- Mechanical, electrical,

Modelica Physical systems modeling Block-based
Multi-method
Easy
Process
Open-source,
Causal
Free interface
& loop
open-source
and modeling,
operations
modeling,
equation-based,
for
simulation,
complex
alternative
scenario
control Policy
Engineering,
Supply
Mechanical,
planning,
chain,electrical,
robotics,
logistics,
systems signal
Software
MATLAB/Simulink
AnyLogic
NetLogo
Arena
Modelica
Vensim
Stella
OpenModelica
Architect Simulation
Continuous,
Agent-based,
Agent-based
Discrete-event
System
Physicaldynamics
systems
modeling
Type(s)
hybrid domain Visual,
discrete-event
modeling Strengthsmodeling thermodynamic
Special intuitive systems
Use Casesocial
Ecology,
Manufacturing,
Education,
Engineering,
Examples
business
energy
science
healthcare
strategy
systems
systems
interactive
behavior
simulation
multi-domain
analysis
to Modelicamodeling
3D modeling processing
epidemiology
thermodynamic
thinking systems
Causal loop modeling, scenario Policy planning, systems
Vensim System dynamics
analysis thinking
Stella Architect System dynamics Visual, intuitive modeling Education, business strategy
Free & open-source alternative to
OpenModelica Physical modeling Engineering, energy systems
Modelica
Python (SimPy, PySD, Full control, custom simulations via
Various Flexible, research prototyping
Mesa) code

[Link]
Key Functionalities:
Automation & Scripting
Automate Data Tasks
Collect and clean data efficiently.

Scripting Languages
Python, R, and MATLAB are popular choices.

Use Case
Python scripts scrape research papers swiftly.

Impact
Data processing time reduced by 40%.

[Link]
Automation & Scripting contd.
Automation in research is pivotal in today's fast-paced environment. It involves using technology strategically to streamline processes, enhance
efficiency, and boost accuracy in data collection and analysis. This shift from traditional to automated methods has transformed how data is
gathered and analyzed, ensuring speed, precision, and scalability while eliminating errors. Automating research workflows offers numerous
advantages, including enabling researchers to focus on interpretation rather than repetitive tasks, delivering timely and precise insights, and
ultimately enhancing productivity and effectiveness in research endeavors, fostering innovation and excellence.

Benefits of Automation in Research:

❖ Automation has transformed studies, boosting productivity. Automated processes cut human error, enhancing accuracy for reliable
results. This accuracy elevates data quality, ensuring consistent findings for informed decisions.
❖ Research automation streamlines studies in diverse fields. Automation tools in market research expedite data collection, analysis, and
reporting, enhancing the research process. Qualitative research benefits from automation by spotting patterns in large datasets, offering
valuable insights into consumer behavior. The adaptability of automation tools in various research areas underscores their role in
enhancing results.
❖ In various sectors, automation transforms research processes. In healthcare, it speeds up data collection, ensures accurate patient
information, and boosts treatment outcomes. Environmental science benefits from real-time monitoring through automation, aiding
conservation. Automating tasks like data cleaning enables more in-depth analysis, providing richer insights.
❖ Automation in research revolutionizes productivity, accuracy, and innovation across diverse fields. Real-world case studies prove its
benefits, reshaping research by offering a more efficient approach for impactful insights.

Current Trends in Automation for Research Methodologies:

Automation in research is advancing rapidly with cutting-edge technologies like artificial intelligence and machine learning revolutionizing
processes by analyzing data and identifying patterns. This synergy between human intellect and automation is driving unprecedented
advancements. Adoption of automation tools is predicted to rise across sectors, with a growing demand for streamlined workflows, improved
accuracy, and enhanced productivity among researchers.
[Link]
Automation & Scripting contd.
1. Script-Based Workflow Automation 4. Batch Processing & Loop Execution
Automate repetitive tasks (data import, cleaning, analysis, visualization) Execute scripts over multiple files or datasets (looping)
Schedule simulations or batch analyses Automate model runs with varying parameters (simulation,
Create pipelines from raw data to final output (e.g., Jupyter, R scripts) sensitivity analysis)
Export results programmatically (e.g., batch plots, reports)

2. Custom Function & Macro Creation 5. Integration & Interoperability

Define reusable functions, macros, or modules Script-based integration between tools (e.g., Python with
Automate UI interactions (e.g., macros in Excel or SPSS syntax) Excel, R with SPSS)
Parameterize workflows (custom inputs without editing code) API usage to connect with web services, databases, or remote
tools
Embed scripting in larger software (e.g., Python in Power BI,
VBA in Excel)

3. Reproducibility & Version Control 6. Automated Reporting & Output Generation

Re-run the same process with consistent outputs Generate reports dynamically (PDF, HTML, Word)
Integration with Git or other version control systems Auto-generate graphs, tables, and dashboards
Documented and shareable code notebooks (e.g., R Markdown, Jupyter) Use of templates and markdown for automation

[Link]
Automation & Scripting contd.
7. Supported Scripting Languages by
Tool

Tool Scripting Language(s) Automation Strength

Python (Jupyter) Python Full-featured; ideal for automation, ML, and pipelines

R (RStudio) R, RMarkdown Statistical automation, reproducible research

MATLAB MATLAB Scientific scripting, simulations

SPSS SPSS Syntax, Python, R Automating statistical tasks
Stata Stata Do-files, Mata Looping, macros, reproducibility
Excel VBA, Python (via plug-ins) Custom macros, form automation
SAS SAS Scripting, Macro Language Complex data processing, batch jobs
Power BI DAX, Power Query (M), Python/R Data transformation, embedded scripts

Tableau Tableau Calc, TabPy (Python) Custom calculations and external script execution

Benefits of Scripting & Automation:

❖ Saves time and reduces manual errors
❖ Ensures reproducibility and consistency
❖ Allows for scalable and complex workflows
[Link]
Software Spotlight: R and RStudio
R Language CRAN Ecosystem RStudio IDE
Open-source with rich Thousands of packages for Enhances productivity with powerful tools.
statistical capabilities. diverse analyses. Used by 70% of statisticians worldwide.
R was created for statistical analysis and data visualization. It started in the early 1990s when researchers needed a tool that could handle large datasets, run
complex computations and display results clearly in graphs and charts. R provides a user-friendly environment and when used with tools like RStudio, we can easily
run analyses, create reproducible research documents with R Markdown and build statistical models.

What is R Programming Language?

R is a tool for data manipulation and analysis, it is both a programming language and a working environment.
R is designed to perform statistical calculations and generate visual data representations. It includes a wide
array of statistical models, such as regression, time-series analysis, clustering and advanced machine
learning techniques.
Its built-in functions make it easy to work with tables, matrices and arrays, so we can quickly clean data,
change its format and summarize it. It has a large collection of add-on packages like ggplot2 for making
graphs, dplyr for organizing data and caret for building simple machine learning models.
Why Use R Language?
➢ R is free and open for everyone to use.
➢ It was built for working with data, making it easy to clean and analyze numbers.
➢ R comes with many tools to create clear graphs and charts.
➢ There are many add-on packages available.
➢ It works well with other software and data formats, making it a flexible choice.

Features of R Programming Language:

❖ Cross-Platform Support: R works on Windows, Mac and Linux, so we can use it no matter what computer we have.
❖ Easy-to-Learn Syntax: The language is designed with a simple style that helps beginners pick it up quickly.
❖ Data Exploration: We can try out code and see results right away, which makes it easy to test ideas.
❖ Community Resources: There are many forums, tutorials and help guides available when we need assistance.
❖ Regular Updates: New features and packages are added often, keeping R up-to-date with the latest tools.
❖ Prototyping: It lets us quickly set up and test data models before running them on larger datasets. [Link]
Software Spotlight: Python with SciPy, NumPy,
Pandas
Python Language SciPy Library NumPy & Pandas
Versatile and widely used for data Enables scientific and technical • Numerical array computing
science projects. computing tasks. • Efficient data handling and
manipulation

[Link]
Python with SciPy, NumPy, Pandas………..
Overview
Python is a high-level, open-source programming language that supports scientific computing, data analysis, machine learning, and automation
through a rich ecosystem of libraries. Among these, NumPy, Pandas, and SciPy form a core trio for numerical and analytical work.
1. NumPy (Numerical Python)
Key Functionalities: Example:
➢ Multidimensional array object (ndarray) for efficient numerical computation
import numpy as np
➢ Vectorized operations (fast math on arrays without explicit loops)
a = [Link]([[1, 2], [3, 4]])
➢ Linear algebra, Fourier transforms, and random number capabilities inverse = [Link](a)
➢ Foundation for other libraries (e.g., Pandas, TensorFlow, SciPy)
2. Pandas (Panel Data Library)
Key Functionalities: Example:
➢ Powerful DataFrame and Series objects for tabular data
import pandas as pd
➢ Data cleaning, filtering, transformation df = pd.read_csv('[Link]')
➢ Grouping, pivot tables, and time series analysis df_grouped = [Link]('category')['value'].mean()
➢ Easy import/export (CSV, Excel, SQL, JSON)
3. SciPy (Scientific Python)
Key Functionalities: Example:
➢ Built on NumPy, provides algorithms for:
from scipy import stats
•Optimization
•Integration data = [1, 2, 3, 4, 5]
•Interpolation t_stat, p_val =
•Signal processing stats.ttest_1samp(data, 3)
•Statistical analysis
•Sparse matrix operations [Link]

➢ Modules: [Link], [Link], [Link], etc.

contd………..
Combined Workflow Use Case:
➢ Load & clean data with Pandas
➢ Compute stats or matrix ops with NumPy
➢ Run optimization or analysis with SciPy
➢ Visualize results with libraries like Matplotlib or Seaborn

Strengths
➢ Open-source and free
➢ Highly extensible (integrates with ML, visualization, web apps)
➢ Supported by Jupyter for interactive coding

Considerations
➢ Requires coding knowledge
➢ Not GUI-based (steeper learning curve for beginners)
➢ Best when used with version control (e.g., Git)

[Link]
Software Spotlight: MATLAB
1 Numerical Computing
Proprietary environment for matrix-based computations.

2 Toolboxes
Specialized for signal processing, image analysis, and more.

3 Simulink
Graphical environment for modeling and simulation.

4 Use Cases
Ideal for algorithm development and prototyping.

[Link]
 Overview:
MATLAB is a proprietary, high-performance programming platform developed by MathWorks, designed primarily
for engineering, scientific computing, and system simulation. It is built around matrix and array mathematics and
is known for its robust toolboxes and intuitive visualization capabilities.

Key Functionalities:
3. Data Analysis & Visualization
1. Numerical Computing & Matrix Operations
➢ Import data from Excel, CSV, sensors, databases
➢ Core strength in matrix manipulation
➢ Built-in plotting functions (2D/3D, surface, contour, histogram)
➢ Advanced linear algebra and calculus
➢ Interactive dashboards, GUIs, and animations
➢ Signal processing, control systems, and FFT
➢ Time series analysis and curve fitting
➢ Built-in solvers for algebraic and differential equations
Example: Example:
A = [1, 2; 3, 4]; x = 0:0.1:10;
invA = inv(A); % Inverse of matrix A y = sin(x);
plot(x, y);
2. Modeling & Simulation (Simulink)
➢ Simulink: graphical environment for model-based 4. Toolboxes (Domain-Specific Add-ons)
design ➢ Signal Processing Toolbox
➢ Simulate dynamic systems (mechanical, electrical, ➢ Image Processing Toolbox
control systems) ➢ Control System Toolbox
➢ Block diagrams, flowcharts, and physical modeling ➢ Statistics and Machine Learning Toolbox
➢ Real-time testing and hardware-in-the-loop (HIL) ➢ Optimization Toolbox
simulation ➢ Deep Learning Toolbox
Toolboxes provide prebuilt functions, apps, and models tailored for specific fields.
[Link]
 Common Use Cases

5. Automation & Scripting Domain Example Applications

➢ Script files (.m files) to automate workflows Control systems, signal processing,
Engineering
➢ Function definitions, loops, and conditional logic robotics
➢ Batch processing of simulations or analyses Physics & Math Numerical modelling, simulations
➢ GUI development with App Designer
Finance Risk modelling, forecasting
Education Simulation-based learning
6. Integration & Interoperability
➢ Interfacing with C/C++, Java, Python, and Excel Processing, filtering, feature
Image & Audio
➢ MATLAB Engine for Python (run MATLAB from Python) extraction
➢ Export reports and code to LaTeX, Word, or PDF
➢ Simulink integration with hardware (Arduino, Raspberry Pi, etc.)

Strengths
➢ Intuitive for engineers and scientists
➢ Excellent documentation and support
➢ Ideal for prototyping, testing, and educational use
➢ Strong visualization and modeling tools

Limitations
➢ Commercial license required (not open-source)
➢ Less flexible than Python for web/ML deployment
➢ May be overkill for small-scale data analysis [Link]
Software Spotlight: Specialized Tools

NVivo SAS Mathematica

Qualitative data analysis and research Advanced statistical analysis and Powerful symbolic and numerical
organization. business intelligence. computation.

[Link]
Trends and Future
Directions

AI-powered Low-code Cloud

Assistants Platforms Environments
Enhance coding speed Enable research with Facilitate collaboration
and accuracy. minimal programming and scalability.
effort.

Big Data
Integration
Handle massive
datasets seamlessly.
[Link]
Choosing the Right Software
Consider Goals
Match software to research objectives and expertise.

Budget & Trials

Explore free trials and open-source options.

Support & Docs

Strong community and documentation aid success.

Maintainability
Choose tools that sustain long-term projects.

[Link]