COMP 448/548: Medical Image Analysis
Introduction
Çiğdem Gündüz Demir
cgunduz@[Link]
Outline for today
§ Lecture 1: Introduction
§ Course outline
§ Course details
§ Coursework
Course objectives
§ Today, imaging systems are
extensively used in medicine MR scan showing
tumor in brain
SPECT-CT bone scan
showing joint inflammation
Fundus photography for
diabetic retinopathy diagnosis
and biology research FISH to
determine HER2
§ Their primary use is to protein status in
breast cancer
visualize human body at
different levels Skin photography
for melanoma
Corneal in vivo confocal
microscope photography of
screening endothelial cells for diagnosis
§ Images are visually analyzed
by clinicians/biologists to
make decisions
§ These analyses rely on visual
interpretations/judgments Micrograph of HE
stained biopsy for colon
Micrograph of
peripheral blood smear
Micrograph of IHC stained biopsy
to localize tumor buddings, a
adenocarcinoma grading for leukemia diagnosis prognostic indicator for cancer
Then, why do we need computers?
1. Some tasks are time-consuming Find the ratio of
senescent cells in this
Count endothelial micrograph belonging
cells in this to an in vitro
corneal in vivo experiment
confocal
micrography
Localize tumor buddings Do more …
Keep doing …
Then, why do we need computers?
2. There are too many cases, but limited manpower
Analyze many X-ray and CT images to diagnose covid-19 infected patients
• In theranostic evaluation of pancreatic neuroendocrine tumors (PanNETs), the proliferative activity of the tumor cells
has been one of the most valuable parameters.
• Ki-67 labeling index, the most widely-used and most reliable method to count proliferative activity has become a part
of all the guidelines for pathologic evaluation.
• In the oncology literature, 10% is being increasingly used as the cut-off for placing the patient for closer surveillance
and possible adjuvant therapy.
Then, why do we need computers?
3. There may be variations in image interpretations among different clinicians
(inter-observer variability)
Which ones do you
call chair and which
ones armchair?
Then, why do we need computers?
3. There may be variations in image interpretations among different clinicians
(inter-observer variability)
Which ones do you
call chair and which
ones armchair?
4. Moreover, interpretations rely on qualitative visual analyses, but no
quantitative metrics, which may make consistent decision-making difficult
(intra-observer variability)
Then, why do we need computers?
3. There may be variations in image interpretations among different clinicians
(inter-observer variability)
4. Moreover, interpretations rely on qualitative visual analyses, but no
quantitative metrics, which may make consistent decision-making difficult
(intra-observer variability)
Now grade cancer in tissues below based on the degree of their deformations.
This is how a normal colon Low-grade or high-grade?
tissue looks like
HER2 gene amplification
tested by FISH. Red
signals: HER2 gene
copies, green signals:
chromosome enumeration
probe 17 copies.
Then, why do we need computers?
1. Some tasks are time-consuming
2. There are too many cases, but limited manpower
3. There may be variations in image interpretations among different clinicians
(inter-observer variability)
4. Moreover, interpretations rely on qualitative visual analyses, but no
quantitative metrics, which may make consistent decision-making difficult
(intra-observer variability)
Thus, there is need for (semi)automated systems for medical image
analysis as they facilitate rapid analyses with better reproducibility
Is there any relation between the
phenotype and the existence of a
particular mutation?
Are immune cells important?
Computational analysis tools
§ The goal is to go from medical images to understanding
Diagnosis
Screening
Treatment planning
Prognosis
Computational analysis tools
§ The goal is to go from medical images to understanding
for different medical imaging modalities
§ Pathology
§ Radiology
§ Nuclear medicine
§ Ophthalmology Lecture 2:
Imaging modalities
§ Dermatology
§ Microscopy
§ …
Computational analysis tools
§ For computers to understand images, they must
§ Represent an image with mathematical features and
§ Use these features in the design of their algorithms Rest of the lectures:
Computer vision & Computational
Handcrafted feature
extraction
machine learning techniques for
algorithms
Input image Output/result
feature extraction
Deep learning aims to learn features directly on image data and algorithm design
§ Most commonly studied algorithms are for
segmentation and classification
Lecture 3:
§ However, feature extraction and algorithm design Design pipeline
might be, indeed most of the time are, challenging and challenges
Course objective and outcomes
§ The objective of this course is to provide you with the basic
concepts and computational and mathematical methods in
medical image analysis
§ By the end of this course, you will be equipped with:
§ Basic understanding of medical imaging modalities
§ Familiarity with medical image analysis applications and their challenges
§ Traditional image processing techniques for medical image analysis, and
§ Deep learning for medical image analysis
Course outline
Week Subject Homework
1 Introduction, an overview of imaging modalities
2 An overview of applications and challenges in medical image analysis.
Medical image segmentation – traditional methods
3-4 (Preprocessing and postprocessing: Filters for medical image enhancement, mathematical morphology,
distance transforms. Thresholding, clustering, segmentation techniques. Segmentation evaluation.)
Medical image classification – traditional methods
5 (Handcrafted feature extraction: morphological, textural, and structural features. Basics of classifiers and HW1 out
neural networks.)
Medical image classification – deep learning
6-7 (Basics of convolutional neural networks. Convolutional neural networks for medical image classification.
Graph neural networks for tissue classification. Transfer learning.)
Medical image segmentation – deep learning
(Encoder-decoder networks, semantic segmentation in biopsy images, tumor/organ segmentation in CT HW2 out
8-9
images, retinal-layer segmentation in OCT images. Multi-task networks for instance segmentation in
microscopy images. Loss functions. Mixed supervision.)
10 Generative models for medical image synthesis, reconstruction and artifact correction in radiologic images.
Self-supervised learning for medical image analysis. Multiple instance learning with applications in whole HW3 out
11
slide pathology image analysis.
12 Foundation models for medical images. Multimodal models for integrated diagnosis
13-14 Case studies from real-world applications.
Course details
§ Instructor : Çiğdem Gündüz Demir
ENG 108A
cgunduz@[Link]
Office hours: through Zoom, by appointment
§ TAs : Selahattin Cansız, scansiz22@[Link]
Onur Çakı, ocaki13@[Link]
Office hours: through Zoom, by appointment
§ Lectures : MoWe 8:30AM – 10:40AM
§ Textbook : No required textbook
Course slides and extra material
will be available
Prerequisites
§ Basic background in calculus, probability, and linear algebra
§ Programming ability with any programming language
§ Basic knowledge of Python would be helpful
§ No knowledge of computer vision and machine learning is necessary
Grading Policy
§ Homework: 30% (3 homework assignments), individually or in a group of two
§ Survey: 10%, working in a group of two or three
§ Term project: 20%, working in a group of two or three
§ Final: 30%, individually
§ Attendance and participation: 10%
Attendance and Participation: To pass the course, students must attend at least 70 percent of the
classes. Failure to meet this requirement (missing 10 or more lectures) will result in an automatic
F, regardless of performance in other components. Additionally, attendance and active
participation will account for 10 percent of the total course grade.
Academic Integrity: This course follows the Koç University Policy on Academic Integrity and the
Rules and Regulations of the Higher Education Council. Violations of these rules in the exam,
survey, project or homework assignments will be punished subject to these policies.
Using AI Tools (ChatGPT or other similar tools): Everybody should carefully read and understand
the Koç University Policy Statement Regarding the Use of Artificial Intelligence Tools. Other uses
will be considered as cheating. [Link]
Coursework
Homework assignments and late policy
§ Homework assignments will be posted on the course website
§ This will be small programming assignments that help you
understand how you can apply the methods that you will learn
in class to real-world applications
§ For the late assignments, 10 percent of the grade will be
deducted per day after the assignment’s due date
§ You may work individually or in a group of two
Coursework
Survey
§ You will work in a group of two and three
§ Each group will prepare a survey on the topic of their interest
(of course related to medical image analysis) by
§ Reading at least 8-10 scientific papers and
§ Writing a short report (maximum of 3 pages including citations)
Coursework
In your survey report,
§ Give the problem/topic definition
§ Discuss the motivation behind the studies working on this problem/topic (just try to
answer the question of "why have all these studies worked on this problem? is it really
important?"), and then explain the studies
§ While explaining the studies, do NOT list the studies and do NOT explain them one by
one. Instead, understand the contribution and methodology of each study, try to group
the studies according to their contributions and methodologies, and then explain/discuss
the studies as groups (like writing a good introduction section to a scientific paper)
§ The quality of the survey as well as those of the selected papers will affect your grade
(select good papers published in prestigious conferences and journals)
§ Additionally, the format, structure, and writing style of your report (including writing the
citations properly) will be a part of your grade
Coursework
Project
§ You will also work in a group of two or three, preferably with the same
groupmates
§ Each group is expected to select a publicly available medical image dataset
and to apply the methods discussed in class for solving the problem defined
on the selected dataset
§ Check [Link]
§ There are more publicly available datasets, search for one you will be interested in
§ Quality of your selection and the methods applied will affect your grade
§ At the end of the semester, as a group, you will write a report
(maximum of 5 pages)
Coursework
In your project report,
§ Give the details of your selected dataset and the problem defined on this
dataset
§ Explain the methods that you will have applied together with the
experimental results you will have obtained using these methods
§ Discuss and interpret these results, explaining the advantages and limitations
of these methods on your selected dataset
§ The content of your report as well as its format, structure, and writing style
will affect your grade
Thank you!
Next lecture:
An overview of imaging modalities