Indian Legal Text Summarization: A Text

Normalisation-based Approach
Satyajit Ghosh Mousumi Dutta Tanaya Das
Department of CSE Department of CSE Department of CSE
Adamas University Adamas University Adamas University
Kolkata, India Kolkata, India Kolkata, India
[Link]@[Link] mousumidutta@[Link] tanayadas.das23@[Link]
arXiv:2206.06238v1 [[Link]] 13 Jun 2022

Abstract—In the Indian court system, pending cases have long use those tools and techniques for other countries. To train
been a problem. There are more than 4 crore cases outstand- a model for domain-specific text summarization, we need
ing. Manually summarising hundreds of documents is a time- a lot of data. As there is no publicly available dataset for
consuming and tedious task for legal stakeholders. Many state-of-
the-art models for text summarization have emerged as machine summarization of Indian legal documents present, so we have
learning has progressed. Domain-independent models don’t do proposed a different methodology to summarize them. Text
well with legal texts, and fine-tuning those models for the Indian summarization techniques are classified into two categories.
Legal System is problematic due to a lack of publicly available An abstractive summarization recognises the language in the
datasets. To improve the performance of domain-independent text and adds novel words to the summary if necessary [7],
models, the authors have proposed a methodology for normalising
legal texts in the Indian context. The authors experimented [8] and in extractive summarization, a summary is formed
with two state-of-the-art domain-independent models for legal from a subset of sentences [9]. In this paper, The Indian legal
text summarization, namely BART and PEGASUS. BART and texts have been normalised as a general text based on our
PEGASUS are put through their paces in terms of extractive proposed methodology. Next, two domain-independent models
and abstractive summarization to understand the effectiveness of have been used for summarization. The authors have tried
the text normalisation approach. Summarised texts are evaluated
by domain experts on multiple parameters and using ROUGE to conduct extractive summarization using BART [10] and
metrics. It shows the proposed text normalisation approach is abstractive summarization with PEGASUS [11]. Section II
effective in legal texts with domain-independent models. presents the related work behind the proposed methodology.
Index Terms—Legal text summarization, Natural Language Section III discusses a detailed explanation of the proposed
Processing, BART, PEGASUS methodology. Section IV presents the evaluation of the work
compared to the traditional one. Finally, Section V concludes
I. I NTRODUCTION the paper along with its future applications.
Text summarization is the process of constructing a concise,
cohesive, and fluent summary of a lengthy text document [1]. II. R ELATED W ORK
It gives us a brief context of the story. Statutes (established Legal case judgements are usually lengthy and complicated
laws) and Precedents (prior cases) are the two primary sources due to the use of many domain-specific abbreviations [2].
of law for countries that follow the Common Law System There is a vast amount of research conducted for Legal text
like India [2]. Hence, there are hundreds of prior cases that summarization in different countries like the US, Canada,
lawyers must go through. Legal documents can be lengthy. Australia, and the UK. Most of the research works tried to train
The abbreviations and terminology used in Legal documents deep-learning models using a supervised or semi-supervised
are different from the standard language. Manual drafting of approach for legal text summarization. J. W. Yingjie and Ma
case summaries is a process that takes a lot of time. Automatic in [12] selected three sentences from each topic that are having
summarization of texts is possible because of the advancement best representation of the topic. The proposed summarization
of Artificial Intelligence (AI) and Machine Learning (ML). method blends term description with sentence description
Thousands of hours of man-labour can be reduced with the for each topic using LSA (Latent Semantic Analysis). B.
help of State-of-the-Art Machine Learning models. It is also Samei et al. in [13] introduced a model for multi-document
useful for beginners and ordinary citizens to understand a summarization using graph-based and information-theoretic
judgement. More than 4.70 crore cases are pending in various concepts. A. Farzindar and G. Lapalme in [14] proposed
courts in India [3]. The use of automatic summarization of legal documents summarization based on the exploration of
texts will also significantly reduce the number. Several legal the documents’ architecture and thematic structures. A. Joshi
text summarization techniques and tools have been reported et al. in [15] describes a summary based on the “three-
in the past based on UK [4], Canadian [5] and Australian sentence selection” metrics. These are “content relevance”,
[6] court judgements. As each country has its structure and “sentence novelty”, and “sentence position relevance”. The
abbreviations in their Legal documents it is not suitable to sentence content relevance is measured using a deep auto-
encoder network. S. Polsley et al. in [16] introduced a tool TABLE I
which takes advantage of standard summary methods based on R AW AND N ORMALISED T EXTS
word frequency and generates automated text summarization Example 1
of the legal documents. The tool is evaluated using Recall- Raw Legal Text “. . . they approached the High Court of Kerala by
Oriented Understudy for Gisting Evaluation (ROUGE) and way of W.P. (C) No. 2329 of 2014.”
Normalised Text “. . . they approached the High Court of Kerala by
human scoring. Vijayasanthi et al. in [17] proposed a hybrid way of Writ Petition under 226 and 227 of the
system for automatic text summarization of legal documents. Constitution No. 2329 of 2014.”
Key phrase matching and case-based techniques are involved Example 2
Raw Legal Text “. . . petition filed by the appellant under Section 13 of
in the hybrid system. M. Saravanan and B. Ravindran in Hindu Marriage Act,1955 on grounds of desertion.”
[18] proposed a system for labelling sentences with their Normalised Text “. . . petition filed by the appellant under Section 13
rhetorical roles. The application of probabilistic models for (for divorce) of Hindu Marriage Act,1955 on grounds
of desertion.”
extraction of the key sentences is described by them. N. Bansal
et al. in [19] introduced a Fuzzy Analytical Hierarchical
Process (FAHP) based feature weighting scheme for producing
the Legal abbreviations with their full form and appended
summaries of legal judgements. They found it to be more
the article and section mentions with its summary in the
promising than other traditional approaches. State-of-the-Art
documents. Table I displays the raw legal texts received from
domain-independent models are not tried in the past research
the documents and after step 4 their normalised versions.
for legal text summarization. The objective is to find out the
After normalisation, the text is divided into small fragments
effectiveness of State-of-the-Art domain-independent models
in step 5, and then those fragments are given to the model
for domain-specific tasks like legal case summarization. Its
in step 6. If the entire document is supplied at once, the
findings will help in legal and other sectors where the models
model is unable to understand and extract key points from
can’t be properly trained or fine-tuned due to a lack of
it. Thus the inputs are provided in small fragments to these
data. In the next section, the authors discuss the detailed
models. Those outputs later merged for the actual summarised
methodology for text summarization using State-of-the-Art
[Link] have used two models. The first one is BART. It
domain-independent models.
is used for extractive summarization. BART is described as a
III. M ETHODOLOGY denoising autoencoder which is implemented as a sequence-to-
The public records of the Indian judiciary are disorganised sequence model and a bi-directional encoder [10]. BART can
and noisy [20]. There is no publicly available dataset of Indian be used in many downstream applications. K. M. Hermann et
legal documents summarization. So, the authors proposed a al. in [22] proposed a new methodology to fine-tune BART for
novel methodology without the need for a dataset. Comprehension, Translation and Natural Language Generation
with very less amount of training data. The model is built
for summarization in English texts based on that paper. The
second model is PEGASUS. It is a pre-trained model with
Document Text Extraction and Removing
Collection Cleaning Additional Data
Text Normalization
extracted Gap-sentences for abstractive summarization. Good
Step 2 Step 3 Step 4 abstractive summarization performance can be achieved across
Step 1

Model Model
a broad domain with very little supervision by fine-tuning
Summarized
Output
Model
Input
Fragmentation
PEGASUS [11]. Next, in step 7 the model outputs for all the
Document Merge Model
small fragments are merged and in step 8 finally, we get the
Step 8
summarised document. The model outputs at the next stage are
Step 7 Step 6 Step 5
evaluated by different Legal experts on multiple parameters.
IV. E VALUATION
Fig. 1. Block diagram of the proposed methodology
The traditional way of evaluating summaries is to use
the ROUGE scores or expert evaluation. We have calculated
Fig.1. depicts our proposed methodology. In step 1 we
ROUGE-1, ROUGE-2, ROUGE-3 and ROUGE-L scores . To
collected the Indian Legal documents from sources such as
calculate the scores we have compared model summaries gen-
SCI, IndianKanoon, Manupatra and ILDC [21]. In step 2, we
erated from raw texts and normalised texts with the summaries
have extracted the texts using Optical Character Recognition
provided by Legal experts.
(OCR) from the documents. Next, the noise is removed from
the extracted text using basic clean up techniques e.g., White-
TABLE II
space removal, Spelling correction. After that, in step 3 the AVERAGE ROUGE SCORES AFTER COMPARING N ORMALISED TEXT
additional information at the beginning before the actual
judgement is removed. Then the Legal texts are normalised Type Precision Recall F-Score
ROUGE-1 0.46 0.55 0.48
in step 4. We have constructed a dictionary with Legal abbre- ROUGE-2 0.28 0.36 0.31
viations and their full form. A separate dictionary is created ROUGE-3 0.21 0.27 0.24
for articles, sections, and their summary. We have replaced ROUGE-L 0.35 0.45 0.39
 TABLE III TABLE IV
AVERAGE ROUGE SCORES AFTER COMPARING R AW TEXT S UMMARY FROM R AW AND N ORMALISED L EGAL T EXTS

Type Precision Recall F-Score Example 1

ROUGE-1 0.45 0.4 0.42 Raw Texts Summary “The road was constructed but no com-
ROUGE-2 0.26 0.24 0.25 pensation was paid. The Panchayat denied
ROUGE-3 0.18 0.17 0.18 of having given any assurance regarding
ROUGE-L 0.33 0.3 0.31 adequate compensation to be paid to the
appellants.”
Normalized Texts Summary “The road was constructed but no com-
Table II and Table III shows the performances of summariz- pensation was paid. The appellants made
various representations starting from the
ing the samples in terms of ROUGE-1, ROUGE-2, ROUGE- time, construction was going on and even
3 and ROUGE-L Precision, Recall and F-scores. Comparing after the construction work was com-
these two tables we can observe that average scores on all the pleted. When no heed was paid to their
request, they approached the High Court
three parameters are increased in normalised texts. The com- of Kerala.”
parison is done using BART outputs as PEGASUS is evaluated Example 2
poorly by legal experts which are discussed below. Perfect Raw Texts Summary “The Family Court ruled that the marriage
scores for extractive summarization are both theoretically and between the parties is irretrievably broken
down. The appeal is also pending before
computationally very difficult to achieve using ROUGE [23]. this Court for the last 12 years (@ SLP
Thus we have also evaluated our results by experts. Random of the year 2010). The appellant is stated
samples have been taken from our data sources and as per our to have got married after the decree of
divorce was granted.”
proposed methodology, those samples are processed. We have Normalized Texts Summary “The Family Court ruled that the marriage
sent those outputs to Legal experts for evaluation. We asked between the parties is irretrievably broken
them to evaluate the summaries based on three parameters on down. The appeal is also pending before
this Court for the last 12 years (@ Special
a scale of 1 to 10. The parameters are conciseness, accuracy, Leave Petition of the year 2010). The
and detail preservation. PEGASUS model is only able to appellant is stated to have got married
summarize the contents with more than 7-point conciseness after the divorce was granted.”
and accuracy in 20% samples. In some samples, it gives
completely out of context summaries. In the case of BART on
As shown in Fig. 3. on average 75% decrease in the length
over 60% of the samples, it received more than a 7-point score
of the normalized text is observed by summarization using
in all the 3 parameters from the Legal experts. Fig.2. depicts
BART from our samples.
average scores given by Legal experts on different parameters
in our samples.

Normalized Text Summarized Text

100%

10
9 9 99 9 9
9 75%
8 8
8
7 7 7 7 7
7
Length

6 6 6 50%

6
5 5
5
4 4 25%
4
3 3 3
3
2 22
2 0%
Sample 1 Sample 2 Sample 3 Sample 4 Sample 5

1
0 000
0
Fig. 3. Summarized vs. Normalized Legal Text Length
PEGASUS

PEGASUS

PEGASUS

PEGASUS

PEGASUS
BART

BART

BART

BART

BART

Sample 1 Sample 2 Sample 3 Sample 4 Sample 5 It can significantly improve the efficacy of the legal system.
Conciseness Accuracy Detail Preservation Beginners and common people can also take advantage of it
to understand different aspects of a legal case without any
Fig. 2. Evaluation by Legal Experts specific domain knowledge.

Both normalised text and raw Legal texts are provided to V. C ONCLUSION
the BART model and the summary outputs are then shown In this paper, the authors have experimented with two State-
to naive persons. The summaries from the normalised texts of-the-Art machine learning models for Indian Legal text
are evaluated as more conscious and easy to understand by summarization. BART performed well in summarizing Legal
them. Table IV compares the raw and normalised legal text texts and decreased the length of the document up to 75%.
summaries. PEGASUS is used for abstractive summarization, but it did not
work well most of the time. Thus, the normalization methodol- [22] K. M. Hermann, T. Kocisky, E. Grefenstette, L. Espeholt, W. Kay, M. Su-
ogy is effective for extractive summarization but certainly not leyman, and P. Blunsom, “Teaching machines to read and comprehend,”
vol. 28, Curran Associates, Inc., 2015.
that much useful for abstractive summarization. The ROUGE [23] N. Schluter, “The limits of automatic summarisation according to rouge,”
metrics and expert evaluation show that even without domain- pp. 41–45, Association for Computational Linguistics, 4 2017.
specific training State-of-the-Art machine learning models can
be used for various domain-specific fields after normalizing
the raw texts. All the supplementary files are available at
[Link]
ACKNOWLEDGMENT
We are grateful to Prasenjit Das from the Department of
Law, Adamas University and Souvik Ghosh from Amity Law
School, Kolkata for evaluating our results.
R EFERENCES
[1] M. C. Popescu, L. Grama, and C. Rusu, “On the use of positive definite
symmetric kernels for summary extraction,” pp. 335–340, 2020.
[2] Kaustubh, R. Subham, P. Nilay, G. Kripabandhu, G. S. B. Paheli, and
Hiware, “A comparative study of summarization algorithms applied to
legal case judgments,” pp. 413–428, Springer International Publishing,
2019.
[3] PTI, “Over 4.70 crore cases pending in various courts: Govt,” 3 2022.
[4] C. Grover, B. Hachey, I. Hughson, and C. Korycinski, “Automatic
summarisation of legal documents,” pp. 243–251, 2003.
[5] A. Farzindar and G. Lapalme, “Letsum, an automatic legal text summa-
rizing,” vol. 120, p. 11, 2004.
[6] S. Polsley, P. Jhunjhunwala, and R. Huang, “Casesummarizer: a system
for automated summarization of legal texts,” pp. 258–262, 2016.
[7] C. Zhu, “Chapter 8 - applications and future of machine reading
comprehension,” 2021.
[8] N. Moratanch and S. Chitrakala, “A survey on abstractive text summa-
rization,” pp. 1–7, 2016.
[9] N. Moratanch and S. Chitrakala, “A survey on extractive text summa-
rization,” pp. 1–6, 2017.
[10] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy,
V. Stoyanov, and L. Zettlemoyer, “Bart: Denoising sequence-to-sequence
pre-training for natural language generation, translation, and comprehen-
sion,” 10 2019.
[11] J. Zhang, Y. Zhao, M. Saleh, and P. J. Liu, “Pegasus: Pre-training with
extracted gap-sentences for abstractive summarization,” 12 2019.
[12] J. W. Yingjie and Ma, “A comprehensive method for text summariza-
tion based on latent semantic analysis,” pp. 394–401, Springer Berlin
Heidelberg, 2013.
[13] B. Samei, M. Estiagh, M. Eshtiagh, F. Keshtkar, and S. Hashemi, “Multi-
document summarization using graph-based iterative ranking algorithms
and information theoretical distortion measures,” 2014.
[14] A. Farzindar and G. Lapalme, “Legal text summarization by exploration
of the thematic structure and argumentative roles,” pp. 27–34, 2004.
[15] A. Joshi, E. Fidalgo, E. Alegre, and L. Fernández-Robles, “Summcoder:
An unsupervised framework for extractive text summarization based
on deep auto-encoders,” Expert Systems with Applications, vol. 129,
pp. 200–215, 2019.
[16] S. Polsley, P. Jhunjhunwala, and R. Huang, “Casesummarizer: A system
for automated summarization of legal texts,” pp. 258–262, The COLING
2016 Organizing Committee, 12 2016.
[17] Vijayasanthi, P. R. G. S. V., B. R. K. S. Deepthi, and Puli, “An auto-
matic legal document summarization and search using hybrid system,”
pp. 229–236, Springer Berlin Heidelberg, 2013.
[18] M. Saravanan and B. Ravindran, “Identification of rhetorical roles
for segmentation and summarization of a legal judgment,” Artificial
Intelligence and Law, vol. 18, pp. 45–76, 2010.
[19] N. Bansal, A. Sharma, and R. K. Singh, “Fuzzy ahp approach for legal
judgement summarization,” Journal of Management Analytics, vol. 6,
pp. 323–340, 2019.
[20] V. Parikh, V. Mathur, P. Mehta, N. Mittal, and P. Majumder, “Lawsum: A
weakly supervised approach for indian legal document summarization,”
10 2021.
[21] V. Malik, R. Sanjay, S. K. Nigam, K. Ghosh, S. K. Guha, A. Bhat-
tacharya, and A. Modi, “Ildc for cjpe: Indian legal documents corpus
for court judgment prediction and explanation,” 5 2021.