Multi-word

Report
A comparative study of TF*IDF, LSI and
multi-words for text classification
Presenter : JIAN-REN CHEN
Authors
: Wen Zhang, Taketoshi Yoshida, Xijin Tang
2011.ESWA
Intelligent Database Systems Lab
Outlines
 Motivation
 Objectives
 Methodology
 Experiments
 Conclusions
 Comments
Intelligent Database Systems Lab
Motivation
Although TF*IDF, LSI and multi-word have been proposed for a
long time, there is no comparative study on these indexing
methods, and no results are reported concerning their
classification performances.
Intelligent Database Systems Lab
Objectives
• A comparative study of TF*IDF, LSI and multi-words
for text classification.
- information retrieval
- text categorization
• indexing term:
 semantic quality
 statistical quality
Intelligent Database Systems Lab
Methodology - TF*IDF
1)
2)
3)
4)
wi,j : the weight for term i in document j
N : the number of documents in the collection
tfi,j : is the term frequency of term i in document j
dfi : is the document frequency of term i in the collection
Terms (keywords) of the document collection
documents
Intelligent Database Systems Lab
Methodology - LSI
Given a term-document matrix X = [x1 , x2 , ... , xn ] є Rm
and suppose the rank of X is r, LSI decomposes the X using SVD as follows:
1.
2. Xk=Uk’ΣkVkT’
Terms (keywords) of the document collection
documents
Intelligent Database Systems Lab
Methodology - Multi-word
its occurrence frequency
should be at least twice in
a document.
the length of the multi-word
should be between 2 and 6
Intelligent Database Systems Lab
Experiments - Datasets
Chinese corpus: TanCorpV1.0
14150 documents
20 categories
Select
1200 documents
agriculture
5,468,301
individual words
219,115 sentences
history
politics
economy
English corpus: Reuters-22173 distribution 1.0
22173 documents
135 categories
Select
2032 documents
Crude (520)
50,837 sentences
agriculture (574)
Trade (514)
281,111
individual words
Interest (424)
Intelligent Database Systems Lab
Experiments - Evaluation
Intelligent Database Systems Lab
Experiments - Chinese
Intelligent Database Systems Lab
Experiments - English
Intelligent Database Systems Lab
Experiments – t-test
Intelligent Database Systems Lab
Comparison
information
retrieval
TF*IDF
Chinese
LSI
English
multi-word
text
categorization
computation
complexity
O(n m)
best
O(n2r3)
O(ms2)
Intelligent Database Systems Lab
Conclusions
• LSI can produce better indexing in discriminative
power.
• LSI and multi-word have better semantic quality
than TF*IDF, and TF*IDF has better statistical quality
than the other two methods.
• The number of dimension is still a decisive factor for
indexing when we use different indexing methods for
classification.
Intelligent Database Systems Lab
Comments
• Advantages
- Compare with TF*IDF, LSI and multi-words
• Disadvantage
- semantic quality and statistical quality are considered
merely by our intuition instead of theory
• Applications
- text mining
Intelligent Database Systems Lab

similar documents