Get highest frequency terms from Lucene index
Solution 1
Have a look at this: http://sujitpal.blogspot.com/2009/02/summarization-with-lucene.html
The class in this page hascomputeTopTermQuery
method which you should be easily able to retrofit for going over multiple indexes.
Solution 2
A very simple way would be to use Luke. On the 'Overview' tab, there is a 'Show top terms' button that can be used for what you need.
Related videos on Youtube
Julia
Updated on June 04, 2022Comments
-
Julia almost 2 years
i need to extract terms with highest frequencies from several lucene indexes, to use them for some semantic analysis.
So, I want to get maybe top 30 most occuring terms(still did not decide on threshold, i will analyze results) and their per-index counts. I am aware that I might lose some precision because of potentionally dropped duplicates, but for now, lets say i am ok with that.
So for the proposed solutions, (needless to say maybe) speed is not important, since I would do static analysis, I would put accent on simplicity of implementation because im not so skilled with Lucene and cant wrap my mind around some concepts of it..
I can not find any code samples from something similar, so all concrete advices (code, pseudocode, links to code samples...) Appreciate all the advices!
Thank you!
-
Julia almost 14 yearsThanx! Exactly what I needed!
-
Thangnv over 10 yearsHi mindas! I using lucene 4.4 so it don't have terms() method ? help me, please!
-
mindas over 10 years@Thangnv you might want to open a separate thread for this, or email Lucene Java mailing list. I don't have Lucene 4.4 running on hand. And the time has been on short supply these days ;-(