How fast lucene

See the original article here. Thanks for visiting DZone today,. Edit Profile. Sign Out View Profile. Over 2 million developers have joined DZone. Lucene's indexing is fast! So how exactly is this different than a commit? Thanks for that nice article! Hi Pascal, A commit must also fsync the new files, so that the bits are all guaranteed to be on stable storage, which can be extremely costly seconds. Hi Mike, Great post. Please let me know, is there an e-mail address I can contact you in private?

Mike - I've looked at your code and bought LIA. So I hope I earned a question or two : The NRTManager - does it work well in the scenario that you open one IndexWriter, use that mostly for indexing in one thread, but occasionally for deleting documents in another, and use the IndexSearcher provided here for all else?

That appears to be yes, and indeed its primary purpose but the code confuses me in one respect: The "real questions" - NRTManager never closes any of the old IndexReader or IndexSearcher instances. Isn't this a problem? And didn't we always get told best practice was to reuse the indexsearcher :. Hi mjb, You can of course ask questions without buying the book! Note also that the next Lucene release 3. So the readers are getting closed and we have good stress tests in Lucene that should fail if they were not getting closed Well I guess if the node is read only that's one reason, but is there another.

Not a dumb question; it's a great one! I'm planning to do a blog post explaining the difference Otherwise, use SearcherManager. For example say you use Lucene to search discussion threads on a forum. When user X adds a new comment on a discussion thread, you then go and add that comment into the Lucene index.

But then when user X searches, rather than just pulling the "current" NRT reader, which may not yet reflect this user's added comment, you should pull a reader that's guaranteed to reflect the change from the comment the user just added. For this tying certain searches to certain changes in the index , you need to use NRTManager.

I recently looked at www. I read a lot about Funnelback as well but not sure about their solution either. Mike - this might be clearer in the docs. Everything mostly worked as expected. A searcher with false passed didn't see the deletes until after the sleep the nrtbackground thread is set to about 30 s.

But it surprised and confused me that the searcher two ALSO didn't see the deletes until after the sleep, unless you force the issue with maybeReopen true. I guess I would have thought the getSearcherManager true would have automagically forced a separate IndexSearcher in this case? Do you follow me? Code got mangled: Here it is java. Apache Lucene Java. Pages Blog. Space shortcuts Meeting notes. Page tree. Browse pages.

A t tachments 0 Page History People who can view. Copy Page Tree. Pages Home Old Moin wiki. Jira links. How to make searching faster Here are some things to try to speed up the seaching speed of your Lucene application. Be sure you really need to speed things up. Many of the ideas here are simple to try, but others will necessarily add some complexity to your application.

So be sure your searching speed is indeed too slow and the slowness is indeed within Lucene. Make sure you are using the latest version of Lucene. Use a local filesystem. Remote filesystems are typically quite a bit slower for searching. If the index must be remote, try to mount the remote filesystem as a "readonly" mount. In the case of a title Field, the field name is title and the value is the title of that content item.

Indexing in Lucene thus involves creating Documents comprising of one or more Fields, and adding these Documents to an IndexWriter. Searching requires an index to have already been built. Lucene has its own mini-language for performing searches.

Read more about the Lucene Query Syntax. The Lucene query language allows the user to specify which field s to search on, which fields to give more weight to boosting , the ability to perform boolean queries AND, OR, NOT and other functionality.