Mailing List
Home
Forum Home
Maven - Project building tool
Axis - Java SOAP implementation
Lucene - Full-featured text search engine APIs
Cocoon - MVC web framework based on XML/XSL
Fop - Create PDF, PCL, PS, SVG, XML driven by XSL formatting objects.
Log4J - A log library
POI - Java Excel, Word and other Microsoft Office files manipulating library
Oracle database error code ...
Subjects
log4j warning: No appenders could be found
java security AccessControlException: access denied (java io FilePermission clie
java lang InstantiationException: org apache tools ant Main
Apache Axis Tutorial
Subject: Struts <logic iterate >
log4j properties How to parse outpu to multiple files
configuring log4j with BEA Weblogic 8 1
How to use XSL FOP Java together
JSP precompile
Proposal: Adding jar manifest classpath in jar and war plugins
Servlet File Download dialog problem (IE6,Adobe 6 0)
java security AccessControlException: access denied (java io FilePermission
Unsupported major minor version 48 0 problem while running the an
   telope task
Subject: axis wsdl2java Ant Task usage
net sf hibernate MappingException: Error reading resource: test/User hbm xml
Building EAR ANT Script for websphere 5 0
CREATING WAR Files
Classpath problem
jsp data into Excel
Jboss 3 2 3+ vs Tomcat Axis Question
RE: How to include jars and add them into the MANIFEST MF/Class Path
attribute
Printing problem
Subject: InstantiationException
Couldn 't find trusted certificate
Please : How can one install ant 1 6 0 under Eclipse 2 1 ?
Excel: Too many different cell formats
Subject: AXIS: tomcat timeout ?
1 3 final: now giving me java io FileNotFoundException (Too many
open files)
XDoclet, Struts and Maven: Where to start? SOLUTION
Subject: Running junit tests fails
 
Applying SpellChecker to a phrase

Applying SpellChecker to a phrase

2007-12-03       - By smokey

 Back
Reply:     1     2     3  

I have not tried this yet. I am trying to understand the best practices from
others who have experiences with SpellChecker before actually implementing
it.

If I understand it correctly, the spell check class suggests alternate but
similar words for a single input term. So I believe I will have to parse the
phrase string and apply spell checker for each member term to construct the
final expanded query. I don't think there is a higher level support that
lets me apply spell check to a phase and do query.toString() to examine how
it internally expanded the query (although it would have been nice to have
something like that - has anyone written or found such class?)

As for performance, we're dealing with hundreds of indexes where each index
typically grows well above 1G in size, so performance is the single most
important factor to consider.

On Dec 3, 2007 8:12 AM, Erick Erickson <erickerickson@(protected)> wrote:

> Have you actually tried this and done a query.toString() to see
> how this is actually expanded? Not that I'm all that familiar
> with SpellChecker, but before presuming how things work
> you would get answers faster if you ran a test.....
>
> And, why do you care about performance? I know that's
> a silly question, but you haven't supplied any parameters
> about your index and usage to give us a clue whether this
> matters. If your index is 3M, you'll never see the difference
> between the two ways of expanding the query. If your
> index is distributed over 10 machines and is 1T, you really,
> really, really care.
>
> And under any circumstances, you can always generate
> your own query of the second form by a bit of pre-processing.
>
> More info please.....
>
> Best
> Erick
>
> On Dec 2, 2007 10:14 PM, smokey <smokeystu@(protected)> wrote:
>
> > Suppose I have an index containing the terms impostor, imposter, fraud,
> > and
> > fruad, then presumably regardless of whether I spell impostor and fraud
> > correctly, Lucene SpellChecker will offer the improperly spelled
> versions
> > as
> > corrections. This means that the phrase "The login fraud involves an
> > impostor" would need to expand to:
> >
> > "The login fraud involves an impostor" OR "The login fruad involves an
> > impostor" OR "The login fraud involves an imposter" OR "The login fruad
> > involves an imposter" to cover all cases and thus find all possible
> > matches.
> >
> > However, that feels like an aweful a lot of matches to perform on the
> > index.
> > A more efficient approach would be to expand the query to "The login
> > (fraud
> > OR fruad) involves an (impostor OR imposter)", which should be logically
> > equivalent to the first (longer) query.
> >
> > So my question is
> > (1) if others have generated the "The login (fraud OR fruad) involves an
> > (impostor OR imposter)" types of queries when applying SpellChecker to a
> > phrase, and agreed that this indeed performs better than the first one.
> > (2) if others have observed any problems in doing so in terms of
> > performance
> > or anything else
> >
> > Any information would be appreciated.
> >
>