summaryrefslogtreecommitdiff
path: root/lib/parser
AgeCommit message (Collapse)Author
2017-12-03parser: promote single value to a range for range-fieldsdjcb
Treat e.g. 'date:20170101' as 'date:20170101..20170101', just like the Xapian parser does.
2017-11-04parser: small regex optimizationdjcb
2017-11-04parser/utils: enforce 64-bit times on 32-bit platformsdjcb
don't assume a 64-bit platform.
2017-11-04parser: handle implicit 'and not'djcb
2017-10-31parser: fix and-not precedencedjcb
For now, don't treat "and not" specially; this gets us back into a somewhat working state. At some point, we probably _do_ want to special-case and_not though (since Xapian supports it).
2017-10-29mu: some optimizationsdjcb
add fast-path for (common) plain-ascii. fix silly static misuse. should improve indexing with some single-digit percentage.
2017-10-28tokenizer: clean unicode-awaredjcb
2017-10-28parser: add more testsdjcb
2017-10-27phrases: only allow for index fieldsdjcb
2017-10-27parser: fix some post-c++14 codedjcb
don't require anything post c++14
2017-10-27query-parser: cleanup source stringdjcb
Ensure there's no non-' ' whitespace, and no trailing/leading spaces.
2017-10-26query-parser: support phrase queriesdjcb
2017-10-25integrate new query parserdjcb
2017-10-24lib: implement new query parserdjcb
mu's query parser is the piece of software that turns your queries into something the Xapian database can understand. So, if you query "maildir:/inbox and subject:bla" this must be translated into a Xapian::Query object which will retrieve the sought after messages. Since mu's beginning, almost a decade ago, this parser was based on Xapian's default Xapian::QueryParser. It works okay, but wasn't really designed for the mu use-case, and had a bit of trouble with anything that's not A..Z (think: spaces, special characters, unicode etc.). Over the years, mu added quite a bit of pre-processing trickery to deal with that. Still, there were corner cases and bugs that were practically unfixable. The solution to all of this is to have a custom query processor that replaces Xapian's, and write it from the ground up to deal with the special characters etc. I wrote one, as part of my "future, post-1.0 mu" reseach project, and I have now backported it to the mu 0.9.19. From a technical perspective, this is a major cleanup, and allows us to get rid of much of the fragile preprocessing both for indexing and querying. From and end-user perspective this (hopefully) means that many of the little parsing issues are gone, and it opens the way for some new features. From an end-user perspective: - better support for special characters. - regexp search! yes, you can now search for regular expressions, e.g. subject:/h.ll?o/ will find subjects with hallo, hello, halo, philosophy, ... As you can imagine, this can be a _heavy_ operation on the database, and might take quite a bit longer than a normal query; but it can be quite useful.