summaryrefslogtreecommitdiff
path: root/lib/mu-query.cc
AgeCommit message (Collapse)Author
2024-05-08mu-query: minor cleanupsDirk-Jan C. Binnema
2024-05-01lib: centralize xapian include, enable move semanticsDirk-Jan C. Binnema
Only include xapian.h in one place, so we can have consistent options. With that in place, we can enable C++ move semantics. We don't do anything with that yet, but we check in the meson.build file to see if we have the required xapian version.
2023-09-12mu-query: remove unnused move ctorDirk-Jan C. Binnema
2023-09-09support xapian ngramsDirk-Jan C. Binnema
Xapian supports an "ngrams" option to help with languages/scripts without explicit wordbreaks, such as Chinese / Japanese / Korean. Add some plumbing for supporting this in mu as well. Experimental for now.
2023-09-09lib: implement new query parserDirk-Jan C. Binnema
Implement a new query parser; the results should be very similar to the old one, but it adds an Sexp middle-representation, so users can see how a query is interpreted.
2023-08-04mu-query: small optimization tweaksDirk-Jan C. Binnema
2023-07-20migrate some more code to mu_format / join_pathsDirk-Jan C. Binnema
Let's modernize a bit.
2023-07-05update to use fmt-based apisDirk-Jan C. Binnema
Not complete, but a first big stab converting users of Mu::Error and various g_warning & friends, format to the new libfmt-based APIs.
2023-07-02all: update for API changes (config etc.)Dirk-Jan C. Binnema
Use the new & improved APIs.
2022-06-29query: exclude some test code from coverageDirk-Jan C. Binnema
2022-04-30lib: move 3rd party code to thirdparty; add tabulate.hppDirk-Jan C. Binnema
Move 3rd-party code to their own dir, and add the nice tabulate.hpp
2022-04-30query: update query subsys to use MessageDirk-Jan C. Binnema
2022-03-28lib+guile: use Mu::Option, not std::optionalDirk-Jan C. Binnema
We need the extensions, and/or let's use _one_ optional implementation everywhere.
2022-03-26many: update for lib/message updatesDirk-Jan C. Binnema
Adapt to the new names / directory. Big commit, but mostly just very boring renaming.
2022-03-04lib/query,parser: update to use mu-message-fieldsDirk-Jan C. Binnema
2022-01-30store/query: access query only through storeDirk-Jan C. Binnema
Make Mu::Query only accessible through store, so we can lock the db for the duration of a (full, multipass) query.
2021-10-23mu-query: honor maxnum in threaded queriesDirk-Jan C. Binnema
2021-10-20clang-format: update c/cc coding styleDirk-Jan C. Binnema
Update all cc code using .clang-format; please do so as well for future PRs etc.; emacs has a handy 'clang-format' mode to make this automatic. For comparing old changes with git blame, we can disregard this one using --ignore-rev (see https://www.moxio.com/blog/43/ignoring-bulk-change-commits-with-git-blame )
2021-10-18lib: replace CATCH_BLOCK macros with template magicDirk-Jan C. Binnema
2021-07-11query: fix include-related.Nicolas Avrutin
mu-query.cc: - make_related_enquire: don't include first query in qvec, we already have all thread IDs we need to query in thread_ds. - run_related: always sort first query by date, explained by the comment. - run_related: include qflags (in particular ascending vs descending) in leader_qflags. - run_theaded: don't limit results to maxnum, that results in threads potentially being cut off. mu-server.cc: - output_sexp: don't limit results to maxnum so as to match the behaviour of mu find (and avoid cuttong off threads). Fixes #1924 and #1911.
2021-06-17query-results: remove GatherThreadIdsDirk-Jan C. Binnema
We can't really do that in the match-decider, since we get _all_ messages there, not the <n>-limited. And some whitespace changes.
2021-06-04query: gather related ids from mset, not match-deciderDirk-Jan C. Binnema
Since we only want the (smaller) set from the mset, not the (full) set that the match-decider sees.
2021-03-17query: Better debug loggingDirk-Jan C. Binnema
2021-02-15query: honor maxnum in threaded searchesDirk-Jan C. Binnema
2021-02-15query: avoid expected warningDirk-Jan C. Binnema
Remove some debug leftover.
2021-02-15query: filter out docs without query-matchesDirk-Jan C. Binnema
This avoid including unwanted messages in threaded results. Also some cleanups.
2021-02-13mu-query: tweaks threaded/threaded queryingDirk-Jan C. Binnema
2021-02-12query: improve thread-key-makerDirk-Jan C. Binnema
For threading, we still get the _full_ set of messages (since the mset is limited, but not the enquire); so no need to warn about docids we haven't seen before. Also, ensure the unwanted docids are sorted after the wanted ones. Fixes: #1926.
2021-02-10mu-query: sort "related" input by dateDirk-Jan C. Binnema
When doing 'related' searches, sort the initial (bounded) query results by date, so we get the newest (or oldest)
2021-02-10lib: improve threaded sortingDirk-Jan C. Binnema
Improve threaded sorting: - rework the recursive algo (which should a crash noted earlier) - sort by thread key (as in 1.5.7) Fixes: #1923 Fixes: #1922 Fixes: #1911 Fixes: #1916
2021-01-25query: in related/threaded mode, search allDirk-Jan C. Binnema
In the related query, search _all_ messages, not just the amount asked; this increases the chance of full threads.
2021-01-20query: Rework querying/threading machineryDirk-Jan C. Binnema
Rewrite the query machinery in c++: - use an MSet decorator instead of the mu-msg-iter stuff - use mu-query-decider to mark duplicates/unreadable/related messages - use mu-query-threader to replace the older container/thread code Algorithm did not substantially change, but the implementation details did.
2020-11-04lib/query: refactor & reworkDirk-Jan C. Binnema
- Move the lib/query/ stuff up a level into lib/ - Associate directly with the Query object - Rework the Query object to be C++ rather than mixed with C - Update all dependencies, tests
2020-10-31lib: improve error handling, cleanupsDirk-Jan C. Binnema
2020-07-25fix compilation error / clang warningsDirk-Jan C. Binnema
2020-05-30rework logging systemDirk-Jan C. Binnema
reimplement the old mu-log.[ch] into mu-logging.{cc,hh} If available (and using an appropriately equipped glib), log to the systemd journal Only g_criticals have stderr output, all the other g_* go to the log file / journal.
2020-03-15mu4e: use faster count queries, document differencesDirk-Jan C. Binnema
Use faster queries for counting read/unread messages; document why the results might differ from what you get doing a normal search.
2020-02-20lib/parser -> lib/queryDirk-Jan C. Binnema
And update the names to follow the mu- convention.
2020-01-22mu-query: implement mu_query_count_runDirk-Jan C. Binnema
To get the number of matches and nothing else.
2020-01-01lib: Use Mu:Error exceptionDirk-Jan C. Binnema
Where applicable.
2019-12-22move parser/utils to utils, Mux->MuDirk-Jan C. Binnema
Move the parser utils to utils/ and rename the Mux namespace into Mu.
2019-12-22update for lib/utils splitDirk-Jan C. Binnema
Update sources with the new paths
2019-07-30lib/mu-store: rework implementationDirk-Jan C. Binnema
* mu-store.h, mu-store-read.cc, mu-store-write.cc, mu-store-priv.hh have been reworked in mu-store.{cc,hh}, it the mix of c/c++ improved * update all the dependent modules * make it easier to upgrade an database in place (without user intervention) * remove the xbatch-size option
2018-07-09Perform threading calculation on related set instead of entire result.Nicolas Avrutin
The current threading algorithm is applied to the entire result of a query, even if maxnum is specified, and then the result of the threading algorithm is truncated to maxnum. The improves threading results by returning the entire thread even when only a single message makes it into the top maxnum results. This commit applies the threading algorithm to the related message set of the maxnum-truncated query result instead of to the entire query result. For a given set of messages, the set of messages which will share threads with any of the original messages is exactly the related message sets. Put another way, either any messages returned by the original query but removed by the maxnum truncation will also be returned by the related message query, or they would not have been needed anyway because they would not be members of any visible thread. To maintain backward compatibility and allow threading to be used without including related messages, the related message set is found for the threading calculation, but any messages which would not have matched the original query are then pruned, resulting in a superset of the truncated query, but a subset of the untruncated query. This does not improve (or degrade) the run time of a threading calculation when maxnum is not set, but significant improves it when maxnum is set by making it scale (roughly) linearly in terms of maxnum. On a maildir with ~200k messages and maxnum set to 500 (the default), the run time of a threading calculation is lowered from ~1m to ~0.1s. Perform threading calculation on related set instead of entire result. The current threading algorithm is applied to the entire result of a query, even if maxnum is specified, and then the result of the threading algorithm is truncated to maxnum. The improves threading results by returning the entire thread even when only a single message makes it into the top maxnum results. This commit applies the threading algorithm to the related message set of the maxnum-truncated query result instead of to the entire query result. For a given set of messages, the set of messages which will share threads with any of the original messages is exactly the related message sets. Put another way, either any messages returned by the original query but removed by the maxnum truncation will also be returned by the related message query, or they would not have been needed anyway because they would not be members of any visible thread. To maintain backward compatibility and allow threading to be used without including related messages, the related message set is found for the threading calculation, but any messages which would not have matched the original query are then pruned, resulting in a superset of the truncated query, but a subset of the untruncated query. This does not improve (or degrade) the run time of a threading calculation when maxnum is not set, but significant improves it when maxnum is set by making it scale (roughly) linearly in terms of maxnum. On a maildir with ~200k messages and maxnum set to 500 (the default), the run time of a threading calculation is lowered from ~1m to ~0.1s.
2017-12-03mu: support 'raw' query (internally)djcb
Allow for passing 'raw' queries to xapian, without any parsing.
2017-11-02query: don't be too eager matching field namesdjcb
only match shortcuts when the field length was 1.
2017-10-30mu: fix clang builddjcb
casting error
2017-10-27server: update query processingdjcb
fix quoting issues with mu4e
2017-10-25update commands for new query parserdjcb
2017-10-25integrate new query parserdjcb