From debbugs-submit-bounces@debbugs.gnu.org Fri Feb 28 07:36:25 2020 Received: (at 39258) by debbugs.gnu.org; 28 Feb 2020 12:36:25 +0000 Received: from localhost ([127.0.0.1]:60742 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7esS-0002QM-O5 for submit@debbugs.gnu.org; Fri, 28 Feb 2020 07:36:24 -0500 Received: from mail-qk1-f196.google.com ([209.85.222.196]:38375) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j7esR-0002Q3-81 for 39258@debbugs.gnu.org; Fri, 28 Feb 2020 07:36:23 -0500 Received: by mail-qk1-f196.google.com with SMTP id z19so2747084qkj.5 for <39258@debbugs.gnu.org>; Fri, 28 Feb 2020 04:36:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ql2er9hVSfXLsVg7SCcUz2vhrbkIG/mO0k+XJT6hGfY=; b=ND/PEar5AF11ptlSDxx4urVpmUTf3yz06oMpUdYsGRuPrZhkRsiUfmisWnpBy91XK6 6cY/+5wgMJDy5K/mGa6yCsgqpAYIzaJ0mRAkomXtbpjd5jrRS04ua/bI+8uLM+HXEDWY dYeBZArdF561gfkPQQG/DQ5kEksHYqhvST85/Cpb9IEKwbrKbeZC8RaAl6dN7OsgKkDE LbvotAW+w3on+k63YaY6OmrtCXc1+nEO43MEjOfUVG8HFi5KeRabMnq8Z4ltrYS1FHSa eVo/mb/rg8oQVsySxEjQTjZpqmQ1HKIDiWOdhxg4YlnSKbYQF3X5pp63VLq3vrFDYY4e 0asQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ql2er9hVSfXLsVg7SCcUz2vhrbkIG/mO0k+XJT6hGfY=; b=M8CrjwjzP0ogvzMuQqGjUX9EaBvaELImAcGanWr5tFNOt0PFO3gsQQou7NwpGyU4s5 Dnl1HxK/Yv/xf9lIU//KNMARfpJ2Wz1RCRN8pzbd3ilDatTny0yEX/rGicdkM6Asre18 DXk3Y3qHJhg/Ju1uUW0kS8KboxwY5o4nFrUdVxt/5wWZgsJCmYAdxobV/r/DZgE0XxBd UIpoxIwKgr15F7kRINC3E9NNxai6Ngx/Csq8wKTWeYJ6GyD5AbmboJOJm5xUaK7YUJyv 8O3WiWRm6vx/dLEJzdehfHGWC/YpciX4xLhD5POfQNgtslWfzOP/3D1G3t91RJoyEE1a rBKA== X-Gm-Message-State: APjAAAVrw1H4sZ5b4b7uQgo/PmiG31IteFdrrULGYWANEjCbUm6klt6G BsF2zwq5pNKldCMoARkfmY/XIWiqXoJ+pYGiZZDitg== X-Google-Smtp-Source: APXvYqwajaO8P+myNqSuUgKjfMI8P9Ah2ShKWY8GFKYITMKKQuN4I8BQMd7PVGXY5uqfviqwHW6eFbc2qJcw7W9izvM= X-Received: by 2002:a05:620a:a45:: with SMTP id j5mr4310682qka.232.1582893377553; Fri, 28 Feb 2020 04:36:17 -0800 (PST) MIME-Version: 1.0 References: <20200227204150.30985-1-arunisaac@systemreboot.net> In-Reply-To: <20200227204150.30985-1-arunisaac@systemreboot.net> From: zimoun Date: Fri, 28 Feb 2020 13:36:06 +0100 Message-ID: Subject: Re: [PATCH 0/4] Xapian for Guix package search To: Arun Isaac Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 39258 Cc: =?UTF-8?Q?Ludovic_Court=C3=A8s?= , Pierre Neidhardt , 39258@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Arun, Really cool! Thank you! On Thu, 27 Feb 2020 at 21:42, Arun Isaac wrote: > * Speed improvement > > Despite search-package-index in gnu/packages.scm taking only around 1.5ms, I > see an overall speedup in `guix search` of only a factor of 2 -- from around > 2s to around 1s. I wonder what else in `guix search` is taking up so much > time. Interesting... maybe an hidden 'fold-packages'? Well, I have not yet looked into your code. > * Currently indexing only the package descriptions > > In this patchset, I have only indexed the package descriptions. In the next > version of this patchset, I will index all other terms as specified in > %package-metrics of guix/ui.scm. Yes, it appears to me a detail that should be easy to fix. I mean, it does not seems blocking. > * Should I add guile-xapian as a propagated input to guix in > gnu/packages/package-management.scm? IMHO, yes. I mean, I guess. :-) > * Drop regexp search support > > In this patchset, I have retained the older regexp search support. But, I > think we should drop it and only have xapian search. In cases where the search > index is not authoritative, we can build an in-memory xapian search index on > the fly and use it to search. This will slow down the search, but will ensure > our search results are consistent and do not depend on the authoritativeness > of the search index. I understand why you have turned off the regexp support. It is not necessary at the first experimentation to see if it is worth the addition or not. So, before investigating how some better regexp could be used with Xapian, let start to benchmark Xapian vs plain 'fold-packages'. > * Commit messages > > Except for patch 1, I am not sure what prefixes (build-self, gnu, etc.) to use > in the first line of the commit message. Some advice there would be helpful. I cannot help. )-: All the best, simon