From debbugs-submit-bounces@debbugs.gnu.org Wed Jan 29 18:34:15 2020 Received: (at 39258) by debbugs.gnu.org; 29 Jan 2020 23:34:15 +0000 Received: from localhost ([127.0.0.1]:34709 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iwwqc-0003Yo-So for submit@debbugs.gnu.org; Wed, 29 Jan 2020 18:34:15 -0500 Received: from mail-qt1-f172.google.com ([209.85.160.172]:46296) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1iwwqb-0003Yb-IF for 39258@debbugs.gnu.org; Wed, 29 Jan 2020 18:34:14 -0500 Received: by mail-qt1-f172.google.com with SMTP id e25so944644qtr.13 for <39258@debbugs.gnu.org>; Wed, 29 Jan 2020 15:34:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=LoCPANSQcRYKL7WvsA1UPP8c/yFpkPntg+GlDHw5LfQ=; b=nOUNoINq2LgFQVkvghW+wmN65dXoTfG1RDsbw8g7PjauqVkLS7p9IWwrzalMmYRar7 OtNPHgQXtBJtLcSovSsPJH099NyJUegPvYCMqqjFDCm89XO0WcFHtDNONMgGwNDtRFle uqE9G0S7u2TnQ/x8eu8O/9JF384W5pPMlpLdqXJiGsPtxO+MGJloZWtIeJYo+tEQNmtk Fp4ka7Dx9+XjA560MD7DSdc3SZmtTeJ7a3uLaqSBF2Yz8DYlwelxMm7v5KC9gUguAdi8 y9IPzjedL2nDLgiU14/ojUNsBaEpJon1ESUV6IKUna3Uu/2P7Phy22bV/57QKPzN+YUM oL/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LoCPANSQcRYKL7WvsA1UPP8c/yFpkPntg+GlDHw5LfQ=; b=DP2QMenHuANFHPaLyyYUFhBJ6GezBr7LyW/pLw0d6RhnaU8WRafcTfBufS6CTw05Gk PoQ4olKqYZo2Fo6PR97s6SbOnewE8gfjASNjAsxHCale4jlC2GRaf/sJ9BlKV0Kgbfii wFOLbd8OTEP0fP/DBxPv8nJulafFhdEqQTjmtD0ho2xOY5RLjGGq8JrpHeUYZeyfH0fA xfcDPUJgxVisimDAPCRU6muua4RHJHA/TOwEi5/XWGQ6WIxVshkyohek+/XYQuLMDFJl js9tmxT4VRLumNVJS9dvUFashX08tHH39whk4j/mdi5O2KAVB85H7Fo+1Lbrg2bsQHq4 1LMw== X-Gm-Message-State: APjAAAUbFmYBaK9yF2sWO1W3E8DUMLZL0fToJWM9m3aha4D2HhSHl7Wu rNIZ3JieuYD4lcnO/HlprbtTryWyP7umMXtT6McRLfF8eSk= X-Google-Smtp-Source: APXvYqxvZjie7g6Bak2iifeJhhT+gOSKffcNZJUzUXfyzaz3AIMKTDA+MP1eofrQA9rGFDBNoVWY1XlIIVUcIV43QkU= X-Received: by 2002:ac8:2e53:: with SMTP id s19mr1967828qta.354.1580340847857; Wed, 29 Jan 2020 15:34:07 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: zimoun Date: Thu, 30 Jan 2020 00:33:56 +0100 Message-ID: Subject: Re: [bug#39258] Faster guix search using an sqlite cache To: Arun Isaac Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 39258 Cc: 39258@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi Arun, Thank you for the patch! Cool! :-) I have not tested it yet. Sorry. On Thu, 23 Jan 2020 at 20:53, Arun Isaac wrote: > At the moment, I am having some difficulty populating the sqlite > database. generate-package-cache populates the database correctly when > invoked from a normal guile REPL using geiser, but fails to do so when > run by the guix daemon during guix pull. [...] > On examining package-cache.sqlite, I find that no records have been > written. And, there is a lingering journal file that shouldn't be > there. For some reason, populating the sqlite database does not work > with guix pull. sqlite probably crashes and leaves the journal file. Hum? weird... Is it possible that a module is loaded when Guile repl is used and not with Guix pull? What about "guix repl"? > If I try to populate the database with each package record being > inserted in its own transaction, at least some of the insertions You mean 'commit' the database after each insertion, right? > work. But the journal file still lingers. My unverified guess is that > everything except the last transaction was successful. And this does not happen with the repl, right? > Any ideas what's going on? I have no idea. Weird. What about adding 'last-insert-row-id' from 'guix/store/database.scm'? I mean without really understanding and just grepping 'sqlite-finalize' spots that 'last-insert-row-id' seems often around. :-) > Also, inserting each package in its own transaction is ridiculously slow > and so that is out of the question. See https://www.sqlite.org/faq.html#q19 Agree that it is not an option. :-) Otherwise, 'list->string' is defined twice. And the first one is not necessary, I guess. The docstring of 'cache-lookup' is not coherent anymore. :-) Cheers, simon