[PATCH 2/2] scripts: time-machine: Error when attempting to visit too old commits.

  • Done
  • quality assurance status badge
Details
4 participants
  • Josselin Poiret
  • Ludovic Courtès
  • Maxim Cournoyer
  • Simon Tournier
Owner
unassigned
Submitted by
Maxim Cournoyer
Severity
normal
M
M
Maxim Cournoyer wrote on 20 Jul 2023 18:34
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
a058a4de3330a6f9cf41456e0781c3865dff0553.1689823648.git.maxim.cournoyer@gmail.com
* doc/guix.texi (Invoking guix time-machine): Document limitation.
* guix/scripts/time-machine.scm (%oldest-possible-commit): New variable.
(guix-time-machine): Raise an error when the channel commit is too old.

Suggested-by: Simon Tournier <zimon.toutoune@gmail.com>
---
doc/guix.texi | 6 ++++++
guix/scripts/time-machine.scm | 23 ++++++++++++++++++++++-
2 files changed, 28 insertions(+), 1 deletion(-)

Toggle diff (86 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 1d8ebcd72f..30fef813c0 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5056,6 +5056,12 @@ Invoking guix time-machine
large number of packages; the result is cached though and subsequent
commands targeting the same commit are almost instantaneous.
+Due to @command{guix time-machine} relying on the ``inferiors''
+mechanism (@pxref{Inferiors}), the oldest commit it can travel to is
+commit @samp{2ca299caf} (``Add (guix inferior) and (guix scripts
+repl).''), dated July 10@sup{th}, 2018. An error is returned when
+attempting to navigate to older commits.
+
@quotation Note
The history of Guix is immutable and @command{guix time-machine}
provides the exact same software as they are in a specific Guix
diff --git a/guix/scripts/time-machine.scm b/guix/scripts/time-machine.scm
index d7c71ef705..36a40a1538 100644
--- a/guix/scripts/time-machine.scm
+++ b/guix/scripts/time-machine.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2019 Konrad Hinsen <konrad.hinsen@fastmail.net>
;;; Copyright © 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -19,13 +20,15 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix scripts time-machine)
+ #:use-module (guix channels)
+ #:use-module (guix diagnostics)
#:use-module (guix ui)
#:use-module (guix scripts)
#:use-module (guix inferior)
#:use-module (guix store)
#:use-module (guix status)
#:use-module ((guix git)
- #:select (with-git-error-handling))
+ #:select (update-cached-checkout with-git-error-handling))
#:use-module ((guix utils)
#:select (%current-system))
#:use-module ((guix scripts pull)
@@ -38,9 +41,16 @@ (define-module (guix scripts time-machine)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-34)
#:use-module (srfi srfi-37)
+ #:use-module (srfi srfi-71)
#:export (guix-time-machine))
+;;; The commit introducing the 'inferiors' mechanism; it is the oldest commit
+;;; that can be travelled to.
+(define %oldest-possible-commit
+ "2ca299caf64489f4e1e665ec1158fb0309b0b565")
+
;;;
;;; Command-line options.
@@ -139,9 +149,20 @@ (define-command (guix-time-machine . args)
(with-git-error-handling
(let* ((opts (parse-args args))
(channels (channel-list opts))
+ (guix-channel (find guix-channel? channels))
(command-line (assoc-ref opts 'exec))
+ (ref (assoc-ref opts 'ref))
+ (checkout commit relation (update-cached-checkout
+ (channel-url guix-channel)
+ #:ref (or ref '())
+ #:starting-commit
+ %oldest-possible-commit))
(substitutes? (assoc-ref opts 'substitutes?))
(authenticate? (assoc-ref opts 'authenticate-channels?)))
+ (unless (memq relation '(ancestor self))
+ (raise (formatted-message
+ (G_ "cannot travel past commit `~a' from July 10th, 2018")
+ (string-take %oldest-possible-commit 12))))
(when command-line
(let* ((directory
(with-store store
--
2.41.0
M
M
Maxim Cournoyer wrote on 22 Jul 2023 04:00
(address . 64746@debbugs.gnu.org)
87lef8hfil.fsf_-_@gmail.com
Hello,

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

Toggle quote (70 lines)
> * doc/guix.texi (Invoking guix time-machine): Document limitation.
> * guix/scripts/time-machine.scm (%oldest-possible-commit): New variable.
> (guix-time-machine): Raise an error when the channel commit is too old.
>
> Suggested-by: Simon Tournier <zimon.toutoune@gmail.com>
> ---
> doc/guix.texi | 6 ++++++
> guix/scripts/time-machine.scm | 23 ++++++++++++++++++++++-
> 2 files changed, 28 insertions(+), 1 deletion(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 1d8ebcd72f..30fef813c0 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -5056,6 +5056,12 @@ Invoking guix time-machine
> large number of packages; the result is cached though and subsequent
> commands targeting the same commit are almost instantaneous.
>
> +Due to @command{guix time-machine} relying on the ``inferiors''
> +mechanism (@pxref{Inferiors}), the oldest commit it can travel to is
> +commit @samp{2ca299caf} (``Add (guix inferior) and (guix scripts
> +repl).''), dated July 10@sup{th}, 2018. An error is returned when
> +attempting to navigate to older commits.
> +
> @quotation Note
> The history of Guix is immutable and @command{guix time-machine}
> provides the exact same software as they are in a specific Guix
> diff --git a/guix/scripts/time-machine.scm b/guix/scripts/time-machine.scm
> index d7c71ef705..36a40a1538 100644
> --- a/guix/scripts/time-machine.scm
> +++ b/guix/scripts/time-machine.scm
> @@ -2,6 +2,7 @@
> ;;; Copyright © 2019 Konrad Hinsen <konrad.hinsen@fastmail.net>
> ;;; Copyright © 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
> ;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
> +;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -19,13 +20,15 @@
> ;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
>
> (define-module (guix scripts time-machine)
> + #:use-module (guix channels)
> + #:use-module (guix diagnostics)
> #:use-module (guix ui)
> #:use-module (guix scripts)
> #:use-module (guix inferior)
> #:use-module (guix store)
> #:use-module (guix status)
> #:use-module ((guix git)
> - #:select (with-git-error-handling))
> + #:select (update-cached-checkout with-git-error-handling))
> #:use-module ((guix utils)
> #:select (%current-system))
> #:use-module ((guix scripts pull)
> @@ -38,9 +41,16 @@ (define-module (guix scripts time-machine)
> #:use-module (srfi srfi-1)
> #:use-module (srfi srfi-11)
> #:use-module (srfi srfi-26)
> + #:use-module (srfi srfi-34)
> #:use-module (srfi srfi-37)
> + #:use-module (srfi srfi-71)
> #:export (guix-time-machine))
>
> +;;; The commit introducing the 'inferiors' mechanism; it is the oldest commit
> +;;; that can be travelled to.
> +(define %oldest-possible-commit
> + "2ca299caf64489f4e1e665ec1158fb0309b0b565")

I just tried travelling to that assumed oldest commit (because it
corresponds to the introduction of the inferiors mechanism), but it
fails like:

Toggle snippet (53 lines)
Computing Guix derivation for 'x86_64-linux'... Backtrace:
- 5 (primitive-load "/gnu/store/b70mihsj9xx0xxp6izliqb5vm4…")
In ice-9/eval.scm:
155:9 4 (_ _)
159:9 3 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
173:47 2 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
In ./guix/self.scm:
932:4 1 (guix-derivation "/gnu/store/yfn2s94i5bvwr7j7r6xcnivwg…" …)
903:2 0 (guile-for-build "3.0")

./guix/self.scm:903:2: In procedure guile-for-build:
Throw to key `match-error' with args `("match" "no matching pattern" "3.0")'.
Backtrace:
In ice-9/boot-9.scm:
1752:10 19 (with-exception-handler _ _ #:unwind? _ # _)
In guix/store.scm:
659:37 18 (thunk)
In guix/status.scm:
839:4 17 (call-with-status-report _ _)
In guix/store.scm:
1298:8 16 (call-with-build-handler #<procedure 7ff1daabfd20 at g…> …)
In guix/monads.scm:
576:2 15 (run-with-store #<store-connection 256.99 7ff1dab842d0> …)
In guix/inferior.scm:
927:8 14 (_ _)
In guix/channels.scm:
982:2 13 (_ _)
924:2 12 (_ _)
In guix/store.scm:
1883:0 11 (_ _)
1996:8 10 (_ _)
In guix/channels.scm:
675:14 9 (_ #<store-connection 256.99 7ff1dab842d0>)
In guix/monads.scm:
576:2 8 (run-with-store #<store-connection 256.99 7ff1dab842d0> …)
In guix/store.scm:
1298:8 7 (call-with-build-handler _ _)
1298:8 6 (call-with-build-handler #<procedure 7ff1db8108b8 at g…> …)
In guix/channels.scm:
690:14 5 (_)
In guix/monads.scm:
576:2 4 (run-with-store #<store-connection 256.99 7ff1dab842d0> …)
In ice-9/eval.scm:
191:27 3 (_ #(#(#<directory (build-self) 7ff1da984aa0> #<pr…>) …))
In ice-9/boot-9.scm:
2007:7 2 (error _ . _)
1685:16 1 (raise-exception _ #:continuable? _)
1685:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
invalid build result (#<derivation /gnu/store/rj2g4x23lqyaq16471qm94xp90slxp3h-compute-guix-derivation.drv => /gnu/store/b70mihsj9xx0xxp6izliqb5vm462yifl-compute-guix-derivation 7ff1d8b55000> "")

Is this a bug or expected?

--
Thanks,
Maxim
L
L
Ludovic Courtès wrote on 8 Aug 2023 17:58
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87leelpluk.fsf_-_@gnu.org
Hi!

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (22 lines)
>> +;;; The commit introducing the 'inferiors' mechanism; it is the oldest commit
>> +;;; that can be travelled to.
>> +(define %oldest-possible-commit
>> + "2ca299caf64489f4e1e665ec1158fb0309b0b565")
>
> I just tried travelling to that assumed oldest commit (because it
> corresponds to the introduction of the inferiors mechanism), but it
> fails like:
>
> Computing Guix derivation for 'x86_64-linux'... Backtrace:
> - 5 (primitive-load "/gnu/store/b70mihsj9xx0xxp6izliqb5vm4…")
> In ice-9/eval.scm:
> 155:9 4 (_ _)
> 159:9 3 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
> 173:47 2 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
> In ./guix/self.scm:
> 932:4 1 (guix-derivation "/gnu/store/yfn2s94i5bvwr7j7r6xcnivwg…" …)
> 903:2 0 (guile-for-build "3.0")
>
> ./guix/self.scm:903:2: In procedure guile-for-build:
> Throw to key `match-error' with args `("match" "no matching pattern" "3.0")'.

I would pick ‘v0.15.0’ (= 359fdda40f754bbf1b5dc261e7427b75463b59be) as
the oldest commit one can travel to; it’s a bit newer than the one
above, but it fails in the same way (to my surprise). It would be
interesting to investigate.

That said, we could just as well pick ‘v1.0.0’, which is the official
warranty-void limit, and which seems to work (it needs to build things,
though…).

Ludo’.
M
M
Maxim Cournoyer wrote on 10 Aug 2023 16:47
(name . Ludovic Courtès)(address . ludo@gnu.org)
87leejeyy9.fsf@gmail.com
Hi Ludo!

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (35 lines)
> Hi!
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>>> +;;; The commit introducing the 'inferiors' mechanism; it is the oldest commit
>>> +;;; that can be travelled to.
>>> +(define %oldest-possible-commit
>>> + "2ca299caf64489f4e1e665ec1158fb0309b0b565")
>>
>> I just tried travelling to that assumed oldest commit (because it
>> corresponds to the introduction of the inferiors mechanism), but it
>> fails like:
>>
>> Computing Guix derivation for 'x86_64-linux'... Backtrace:
>> - 5 (primitive-load "/gnu/store/b70mihsj9xx0xxp6izliqb5vm4…")
>> In ice-9/eval.scm:
>> 155:9 4 (_ _)
>> 159:9 3 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
>> 173:47 2 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
>> In ./guix/self.scm:
>> 932:4 1 (guix-derivation "/gnu/store/yfn2s94i5bvwr7j7r6xcnivwg…" …)
>> 903:2 0 (guile-for-build "3.0")
>>
>> ./guix/self.scm:903:2: In procedure guile-for-build:
>> Throw to key `match-error' with args `("match" "no matching pattern" "3.0")'.
>
> I would pick ‘v0.15.0’ (= 359fdda40f754bbf1b5dc261e7427b75463b59be) as
> the oldest commit one can travel to; it’s a bit newer than the one
> above, but it fails in the same way (to my surprise). It would be
> interesting to investigate.
>
> That said, we could just as well pick ‘v1.0.0’, which is the official
> warranty-void limit, and which seems to work (it needs to build things,
> though…).

I tried building v1.0.0 but it failed the same as earlier attempts on
Python 2, which has a SSL test failing due to now-expired certificates:

Toggle snippet (58 lines)
@ build-log 18017 117
/gnu/store/gfprsx2m62cvqbh7ysc9ay9slhijvmal-module-import/guix/build/gnu-build-system.scm:369:6: In procedso ure check:
@ build-log 18017 167
Throw to key `srfi-34' with args `(#<condition &invoke-error [program: "make" arguments: ("test" "-j" "24" ) exit-status: 2 term-signal: #f stop-signal: #f] 9a4900>)'. 40
\@ build-log 18017 101
builder for `/gnu/store/sp947xhp8dqfzn3zd2m0aq4ia5qvklbl-python2-2.7.15.drv' failed with exit code 1
@ build-log 18017 183 -o
@ build-failed /gnu/store/sp947xhp8dqfzn3zd2m0aq4ia5qvklbl-python2-2.7.15.drv - 1 builder for `/gnu/store/h-sp947xhp8dqfzn3zd2m0aq4ia5qvklbl-python2-2.7.15.drv' failed with exit code 1
@ build-log 18017 189
derivation '/gnu/store/sp947xhp8dqfzn3zd2m0aq4ia5qvklbl-python2-2.7.15.drv' offloaded to '10.0.0.7' failede-: build of `/gnu/store/sp947xhp8dqfzn3zd2m0aq4ia5qvklbl-python2-2.7.15.drv' failed ar
@ build-failed /gnu/store/sp947xhp8dqfzn3zd2m0aq4ia5qvklbl-python2-2.7.15.drv - 1 builder for `/gnu/store/cesp947xhp8dqfzn3zd2m0aq4ia5qvklbl-python2-2.7.15.drv' failed with exit code 100
Backtrace:
In ./guix/gexp.scm:
573:13 19 (_ _)
In ./guix/store.scm:
1667:8 18 (_ _)
1667:8 17 (_ _) so
In ./guix/gexp.scm:
708:2 16 (_ _)
In ./guix/monads.scm:
482:9 15 (_ _)
In ./guix/gexp.scm:
573:13 14 (_ _)
In ./guix/store.scm:
1667:8 13 (_ _)
In ./guix/gexp.scm:
708:2 12 (_ _)
In ./guix/monads.scm: 9}
482:9 11 (_ _)
In ./guix/gexp.scm:
573:13 10 (_ _)
In ./guix/store.scm: 37
1667:8 9 (_ _) 36
In ./guix/gexp.scm: 1c
708:2 8 (_ _) 2!
In ./guix/monads.scm: 3I
482:9 7 (_ _) \1
|
In ./guix/gexp.scm:
573:13 6 (_ _)
In ./guix/store.scm:
1667:8 5 (_ _)
1690:38 4 (_ #<store-connection 256.99 7f6b38e620c0>)
In ./guix/packages.scm:
936:16 3 (cache! #<weak-table 406/883> #<package guile-gcrypt@0?> ?)
1255:22 2 (thunk)
1188:25 1 (bag->derivation #<store-connection 256.99 7f6b38e620c0> ?)
In srfi/srfi-1.scm:
592:17 0 (map1 (("source" #<origin #<<git-reference> url: "?>) ?))

srfi/srfi-1.scm:592:17: In procedure map1:
Throw to key `srfi-34' with args `(#<condition &store-protocol-error [message: "build of `/gnu/store/sp947xhp8dqfzn3zd2m0aq4ia5qvklbl-python2-2.7.15.drv' failed" status: 100] 7f6b3a5b2c60>)'.
guix time-machine: erreur : You found a bug: the program '/gnu/store/d12x45lz2dv3mgp594kzjv0d121g0ncs-compute-guix-derivation'
failed to compute the derivation for Guix (version: "6298c3ffd9654d3231a6f25390b056483e8f407c"; system: "xso86_64-linux";
host version: "985638aea14720e16ed5fd94a0e1382a57dec7ac"; pull-version: 1).
Please report it by email to <bug-guix@gnu.org>.

Since we can't retroactively fix this kind of problem, it means we
should find the oldest commit which is immune to that problem?

--
Thanks,
Maxim
L
L
Ludovic Courtès wrote on 10 Aug 2023 18:56
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87bkfekf95.fsf@gnu.org
Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (3 lines)
> Since we can't retroactively fix this kind of problem, it means we
> should find the oldest commit which is immune to that problem?

Timebombs (typically expired SSL certificates as in the Python case you
mention) can be worked around. Currently it’s inconvenient at best
because you need to manually set up a VM or physical machine with its
date set to an older date, but it’s feasible.

For other problems, there’s (guix quirks), which can, to some extent,
let us retroactively fix problems, though that should be rare.

‘etc/time-travel-manifest.scm’, is supposed to test these things, but
apparently it’s been stuck for a while. Needs investigation!

Ludo’.
J
J
Josselin Poiret wrote on 11 Aug 2023 09:19
87ttt6rqp7.fsf@jpoiret.xyz
Hi everyone,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (3 lines)
> For other problems, there’s (guix quirks), which can, to some extent,
> let us retroactively fix problems, though that should be rare.

Is there any established policy around what (guix quirks) is allowed to
do? To be more specific, can/will it ever change derivation hashes?

Best,
--
Josselin Poiret
-----BEGIN PGP SIGNATURE-----

iQHEBAEBCgAuFiEEOSSM2EHGPMM23K8vUF5AuRYXGooFAmTV4RQQHGRldkBqcG9p
cmV0Lnh5egAKCRBQXkC5FhcairSeC/9A0TE/0bKahmqDPg5nqg4FjDvOANnuTEcH
GBN4Bd+6azISKSJAmQdl6MhoA7L8cbpyPStjcJ2op+c9u/U2PVpYuUqUQsHtvNWw
5vYxUcwLhUt/RVQ5lRjQi5RJLpISl00q7UtYXDWQqQ+st3z1BFYD1rG1zI3cYWMY
pOOhDnYx+cQy4l3pgVM+V/v5HhhzxEetcFi+ks5B5hTMMILJ5RGw797xp90k8RAx
7zqmjoD+d31vNtvGWYJqvVQD33COEwV51gbH0Yz8J6TWW7+WEIT64UXGRWgd/6B6
jnsa2pP5cfhUDYbbc1TDYbKfN26i81n+Dy3xech+WqxJECGuw7xCDUPQvBMyqqBp
01TsMha6aI7OvBHowpMgzi3219ojPg+dv9Vby8RwV8wh8hCSDN0frRR7Z/aaFo5x
cSIFI/XRZcZnoZtnWuWWSQgxaUC0beXPOB8UDrbL+1kR3JcKdDsvsZjisbUxK/ae
b6oiHeIPcTSl6Pdh3u9vheApdpY54wc=
=Y8SB
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 12 Aug 2023 22:32
(name . Josselin Poiret)(address . dev@jpoiret.xyz)
87ttt4dmt2.fsf@gnu.org
Hi,

Josselin Poiret <dev@jpoiret.xyz> skribis:

Toggle quote (8 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> For other problems, there’s (guix quirks), which can, to some extent,
>> let us retroactively fix problems, though that should be rare.
>
> Is there any established policy around what (guix quirks) is allowed to
> do? To be more specific, can/will it ever change derivation hashes?

It should not change derivations, which means it can only touch
auxiliary files like ‘build-aux/build-self.scm’.

Ludo’.
L
L
Ludovic Courtès wrote on 15 Aug 2023 18:14
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87pm3o8epw.fsf_-_@gnu.org
Hello,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (15 lines)
> @@ -139,9 +149,20 @@ (define-command (guix-time-machine . args)
> (with-git-error-handling
> (let* ((opts (parse-args args))
> (channels (channel-list opts))
> + (guix-channel (find guix-channel? channels))
> (command-line (assoc-ref opts 'exec))
> + (ref (assoc-ref opts 'ref))
> + (checkout commit relation (update-cached-checkout
> + (channel-url guix-channel)
> + #:ref (or ref '())
> + #:starting-commit
> + %oldest-possible-commit))
> (substitutes? (assoc-ref opts 'substitutes?))
> (authenticate? (assoc-ref opts 'authenticate-channels?)))

Following your question earlier today on IRC, I realized that this would
unconditionally add a Git checkout update every time ‘time-machine’ is
started.

This would noticeably degrade performance in the cache-hit case (when
running a cached channel set). Currently, on cache hits,
‘cached-channel-instance’ directly returns ~/.cache/guix/profiles/xyz
without performing any Git or (guix store) operation.

This is important because it guarantees that one can run ‘guix
time-machine -C channels.scm -- COMMAND’ at pretty much the same speed
as ‘guix COMMAND’.

Perhaps the solution would be for ‘cached-channel-instance’ to have a
new #:validate-channel procedure (or similar) that would be passed
channel/commit/relation and would have the opportunity to validate,
before the thing is in cache.

How does that sound?

Ludo’.
M
M
Maxim Cournoyer wrote on 15 Aug 2023 20:57
(name . Ludovic Courtès)(address . ludo@gnu.org)
87h6p0jfqe.fsf_-_@gmail.com
Hello,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (35 lines)
> Hi!
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>>> +;;; The commit introducing the 'inferiors' mechanism; it is the oldest commit
>>> +;;; that can be travelled to.
>>> +(define %oldest-possible-commit
>>> + "2ca299caf64489f4e1e665ec1158fb0309b0b565")
>>
>> I just tried travelling to that assumed oldest commit (because it
>> corresponds to the introduction of the inferiors mechanism), but it
>> fails like:
>>
>> Computing Guix derivation for 'x86_64-linux'... Backtrace:
>> - 5 (primitive-load "/gnu/store/b70mihsj9xx0xxp6izliqb5vm4…")
>> In ice-9/eval.scm:
>> 155:9 4 (_ _)
>> 159:9 3 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
>> 173:47 2 (_ #(#(#(#(#(#(#(#(#(#(#(…) …) …) …) …) …) …) …) …) …) …))
>> In ./guix/self.scm:
>> 932:4 1 (guix-derivation "/gnu/store/yfn2s94i5bvwr7j7r6xcnivwg…" …)
>> 903:2 0 (guile-for-build "3.0")
>>
>> ./guix/self.scm:903:2: In procedure guile-for-build:
>> Throw to key `match-error' with args `("match" "no matching pattern" "3.0")'.
>
> I would pick ‘v0.15.0’ (= 359fdda40f754bbf1b5dc261e7427b75463b59be) as
> the oldest commit one can travel to; it’s a bit newer than the one
> above, but it fails in the same way (to my surprise). It would be
> interesting to investigate.
>
> That said, we could just as well pick ‘v1.0.0’, which is the official
> warranty-void limit, and which seems to work (it needs to build things,
> though…).

I've picked v1.0.0. It already seems hard enough to get there (you'd
have to build Python 2 with the hardware clock set to a value in the
past to avoid time bombs in its test suite).

As discussed with Ludovic, it's best to avoid doing work on critical
path, that is, when there is a cache hit for the channel. I've
implemented their idea to delay such work to within
'cached-channel-instance', where we know if there's a cache hit or not.
Here's a simple 'strace -c' benchmark:

Without this change (Guix commit
985638aea14720e16ed5fd94a0e1382a57dec7ac), on a warm cache, it results
on something like:

Toggle snippet (8 lines)
$ strace -c guix time-machine --commit=v1.3.0 -- \
environment --ad-hoc hello -- hello
% time seconds usecs/call calls errors syscall
[...]
100,00 0,072028 4 16963 2026 total

With this change (v2 incoming) installed:
$ strace -c ./pre-inst-env guix time-machine --commit=v1.3.0 -- \
environment --ad-hoc hello -- hello
% time seconds usecs/call calls errors syscall
[...]
100,00 0,074576 4 18005 2700 total
Toggle snippet (9 lines)
It seems a small cost to pay for the increased user friendliness.

What do you think?

--
Thanks,
Maxim
M
M
Maxim Cournoyer wrote on 15 Aug 2023 21:44
[PATCH v2 1/3] git: Clarify commit relation reference in doc.
(address . 64746@debbugs.gnu.org)(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
340786ccd3d40c0c8906028dc20af5ae9b0aa6b9.1692128648.git.maxim.cournoyer@gmail.com
* guix/git.scm (update-cached-checkout): Clarify that it is the relation of
STARTING-COMMIT that is returned, relative to the new commit, not the other
way around.
---

(no changes since v1)

guix/git.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (17 lines)
diff --git a/guix/git.scm b/guix/git.scm
index be20cde019..dbc3b7caa7 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -439,7 +439,7 @@ (define* (update-cached-checkout url
#:recursive? recursive?)))
"Update the cached checkout of URL to REF in CACHE-DIRECTORY. Return three
values: the cache directory name, and the SHA1 commit (a string) corresponding
-to REF, and the relation of the new commit relative to STARTING-COMMIT (if
+to REF, and the relation of STARTING-COMMIT relative to the new commit (if
provided) as returned by 'commit-relation'.
REF is pair whose key is [branch | commit | tag | tag-or-commit ] and value

base-commit: a4bed14c438dc0cbc1c1885a38f8409c7fef7957
--
2.41.0
M
M
Maxim Cournoyer wrote on 15 Aug 2023 21:44
[PATCH v2 2/3] pull: Tag commit argument with 'tag-or-commit.
(address . 64746@debbugs.gnu.org)(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
a76e62f54c649713083ff3e51442c49f66857b22.1692128648.git.maxim.cournoyer@gmail.com
For compatibility with (guix git) procedures.

* guix/scripts/pull.scm (channel-list): Also accept tag-or-commit tagged
refspec.
---

New commit.

guix/scripts/pull.scm | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)

Toggle diff (25 lines)
diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm
index ecd264d3fa..9b78d4b5ca 100644
--- a/guix/scripts/pull.scm
+++ b/guix/scripts/pull.scm
@@ -166,7 +166,7 @@ (define %options
(alist-delete 'repository-url result))))
(option '("commit") #t #f
(lambda (opt name arg result)
- (alist-cons 'ref `(commit . ,arg) result)))
+ (alist-cons 'ref `(tag-or-commit . ,arg) result)))
(option '("branch") #t #f
(lambda (opt name arg result)
(alist-cons 'ref `(branch . ,arg) result)))
@@ -774,7 +774,8 @@ (define (channel-list opts)
(if (guix-channel? c)
(let ((url (or url (channel-url c))))
(match ref
- (('commit . commit)
+ ((or ('commit . commit)
+ ('tag-or-commit . commit))
(channel (inherit c)
(url url) (commit commit) (branch #f)))
(('branch . branch)
--
2.41.0
M
M
Maxim Cournoyer wrote on 15 Aug 2023 21:44
[PATCH v2 3/3] scripts: time-machine: Error when attempting to visit too old commits.
(address . 64746@debbugs.gnu.org)
7ce3ce800ab5dde1cdaaf272a54738be6256c1b3.1692128648.git.maxim.cournoyer@gmail.com
* doc/guix.texi (Invoking guix time-machine): Document limitation.
* guix/inferior.scm (cached-channel-instance): New VALIDATE-CHANNELS
argument. Use it to validate channels when there are no cache hit.
* guix/scripts/time-machine.scm
(%options): Tag the given reference with 'tag-or-commit instead of 'commit.
(%oldest-possible-commit): New variable.
(guix-time-machine) <validate-guix-channel>: New nested procedure. Pass it to
the 'cached-channel-instance' call.
* tests/guix-time-machine.sh: New test.
* Makefile.am (SH_TESTS): Register it.

Suggested-by: Simon Tournier <zimon.toutoune@gmail.com>
Reviewed-by: Ludovic Courtès <ludo@gnu.org>

---

Changes in v2:
- Test it
- Delay validation work to inside cached-channel-instance, when there's no
cache hit
- Expound doc to mention possible problems and possible workarounds

Makefile.am | 1 +
doc/guix.texi | 14 +++++++++
guix/inferior.scm | 57 ++++++++++++++++++++---------------
guix/scripts/time-machine.scm | 38 +++++++++++++++++++++--
tests/guix-time-machine.sh | 28 +++++++++++++++++
5 files changed, 110 insertions(+), 28 deletions(-)
create mode 100644 tests/guix-time-machine.sh

Toggle diff (243 lines)
diff --git a/Makefile.am b/Makefile.am
index 5ffcb6a12d..4228c07803 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -615,6 +615,7 @@ SH_TESTS = \
tests/guix-refresh.sh \
tests/guix-shell.sh \
tests/guix-shell-export-manifest.sh \
+ tests/guix-time-machine.sh \
tests/guix-graph.sh \
tests/guix-describe.sh \
tests/guix-repl.sh \
diff --git a/doc/guix.texi b/doc/guix.texi
index b50feed4c4..a3754b7019 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5060,6 +5060,20 @@ Invoking guix time-machine
large number of packages; the result is cached though and subsequent
commands targeting the same commit are almost instantaneous.
+Due to @command{guix time-machine} relying on the ``inferiors''
+mechanism (@pxref{Inferiors}), the oldest commit it can travel to is
+commit @samp{6298c3ff} (``v1.0.0''), dated May 1@sup{st}, 2019, which is
+the first release that included the inferiors mechanism. An error is
+returned when attempting to navigate to older commits. Note that
+although it should technically be possible to travel to such an old
+commit, the ease to do so will largely depend on the availability of
+binary substitutes. When traveling to a distant past, some packages may
+not easily build from source anymore. One such example are old versions
+of Python 2 which had time bombs in its test suite, in the form of
+expiring SSL certificates. This particular problem can be worked around
+by setting the hardware clock to a value in the past before attempting
+the build.
+
@quotation Note
The history of Guix is immutable and @command{guix time-machine}
provides the exact same software as they are in a specific Guix
diff --git a/guix/inferior.scm b/guix/inferior.scm
index 5dfd30a6c8..fca6fb4b22 100644
--- a/guix/inferior.scm
+++ b/guix/inferior.scm
@@ -871,11 +871,15 @@ (define* (cached-channel-instance store
#:key
(authenticate? #t)
(cache-directory (%inferior-cache-directory))
- (ttl (* 3600 24 30)))
+ (ttl (* 3600 24 30))
+ validate-channels)
"Return a directory containing a guix filetree defined by CHANNELS, a list of channels.
-The directory is a subdirectory of CACHE-DIRECTORY, where entries can be reclaimed after TTL seconds.
-This procedure opens a new connection to the build daemon. AUTHENTICATE?
-determines whether CHANNELS are authenticated."
+The directory is a subdirectory of CACHE-DIRECTORY, where entries can be
+reclaimed after TTL seconds. This procedure opens a new connection to the
+build daemon. AUTHENTICATE? determines whether CHANNELS are authenticated.
+VALIDATE-CHANNELS, if specified, must be a one argument procedure accepting a
+list of channels that can be used to validate the channels; it should raise an
+exception in case of problems."
(define commits
;; Since computing the instances of CHANNELS is I/O-intensive, use a
;; cheaper way to get the commit list of CHANNELS. This limits overhead
@@ -923,27 +927,30 @@ (define* (cached-channel-instance store
(if (file-exists? cached)
cached
- (run-with-store store
- (mlet* %store-monad ((instances
- -> (latest-channel-instances store channels
- #:authenticate?
- authenticate?))
- (profile
- (channel-instances->derivation instances)))
- (mbegin %store-monad
- ;; It's up to the caller to install a build handler to report
- ;; what's going to be built.
- (built-derivations (list profile))
-
- ;; Cache if and only if AUTHENTICATE? is true.
- (if authenticate?
- (mbegin %store-monad
- (symlink* (derivation->output-path profile) cached)
- (add-indirect-root* cached)
- (return cached))
- (mbegin %store-monad
- (add-temp-root* (derivation->output-path profile))
- (return (derivation->output-path profile)))))))))
+ (begin
+ (when (procedure? validate-channels)
+ (validate-channels channels))
+ (run-with-store store
+ (mlet* %store-monad ((instances
+ -> (latest-channel-instances store channels
+ #:authenticate?
+ authenticate?))
+ (profile
+ (channel-instances->derivation instances)))
+ (mbegin %store-monad
+ ;; It's up to the caller to install a build handler to report
+ ;; what's going to be built.
+ (built-derivations (list profile))
+
+ ;; Cache if and only if AUTHENTICATE? is true.
+ (if authenticate?
+ (mbegin %store-monad
+ (symlink* (derivation->output-path profile) cached)
+ (add-indirect-root* cached)
+ (return cached))
+ (mbegin %store-monad
+ (add-temp-root* (derivation->output-path profile))
+ (return (derivation->output-path profile))))))))))
(define* (inferior-for-channels channels
#:key
diff --git a/guix/scripts/time-machine.scm b/guix/scripts/time-machine.scm
index d7c71ef705..e4fe511382 100644
--- a/guix/scripts/time-machine.scm
+++ b/guix/scripts/time-machine.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2019 Konrad Hinsen <konrad.hinsen@fastmail.net>
;;; Copyright © 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -19,13 +20,15 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix scripts time-machine)
+ #:use-module (guix channels)
+ #:use-module (guix diagnostics)
#:use-module (guix ui)
#:use-module (guix scripts)
#:use-module (guix inferior)
#:use-module (guix store)
#:use-module (guix status)
#:use-module ((guix git)
- #:select (with-git-error-handling))
+ #:select (update-cached-checkout with-git-error-handling))
#:use-module ((guix utils)
#:select (%current-system))
#:use-module ((guix scripts pull)
@@ -38,9 +41,17 @@ (define-module (guix scripts time-machine)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-34)
#:use-module (srfi srfi-37)
+ #:use-module (srfi srfi-71)
#:export (guix-time-machine))
+;;; The required inferiors mechanism relied on by 'guix time-machine' was
+;;; firmed up in v1.0.0; it is the oldest, safest commit that can be travelled
+;;; to.
+(define %oldest-possible-commit
+ "6298c3ffd9654d3231a6f25390b056483e8f407c") ;v1.0.0
+
;;;
;;; Command-line options.
@@ -81,7 +92,7 @@ (define %options
(alist-delete 'repository-url result))))
(option '("commit") #t #f
(lambda (opt name arg result)
- (alist-cons 'ref `(commit . ,arg) result)))
+ (alist-cons 'ref `(tag-or-commit . ,arg) result)))
(option '("branch") #t #f
(lambda (opt name arg result)
(alist-cons 'ref `(branch . ,arg) result)))
@@ -140,8 +151,27 @@ (define-command (guix-time-machine . args)
(let* ((opts (parse-args args))
(channels (channel-list opts))
(command-line (assoc-ref opts 'exec))
+ (ref (assoc-ref opts 'ref))
(substitutes? (assoc-ref opts 'substitutes?))
(authenticate? (assoc-ref opts 'authenticate-channels?)))
+
+ (define (validate-guix-channel channels)
+ "Finds the Guix channel among CHANNELS, and validates that REF as
+captured from the closure, a git reference specification such as a commit hash
+or tag associated to CHANNEL, is valid and new enough to satisfy the 'guix
+time-machine' requirements. A `formatted-message' condition is raised
+otherwise."
+ (let* ((guix-channel (find guix-channel? channels))
+ (checkout commit relation (update-cached-checkout
+ (channel-url guix-channel)
+ #:ref (or ref '())
+ #:starting-commit
+ %oldest-possible-commit)))
+ (unless (memq relation '(ancestor self))
+ (raise (formatted-message
+ (G_ "cannot travel past commit `~a' from May 1st, 2019")
+ (string-take %oldest-possible-commit 12))))))
+
(when command-line
(let* ((directory
(with-store store
@@ -153,6 +183,8 @@ (define-command (guix-time-machine . args)
#:dry-run? #f)
(set-build-options-from-command-line store opts)
(cached-channel-instance store channels
- #:authenticate? authenticate?)))))
+ #:authenticate? authenticate?
+ #:validate-channels
+ validate-guix-channel)))))
(executable (string-append directory "/bin/guix")))
(apply execl (cons* executable executable command-line))))))))
diff --git a/tests/guix-time-machine.sh b/tests/guix-time-machine.sh
new file mode 100644
index 0000000000..8b62ef75ea
--- /dev/null
+++ b/tests/guix-time-machine.sh
@@ -0,0 +1,28 @@
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+#
+# Test the 'guix time-machine' command-line utility.
+#
+
+guix time-machine --version
+
+# Visiting a commit older than v1.0.0 fails.
+! guix time-machine --commit=v0.15.0
+
+exit 0
--
2.41.0
S
S
Simon Tournier wrote on 16 Aug 2023 16:46
Re: [bug#64746] [PATCH 2/2] scripts: time-machine: Error when attempting to visit too old commits.
87sf8jf3ja.fsf@gmail.com
Hi,

Thanks Maxim for this improvement.


On Tue, 15 Aug 2023 at 18:14, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (4 lines)
> Following your question earlier today on IRC, I realized that this would
> unconditionally add a Git checkout update every time ‘time-machine’ is
> started.

Please note that if git.savannah.gnu.org is not reachable, then “guix
time-machine” fails.

Let start with the regular:

Toggle snippet (23 lines)
$ guix describe
Generation 26 Jul 12 2023 09:13:39 (current)
guix 4a027d2
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 4a027d2b0ee68e39f21f6802a8cd1751d3065330

$ guix time-machine --commit=4a027d2 -- describe
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
building /gnu/store/sg8ca36rlbh4il6jy8dk2gr33lxm4z8q-compute-guix-derivation.drv...
Computing Guix derivation for 'x86_64-linux'... |
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
The following derivations will be built:
[...]
building profile with 1 package...
guix 4a027d2
repository URL: https://git.savannah.gnu.org/git/guix.git
commit: 4a027d2b0ee68e39f21f6802a8cd1751d3065330

So far, so good. Here all is cached and so on. Now, let make
git.savannah.gnu.org unreachable by tweaking some stuff. Then,

Toggle snippet (4 lines)
$ guix time-machine --commit=4a027d2 -- describe
guix time-machine: error: Git error: failed to resolve address for git.savannah.gnu.org: Name or service not known

Well, even if this patch will add another inefficiency, there is already
one. :-) I would say without having looking closely to the code that the
issue comes because something™ tries to connect before checking if the
commit is already in the local checkout.


Toggle quote (5 lines)
> This would noticeably degrade performance in the cache-hit case (when
> running a cached channel set). Currently, on cache hits,
> ‘cached-channel-instance’ directly returns ~/.cache/guix/profiles/xyz
> without performing any Git or (guix store) operation.

As shown above, I guess there is bug…


Toggle quote (4 lines)
> This is important because it guarantees that one can run ‘guix
> time-machine -C channels.scm -- COMMAND’ at pretty much the same speed
> as ‘guix COMMAND’.

Yes but I guess it could also be improved as shown above. :-)

The check if the requested commit is newer than the
%oldest-possible-commit should use the Git history graph closure
similarly as the authentication mechanism, no?

And this check should come after checking the cache, no?

Cheers,
simon
S
S
Simon Tournier wrote on 16 Aug 2023 17:02
Re: [bug#64746] [PATCH v2 2/3] pull: Tag commit argument with 'tag-or-commit.
87o7j7f2tb.fsf@gmail.com
Hi Maxim,

On Tue, 15 Aug 2023 at 15:44, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
Toggle quote (6 lines)
> For compatibility with (guix git) procedures.
>
> * guix/scripts/pull.scm (channel-list): Also accept tag-or-commit tagged
> refspec.
> ---

I am not sure to understand these both changes.

Toggle quote (15 lines)
>
> guix/scripts/pull.scm | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm
> index ecd264d3fa..9b78d4b5ca 100644
> --- a/guix/scripts/pull.scm
> +++ b/guix/scripts/pull.scm
> @@ -166,7 +166,7 @@ (define %options
> (alist-delete 'repository-url result))))
> (option '("commit") #t #f
> (lambda (opt name arg result)
> - (alist-cons 'ref `(commit . ,arg) result)))
> + (alist-cons 'ref `(tag-or-commit . ,arg) result)))

Well, why not. :-)

Toggle quote (11 lines)
> (option '("branch") #t #f
> (lambda (opt name arg result)
> (alist-cons 'ref `(branch . ,arg) result)))
> @@ -774,7 +774,8 @@ (define (channel-list opts)
> (if (guix-channel? c)
> (let ((url (or url (channel-url c))))
> (match ref
> - (('commit . commit)
> + ((or ('commit . commit)
> + ('tag-or-commit . commit))

Here, why not also add 'tag?. Hum, I am missing why this ’tag-or-commit
would be required.

Cheers,
simon
S
S
Simon Tournier wrote on 16 Aug 2023 17:39
Re: [bug#64746] [PATCH v2 3/3] scripts: time-machine: Error when attempting to visit too old commits.
87jztvf13c.fsf@gmail.com
Hi Maxim,

For the record, I have documented [1] the various roadblocks when using
“guix time-machine” rebuilding all from source. Time-bomb is one among
other annoyances – I have in mind the complete bootstrap.

Well, as Ludo pointed, the CI is currently building all the past
releases. Waiting the fixes for all the bugs, I suggest that we retain
the substitutes for the release. I mean that,

guix time-machine --commit=v1.X.0 -- help

just works when substitutes are available. If the project is lacking
disk space, the University of Montpellier (France) is proposing to store
some binary artifact outputs. At least, they were proposing back on
past September in 10 Years of Guix event. :-) There is discussion on
guix-sysadmin, I guess. Maybe we could resume this discussion and
complete the last steps. WDYT?



On Tue, 15 Aug 2023 at 15:44, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
Toggle quote (11 lines)
> * doc/guix.texi (Invoking guix time-machine): Document limitation.
> * guix/inferior.scm (cached-channel-instance): New VALIDATE-CHANNELS
> argument. Use it to validate channels when there are no cache hit.
> * guix/scripts/time-machine.scm
> (%options): Tag the given reference with 'tag-or-commit instead of 'commit.
> (%oldest-possible-commit): New variable.
> (guix-time-machine) <validate-guix-channel>: New nested procedure. Pass it to
> the 'cached-channel-instance' call.
> * tests/guix-time-machine.sh: New test.
> * Makefile.am (SH_TESTS): Register it.

Cool! That’s a nice usability improvement.

Toggle quote (14 lines)
> diff --git a/doc/guix.texi b/doc/guix.texi
> index b50feed4c4..a3754b7019 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -5060,6 +5060,20 @@ Invoking guix time-machine
> large number of packages; the result is cached though and subsequent
> commands targeting the same commit are almost instantaneous.
>
> +Due to @command{guix time-machine} relying on the ``inferiors''
> +mechanism (@pxref{Inferiors}), the oldest commit it can travel to is
> +commit @samp{6298c3ff} (``v1.0.0''), dated May 1@sup{st}, 2019, which is
> +the first release that included the inferiors mechanism. An error is
> +returned when attempting to navigate to older commits.

There is also some issue with bootstrapping depending on your hardware.

About time-bomb, there are also gnutls and openssl or libgit2. It was
probably transparent for you because there are substitutable, I guess.
While Python 2 had probably been removed for some reasons.

Well, I would move the workaround to some dedicated block and move this
comment after the note about security

Toggle snippet (32 lines)
@quotation Note
The history of Guix is immutable and @command{guix time-machine}
provides the exact same software as they are in a specific Guix
revision. Naturally, no security fixes are provided for old versions
of Guix or its channels. A careless use of @command{guix time-machine}
opens the door to security vulnerabilities. @xref{Invoking guix pull,
@option{--allow-downgrades}}.
@end quotation
+Due to @command{guix time-machine} relying on the ``inferiors''
+mechanism (@pxref{Inferiors}), the oldest commit it can travel to is
+commit @samp{6298c3ff} (``v1.0.0''), dated May 1@sup{st}, 2019, which is
+the first release that included the inferiors mechanism. An error is
+returned when attempting to navigate to older commits.
+
+@quotation Note
+Although it should technically be possible to travel to such an old
+revision, the ease to do so will largely depend on the availability of
+binary substitutes. When traveling to a distant past, some packages may
+not easily build from source anymore. One such example are old versions
+of Python 2 which had time bombs in its test suite, in the form of
+expiring SSL certificates. This particular problem can be worked around
+by setting the hardware clock to a value in the past before attempting
+the build.
+@end quotation
+
The general syntax is:
@example


Toggle quote (5 lines)
> new file mode 100644
> index 0000000000..8b62ef75ea
> --- /dev/null
> +++ b/tests/guix-time-machine.sh

[...]

Toggle quote (3 lines)
> +# Visiting a commit older than v1.0.0 fails.
> +! guix time-machine --commit=v0.15.0

Cool to add test. But this test needs a network access:

Toggle snippet (4 lines)
$ ./pre-inst-env guix time-machine --commit=v0.15.0 -- describe
guix time-machine: error: Git error: failed to resolve address for git.savannah.gnu.org: Name or service not known

It’s as I said elsewhere. :-) Well, I have not investigated more but I
guess a bug in some Git manipulation.


Cheers,
simon
M
M
Maxim Cournoyer wrote on 16 Aug 2023 20:41
Re: [bug#64746] [PATCH 2/2] scripts: time-machine: Error when attempting to visit too old commits.
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87fs4ij0cs.fsf@gmail.com
Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (46 lines)
> Hi,
>
> Thanks Maxim for this improvement.
>
>
> On Tue, 15 Aug 2023 at 18:14, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Following your question earlier today on IRC, I realized that this would
>> unconditionally add a Git checkout update every time ‘time-machine’ is
>> started.
>
> Please note that if git.savannah.gnu.org is not reachable, then “guix
> time-machine” fails.
>
> Let start with the regular:
>
> $ guix describe
> Generation 26 Jul 12 2023 09:13:39 (current)
> guix 4a027d2
> repository URL: https://git.savannah.gnu.org/git/guix.git
> branch: master
> commit: 4a027d2b0ee68e39f21f6802a8cd1751d3065330
>
> $ guix time-machine --commit=4a027d2 -- describe
> Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
> building /gnu/store/sg8ca36rlbh4il6jy8dk2gr33lxm4z8q-compute-guix-derivation.drv...
> Computing Guix derivation for 'x86_64-linux'... |
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
> The following derivations will be built:
> [...]
> building profile with 1 package...
> guix 4a027d2
> repository URL: https://git.savannah.gnu.org/git/guix.git
> commit: 4a027d2b0ee68e39f21f6802a8cd1751d3065330
>
>
> So far, so good. Here all is cached and so on. Now, let make
> git.savannah.gnu.org unreachable by tweaking some stuff. Then,
>
> $ guix time-machine --commit=4a027d2 -- describe
> guix time-machine: error: Git error: failed to resolve address for git.savannah.gnu.org: Name or service not known
>

[...]

Toggle quote (6 lines)
> The check if the requested commit is newer than the
> %oldest-possible-commit should use the Git history graph closure
> similarly as the authentication mechanism, no?
>
> And this check should come after checking the cache, no?

Interesting finding! I think it'd make sense to raise this issue
separately and discuss its resolution there, too keep things focused and
discoverable :-).

--
Thanks,
Maxim
M
M
Maxim Cournoyer wrote on 16 Aug 2023 20:47
Re: [bug#64746] [PATCH v2 2/3] pull: Tag commit argument with 'tag-or-commit.
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87bkf6j02r.fsf@gmail.com
Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (28 lines)
> Hi Maxim,
>
> On Tue, 15 Aug 2023 at 15:44, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>> For compatibility with (guix git) procedures.
>>
>> * guix/scripts/pull.scm (channel-list): Also accept tag-or-commit tagged
>> refspec.
>> ---
>
> I am not sure to understand these both changes.
>
>>
>> guix/scripts/pull.scm | 5 +++--
>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm
>> index ecd264d3fa..9b78d4b5ca 100644
>> --- a/guix/scripts/pull.scm
>> +++ b/guix/scripts/pull.scm
>> @@ -166,7 +166,7 @@ (define %options
>> (alist-delete 'repository-url result))))
>> (option '("commit") #t #f
>> (lambda (opt name arg result)
>> - (alist-cons 'ref `(commit . ,arg) result)))
>> + (alist-cons 'ref `(tag-or-commit . ,arg) result)))
>
> Well, why not. :-)

The reason is to standardize the API of (guix pull) and (guix git),
whose procedure had a different expectation for 'ref' objects.

--
Thanks,
Maxim
M
M
Maxim Cournoyer wrote on 17 Aug 2023 03:41
Re: [bug#64746] [PATCH v2 3/3] scripts: time-machine: Error when attempting to visit too old commits.
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87y1iah2d0.fsf@gmail.com
Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (19 lines)
> Hi Maxim,
>
> For the record, I have documented [1] the various roadblocks when using
> “guix time-machine” rebuilding all from source. Time-bomb is one among
> other annoyances – I have in mind the complete bootstrap.
>
> Well, as Ludo pointed, the CI is currently building all the past
> releases. Waiting the fixes for all the bugs, I suggest that we retain
> the substitutes for the release. I mean that,
>
> guix time-machine --commit=v1.X.0 -- help
>
> just works when substitutes are available. If the project is lacking
> disk space, the University of Montpellier (France) is proposing to store
> some binary artifact outputs. At least, they were proposing back on
> past September in 10 Years of Guix event. :-) There is discussion on
> guix-sysadmin, I guess. Maybe we could resume this discussion and
> complete the last steps. WDYT?

That make sense. The current means this can be achieved is by having a
jobset for each release in Cuirass, as hinted in doc/release.org from
the guix-maintenance repo:

Toggle snippet (9 lines)
** Adding a Cuirass jobset for branch =version-X.Y.Z=

This jobset will have to be kept until the next release, so that
substitutes remain available. The easiest way to add a new jobset is
directly via the web interface of Cuirass. To be allowed to do so,
you must authenticate with the Cuirass instance via a private TLS
certificate imported into your browser.

They should be added declaratively in the guix-maintenance repo to avoid
loosing them. Would you like to give it a try?

[...]

Toggle quote (49 lines)
>> diff --git a/doc/guix.texi b/doc/guix.texi
>> index b50feed4c4..a3754b7019 100644
>> --- a/doc/guix.texi
>> +++ b/doc/guix.texi
>> @@ -5060,6 +5060,20 @@ Invoking guix time-machine
>> large number of packages; the result is cached though and subsequent
>> commands targeting the same commit are almost instantaneous.
>>
>> +Due to @command{guix time-machine} relying on the ``inferiors''
>> +mechanism (@pxref{Inferiors}), the oldest commit it can travel to is
>> +commit @samp{6298c3ff} (``v1.0.0''), dated May 1@sup{st}, 2019, which is
>> +the first release that included the inferiors mechanism. An error is
>> +returned when attempting to navigate to older commits.
>
> There is also some issue with bootstrapping depending on your hardware.
>
> About time-bomb, there are also gnutls and openssl or libgit2. It was
> probably transparent for you because there are substitutable, I guess.
> While Python 2 had probably been removed for some reasons.
>
> Well, I would move the workaround to some dedicated block and move this
> comment after the note about security
>
> @quotation Note
> The history of Guix is immutable and @command{guix time-machine}
> provides the exact same software as they are in a specific Guix
> revision. Naturally, no security fixes are provided for old versions
> of Guix or its channels. A careless use of @command{guix time-machine}
> opens the door to security vulnerabilities. @xref{Invoking guix pull,
> @option{--allow-downgrades}}.
> @end quotation
>
> +Due to @command{guix time-machine} relying on the ``inferiors''
> +mechanism (@pxref{Inferiors}), the oldest commit it can travel to is
> +commit @samp{6298c3ff} (``v1.0.0''), dated May 1@sup{st}, 2019, which is
> +the first release that included the inferiors mechanism. An error is
> +returned when attempting to navigate to older commits.
> +
> +@quotation Note
> +Although it should technically be possible to travel to such an old
> +revision, the ease to do so will largely depend on the availability of
> +binary substitutes. When traveling to a distant past, some packages may
> +not easily build from source anymore. One such example are old versions
> +of Python 2 which had time bombs in its test suite, in the form of
> +expiring SSL certificates. This particular problem can be worked around
> +by setting the hardware clock to a value in the past before attempting
> +the build.
> +@end quotation

Good suggestion, done!

Toggle quote (24 lines)
> The general syntax is:
>
> @example
>
>
>
>> new file mode 100644
>> index 0000000000..8b62ef75ea
>> --- /dev/null
>> +++ b/tests/guix-time-machine.sh
>
> [...]
>
>> +# Visiting a commit older than v1.0.0 fails.
>> +! guix time-machine --commit=v0.15.0
>
> Cool to add test. But this test needs a network access:
>
> $ ./pre-inst-env guix time-machine --commit=v0.15.0 -- describe
> guix time-machine: error: Git error: failed to resolve address for git.savannah.gnu.org: Name or service not known
>
> It’s as I said elsewhere. :-) Well, I have not investigated more but I
> guess a bug in some Git manipulation.

From my testing, the test doesn't require networking, perhaps because it
doesn't use the --commit or --branch arguments. I've tested that with
this in /etc/hosts, with the first IP being bogus:

Toggle snippet (3 lines)
192.168.254.254 git.savannah.gnu.org savannah

I've investigated a bit, and it seems that reaching to the network is
only done when using tags or branch names, not exact commits. That is
expected I think, since tags or branch names are not immutable in Git,
while a commit ID is.

I've now installed this change; thanks for the review and suggestions!

--
Thanks,
Maxim
Closed
S
S
Simon Tournier wrote on 17 Aug 2023 16:45
Re: [bug#64746] [PATCH v2 2/3] pull: Tag commit argument with 'tag-or-commit.
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87a5up4tie.fsf@gmail.com
Hi Maxim,

On Wed, 16 Aug 2023 at 14:47, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (5 lines)
>>> (option '("commit") #t #f
>>> (lambda (opt name arg result)
>>> - (alist-cons 'ref `(commit . ,arg) result)))
>>> + (alist-cons 'ref `(tag-or-commit . ,arg) result)))

[...]

Toggle quote (8 lines)
> (match ref
> - (('commit . commit)
> + ((or ('commit . commit)
> + ('tag-or-commit . commit))

> The reason is to standardize the API of (guix pull) and (guix git),
> whose procedure had a different expectation for 'ref' objects.

My point is that this ’or’ is useless, IIUC. Well, I have removed it in
the series fixing the annoyance with the network access of “guix
time-machine”.

Cheers,
simon
M
M
Maxim Cournoyer wrote on 17 Aug 2023 20:16
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87jzttfsb8.fsf@gmail.com
Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (23 lines)
> Hi Maxim,
>
> On Wed, 16 Aug 2023 at 14:47, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>>>> (option '("commit") #t #f
>>>> (lambda (opt name arg result)
>>>> - (alist-cons 'ref `(commit . ,arg) result)))
>>>> + (alist-cons 'ref `(tag-or-commit . ,arg) result)))
>
> [...]
>
>> (match ref
>> - (('commit . commit)
>> + ((or ('commit . commit)
>> + ('tag-or-commit . commit))
>
>> The reason is to standardize the API of (guix pull) and (guix git),
>> whose procedure had a different expectation for 'ref' objects.
>
> My point is that this ’or’ is useless, IIUC. Well, I have removed it in
> the series fixing the annoyance with the network access of “guix
> time-machine”.

It isn't, unless you meant after applying further changes :-) You should
be able to see the problem by reverting that commit and running 'guix
time-machine --commit=v1.4.0 -- describe', for example.

--
Thanks,
Maxim
S
S
Simon Tournier wrote on 17 Aug 2023 20:47
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
CAJ3okZ3PAcWkLFUROuQ8nBDGS=FriAYw6WLZ7+xAeH1ypzrvzA@mail.gmail.com
Re Maxim,

On Thu, 17 Aug 2023 at 20:16, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (25 lines)
> > On Wed, 16 Aug 2023 at 14:47, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
> >
> >>>> (option '("commit") #t #f
> >>>> (lambda (opt name arg result)
> >>>> - (alist-cons 'ref `(commit . ,arg) result)))
> >>>> + (alist-cons 'ref `(tag-or-commit . ,arg) result)))
> >
> > [...]
> >
> >> (match ref
> >> - (('commit . commit)
> >> + ((or ('commit . commit)
> >> + ('tag-or-commit . commit))
> >
> >> The reason is to standardize the API of (guix pull) and (guix git),
> >> whose procedure had a different expectation for 'ref' objects.
> >
> > My point is that this ’or’ is useless, IIUC. Well, I have removed it in
> > the series fixing the annoyance with the network access of “guix
> > time-machine”.
>
> It isn't, unless you meant after applying further changes :-) You should
> be able to see the problem by reverting that commit and running 'guix
> time-machine --commit=v1.4.0 -- describe', for example.

Yes for sure because you introduced this in guix/scripts/time-machine.scm,

(lambda (opt name arg result)
- (alist-cons 'ref `(commit . ,arg) result)))
+ (alist-cons 'ref `(tag-or-commit . ,arg) result)))

with the previous commit 79ec651a286c71a3d4c72be33a1f80e76a560031.

Well, I feel there is a misunderstanding. :-) And maybe I am missing
something... IIUC, the option parser:

(option '("commit") #t #f
(lambda (opt name arg result)
(alist-cons 'ref `(commit . ,arg) result)))

implemented in guix/scrtips/pull.scm provides a way to construct this
REF. This way should be compliant with the other one in
guix/scripts/time-machine.scm -- at least for being consistent. And I
am missing the reason why you introduced this difference.

If the both option parsers use the same way, then the 'or' is useless.

As I said initially commenting this patch v2 2/3:

Toggle snippet (9 lines)
> For compatibility with (guix git) procedures.
>
> * guix/scripts/pull.scm (channel-list): Also accept tag-or-commit tagged
> refspec.
> ---

I am not sure to understand these both changes.

Anyway, my other message [1] in #65352 contains the two alternatives
for closing this discussion. :-)



Cheers,
simon
M
M
Maxim Cournoyer wrote on 23 Aug 2023 04:54
Re: bug#64746: [PATCH 2/2] scripts: time-machine: Error when attempting to visit too old commits.
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87y1i2a2p4.fsf_-_@gmail.com
Hi again,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (37 lines)
> Re Maxim,
>
> On Thu, 17 Aug 2023 at 20:16, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>> > On Wed, 16 Aug 2023 at 14:47, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>> >
>> >>>> (option '("commit") #t #f
>> >>>> (lambda (opt name arg result)
>> >>>> - (alist-cons 'ref `(commit . ,arg) result)))
>> >>>> + (alist-cons 'ref `(tag-or-commit . ,arg) result)))
>> >
>> > [...]
>> >
>> >> (match ref
>> >> - (('commit . commit)
>> >> + ((or ('commit . commit)
>> >> + ('tag-or-commit . commit))
>> >
>> >> The reason is to standardize the API of (guix pull) and (guix git),
>> >> whose procedure had a different expectation for 'ref' objects.
>> >
>> > My point is that this ’or’ is useless, IIUC. Well, I have removed it in
>> > the series fixing the annoyance with the network access of “guix
>> > time-machine”.
>>
>> It isn't, unless you meant after applying further changes :-) You should
>> be able to see the problem by reverting that commit and running 'guix
>> time-machine --commit=v1.4.0 -- describe', for example.
>
> Yes for sure because you introduced this in guix/scripts/time-machine.scm,
>
> (lambda (opt name arg result)
> - (alist-cons 'ref `(commit . ,arg) result)))
> + (alist-cons 'ref `(tag-or-commit . ,arg) result)))
>
> with the previous commit 79ec651a286c71a3d4c72be33a1f80e76a560031.

It's a bit convoluted, but there are three things involved:

1. (guix scripts time-machine)
2. (guix pull)
3. (guix git)

They are involved in that order, if I remember correctly.

Now the important part is that the update-cached-checkout from (guix
git), newly used in (guix scripts time-machine), should be passed a
tag-or-commit ref and not a commit one if we want to support both tags
or commits (otherwise tags would throw an error about not respecting a
git hash format).

Since 'guix time-machine --commit' is documented as accepting a tag or
commit, it makes sense to tag it as such at that point.

I hope this clarifies it?

--
Thanks,
Maxim
S
S
Simon Tournier wrote on 23 Aug 2023 10:27
Re: [bug#64746] [PATCH 2/2] scripts: time-machine: Error when attempting to visit too old commits.
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
86o7iy40zc.fsf@gmail.com
Hi Maxim,

On Tue, 22 Aug 2023 at 22:54, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (4 lines)
> 1. (guix scripts time-machine)
> 2. (guix pull)
> 3. (guix git)

[...]

Toggle quote (2 lines)
> I hope this clarifies it?

All is clear since my initial comment [1] about this patch. ;-) And my
proposal for improving the asymmetry your patch introduced is summarized
by the alternative options I wrote in [2].

Therefore, I will continue the discussion in #65352. :-)


Cheers,
simon
?