Installer does not show what is being downloaded

  • Done
  • quality assurance status badge
Details
4 participants
  • Josselin Poiret
  • Julien Lepiller
  • Ludovic Courtès
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Julien Lepiller
Severity
important
Merged with
J
J
Julien Lepiller wrote on 8 Oct 2022 12:34
(address . bug-guix@gnu.org)
960E16D9-C67D-42E0-B3DE-75CA3041F738@lepiller.eu
I installed Guix on a new drive, so I tried the installer from the latest image. After reviewing the system config, the installer goes to a black screen where guix output is shown. Although I can see messages about substitutes being updated, and how much will be downloaded, all download lines are replaced with a seemingly random number of dots:

substitute: mise à jour des substituts depuis «…»... 100.0 %
4.5 Mo seront téléchargés.
.
...
.
.
......

Maybe this is related to the translation?
Attachment: file
J
J
Josselin Poiret wrote on 8 Oct 2022 20:09
87sfjy4276.fsf@jpoiret.xyz
Hi Julien,

Julien Lepiller <julien@lepiller.eu> writes:
Toggle quote (12 lines)
> I installed Guix on a new drive, so I tried the installer from the latest image. After reviewing the system config, the installer goes to a black screen where guix output is shown. Although I can see messages about substitutes being updated, and how much will be downloaded, all download lines are replaced with a seemingly random number of dots:
>
> substitute: mise à jour des substituts depuis «…»... 100.0 %
> 4.5 Mo seront téléchargés.
> .
> ...
> .
> .
> ......
>
> Maybe this is related to the translation?

I agree that this does look pretty bad, but this is because we capture
the guix command output to a log file so that we're able to include it
in the installer dump if anything goes wrong. Guix detects it's not
talking directly to a terminal and so doesn't try to use terminal
specific escape codes to display the download progress. I don't know
how we could improve on that front though, since in general capturing
the guix command output helps with debugging issues.

Best,
--
Josselin Poiret
L
L
Ludovic Courtès wrote on 9 Oct 2022 18:56
(name . Josselin Poiret)(address . dev@jpoiret.xyz)
87bkql541o.fsf@gnu.org
Hello!

Josselin Poiret <dev@jpoiret.xyz> skribis:

Toggle quote (21 lines)
> Julien Lepiller <julien@lepiller.eu> writes:
>> I installed Guix on a new drive, so I tried the installer from the latest image. After reviewing the system config, the installer goes to a black screen where guix output is shown. Although I can see messages about substitutes being updated, and how much will be downloaded, all download lines are replaced with a seemingly random number of dots:
>>
>> substitute: mise à jour des substituts depuis «…»... 100.0 %
>> 4.5 Mo seront téléchargés.
>> .
>> ...
>> .
>> .
>> ......
>>
>> Maybe this is related to the translation?
>
> I agree that this does look pretty bad, but this is because we capture
> the guix command output to a log file so that we're able to include it
> in the installer dump if anything goes wrong. Guix detects it's not
> talking directly to a terminal and so doesn't try to use terminal
> specific escape codes to display the download progress. I don't know
> how we could improve on that front though, since in general capturing
> the guix command output helps with debugging issues.

This is a duplicate of https://issues.guix.gnu.org/55360, marked as
important.

I think capturing all the output as-is is overkill; one option I
suggested in the bug report above is to syslog specific events of
interest during ‘guix system init’.

Thoughts?

Ludo’.
L
L
Ludovic Courtès wrote on 9 Oct 2022 18:56
control message for bug #58375
(address . control@debbugs.gnu.org)
87a665541i.fsf@gnu.org
severity 58375 important
quit
L
L
Ludovic Courtès wrote on 9 Oct 2022 18:56
control message for bug #55360
(address . control@debbugs.gnu.org)
878rlp5418.fsf@gnu.org
merge 55360 58375
quit
M
M
Mathieu Othacehe wrote on 13 Oct 2022 09:04
Re: bug#58375: Installer does not show what is being downloaded
(name . Ludovic Courtès)(address . ludo@gnu.org)
87lepk5hnh.fsf_-_@gnu.org
Hey Ludo!

Toggle quote (4 lines)
> I’m not sure it’s a good idea for ‘guix system init’: we’d be logging
> mostly progress bars, package names, and the likes to syslog—not super
> useful. So I’d suggest not capturing stdout of ‘guix system init’.

In the bug report https://issues.guix.gnu.org/57983capturing the 'guix
system init' output highlighted a "guix substitute" crash. So it does
seem like a useful mechanism, especially while 56005 is still open.

Now the current situation is also not really acceptable. What about
hiding the "guix system init" output completely and display a progress
bar page instead?

Thanks,

Mathieu
L
L
Ludovic Courtès wrote on 13 Oct 2022 15:21
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87zgdzq2oe.fsf@gnu.org
Hi!

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (8 lines)
>> I’m not sure it’s a good idea for ‘guix system init’: we’d be logging
>> mostly progress bars, package names, and the likes to syslog—not super
>> useful. So I’d suggest not capturing stdout of ‘guix system init’.
>
> In the bug report https://issues.guix.gnu.org/57983 capturing the 'guix
> system init' output highlighted a "guix substitute" crash. So it does
> seem like a useful mechanism, especially while 56005 is still open.

installer-specific, so it’s annoying that we have to prepare for that,
but we can’t deny such bugs exist and prevent installation.

If we really want to capture the output of ‘guix system init’, then we
need to open a pseudo-terminal with ‘openpty’ & co. instead of ‘pipe’ in
‘run-external-command-with-handler’. That may be relatively easy
actually.

But then I think this should be used sparsely, maybe only for ‘guix
system init’.

Toggle quote (2 lines)
> Now the current situation is also not really acceptable.

Nope. :-)

Toggle quote (3 lines)
> What about hiding the "guix system init" output completely and display
> a progress bar page instead?

I don’t think we can do that (with grafts, only part of the build plan
is known upfront so we don’t even know beforehand how many items will be
built/downloaded).

Also, I think there are two strategies: either we run ‘guix system
init’, in which case we let its output through, or we integrate ‘guix
system init’ functionality in the installer so we have more fine-grain
control over the process, in which case we could also have more
graphical output or something.

That second solution is a lot of work, though.

Thanks,
Ludo’.
M
M
Mathieu Othacehe wrote on 14 Oct 2022 17:44
(name . Ludovic Courtès)(address . ludo@gnu.org)
871qrabeav.fsf_-_@gnu.org
Hey,

Toggle quote (5 lines)
> If we really want to capture the output of ‘guix system init’, then we
> need to open a pseudo-terminal with ‘openpty’ & co. instead of ‘pipe’ in
> ‘run-external-command-with-handler’. That may be relatively easy
> actually.

So I implemented your proposal. It seems to be working quite well. As
discussed on #guix, we could avoid to dump the download bars to the
syslog if the "guix system init" command succeeds. However, it seems
quite tricky in the current implementation where the syslog dumping is
actually a hook (%syslog-line-hook).

Fixing this issue, I also realized that when the "guix system init"
command fails, the user is only offered to resume the installation or
restart it.

In cases where "guix system init" failed because of a network issue, or
because a partition was too small, restarting/resuming seems like the
right thing to do.

However, when the installer failed because "guix system init" crashed or
segfaulted, restarting/resuming won't probably help, and dumping the
crash is probably the best way to get help. That's why I added in a
second patch, a new button "Report the failure" to the
"run-install-failed-page".

Thanks,

Mathieu
Attachment: 0001-installer-Run-the-guix-system-init-command-in-a-PTY.patch
From 159b82a013cad8c3c698031cce9ee07956153da3 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@gnu.org>
Date: Fri, 14 Oct 2022 17:33:28 +0200
Subject: [PATCH 2/2] installer: Propose the user to report a "guix system
init" failure.

* gnu/installer/newt/final.scm (run-install-failed-page): Add a "Report the
failure" button.
---
gnu/installer/newt/final.scm | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

Toggle diff (30 lines)
diff --git a/gnu/installer/newt/final.scm b/gnu/installer/newt/final.scm
index 7c3f73ee82..6e55be5067 100644
--- a/gnu/installer/newt/final.scm
+++ b/gnu/installer/newt/final.scm
@@ -80,16 +80,21 @@ (define (run-install-success-page)
(define (run-install-failed-page)
(match (current-clients)
(()
- (match (choice-window
+ (match (ternary-window
(G_ "Installation failed")
(G_ "Resume")
(G_ "Restart the installer")
+ (G_ "Report the failure")
(G_ "The final system installation step failed. You can resume from \
a specific step, or restart the installer."))
(1 (abort-to-prompt 'installer-step 'abort))
(2
;; Keep going, the installer will be restarted later on.
- #t)))
+ #t)
+ (3 (raise
+ (condition
+ (&message
+ (message "User abort.")))))))
(_
(send-to-clients '(installation-failure))
#t)))
--
2.37.3
L
L
Ludovic Courtès wrote on 18 Oct 2022 17:17
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87czapma90.fsf@gnu.org
Hi Mathieu,

Mathieu Othacehe <othacehe@gnu.org> skribis:

Toggle quote (6 lines)
> So I implemented your proposal. It seems to be working quite well. As
> discussed on #guix, we could avoid to dump the download bars to the
> syslog if the "guix system init" command succeeds. However, it seems
> quite tricky in the current implementation where the syslog dumping is
> actually a hook (%syslog-line-hook).

Yes.

I haven’t actually tested the patch but it LGTM. One thing to check is
whether ‘terminal-window-size’ returns something sensible for the
pseudo-terminal; it could be that we need an extra ioctl so the
pseudo-terminal has the same size as the actual terminal.

Toggle quote (14 lines)
> Fixing this issue, I also realized that when the "guix system init"
> command fails, the user is only offered to resume the installation or
> restart it.
>
> In cases where "guix system init" failed because of a network issue, or
> because a partition was too small, restarting/resuming seems like the
> right thing to do.
>
> However, when the installer failed because "guix system init" crashed or
> segfaulted, restarting/resuming won't probably help, and dumping the
> crash is probably the best way to get help. That's why I added in a
> second patch, a new button "Report the failure" to the
> "run-install-failed-page".

Neat!

Toggle quote (13 lines)
> From c6286404e9c4c0dc302c3d398a8f27b050cf4ce0 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe@gnu.org>
> Date: Fri, 14 Oct 2022 17:28:27 +0200
> Subject: [PATCH 1/2] installer: Run the "guix system init" command in a PTY.
>
> Fixes: <https://issues.guix.gnu.org/55360>
>
> * gnu/installer/utils.scm (run-external-command-with-handler/tty): New
> procedure.
> (run-external-command-with-line-hooks, run-command): Add a TTY? argument.
> * gnu/installer/final.scm (install-system): Call run-command with TTY?
> argument set to #true.

LGTM, modulo the terminal size issue mentioned above.

Toggle quote (9 lines)
> From 159b82a013cad8c3c698031cce9ee07956153da3 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe@gnu.org>
> Date: Fri, 14 Oct 2022 17:33:28 +0200
> Subject: [PATCH 2/2] installer: Propose the user to report a "guix system
> init" failure.
>
> * gnu/installer/newt/final.scm (run-install-failed-page): Add a "Report the
> failure" button.

LGTM.

Thank you!

Ludo’.
M
M
Mathieu Othacehe wrote on 20 Oct 2022 16:40
(name . Ludovic Courtès)(address . ludo@gnu.org)
875ygepnhz.fsf@gnu.org
Hey,

Thanks for having a look!

Toggle quote (5 lines)
> I haven’t actually tested the patch but it LGTM. One thing to check is
> whether ‘terminal-window-size’ returns something sensible for the
> pseudo-terminal; it could be that we need an extra ioctl so the
> pseudo-terminal has the same size as the actual terminal.

Well it returns 0 for all fields, but I tested on several screen sizes
and everything seems fine so I went ahead.

While testing I noticed two new issues though:

1. When the disk is GPT partitionned there is no confirmation page in
"run-label-page". Something I missed in #57232.

2. When there is an exception in run-external-command-with-handler/tty
for instance, the backtrace page is displayed in the PTY and the
keyboard shortcuts do not work anymore.

I'll address point 1 shortly but could use some advice for point 2.

Thanks,

Mathieu
Closed
?