futurile/guix-play
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

doc: Add the "Full-source Bootstrap".

Browse Source 
* doc/guix.texi (Reduced Binary Seed Bootstrap): Update and rename to...
(Full-Source Bootstrap): ...this.
* doc/images/gcc-core-mesboot0-graph.dot: Regenerate.
This commit is contained in:
Jan (janneke) Nieuwenhuizen 2021-01-02 10:47:15 +01:00
parent 454bb06e83
commit eeba993f57
No known key found for this signature in database
GPG Key ID: F3C1A0D9C1D65273
2 changed files with 117 additions and 145 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

75
doc/guix.texi
View File

@ -36,7 +36,7 @@ Copyright @copyright{} 2016, 2017, 2018, 2021 Chris Marusich@*
Copyright @copyright{} 2016, 2017, 2018, 2019, 2020, 2021, 2022 Efraim Flashner@*
Copyright @copyright{} 2016 John Darrington@*
Copyright @copyright{} 2016, 2017 Nikita Gillmann@*
Copyright @copyright{} 2016, 2017, 2018, 2019, 2020 Jan Nieuwenhuizen@*
Copyright @copyright{} 2016, 2017, 2018, 2019, 2021, 2022 Jan Nieuwenhuizen@*
Copyright @copyright{} 2016, 2017, 2018, 2019, 2020, 2021 Julien Lepiller@*
Copyright @copyright{} 2016 Alex ter Weele@*
Copyright @copyright{} 2016, 2017, 2018, 2019, 2020, 2021 Christopher Baines@*
@ -394,7 +394,7 @@ Installing Debugging Files
Bootstrapping
* Reduced Binary Seed Bootstrap:: A Bootstrap worthy of GNU.
* Full-source Bootstrap:: A Bootstrap worthy of GNU.
* Preparing to Use the Bootstrap Binaries:: Building that what matters most.
@end detailmenu
@ -39491,12 +39491,12 @@ re-create them if needed (@pxref{Preparing to Use the Bootstrap
Binaries}).
@menu
* Reduced Binary Seed Bootstrap:: A Bootstrap worthy of GNU.
* Full-source Bootstrap:: A Bootstrap worthy of GNU.
* Preparing to Use the Bootstrap Binaries:: Building that what matters most.
@end menu
@node Reduced Binary Seed Bootstrap
@section The Reduced Binary Seed Bootstrap
@node Full-source Bootstrap
@section The Full-source Bootstrap
Guix---like other GNU/Linux distributions---is traditionally bootstrapped from
a set of bootstrap binaries: Bourne shell, command-line tools provided by GNU
@ -39511,28 +39511,47 @@ about 250MB (@pxref{Bootstrappable Builds,,, mes, GNU Mes}). Auditing
or even inspecting these is next to impossible.
For @code{i686-linux} and @code{x86_64-linux}, Guix now features a
``Reduced Binary Seed'' bootstrap @footnote{We would like to say: ``Full
Source Bootstrap'' and while we are working towards that goal it would
be hyperbole to use that term for what we do now.}.
@dfn{full-source bootstrap}. This bootstrap is rooted in
@file{hex0-seed} from the @url{https://savannah.gnu.org/projects/stage0,
Stage0} package. The hex0 program is minimalist assembler: it reads
space-separated hexadecimal digits (nibbles) from a file, possibly
including comments, and emits on standard output the bytes corresponding
to those hexadecimal numbers. The source code of this initial hex0
program is a file called
@c XXX TODO: udpate to savannah url, once accepted there
@url{https://github.com/oriansj/bootstrap-seeds/blob/master/POSIX/x86/hex0_x86.hex0,@file{hex0_x86.hex0}}
and is written in the @code{hex0} language.
The Reduced Binary Seed bootstrap removes the most critical tools---from a
trust perspective---from the bootstrap binaries: GCC, Binutils and the GNU C
Library are replaced by: @code{bootstrap-mescc-tools} (a tiny assembler and
linker) and @code{bootstrap-mes} (a small Scheme Interpreter and a C compiler
written in Scheme and the Mes C Library, built for TinyCC and for GCC).
Hex0 is self-hosting, which means that it can build itself:
Using these new binary seeds the ``missing'' Binutils, GCC, and the GNU
C Library are built from source. From here on the more traditional
bootstrap process resumes. This approach has reduced the bootstrap
binaries in size to about 145MB in Guix v1.1.
@example
./hex0-seed hex0_x86.hex0 hex0
@end example
The next step that Guix has taken is to replace the shell and all its
Hex0 it is the ASCII-equivalent of the binary program and can be
produced by doing something much like:
@example
sed 's/[;#].*$//g' hex0_x86.hex0 | xxd -r -p > hex0
chmod +x hex0
@end example
It is because of this ASCII-binary equivalence that we can bless this
initial 357-byte binary as source, and hence `full-source bootstrap''.
The bootstrap then continues: @code{hex0} builds @code{hex1} and then on
to @code{M0}, @code{hex2}, @code{M1}, @code{mescc-tools} and finally
@code{M2-Planet}. Then, using @code{mescc-tools}, @code{M2-Planet} we
build Mes (@pxref{Top, GNU Mes Reference Manual,, mes, GNU Mes}, a
Scheme interpreter and C compiler in Scheme). Frome here on starts
the more traditional @code{C}-based bootstrap of the GNU System.
Another step that Guix has taken is to replace the shell and all its
utilities with implementations in Guile Scheme, the @emph{Scheme-only
bootstrap}. Gash (@pxref{Gash,,, gash, The Gash manual}) is a
POSIX-compatible shell that replaces Bash, and it comes with Gash Utils
which has minimalist replacements for Awk, the GNU Core Utilities, Grep,
Gzip, Sed, and Tar. The rest of the bootstrap binary seeds that were
removed are now built from source.
Gzip, Sed, and Tar.
Building the GNU System from source is currently only possible by adding
some historical GNU packages as intermediate steps@footnote{Packages
@ -39548,22 +39567,10 @@ The graph below shows the resulting dependency graph for
@code{gcc-core-mesboot0}, the bootstrap compiler used for the
traditional bootstrap of the rest of the Guix System.
@c ./pre-inst-env guix graph -e '(@@ (gnu packages commencement) gcc-core-mesboot0)' | sed -re 's,((bootstrap-mescc-tools|bootstrap-mes|guile-bootstrap).*shape =) box,\1 ellipse,' > doc/images/gcc-core-mesboot0-graph.dot
@c ./pre-inst-env guix graph -e '(@@ (gnu packages commencement) gcc-core-mesboot0)' | sed -re 's,((bootstrap-seeds|guile-bootstrap).*shape =) box,\1 ellipse,' > doc/images/gcc-core-mesboot0-graph.dot
@image{images/gcc-core-mesboot0-graph,6in,,Dependency graph of gcc-core-mesboot0}
The only significant binary bootstrap seeds that remain@footnote{
Ignoring the 68KB @code{mescc-tools}; that will be removed later,
together with @code{mes}.} are a Scheme interpreter and a Scheme
compiler: GNU Mes and GNU Guile@footnote{Not shown in this graph are the
static binaries for @file{bash}, @code{tar}, and @code{xz} that are used
to get Guile running.}.
This further reduction has brought down the size of the binary seed to
about 60MB for @code{i686-linux} and @code{x86_64-linux}.
Work is ongoing to remove all binary blobs from our free software
bootstrap stack, working towards a Full Source Bootstrap. Also ongoing
is work to bring these bootstraps to the @code{arm-linux} and
Work is ongoing to to bring these bootstraps to the @code{arm-linux} and
@code{aarch64-linux} architectures and to the Hurd.
If you are interested, join us on @samp{#bootstrappable} on the Freenode

187
doc/images/gcc-core-mesboot0-graph.dot
View File

@ -1,114 +1,79 @@
digraph "Guix package" {
"139781154519456" [label = "gcc-core-mesboot0@2.95.3", shape = box, fontname = Helvetica];
"139781154519456" -> "139781154519632" [color = darkgoldenrod];
"139781154519456" -> "139781154520688" [color = darkgoldenrod];
"139781154519456" -> "139781034971312" [color = darkgoldenrod];
"139781154519456" -> "139781154520160" [color = darkgoldenrod];
"139781154519456" -> "139781034971664" [color = darkgoldenrod];
"139781154519456" -> "139781154519984" [color = darkgoldenrod];
"139781154519456" -> "139781154519808" [color = darkgoldenrod];
"139781154519456" -> "139781154520512" [color = darkgoldenrod];
"139781154519456" -> "139781034971488" [color = darkgoldenrod];
"139781154519456" -> "139781034972896" [color = darkgoldenrod];
"139781154519456" -> "139781034972368" [color = darkgoldenrod];
"139781154519456" -> "139781034973072" [color = darkgoldenrod];
"139781154519456" -> "139781131267872" [color = darkgoldenrod];
"139781154519632" [label = "binutils-mesboot0@2.14", shape = box, fontname = Helvetica];
"139781154519632" -> "139781154520688" [color = peachpuff4];
"139781154519632" -> "139781034971312" [color = peachpuff4];
"139781154519632" -> "139781154520160" [color = peachpuff4];
"139781154519632" -> "139781034971664" [color = peachpuff4];
"139781154519632" -> "139781154519984" [color = peachpuff4];
"139781154519632" -> "139781154519808" [color = peachpuff4];
"139781154519632" -> "139781154520512" [color = peachpuff4];
"139781154519632" -> "139781034971488" [color = peachpuff4];
"139781154519632" -> "139781034972896" [color = peachpuff4];
"139781154519632" -> "139781034972368" [color = peachpuff4];
"139781154519632" -> "139781034973072" [color = peachpuff4];
"139781154519632" -> "139781131267872" [color = peachpuff4];
"139781154520688" [label = "bash-mesboot0@2.05b", shape = box, fontname = Helvetica];
"139781154520688" -> "139781034971488" [color = magenta];
"139781154520688" -> "139781034971840" [color = magenta];
"139781154520688" -> "139781034972896" [color = magenta];
"139781154520688" -> "139781034972368" [color = magenta];
"139781154520688" -> "139781034973072" [color = magenta];
"139781154520688" -> "139781131267872" [color = magenta];
"139781034971488" [label = "make-mesboot0@3.80", shape = box, fontname = Helvetica];
"139781034971488" -> "139781034971840" [color = dimgrey];
"139781034971488" -> "139781034972896" [color = dimgrey];
"139781034971488" -> "139781034972368" [color = dimgrey];
"139781034971488" -> "139781034973072" [color = dimgrey];
"139781034971488" -> "139781131267872" [color = dimgrey];
"139781034971840" [label = "tcc-boot0@0.9.26-1103-g6e62e0e", shape = box, fontname = Helvetica];
"139781034971840" -> "139781034972016" [color = magenta];
"139781034971840" -> "139781131266816" [color = magenta];
"139781034971840" -> "139781034972896" [color = magenta];
"139781034971840" -> "139781034972368" [color = magenta];
"139781034971840" -> "139781034973072" [color = magenta];
"139781034971840" -> "139781131267872" [color = magenta];
"139781034972016" [label = "mes-boot@0.22", shape = box, fontname = Helvetica];
"139781034972016" -> "139781034972192" [color = cyan3];
"139781034972016" -> "139781131266816" [color = cyan3];
"139781034972016" -> "139781034972896" [color = cyan3];
"139781034972016" -> "139781034972368" [color = cyan3];
"139781034972016" -> "139781034973072" [color = cyan3];
"139781034972016" -> "139781131267872" [color = cyan3];
"139781034972192" [label = "bootstrap-mes-rewired@0.19", shape = ellipse, fontname = Helvetica];
"139781034972192" -> "139781131266640" [color = darkseagreen];
"139781034972192" -> "139781034972896" [color = darkseagreen];
"139781131266640" [label = "bootstrap-mes@0", shape = ellipse, fontname = Helvetica];
"139781034972896" [label = "gash-boot@0.2.0", shape = box, fontname = Helvetica];
"139781034972896" -> "139781034973072" [color = darkviolet];
"139781034972896" -> "139781131267872" [color = darkviolet];
"139781034973072" [label = "bootar@1", shape = box, fontname = Helvetica];
"139781034973072" -> "139781131267872" [color = darkviolet];
"139781131267872" [label = "guile-bootstrap@2.0", shape = ellipse, fontname = Helvetica];
"139781131266816" [label = "bootstrap-mescc-tools@0.5.2", shape = ellipse, fontname = Helvetica];
"139781034972368" [label = "gash-utils-boot@0.1.0", shape = box, fontname = Helvetica];
"139781034972368" -> "139781034973072" [color = darkseagreen];
"139781034972368" -> "139781034972896" [color = darkseagreen];
"139781034972368" -> "139781131267872" [color = darkseagreen];
"139781034971312" [label = "bzip2-mesboot@1.0.8", shape = box, fontname = Helvetica];
"139781034971312" -> "139781034971488" [color = cyan3];
"139781034971312" -> "139781034971840" [color = cyan3];
"139781034971312" -> "139781034972896" [color = cyan3];
"139781034971312" -> "139781034972368" [color = cyan3];
"139781034971312" -> "139781034973072" [color = cyan3];
"139781034971312" -> "139781131267872" [color = cyan3];
"139781154520160" [label = "diffutils-mesboot@2.7", shape = box, fontname = Helvetica];
"139781154520160" -> "139781034971488" [color = red];
"139781154520160" -> "139781034971840" [color = red];
"139781154520160" -> "139781034972896" [color = red];
"139781154520160" -> "139781034972368" [color = red];
"139781154520160" -> "139781034973072" [color = red];
"139781154520160" -> "139781131267872" [color = red];
"139781034971664" [label = "gzip-mesboot@1.2.4", shape = box, fontname = Helvetica];
"139781034971664" -> "139781034971840" [color = red];
"139781034971664" -> "139781034972896" [color = red];
"139781034971664" -> "139781034972368" [color = red];
"139781034971664" -> "139781034973072" [color = red];
"139781034971664" -> "139781131267872" [color = red];
"139781154519984" [label = "patch-mesboot@2.5.9", shape = box, fontname = Helvetica];
"139781154519984" -> "139781034971488" [color = blue];
"139781154519984" -> "139781034971840" [color = blue];
"139781154519984" -> "139781034972896" [color = blue];
"139781154519984" -> "139781034972368" [color = blue];
"139781154519984" -> "139781034973072" [color = blue];
"139781154519984" -> "139781131267872" [color = blue];
"139781154519808" [label = "sed-mesboot0@1.18", shape = box, fontname = Helvetica];
"139781154519808" -> "139781034971488" [color = red];
"139781154519808" -> "139781034971840" [color = red];
"139781154519808" -> "139781034972896" [color = red];
"139781154519808" -> "139781034972368" [color = red];
"139781154519808" -> "139781034973072" [color = red];
"139781154519808" -> "139781131267872" [color = red];
"139781154520512" [label = "tcc-boot@0.9.27", shape = box, fontname = Helvetica];
"139781154520512" -> "139781034971312" [color = darkviolet];
"139781154520512" -> "139781034971488" [color = darkviolet];
"139781154520512" -> "139781034971840" [color = darkviolet];
"139781154520512" -> "139781034972896" [color = darkviolet];
"139781154520512" -> "139781034972368" [color = darkviolet];
"139781154520512" -> "139781034973072" [color = darkviolet];
"139781154520512" -> "139781131267872" [color = darkviolet];
"139803511371136" [label = "gcc-core-mesboot0@2.95.3", shape = box, fontname = sans];
"139803511371136" -> "139803511371312" [color = red];
"139803511371136" -> "139803511372016" [color = red];
"139803511371136" -> "139803511371488" [color = red];
"139803511371136" -> "139803511371664" [color = red];
"139803511371136" -> "139803511371840" [color = red];
"139803511371136" -> "139803507105792" [color = red];
"139803511371136" -> "139803511373600" [color = red];
"139803511371136" -> "139803507105968" [color = red];
"139803511371136" -> "139803749199472" [color = red];
"139803511371312" [label = "binutils-mesboot0@2.20.1a", shape = box, fontname = sans];
"139803511371312" -> "139803511372016" [color = dimgrey];
"139803511371312" -> "139803511371488" [color = dimgrey];
"139803511371312" -> "139803511371664" [color = dimgrey];
"139803511371312" -> "139803511371840" [color = dimgrey];
"139803511371312" -> "139803507105792" [color = dimgrey];
"139803511371312" -> "139803511373600" [color = dimgrey];
"139803511371312" -> "139803507105968" [color = dimgrey];
"139803511371312" -> "139803749199472" [color = dimgrey];
"139803511372016" [label = "gzip-mesboot@1.2.4", shape = box, fontname = sans];
"139803511372016" -> "139803511372192" [color = magenta];
"139803511372016" -> "139803507105792" [color = magenta];
"139803511372016" -> "139803511373600" [color = magenta];
"139803511372016" -> "139803507105968" [color = magenta];
"139803511372016" -> "139803749199472" [color = magenta];
"139803511372192" [label = "tcc-boot0@0.9.26-1136-g5bba73cc", shape = box, fontname = sans];
"139803511372192" -> "139803511372368" [color = magenta];
"139803511372192" -> "139803511372544" [color = magenta];
"139803511372192" -> "139803507105792" [color = magenta];
"139803511372192" -> "139803511373600" [color = magenta];
"139803511372192" -> "139803507105968" [color = magenta];
"139803511372192" -> "139803749199472" [color = magenta];
"139803511372368" [label = "mes-boot@0.24", shape = box, fontname = sans];
"139803511372368" -> "139803511372544" [color = darkviolet];
"139803511372368" -> "139803507105792" [color = darkviolet];
"139803511372368" -> "139803511373600" [color = darkviolet];
"139803511372368" -> "139803507105968" [color = darkviolet];
"139803511372368" -> "139803749199472" [color = darkviolet];
"139803511372544" [label = "stage0-posix@1.4", shape = box, fontname = sans];
"139803511372544" -> "139803511373072" [color = peachpuff4];
"139803511372544" -> "139803507105792" [color = peachpuff4];
"139803511372544" -> "139803511373600" [color = peachpuff4];
"139803511372544" -> "139803507105968" [color = peachpuff4];
"139803511372544" -> "139803749199472" [color = peachpuff4];
"139803511373072" [label = "bootstrap-seeds@1.0.0", shape = ellipse, fontname = sans];
"139803511373072" -> "139803507105968" [color = cyan3];
"139803507105968" [label = "bootar@1b", shape = box, fontname = sans];
"139803507105968" -> "139803749199472" [color = dimgrey];
"139803749199472" [label = "guile-bootstrap@2.0", shape = ellipse, fontname = sans];
"139803507105792" [label = "gash-boot@0.3.0", shape = box, fontname = sans];
"139803507105792" -> "139803507105968" [color = darkviolet];
"139803507105792" -> "139803749199472" [color = darkviolet];
"139803511373600" [label = "gash-utils-boot@0.2.0", shape = box, fontname = sans];
"139803511373600" -> "139803507105968" [color = red];
"139803511373600" -> "139803507105792" [color = red];
"139803511373600" -> "139803749199472" [color = red];
"139803511371488" [label = "patch-mesboot@2.5.9", shape = box, fontname = sans];
"139803511371488" -> "139803511371840" [color = darkseagreen];
"139803511371488" -> "139803511372192" [color = darkseagreen];
"139803511371488" -> "139803507105792" [color = darkseagreen];
"139803511371488" -> "139803511373600" [color = darkseagreen];
"139803511371488" -> "139803507105968" [color = darkseagreen];
"139803511371488" -> "139803749199472" [color = darkseagreen];
"139803511371840" [label = "make-mesboot0@3.80", shape = box, fontname = sans];
"139803511371840" -> "139803511372192" [color = blue];
"139803511371840" -> "139803507105792" [color = blue];
"139803511371840" -> "139803511373600" [color = blue];
"139803511371840" -> "139803507105968" [color = blue];
"139803511371840" -> "139803749199472" [color = blue];
"139803511371664" [label = "tcc-boot@0.9.27", shape = box, fontname = sans];
"139803511371664" -> "139803511371840" [color = peachpuff4];
"139803511371664" -> "139803511372192" [color = peachpuff4];
"139803511371664" -> "139803507105792" [color = peachpuff4];
"139803511371664" -> "139803511373600" [color = peachpuff4];
"139803511371664" -> "139803507105968" [color = peachpuff4];
"139803511371664" -> "139803749199472" [color = peachpuff4];
}