410 lines
17 KiB
ReStructuredText
410 lines
17 KiB
ReStructuredText
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
|
||
|
||
Maintaining Build Output Quality
|
||
********************************
|
||
|
||
Many factors can influence the quality of a build. For example, if you
|
||
upgrade a recipe to use a new version of an upstream software package or
|
||
you experiment with some new configuration options, subtle changes can
|
||
occur that you might not detect until later. Consider the case where
|
||
your recipe is using a newer version of an upstream package. In this
|
||
case, a new version of a piece of software might introduce an optional
|
||
dependency on another library, which is auto-detected. If that library
|
||
has already been built when the software is building, the software will
|
||
link to the built library and that library will be pulled into your
|
||
image along with the new software even if you did not want the library.
|
||
|
||
The :ref:`ref-classes-buildhistory` class helps you maintain the quality of
|
||
your build output. You can use the class to highlight unexpected and possibly
|
||
unwanted changes in the build output. When you enable build history, it records
|
||
information about the contents of each package and image and then commits that
|
||
information to a local Git repository where you can examine the information.
|
||
|
||
The remainder of this section describes the following:
|
||
|
||
- :ref:`How you can enable and disable build history <dev-manual/build-quality:enabling and disabling build history>`
|
||
|
||
- :ref:`How to understand what the build history contains <dev-manual/build-quality:understanding what the build history contains>`
|
||
|
||
- :ref:`How to limit the information used for build history <dev-manual/build-quality:using build history to gather image information only>`
|
||
|
||
- :ref:`How to examine the build history from both a command-line and web interface <dev-manual/build-quality:examining build history information>`
|
||
|
||
Enabling and Disabling Build History
|
||
====================================
|
||
|
||
Build history is disabled by default. To enable it, add the following
|
||
:term:`INHERIT` statement and set the :term:`BUILDHISTORY_COMMIT` variable to
|
||
"1" at the end of your ``conf/local.conf`` file found in the
|
||
:term:`Build Directory`::
|
||
|
||
INHERIT += "buildhistory"
|
||
BUILDHISTORY_COMMIT = "1"
|
||
|
||
Enabling build history as
|
||
previously described causes the OpenEmbedded build system to collect
|
||
build output information and commit it as a single commit to a local
|
||
:ref:`overview-manual/development-environment:git` repository.
|
||
|
||
.. note::
|
||
|
||
Enabling build history increases your build times slightly,
|
||
particularly for images, and increases the amount of disk space used
|
||
during the build.
|
||
|
||
You can disable build history by removing the previous statements from
|
||
your ``conf/local.conf`` file.
|
||
|
||
Understanding What the Build History Contains
|
||
=============================================
|
||
|
||
Build history information is kept in ``${``\ :term:`TOPDIR`\ ``}/buildhistory``
|
||
in the :term:`Build Directory` as defined by the :term:`BUILDHISTORY_DIR`
|
||
variable. Here is an example abbreviated listing:
|
||
|
||
.. image:: figures/buildhistory.png
|
||
:align: center
|
||
:width: 50%
|
||
|
||
At the top level, there is a ``metadata-revs`` file that lists the
|
||
revisions of the repositories for the enabled layers when the build was
|
||
produced. The rest of the data splits into separate ``packages``,
|
||
``images`` and ``sdk`` directories, the contents of which are described
|
||
as follows.
|
||
|
||
Build History Package Information
|
||
---------------------------------
|
||
|
||
The history for each package contains a text file that has name-value
|
||
pairs with information about the package. For example,
|
||
``buildhistory/packages/i586-poky-linux/busybox/busybox/latest``
|
||
contains the following:
|
||
|
||
.. code-block:: none
|
||
|
||
PV = 1.22.1
|
||
PR = r32
|
||
RPROVIDES =
|
||
RDEPENDS = glibc (>= 2.20) update-alternatives-opkg
|
||
RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d
|
||
PKGSIZE = 540168
|
||
FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \
|
||
/etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \
|
||
/usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \
|
||
/usr/share/pixmaps /usr/share/applications /usr/share/idl \
|
||
/usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers
|
||
FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \
|
||
/etc/busybox.links.nosuid /etc/busybox.links.suid
|
||
|
||
Most of these
|
||
name-value pairs correspond to variables used to produce the package.
|
||
The exceptions are ``FILELIST``, which is the actual list of files in
|
||
the package, and ``PKGSIZE``, which is the total size of files in the
|
||
package in bytes.
|
||
|
||
There is also a file that corresponds to the recipe from which the package
|
||
came (e.g. ``buildhistory/packages/i586-poky-linux/busybox/latest``):
|
||
|
||
.. code-block:: none
|
||
|
||
PV = 1.22.1
|
||
PR = r32
|
||
DEPENDS = initscripts kern-tools-native update-rc.d-native \
|
||
virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \
|
||
virtual/libc virtual/update-alternatives
|
||
PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \
|
||
busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \
|
||
busybox-staticdev busybox-dev busybox-doc busybox-locale busybox
|
||
|
||
Finally, for those recipes fetched from a version control system (e.g.,
|
||
Git), there is a file that lists source revisions that are specified in
|
||
the recipe and the actual revisions used during the build. Listed
|
||
and actual revisions might differ when
|
||
:term:`SRCREV` is set to
|
||
${:term:`AUTOREV`}. Here is an
|
||
example assuming
|
||
``buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev``)::
|
||
|
||
# SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
|
||
SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1"
|
||
# SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f"
|
||
SRCREV_meta ="a227f20eff056e511d504b2e490f3774ab260d6f"
|
||
|
||
You can use the
|
||
``buildhistory-collect-srcrevs`` command with the ``-a`` option to
|
||
collect the stored :term:`SRCREV` values from build history and report them
|
||
in a format suitable for use in global configuration (e.g.,
|
||
``local.conf`` or a distro include file) to override floating
|
||
:term:`AUTOREV` values to a fixed set of revisions. Here is some example
|
||
output from this command::
|
||
|
||
$ buildhistory-collect-srcrevs -a
|
||
# all-poky-linux
|
||
SRCREV:pn-ca-certificates = "07de54fdcc5806bde549e1edf60738c6bccf50e8"
|
||
SRCREV:pn-update-rc.d = "8636cf478d426b568c1be11dbd9346f67e03adac"
|
||
# core2-64-poky-linux
|
||
SRCREV:pn-binutils = "87d4632d36323091e731eb07b8aa65f90293da66"
|
||
SRCREV:pn-btrfs-tools = "8ad326b2f28c044cb6ed9016d7c3285e23b673c8"
|
||
SRCREV_bzip2-tests:pn-bzip2 = "f9061c030a25de5b6829e1abf373057309c734c0"
|
||
SRCREV:pn-e2fsprogs = "02540dedd3ddc52c6ae8aaa8a95ce75c3f8be1c0"
|
||
SRCREV:pn-file = "504206e53a89fd6eed71aeaf878aa3512418eab1"
|
||
SRCREV_glibc:pn-glibc = "24962427071fa532c3c48c918e9d64d719cc8a6c"
|
||
SRCREV:pn-gnome-desktop-testing = "e346cd4ed2e2102c9b195b614f3c642d23f5f6e7"
|
||
SRCREV:pn-init-system-helpers = "dbd9197569c0935029acd5c9b02b84c68fd937ee"
|
||
SRCREV:pn-kmod = "b6ecfc916a17eab8f93be5b09f4e4f845aabd3d1"
|
||
SRCREV:pn-libnsl2 = "82245c0c58add79a8e34ab0917358217a70e5100"
|
||
SRCREV:pn-libseccomp = "57357d2741a3b3d3e8425889a6b79a130e0fa2f3"
|
||
SRCREV:pn-libxcrypt = "50cf2b6dd4fdf04309445f2eec8de7051d953abf"
|
||
SRCREV:pn-ncurses = "51d0fd9cc3edb975f04224f29f777f8f448e8ced"
|
||
SRCREV:pn-procps = "19a508ea121c0c4ac6d0224575a036de745eaaf8"
|
||
SRCREV:pn-psmisc = "5fab6b7ab385080f1db725d6803136ec1841a15f"
|
||
SRCREV:pn-ptest-runner = "bcb82804daa8f725b6add259dcef2067e61a75aa"
|
||
SRCREV:pn-shared-mime-info = "18e558fa1c8b90b86757ade09a4ba4d6a6cf8f70"
|
||
SRCREV:pn-zstd = "e47e674cd09583ff0503f0f6defd6d23d8b718d3"
|
||
# qemux86_64-poky-linux
|
||
SRCREV_machine:pn-linux-yocto = "20301aeb1a64164b72bc72af58802b315e025c9c"
|
||
SRCREV_meta:pn-linux-yocto = "2d38a472b21ae343707c8bd64ac68a9eaca066a0"
|
||
# x86_64-linux
|
||
SRCREV:pn-binutils-cross-x86_64 = "87d4632d36323091e731eb07b8aa65f90293da66"
|
||
SRCREV_glibc:pn-cross-localedef-native = "24962427071fa532c3c48c918e9d64d719cc8a6c"
|
||
SRCREV_localedef:pn-cross-localedef-native = "794da69788cbf9bf57b59a852f9f11307663fa87"
|
||
SRCREV:pn-debianutils-native = "de14223e5bffe15e374a441302c528ffc1cbed57"
|
||
SRCREV:pn-libmodulemd-native = "ee80309bc766d781a144e6879419b29f444d94eb"
|
||
SRCREV:pn-virglrenderer-native = "363915595e05fb252e70d6514be2f0c0b5ca312b"
|
||
SRCREV:pn-zstd-native = "e47e674cd09583ff0503f0f6defd6d23d8b718d3"
|
||
|
||
.. note::
|
||
|
||
Here are some notes on using the ``buildhistory-collect-srcrevs`` command:
|
||
|
||
- By default, only values where the :term:`SRCREV` was not hardcoded
|
||
(usually when :term:`AUTOREV` is used) are reported. Use the ``-a``
|
||
option to see all :term:`SRCREV` values.
|
||
|
||
- The output statements might not have any effect if overrides are
|
||
applied elsewhere in the build system configuration. Use the
|
||
``-f`` option to add the ``forcevariable`` override to each output
|
||
line if you need to work around this restriction.
|
||
|
||
- The script does apply special handling when building for multiple
|
||
machines. However, the script does place a comment before each set
|
||
of values that specifies which triplet to which they belong as
|
||
previously shown (e.g., ``i586-poky-linux``).
|
||
|
||
Build History Image Information
|
||
-------------------------------
|
||
|
||
The files produced for each image are as follows:
|
||
|
||
- ``image-files:`` A directory containing selected files from the root
|
||
filesystem. The files are defined by
|
||
:term:`BUILDHISTORY_IMAGE_FILES`.
|
||
|
||
- ``build-id.txt:`` Human-readable information about the build
|
||
configuration and metadata source revisions. This file contains the
|
||
full build header as printed by BitBake.
|
||
|
||
- ``*.dot:`` Dependency graphs for the image that are compatible with
|
||
``graphviz``.
|
||
|
||
- ``files-in-image.txt:`` A list of files in the image with
|
||
permissions, owner, group, size, and symlink information.
|
||
|
||
- ``image-info.txt:`` A text file containing name-value pairs with
|
||
information about the image. See the following listing example for
|
||
more information.
|
||
|
||
- ``installed-package-names.txt:`` A list of installed packages by name
|
||
only.
|
||
|
||
- ``installed-package-sizes.txt:`` A list of installed packages ordered
|
||
by size.
|
||
|
||
- ``installed-packages.txt:`` A list of installed packages with full
|
||
package filenames.
|
||
|
||
.. note::
|
||
|
||
Installed package information is able to be gathered and produced
|
||
even if package management is disabled for the final image.
|
||
|
||
Here is an example of ``image-info.txt``:
|
||
|
||
.. code-block:: none
|
||
|
||
DISTRO = poky
|
||
DISTRO_VERSION = 3.4+snapshot-a0245d7be08f3d24ea1875e9f8872aa6bbff93be
|
||
USER_CLASSES = buildstats
|
||
IMAGE_CLASSES = qemuboot qemuboot license_image
|
||
IMAGE_FEATURES = debug-tweaks
|
||
IMAGE_LINGUAS =
|
||
IMAGE_INSTALL = packagegroup-core-boot speex speexdsp
|
||
BAD_RECOMMENDATIONS =
|
||
NO_RECOMMENDATIONS =
|
||
PACKAGE_EXCLUDE =
|
||
ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; cve_check_write_rootfs_manifest; ssh_allow_empty_password; ssh_allow_root_login; postinst_enable_logging; rootfs_update_timestamp; write_image_test_data; empty_var_volatile; sort_passwd; rootfs_reproducible;
|
||
IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ;
|
||
IMAGESIZE = 9265
|
||
|
||
Other than ``IMAGESIZE``,
|
||
which is the total size of the files in the image in Kbytes, the
|
||
name-value pairs are variables that may have influenced the content of
|
||
the image. This information is often useful when you are trying to
|
||
determine why a change in the package or file listings has occurred.
|
||
|
||
Using Build History to Gather Image Information Only
|
||
----------------------------------------------------
|
||
|
||
As you can see, build history produces image information, including
|
||
dependency graphs, so you can see why something was pulled into the
|
||
image. If you are just interested in this information and not interested
|
||
in collecting specific package or SDK information, you can enable
|
||
writing only image information without any history by adding the
|
||
following to your ``conf/local.conf`` file found in the
|
||
:term:`Build Directory`::
|
||
|
||
INHERIT += "buildhistory"
|
||
BUILDHISTORY_COMMIT = "0"
|
||
BUILDHISTORY_FEATURES = "image"
|
||
|
||
Here, you set the
|
||
:term:`BUILDHISTORY_FEATURES`
|
||
variable to use the image feature only.
|
||
|
||
Build History SDK Information
|
||
-----------------------------
|
||
|
||
Build history collects similar information on the contents of SDKs (e.g.
|
||
``bitbake -c populate_sdk imagename``) as compared to information it
|
||
collects for images. Furthermore, this information differs depending on
|
||
whether an extensible or standard SDK is being produced.
|
||
|
||
The following list shows the files produced for SDKs:
|
||
|
||
- ``files-in-sdk.txt:`` A list of files in the SDK with permissions,
|
||
owner, group, size, and symlink information. This list includes both
|
||
the host and target parts of the SDK.
|
||
|
||
- ``sdk-info.txt:`` A text file containing name-value pairs with
|
||
information about the SDK. See the following listing example for more
|
||
information.
|
||
|
||
- ``sstate-task-sizes.txt:`` A text file containing name-value pairs
|
||
with information about task group sizes (e.g. :ref:`ref-tasks-populate_sysroot`
|
||
tasks have a total size). The ``sstate-task-sizes.txt`` file exists
|
||
only when an extensible SDK is created.
|
||
|
||
- ``sstate-package-sizes.txt:`` A text file containing name-value pairs
|
||
with information for the shared-state packages and sizes in the SDK.
|
||
The ``sstate-package-sizes.txt`` file exists only when an extensible
|
||
SDK is created.
|
||
|
||
- ``sdk-files:`` A folder that contains copies of the files mentioned
|
||
in ``BUILDHISTORY_SDK_FILES`` if the files are present in the output.
|
||
Additionally, the default value of ``BUILDHISTORY_SDK_FILES`` is
|
||
specific to the extensible SDK although you can set it differently if
|
||
you would like to pull in specific files from the standard SDK.
|
||
|
||
The default files are ``conf/local.conf``, ``conf/bblayers.conf``,
|
||
``conf/auto.conf``, ``conf/locked-sigs.inc``, and
|
||
``conf/devtool.conf``. Thus, for an extensible SDK, these files get
|
||
copied into the ``sdk-files`` directory.
|
||
|
||
- The following information appears under each of the ``host`` and
|
||
``target`` directories for the portions of the SDK that run on the
|
||
host and on the target, respectively:
|
||
|
||
.. note::
|
||
|
||
The following files for the most part are empty when producing an
|
||
extensible SDK because this type of SDK is not constructed from
|
||
packages as is the standard SDK.
|
||
|
||
- ``depends.dot:`` Dependency graph for the SDK that is compatible
|
||
with ``graphviz``.
|
||
|
||
- ``installed-package-names.txt:`` A list of installed packages by
|
||
name only.
|
||
|
||
- ``installed-package-sizes.txt:`` A list of installed packages
|
||
ordered by size.
|
||
|
||
- ``installed-packages.txt:`` A list of installed packages with full
|
||
package filenames.
|
||
|
||
Here is an example of ``sdk-info.txt``:
|
||
|
||
.. code-block:: none
|
||
|
||
DISTRO = poky
|
||
DISTRO_VERSION = 1.3+snapshot-20130327
|
||
SDK_NAME = poky-glibc-i686-arm
|
||
SDK_VERSION = 1.3+snapshot
|
||
SDKMACHINE =
|
||
SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs
|
||
BAD_RECOMMENDATIONS =
|
||
SDKSIZE = 352712
|
||
|
||
Other than ``SDKSIZE``, which is
|
||
the total size of the files in the SDK in Kbytes, the name-value pairs
|
||
are variables that might have influenced the content of the SDK. This
|
||
information is often useful when you are trying to determine why a
|
||
change in the package or file listings has occurred.
|
||
|
||
Examining Build History Information
|
||
-----------------------------------
|
||
|
||
You can examine build history output from the command line or from a web
|
||
interface.
|
||
|
||
To see any changes that have occurred (assuming you have
|
||
:term:`BUILDHISTORY_COMMIT` = "1"),
|
||
you can simply use any Git command that allows you to view the history
|
||
of a repository. Here is one method::
|
||
|
||
$ git log -p
|
||
|
||
You need to realize,
|
||
however, that this method does show changes that are not significant
|
||
(e.g. a package's size changing by a few bytes).
|
||
|
||
There is a command-line tool called ``buildhistory-diff``, though,
|
||
that queries the Git repository and prints just the differences that
|
||
might be significant in human-readable form. Here is an example::
|
||
|
||
$ poky/poky/scripts/buildhistory-diff . HEAD^
|
||
Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt):
|
||
/etc/anotherpkg.conf was added
|
||
/sbin/anotherpkg was added
|
||
* (installed-package-names.txt):
|
||
* anotherpkg was added
|
||
Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt):
|
||
anotherpkg was added
|
||
packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras"
|
||
* PR changed from "r0" to "r1"
|
||
* PV changed from "0.1.10" to "0.1.12"
|
||
packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%)
|
||
* PR changed from "r0" to "r1"
|
||
* PV changed from "0.1.10" to "0.1.12"
|
||
|
||
.. note::
|
||
|
||
The ``buildhistory-diff`` tool requires the ``GitPython``
|
||
package. Be sure to install it using Pip3 as follows::
|
||
|
||
$ pip3 install GitPython --user
|
||
|
||
|
||
Alternatively, you can install ``python3-git`` using the appropriate
|
||
distribution package manager (e.g. ``apt``, ``dnf``, or ``zipper``).
|
||
|
||
To see changes to the build history using a web interface, follow the
|
||
instruction in the ``README`` file
|
||
:yocto_git:`here </buildhistory-web/>`.
|
||
|
||
Here is a sample screenshot of the interface:
|
||
|
||
.. image:: figures/buildhistory-web.png
|
||
:width: 100%
|
||
|