58 lines
2.7 KiB
Plaintext
58 lines
2.7 KiB
Plaintext
There are expectations of users of the fetcher code. This file attempts to document
|
|
some of the constraints that are present. Some are obvious, some are less so. It is
|
|
documented in the context of how OE uses it but the API calls are generic.
|
|
|
|
a) network access for sources is only expected to happen in the do_fetch task.
|
|
This is not enforced or tested but is required so that we can:
|
|
|
|
i) audit the sources used (i.e. for license/manifest reasons)
|
|
ii) support offline builds with a suitable cache
|
|
iii) allow work to continue even with downtime upstream
|
|
iv) allow for changes upstream in incompatible ways
|
|
v) allow rebuilding of the software in X years time
|
|
|
|
b) network access is not expected in do_unpack task.
|
|
|
|
c) you can take DL_DIR and use it as a mirror for offline builds.
|
|
|
|
d) access to the network is only made when explicitly configured in recipes
|
|
(e.g. use of AUTOREV, or use of git tags which change revision).
|
|
|
|
e) fetcher output is deterministic (i.e. if you fetch configuration XXX now it
|
|
will match in future exactly in a clean build with a new DL_DIR).
|
|
One specific pain point example are git tags. They can be replaced and change
|
|
so the git fetcher has to resolve them with the network. We use git revisions
|
|
where possible to avoid this and ensure determinism.
|
|
|
|
f) network access is expected to work with the standard linux proxy variables
|
|
so that access behind firewalls works (the fetcher sets these in the
|
|
environment but only in the do_fetch tasks).
|
|
|
|
g) access during parsing has to be minimal, a "git ls-remote" for an AUTOREV
|
|
git recipe might be ok but you can't expect to checkout a git tree.
|
|
|
|
h) we need to provide revision information during parsing such that a version
|
|
for the recipe can be constructed.
|
|
|
|
i) versions are expected to be able to increase in a way which sorts allowing
|
|
package feeds to operate (see PR server required for git revisions to sort).
|
|
|
|
j) API to query for possible version upgrades of a url is highly desireable to
|
|
allow our automated upgrage code to function (it is implied this does always
|
|
have network access).
|
|
|
|
k) Where fixes or changes to behaviour in the fetcher are made, we ask that
|
|
test cases are added (run with "bitbake-selftest bb.tests.fetch"). We do
|
|
have fairly extensive test coverage of the fetcher as it is the only way
|
|
to track all of its corner cases, it still doesn't give entire coverage
|
|
though sadly.
|
|
|
|
l) If using tools during parse time, they will have to be in ASSUME_PROVIDED
|
|
in OE's context as we can't build git-native, then parse a recipe and use
|
|
git ls-remote.
|
|
|
|
Not all fetchers support all features, autorev is optional and doesn't make
|
|
sense for some. Upgrade detection means different things in different contexts
|
|
too.
|
|
|