bitbake: Merge latest upstream#55
Open
Shreejit-03 wants to merge 15 commits into
Open
Conversation
The deb/ipk unpack path selects a data archive member from 'ar -t' output and then passes that member name to a shell command. Previously, any member beginning with data.tar. was selected. Only select known deb/ipk data archive member names when datafile is created. Quote the package path used in the shell command as it can come from the local fetch path. Add local fetcher regression coverage for quoted package filenames, valid compressed data members, and unsupported or unsafe data member names. Signed-off-by: Anders Heimer <anders.heimer@est.tech> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 73ae3a2) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
The striplevel URL parameter is appended to tar_cmd, which is later run through the shell. Validate it as a decimal count before using it in the tar arguments. Signed-off-by: Anders Heimer <anders.heimer@est.tech> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 934fe71) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
BB_GIT_SHALLOW_EXTRA_REFS can include wildcard entries. Matching refs advertised by the remote are later passed to git fetch and update-ref while creating shallow tarballs. Quote the generated command arguments and pass the fetched ref after -- so shell metacharacters and option-like ref names are not interpreted as command syntax or git fetch options. Signed-off-by: Anders Heimer <anders.heimer@est.tech> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e9a06f7) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Use cpio --no-absolute-filenames when unpacking RPM and SRPM archives so absolute paths and parent-directory components in cpio member names are extracted relative to the intended unpack directory. Signed-off-by: Anders Heimer <anders.heimer@est.tech> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1b1a715) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Anders Heimer <anders.heimer@est.tech> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 16ef07c) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Anders Heimer <anders.heimer@est.tech> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f9b817d) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
The YP and OE repos have moved to "https:" instead of "git:" for cloning. YP has removed the "cgit/cgit.cgi" from web URLs for layers. This patch updates the fixture files that bootstrap these values for bitbake and the default layers. Also, fix Wrynose bitbake version. Note that Toaster is dependent on these values also fixed in the Layer Index (#16240). [YOCTO #16304] Signed-off-by: David Reyna <mailto:David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit d73feaf) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Fixes [YOCTO #16268] Adds a missing "raise" statement to download that ensures when there is an IOError/OSError that is is correctly reported and not silently discarded. When an IOError is thrown in this case do_fetch silently passes which then leads to downstream failures in do_unpack when the file doesn't exist. Signed-off-by: Anthony Squires <me@anthonysquires.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0460038) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
This patch fixes an issue when checking if a varflag can be safely excluded. BB_SIGNATURE_EXCLUDE_FLAGS lists variable flags that can be safely excluded from checksum and dependency data for keys in the datastore. When bitbake checks if a varflag must be excluded it checks if the varflag name is part of the string stored in BB_SIGNATURE_EXCLUDE_FLAGS. As an example, if the varflag 'filename' is in BB_SIGNATURE_EXCLUDE_FLAGS, the varflag 'name' will also be excluded because the check will return 'True' when checking if the varflag is part of the string with the varflags to exclude. To fix this issue the string from BB_SIGNATURE_EXCLUDE_FLAGS is converted to a list before checking if a varflag is part of it. Signed-off-by: Marcio Henriques <marcio.henriques@ctw.bmwgroup.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 8ab71d0) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
If the requested branch(es) is not found, a debugging message is displayed to list it, so the expected behavior is something like this: $ bitbake-layers layerindex-fetch meta-firefox NOTE: Starting bitbake server... Loading https://layers.openembedded.org/layerindex/api/;branch=wrynose... ERROR: Remote layer index ['https://layers.openembedded.org/layerindex/'] is empty for branches ['wrynose'] However, the variable currently used is unbound, and the error below comes up: File "/src/bitbake/lib/layerindexlib/restapi.py", line 216, in load_index_web logger.debug("No valid branches (%s) found at url %s." % (branch, url)) ^^^^^^ UnboundLocalError: cannot access local variable 'branch' where it is not associated with a value Replace 'branch' by 'branches' to fix this. Signed-off-by: João Marcos Costa <joaomarcos.costa@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit df3295f) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
The cargo sparse index marks yanked crate versions with "yanked": true. Treat those the same way as non-existing versions so that version checking never promotes a yanked release as the latest one. Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com> Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 0d757e8) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Both latest_versionstring() and latest_versionstring_from_index() would
raise IndexError on versions[-1] if the fetched index returns no usable
versions.
Return ("", "") in that case, consistent with what callers expect when
no upstream version can be determined.
Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ffdc2a0)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
While using PREMIRRORS to gets source code, recipe sbom-cve-check-update-nvd-native
do fetch failed:
$ bitbake sbom-cve-check-update-nvd-native -cfetch -f
...
ERROR: sbom-cve-check-update-nvd-native-1.0-r0 do_fetch: Bitbake Fetcher Error: MalformedUrl('git:///path-to-premirror/git/github.com.fkie-cad.nvd-json-data-feeds.git;branch=main;protocol=file;destsuffix')
ERROR: Logfile of failure stored in: tmp/work/x86_64-linux/sbom-cve-check-update-nvd-native/1.0/temp/log.do_fetch.4135595
ERROR: Task (oe-core/meta/recipes-devtools/sbom-cve-check/sbom-cve-check-update-nvd-native.bb:do_fetch) failed with exit code '1'
...
In oe-core commit [1], destsuffix was configured to empty string through SRC_URI,
but in bitbake commit [2], it removed `=' from destsuffix in which value is empty,
and trigger MalformedUrl failure, see sbom-cve-check-update-nvd-native do_fetch log:
...log.do_fetch...
DEBUG: For url git://github.com/fkie-cad/nvd-json-data-feeds.git;branch=main;protocol=https;destsuffix= returning git:///path-to-premirror/git/github.com.fkie-cad.nvd-json-data-feeds.git;branch=main;protocol=file;destsuffix
...log.do_fetch...
This commit use function _query_str_join for query as usual (if a value is
None then it isn't a key-value pair, but a bare key.), and use function
_param_str_join for params to allow value is empty string and still key-value
pair, after applying this commit
...log.do_fetch...
DEBUG: For url git://github.com/fkie-cad/nvd-json-data-feeds.git;branch=main;protocol=https;destsuffix= returning git:///path-to-premirror/git/github.com.fkie-cad.nvd-json-data-feeds.git;branch=main;protocol=file;destsuffix=
...log.do_fetch...
$ bitbake-selftest -v bb.tests.fetch.URITest.test_uri
test_uri (bb.tests.fetch.URITest) ... ok
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
[1] openembedded/openembedded-core@131e024
[2] openembedded@eac583b
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b3867e6)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
FixedHTTPRedirectHandler copies request headers when checkstatus() follows a redirect, including Authorization from SRC_URI or .netrc. Keep same-origin redirects unchanged, but drop Authorization and Cookie for different-origin targets (scheme, host and effective port), following RFC 9110 redirect guidance for resource-specific headers. This only affects the Python checkstatus() path; normal wget downloads are unchanged. Signed-off-by: Anders Heimer <anders.heimer@est.tech> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1019d5a) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Add local HTTP server tests for Wget.checkstatus() redirects. They check that Authorization is kept for same-origin redirects and dropped when the target has a different origin. Signed-off-by: Anders Heimer <anders.heimer@est.tech> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c687d42) Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge latest from upstream. No conflicts.
Justification
AB#3738533
Testing
@ni/rtos