Send patches - preferably formatted by git format-patch - to patches at archlinux32 dot org.
summaryrefslogtreecommitdiff
path: root/scripts/libmakepkg
AgeCommit message (Collapse)Author
2020-08-10libmakepkg: extend compress.sh to also permit checking validityEli Schwartz
get_compression_command() can now be used to do upfront checks for whether a given extension is known to do something successfully. This is useful when writing tools in which an unknown compression type is a fatal error. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-06-26srcinfo.sh: remove trailing newlineDenton Liu
When a .SRCINFO file is generated via `makepkg --printsrcinfo`, each section is concluded with an empty line. This means that at the end of the file, an empty line remains. This is considered a trailing whitespace error. In fact, `git diff --check` will warn about this, saying "new blank line at EOF." Instead of closing each section off with an empty line, use the empty line to separate sections, omitting the empty line at the end of the file. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-06-11makepkg/repo-add: handle GPGKEY with spacesEli Schwartz
We pass this to gpg -u and this gpg option can accept a number of different formats, not just the historical hexadecimal fingerprint we assumed. We should not barf hard if a format is used which happens to contain spaces. This also fixes a validation bug. When we initially check if the desired key is available, we don't quote spaces, so gpg goes ahead and treats each space-separated string as a *different key* to search for, returning partial matches, and returning success if at least one key is found. But gpg --detach-sign -u will certainly not accept multiple keys! Fixes FS#66949 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-06-11libmakepkg: fix regression in sending plain() output to stderrEli Schwartz
In commit 882e707e40bbade0111cf3bdedbdac4d4b70453b we changed message output to go to stdout by default, unless it was an error. The plain() function doesn't *look* like an error function, but in practice it was -- it's used to continue multiline messages, and all in-tree uses were for warning/error. This is a problem both because we're sending output to the wrong place, and because in some cases, we were performing error logging from a function which would otherwise return a value to be captured in a variable using command substution. Fix this and straighten out the API by providing two functions: one for continuing msg output, and one which wraps this by sending output to stderr, for continuing error output. Change all callers to use the second function.
2020-06-11makepkg: correctly handle missing download clientsEli Schwartz
This was broken in commit 882e707e40bbade0111cf3bdedbdac4d4b70453b, which changed 'plain()' messages to go to stdout, which was then captured as the download client in question: cmdline=("Aborting..."). The result was a very confusing error message e.g. /usr/share/makepkg/source/file.sh: line 72: $'\E[1m': command not found or with makepkg --nocolor: /usr/share/makepkg/source/file.sh: line 72: Aborting...: command not found The problem here is that we checked to see if an asynchronous subshell, in our case <(...), failed, by checking if its captured stdout is non-empty. Which is terrible, and also a limitation of old bash. But bash 4.4 can use wait $! to retrieve the return value of an asynchronous subshell. Now we target that as our minimum, we can sanely handle errors in such functions. Losing error messages on stdout by capturing them in a variable instead of printing them, continues to be a problem, but this will be fixed systematically in a later commit. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-06-11makepkg: guard against undefined git pinned sourcesEli Schwartz
If something like source=(..."#commit=") is used, e.g. due to failed variable expansion, we try to check out an empty refspec as nothing at all, and end up just running "git checkout". This happens because we fail at variable expansion too -- so let's quote our variables properly and make sure git sees this as an empty refspec, so it can error out. Also make sure it is interpreted as a ref instead of a path. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-06-01libmakepkg/strip: don't re-add the same debug source multiple timesEli Schwartz
It's either a waste of work, or triggers edge cases in some packages (like coreutils-8.31) where the source file is readonly and cp gets a permission denied error trying to overwrite it with an identical copy of itself. Also while we are at it, make the variable names be something readable, because I could barely tell what this was doing while editing it. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2020-02-10build-aux/update-copyright 2019 2020Allan McRae
Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-28makepkg: add CRC checksums and set these to be the defaultAllan McRae
Checksums arrays should be filled with values provided by upstream. We currently have md5 set as an unsecure default, and are constantly asked to change it to sha2. However, just changing the default to a stronger checksum gives the user the impression that "makepkg -g" checksums are perfect. Instead, change the default checksum to a CRC, to make it clear that any checksum generated purely by "makepkg -g" is not ideal. Signed-off-by: Allan McRae <allan@archlinux.org>
2020-01-27makepkg: make per-package files containing '$pkgname' consistently workEli Schwartz
Extracting function variables containing arbitrarily scoped variables of arbitrary nature is a disaster, but let's at least cover the common case of using the actual '$pkgname' in an install/changelog file. It's the odd case of actually being basically justified use of disambiguating between the same variable used in multiple different split packages... and also, --printsrcinfo already uses and overwrites the variable 'pkgname' in pkgbuild_extract_to_srcinfo, so this "works" in .SRCINFO but doesn't work in .src.tar.gz It doesn't work in lint_pkgbuild either, but in that case the problem is being too permissive, not too restrictive -- we might end up checking the same file twice, and printing that it is missing twice. Fixes FS#64932 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-12-11libmakepkg: use readelf instead of file for finding ELF file typesEthan Sommer
Signed-off-by: Ethan Sommer <e5ten.arch@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-26makepkg: fix regression that broke extraction of file:// sourcesEli Schwartz
In commit 9c817b654996249b8022e189ee7e2692f4668431 we made these sources extendable, and heuristically determined the correct extraction functions to use. But our fallback for protos that didn't have an exact extract_* function didn't take into account that 'extract_file' matches an actual proto... so we passed the netfile in while the function expected a file. Solution: the function should expect a netfile too, thereby allowing us to delay an attempted resolution of netfile -> file, to the one case where it is actually used. This makes us slightly more efficient in the non-file case, makes our functions a bit more consistent, and makes file:// extraction work again. Fixes FS#64648 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-05libmakepkg: fix empty arguments in parseoptsEthan Sommer
Previously parseopts checked if there was an argument by checking that the string was non-empty, resulting in empty arguments being incorrectly considered non-existent. This change makes parseopts check if arguments exist at all, rather than checking that they are non-empty Signed-off-by: Ethan Sommer <e5ten.arch@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-11-04libmakepkg: add optional argument support to parseoptsEthan Sommer
Adds a "?" suffix that can be used to indicate that an option's argument is optional. This allows options to have a default behaviour when the user doesn't specify one, e.g.: --color=[when] being able to behave like --color=auto when only --color is passed Options with optional arguments given on the command line will be returned in the form "--opt=optarg" and "-o=optarg". Despite that not being the syntax for passing an argument with a shortopt (trying to pass -o=foo would make -o's argument "=foo"), this is done to allow the caller to split the option and its optarg easily Signed-off-by: Ethan Sommer <e5ten.arch@gmail.com> Reviewed-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-30Comma failAllan McRae
2019-10-30makepkg: do not count hard links multiple times when calculating pkg sizeEli Schwartz
Exclude files with hardlinks when cat'ing all the files, and do a second run to look at each file with hardlinks, keep track of the ones we've already operated on, and only cat each inode once. Then use "wc -c" to get the size of all (deduplicated) files the same way we were already doing. Original-patch-by: Ronan Pigott <rpigott@berkeley.edu> Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-30makepkg: protect against unexpected whitespace in filenamesEli Schwartz
zipman: read -r protects against those evil manpages whose filenames contain backslash escapes, (muahahaha?) IFS= read protects against filenames with: - leading whitespace (but no one is actually stupid enough to configure their MAN_DIRS=() in makepkg.conf with such silly directories, *right*?) - trailing whitespace (but likewise, no one should be stupid enough to write an uncompressed manpage for section '1 ' or something) Also fix several other cases where we read filenames without protecting against surrounding whitespace, or without using null-delimited filenames when we could trivially do so. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-23Update copyright yearsAllan McRae
make update-copyright OLD=2018 NEW=2019 Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-22makepkg: don't warn when PACKAGER is unsetmorganamilo
makepkg now complains when PACKAGER is not in the format "name <email>". Hide this warning when PACKAGER is unset but still warn if it is set to something out of format. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07Support file with seccomp enabledEli Schwartz
Not all compression types can be detected in the seccomp sandbox, so we need to disable it. This requires either configuring makepkg to know the sandbox is available, or checking for file >= 5.38 in which the sandbox option is a no-op even when seccomp is disabled. - Requires autoconf-archive for autotools version compare macro. - meson version comparison could be made a lot simpler using meson-git. Fixes FS#58626 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-07makepkg: add rust support for *FLAGS and debug-prefix-mapEli Schwartz
The rust language supports $RUSTFLAGS to be used automatically in all rustc invocations. Allow setting this in makepkg.conf (e.g. for optimization or debuginfo support), and teach debug+strip to pass the rustc command line argument necessary to rewrite source file paths in the debugging symbols. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-04libmakepkg: check if PACKAGER has the expected format for WKD lookupJonas Witschel
pacman should be able to extract an email address from PACKAGER for WKD lookup, so issue a warning if it is not of the form "Example Name <email@address.invalid>". Neither the name nor the email address must contain additional angle brackets. Signed-off-by: Jonas Witschel <diabonas@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-10-04libmakepkg: fix typomorganamilo
Signed-off-by: Allan McRae <allan@archlinux.org>
2019-08-05makepkg: Ignore "<artificial>" source filesAustin Lund
An artificial symbol can be produced when requesting debugging symbols and the compiler has inlined a function. These symbols will give spurious results when listing source files for inclusion in debug packages. This will ignore these symbols and avoid an error that can be generated when creating a debug package. Signed-off-by: Austin Lund <austin.lund@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-06-06makepkg: restrict pkgname and pkgver to asciiAndrew Gregory
pkgname and pkgver are used as directory names within database files. libarchive does not provide a reliable locale-independent method for reading archive file names, causing errors when archive paths include non-ascii characters. This is a first step toward dealing with FS#49342, by hopefully reducing the number of packages with non-ascii data in the wild before updating libalpm to reject them outright. See https://github.com/libarchive/libarchive/wiki/Filenames and https://github.com/libarchive/libarchive/issues/587 Signed-off-by: Andrew Gregory <andrew.gregory.8@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-28makepkg: also move restore_envvars handling into libmakepkgEli Schwartz
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-28makepkg: move config loading into libmakepkgEli Schwartz
When scripting/automating around makepkg, it is sometimes desirable to know how makepkg will be configured to operate. One example is the archlinux devtools, which must forward select makepkg.conf variables into a build chroot (for example PACKAGER) or use those variables itself (for example {SRC,PKG,LOG}DEST). The configuration file can be in up to 3 places, and should be capable of being overridden via environment variables. It is sufficiently complex to represent distinct functionality, and sufficiently useful to merit easy accessibility in other scripts, therefore, let us move it into a publicly exposed utility library. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-28libmakepkg: add lint_config to validate SRCEXT/PKGEXTEli Schwartz
These variables must begin with .src.tar / .pkg.tar respectively, so fail early if those expectations are not matched. This prevents makepkg from creating e.g. package files literally named "./pacman-5.1.3-1-x86_64" which are actually uncompressed tarballs. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-28Support application/gzip MIME type in extractionKevin Mihelich
file 5.37 changed the gzip MIME type from application/x-gzip to application/gzip, so support this when checking to extract source files. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08makepkg: propagate error codes when package failed to sign correctlyEli Schwartz
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08libmakepkg: fix missing or inaccurate interdependenciesEli Schwartz
When the executable checking was refactored into libmakepkg, it carried with it, usage of $E_* error codes, which need to be declared from error.sh but are only available when the parent program already sources error.sh; additionally, message.sh was only loaded in a parent library, but not where it was needed, and option.sh was often loaded when it wasn't needed at all. util.sh, meanwhile, has always depended on message.sh functions. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08scripts: protect against unintended glob matching in [[ ]] RHSEli Schwartz
The right-hand side of the [[ ... = ... ]] keyword is an exception to the general rule that quoting is unnecessary with [[ This is usually not a problem, e.g. in libmakepkg, lint_one_pkgname will already fail if pkgname has an asterisk, but it certainly doesn't hurt to be "more proper" and go with the spec; it is more dangerous in repo-add, which can get caught in an infinite loop instead of safely asserting there is no package named 'foo*'. Reported-by: Rafael Ascensão <rafa.almas@gmail.com> Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08makepkg: correctly handle hg sources with updates on a non-default branchEli Schwartz
The "tip" ref actually signifies the most recently updated branch. hg does not support a default branch named anything other than "default", except by creating a "@" bookmark. The correct way to explicitly update to the default clone ref, is therefore to use one of these, rather than "tip". Fixes FS#62092 Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-05-08libmakepkg: fix migration to schema.sh for integsumsEli Schwartz
One of the callers was changed to use known_hash_algos, one was not. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19libmakepkg: fix reporting of invalid archive extensions in compress.shEli Schwartz
In commit 1825bd6716c2a51c92642e8b96beac0101e83805 this was split out from makepkg, but the warning was not properly migrated; $ext did not ever exist. As a result, no matter what you did, the only possible warning was: ==> WARNING: '' is not a valid archive extension. Fix to filter based on the presence of .tar in the argument, and building the $ext variable for all checking and messaging purposes within the function. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-19makepkg: use "shared" git clones when checking out sourcesEli Schwartz
In order to cache sources offline, makepkg creates *two* copies of every git repo. This is a useful tradeoff for network time, but comes at the cost of increased disk space. Normally, git can smooth this over automagically. Whenever possible, git objects are hardlinked to save space, but this does not work when SRCDEST and BUILDDIR are on separate filesystems. When the repo in question is both very large (linux.git for example is 2.2 GB) and crosses filesystem boundaries, this results in a lot of extra disk space being used; the most likely scenario is where BUILDDIR is a tmpfs for bonus ouch. git(1) has a builtin feature which serves this case handily: the --shared flag will create the info/alternates file instructing git to not copy or hardlink or create objects/packs at all, but merely look for them in an external location (that being the source of the clone). The downside of using shared clones, is that if you modify and drop commits from the original repo, or simply delete the whole repo altogether, you break the copy. But we don't care about that here, because 1) the BUILDDIR copy is meant to be a temporary copy strictly derived via PKGBUILD syntax from the SRCDEST, and must be able to be recreated at any time, 2) if the SRCDEST disappears, makepkg will redownload it, thus restoring the objects needed by the BUILDDIR clone, 3) if the user does non-default things like hacking on the BUILDDIR copy then deleting and re-cloning the SRCDEST may result in momentary breakage, but ultimately should be fine -- the unique objects they created will be stored in the BUILDDIR copy. While it's theoretically possible that upstream will force-push to overwrite the base tree from which makepkg is building (which they should not do), *and* the user deleted their SRCDEST which they should not do, *and* they saved work in makepkg's working directory which they should not do either... ... this is an unlikely chain of events for which we should not care. Using --shared is therefore helpful in immediately useful ways and IMHO has no actual downsides; we should use it. An alternative implementation would be to use worktrees. I've rejected this since it is essentially the same as shared clones, except adding additional restrictions on the branch namespace, and could potentially break existing use cases such as manually handling the SRCDEST in order to share repositories with normal working copies. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-03-07makepkg: add new checksum algorithm via coreutils b2sumEli Schwartz
coreutils 8.26 in December 2016 added this new hashing method which is compatible with the existing md5sum and sha*sum tool usage, while using the blake2 hash algorithm. makepkg uses coreutils to provide source file integrity checks via ${integ}sum binaries and it makes sense to offer this as an additional option. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-21libmakepkg: migrate to schema.sh for integsumsEli Schwartz
We cannot use most of the arrays defined in schema.sh as srcinfo is dependent on the order, but migrate the hashes for now. Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-02-04fix various typosmorganamilo
Signed-off-by: morganamilo <morganamilo@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-31libmakepkg: lint disallowed architecture specific variablesmorganamilo
Variables such as 'pkgdesc_x86_64' are invalid, instead of ignoring them raise an error. This also disallows using 'any' as an architecture specific variable Signed-off-by: morganamilo <morganamilo@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-31libmakepkg: lint disallowed variables in package()morganamilo
makepkg will now error if disallowed variables are set inside of the package function. Disallowed variables are variables that do exist, like 'makedepends' and 'pkgver' but can not be set inside of a package function. Signed-off-by: morganamilo <morganamilo@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-31libmakepkg: centralise random arrays of pkgbuild variablesmorganamilo
Refactor many of the different arrays of pkgbuild variables into scripts/libmakepkg/util/schema.sh.in. Signed-off-by: morganamilo <morganamilo@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-30libmakepkg: Add lzip ext supportChloe Kudryavtsev
lzip is a lossless data compressor designed to replace gzip and bzip2 as the standard general-purpose compressed format. - add .lz (lzip) support to libmakepkg/util/compress.sh:compress_as - add COMPRESSLZ to makepkg.conf.in - document COMPRESSLZ - document PKGEXT with `.tar.lz` Signed-off-by: Chloe Kudryavtsev <toast@toastin.space> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22libmakepkg: move checkdepends to the correct arraymorganamilo
Signed-off-by: morganamilo <morganamilo@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22libmakepkg: Implement extendable signature verificationEli Schwartz
Lookup the existence of matching functions for each protocol, and fallback on the generic file handler. New verification protocols can then be added via thirdparty libmakepkg drop-ins without requiring modifications to verify_signature.sh Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22libmakepkg: implement extendable source protocolsEli Schwartz
Lookup the existence of matching functions for each protocol, and fallback on the generic file handler. New source protocols can then be added via thirdparty libmakepkg drop-ins without requiring modifications to source.sh Fixes FS#49076 Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22libmakepkg: optimize get_protocol to always return proto, not proto+uriEli Schwartz
e.g. git+https:// is commonly used for git repositories cloned over HTTPS, but we assume a proto with a plus in it is actually a protocol followed by some URI handler. So we might as well simplify the return value and not have to always add glob matching everywhere when checking the proto in use. This is required in order to use the proto directly in function calls, which will be used in a followup patch. Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-22libmakepkg: make gettext replacements more templatedEli Schwartz
Making the undescore be translated is probably not something we need translators to think about. Additionally, a number of places which use the same text differ only by the variable being referenced, so simplifying the string means we can drop a redundant translation. Bonus: we save a few bytes here and there. \o/ Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-16libmakepkg: disallow using 'any' with other archesmorganamilo
Error if the arch array contains any and any other values. This also fixes a bug where the check for `$arch == 'any'` which only evaluated the first value in the array, meaning the rest of the values would not be linted. Signed-off-by: morganamilo <morganamilo@gmail.com> Signed-off-by: Allan McRae <allan@archlinux.org>
2019-01-14meson: fix regression that broke building libmakepkgEli Schwartz
In commit f7efa6a93d5361af610827d41045d87c7a72f2b5 we added a new file, and also wired it up to the build systems, but it got added under the wrong name in meson.build Signed-off-by: Eli Schwartz <eschwartz@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>