Since the addition of modules.aliases.bin, depmod has to open that
index multiple times and parse it over and over again:
$ sudo strace -e openat ./tools/depmod 2>&1 | grep modules.builtin.modinfo | wc -l
299
$ time sudo ./tools/depmod
real 0m7.814s
user 0m7.571s
sys 0m0.237s
Rework the logic in depmod so it does everything: open, read and parse. The
format is very straightforward and we don't need to keep it in a data structure
since we only want to add the result to a index. New output:
$ sudo strace -e openat ./tools/depmod 2>&1 | grep modules.builtin.modinfo | wc -l
1
$ time sudo ./tools/depmod
real 0m7.663s
user 0m7.516s
sys 0m0.139s
Indexes still match:
$ cmp /tmp/modules.builtin.alias.bin.new
/tmp/modules.builtin.alias.bin.old; echo $?
0
Fix: https://github.com/kmod-project/kmod/issues/11
Now that libkmod uses modules.builtin.bin again, we don't need to add
the module names in modules.builtin.alias.bin and just add the aliases.
After this change, here are the new sizes for the indexes:
Before After index
21k 6.4K modules.builtin.alias.bin
11k 11K modules.builtin.bin
Due to wrong documentation on kmod_module_get_info() we ended up
checking for 0 as return. Check for > 0 to decided if we want to write
the index to the file, otherwise we would output a 0-sized index on
success.
Configuration in /etc should have higher prio than /run.
Given how rarely configuration in /run is used with depmod, this is
likely not to cause any problems, even if it's a change in behavior.
The file is always created and unless we return an error, the temporary
file is renamed to its final destination. All other places write the
index without checking if the index is empty, so just do the same.
Reported-by: Joe Buehler <aspam@cox.net>
idx is allocated in the beginning but it's not freed if there is
a failure after the allocation.
Change the error path: return immediately if idx allocation fails
and then free it in both success and error path at the end.
Signed-off-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
index_write() relies on fseek/ftell to manage the position to which we
are write and thus needs the file stream to support it.
Right now when trying to write the index to stdout we fail with:
depmod: tools/depmod.c:416: index_write: Assertion `initial_offset >= 0' failed.
Aborted (core dumped)
We have no interest in outputting our index to stdout, so just skip it
like is done with other indexes.
While at it, add/remove some newlines to improve readability.
Reported-by: Yanko Kaneti <yaneti@declera.com>
Fix: b866b2165a ("Lookup aliases in the modules.builtin.modinfo")
New modules.builtin.modinfo duplicates modules.builtin in the built-in
module name search. If it exists, then we can use this file, but if not,
then we need to fallback to the old file.
Signed-off-by: Alexey Gladkov <gladkov.alexey@gmail.com>
In a couple of places depmod concatenates the module directory and filename
with snprintf. This can technically overflow creating an unterminated string if
module directory name is long. Use openat instead as is done elsewhere in
depmod. This avoids the snprintf, the extra buffer on stack, and the gcc
warning. It may even fix a corner case when the module direcotry name is just
under PATH_MAX.
[ Lucas: fix up coding style and closing fd on error path ]
Signed-off-by: Michal Suchanek <msuchanek@suse.de>
Depmod does not use unique filename for temporary files. There is no
guarantee the user does not attempt to run mutiple depmod processes in
parallel. If that happens a temporary file might be created by
depmod(1st), truncated by depmod(2nd), and renamed to final name by
depmod(1st) resulting in corrupted file seen by user.
Due to missing mkstempat() this is more complex than it should be.
Adding PID and timestamp to the filename should be reasonably reliable.
Adding O_EXCL as mkstemp does fails creating the file rather than
corrupting existing file.
Signed-off-by: Michal Suchanek <msuchanek@suse.de>
depmod deletes the module dependency files before moving the temporary
files in their place. This results in user seeing no dependency files
while they are updated. Remove the unlink call. The rename call should
suffice to move the new file in place and unlink the old one. It should
also do both atomically so there is no window when no dependency file
exists.
Signed-off-by: Michal Suchanek <msuchanek@suse.de>
depmod_module_is_higher_priority checks module's path if it is under
module root directory and if so uses relative to the root path to
lookup the module in override and search lists.
Originally only relative path was used in the function, so the
variables with full path and and path length were changed:
newpath += cfg->dirnamelen + 1;
newlen -= cfg->dirnamelen + 1;
oldpath += cfg->dirnamelen + 1;
oldlen -= cfg->dirnamelen + 1;
Commit 7da6884e73 (depmod: implement
external directories support) changed the logic since it need the
full path to the module for comparations as well.
Unfortunately, it introduce a mistake in calculation of the relative
paths replacing '-=' with assignment to a new variable -- the
'cfg->dirnamelen + 1' value must be substracted all together. It
breaks, for example, overrides lookup.
Fix the calculation by putting braces around the value in the
subsctuction expression.
Signed-off-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
d46136bb59 ("depmod: Ignore PowerPC64 ABIv2 .TOC. symbol") adds fake
.TOC. unconditionally but when there is .TOC. in the kernel adding the
fake one breaks resolving .TOC.
Fixes: d46136bb59 ("depmod: Ignore PowerPC64 ABIv2 .TOC. symbol")
Signed-off-by: Michal Suchanek <msuchanek@suse.de>
The idea is to add a configuration keyword, external, which
will list directories for scanning for particular kernel version
mask:
external 4.10 /the/modules/dir /second/modules/dir
And extend "search" keyword to set it's priority with pseudo dir
"external" (as it's done for built-in):
search subdir external subdir2 built-in subdir3
(actually, the version is the same as for override keyword: * or
posix regexp, so example above is a bit incorrect).
All other logic left the same: if there are duplicates, only one
is under consideration and it is unloadable if it is bad.
The resulting modules.dep will contain entries a-la:
/the/modules/dir/module1.ko:
kernel/module2.ko: /the/modules/dir/module1.ko
(here /lib/modules/$(uname -r)/kernel/module2.ko depends of
symbols, provided by /the/modules/dir/module1.ko and external has
higher priority).
modprobe and modinfo understand it out of box.
This is a pretty simple extention of existing logic, since now
depmod already is able to:
a) scan modules with full path from command line without -a
switch;
b) detects broken symbol dependencies and broken modversions,
what assumes, that modules are already are not built for the
existing kernel.
Signed-off-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
The recursive search code used used pretty big, PATH_MAX,
automatic storage buffer for the module directory scanning. Some
time ago there was scratchbuf implemented, which dynamically
reallocates its buffer on demand. The patch takes it in use for
the scanning code also. The initial size is hardcoded to 256
bytes which sounds good enough for most usecases so there should
be not many reallocations.
Signed-off-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
Prepare to implement external directories support.
The patch splits depmod_modules_search() function to two
functions: depmod_modules_search(), called by the high level with
intention to search all possible modules, and
depmod_module_search_path(), which takes path as a parameter and
scans modules under the path only. Initially it is used to scan
the same depmod->cfg->dirname path only.
Signed-off-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
Prepare to implement external directories support.
It's better to isolate behaviour difference under the
cfg_search_add() call, then make the client code aware of it.
In case of external modules/directories support, there will be
one more keyword added, so making the clients aware of it makes
even less sense.
Signed-off-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
The c7ce9f0c80 commit (depmod:
handle nested loops) introduced a bunch of possible memory leaks
in error path. In the real world scenario it is not a problem,
since the utility quits if it detects any of the errors, but from
the programming point of view, it is not nice. So, add the
cleanups.
Signed-off-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
This is a rework of depmod report cycles logic to make it
tolerant to more complex loops.
The patch tries to remember own path for vertexes which makes it
possible to handle configurations with common edges and
non-cyclic modules.
It assumes that the previous dependency calculations can not give
as input something like
mod_a -> mod_b -> <loop>, but
<loop> -> mod_a -> mod_b should be fine.
Signed-off-by: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
Only print actual cyclic dependencies. Print count of all the modules
in cyclic dependency at the end of the function so that dependent
modules which are not in cyclic chain can be ignored.
Printing dependent modules which are not in cyclic chain causes buffer
overflow as m->modnamesz is not included in buffer size calculations
(loop == m is never true). This buffer overflow causes kmod to crash.
Update depmod test to reflect the change as well.
Reported-by: Andreas Färber <afaerber@suse.de>
Signed-off-by: Mian Yousaf Kaukab <yousaf.kaukab@suse.com>
The .TOC. symbol on the PowerPC64 ABIv2 identifies the GOT
pointer, similar to how other architectures use _GLOBAL_OFFSET_TABLE_.
This is not a symbol that needs relocation, and should be ignored
by depmod.
Currently, if a value that doesn't match a kernel version
("%u.%u") is passed in, depmod silently falls back to
using uname. Rather than try and work around the caller passing
bad data, just exit out instead.
In between the start of the program and the call to log_setup_kmod_log,
the only messages that will be printed are the ones at or above the
global default level. Debug messages in this range will never be printed
so remove them.
malloc(0) can return != NULL. We need to pass the pointer to free().
This happens if index__haschildren(node) returned true, but
child_count is set to 0.
Reusing the root variable was a bad idea. Doing so we could call free()
on a variable that was not allocated. For example: "depmod -b / -h".
Since we would jump to cmdline_failed, root would not be duplicated.
Instead of fighting the order in the options, just used the cleanup
attribute and remove the calls to free() on "config_paths" and "root".
Make the includes be libkmod/libkmod.h for code outside of library. This
fixes the broken build after 1315123 ('build-sys: Don't add libkmod
subdirectory to include path').
Instead of repeating all documentation, point to the documentation
available in libkmod-index.c
This also removes INDEX_PRIORITY_MIN that was not being used.
The only user outside of libkmod-util is depmod, which really only needs
to get the string for the extension of uncompressed modules. It doesn't
need to access the array itself.
Since now depmod fails when there are module loops, let's at least give
better error messages, printing the loops we found. Since we may have
more than 1 loop, just printing the modules that are in loop is not
very clear.
Assuming as an example 2 independent loops, this is how the new messages
compare to the old ones:
Before:
depmod: ERROR: Found 5 modules in dependency cycles!
depmod: ERROR: /tmp/test-kmod//lib/modules/3.14.4-1-ARCH/kernel/moduleE.ko in dependency cycle!
depmod: ERROR: /tmp/test-kmod//lib/modules/3.14.4-1-ARCH/kernel/moduleB.ko in dependency cycle!
depmod: ERROR: /tmp/test-kmod//lib/modules/3.14.4-1-ARCH/kernel/moduleC.ko in dependency cycle!
depmod: ERROR: /tmp/test-kmod//lib/modules/3.14.4-1-ARCH/kernel/moduleD.ko in dependency cycle!
depmod: ERROR: /tmp/test-kmod//lib/modules/3.14.4-1-ARCH/kernel/moduleA.ko in dependency cycle!
After:
depmod: ERROR: Found 5 modules in dependency cycles!
depmod: ERROR: Cycle detected: moduleE -> moduleD -> moduleE
depmod: ERROR: Cycle detected: moduleB -> moduleC -> moduleA -> moduleB
In mod->modnamelen we were actually including the '\0', i.e.
strlen(modname) + 1. So rename it to modnamesz and add a comment in
depmod_module_is_higher_priority() to notice why it's correct since the
new one is really using strlen(modname).
Since the beginning depmod just warned about dependency loops and upon
creation of modules.dep{,.bin} it skipped the modules that were part of
a loop. However just skipping the modules may come as a surprise to
kernel module developers: they will need to try to load the module (or
to pay attention to the log messages) to notice thavt the module has not
been put in the index. Also, differently from module-init-tools we were
not skipping modules that depend on modules with dependency loops,
leading to a segfault in depmod.
So this is a summary of the change in behavior with this patch:
Loop 1)
A -> B -> C -
^ |
'------------
Before:
depmod: WARNING: found 3 modules in dependency cycles!
depmod: WARNING: /tmp/test-kmod/lib/modules/3.14.2-1-ARCH/kernel/moduleB.ko in dependency cycle!
depmod: WARNING: /tmp/test-kmod/lib/modules/3.14.2-1-ARCH/kernel/moduleC.ko in dependency cycle!
depmod: WARNING: /tmp/test-kmod/lib/modules/3.14.2-1-ARCH/kernel/moduleA.ko in dependency cycle!
return code: 0
After:
depmod: ERROR: Found 3 modules in dependency cycles!
depmod: ERROR: /tmp/test-kmod/lib/modules/3.14.2-1-ARCH/kernel/moduleB.ko in dependency cycle!
depmod: ERROR: /tmp/test-kmod/lib/modules/3.14.2-1-ARCH/kernel/moduleC.ko in dependency cycle!
depmod: ERROR: /tmp/test-kmod/lib/modules/3.14.2-1-ARCH/kernel/moduleA.ko in dependency cycle!
return code: 2
Loop 2)
A -> B -> C -
^ |
'-------
Before:
depmod: WARNING: found 2 modules in dependency cycles!
depmod: WARNING: /tmp/test-kmod//lib/modules/3.14.2-1-ARCH/kernel/moduleB.ko in dependency cycle!
depmod: WARNING: /tmp/test-kmod//lib/modules/3.14.2-1-ARCH/kernel/moduleC.ko in dependency cycle!
Segmentation fault (core dumped)
After:
depmod: ERROR: Found 2 modules in dependency cycles!
depmod: ERROR: /tmp/test-kmod/lib/modules/3.14.2-1-ARCH/kernel/moduleB.ko in dependency cycle!
depmod: ERROR: /tmp/test-kmod/lib/modules/3.14.2-1-ARCH/kernel/moduleC.ko in dependency cycle!
return code: 2
The segfault above could be fixed, but let's just fail everything
because dependency cycles should be fixed in the modules rather than
just be skipped in the index.
This information can be found in /lib/modules/`uname -r`/modules.softdep, and
has only recently been exported by the kernel.
Also remove the advice about copying modules.softdep to /lib/modules as it is
not clear how to do this correctly with several kernels installed with
potentially conflicting soft dependencies.
Currently e.g. "search foo foobar built-in" will cause unpredictable
results if baz.ko is in both foo/ and foobar/, since "foo" in search may
match both of those directories and the preferred module therefore
depends on processing order.
Fix the code to ensure that the match is performed on full pathname
components only.
Lucas De Marchi
Yauheni Kaliuta
Alexey Gladkov
Michal Suchanek
Mian Yousaf Kaukab
Anton Blanchard
Josh Triplett
Laura Abbott
Colin Walters
Tom Gundersen
Anssi Hannula