On a source tree that is stored in a directory that has a top level path component that is symlinked to another directory, make test fails with the following error. SUITE: "basedir", TEST: "set CCACHE_BASEDIR" - Expected "cache miss" to be 1, got 0 I think it has something do with the symlink in the path component since the same test that is run on a different PC (64 bit Debian GNU/linux) succeeded. The following is a memo I prepared when I experienced the issue on a 32-bit Debian. --- The current ccache git source has a problem. make test fails with the following message. SUITE: "basedir", TEST: "set CCACHE_BASEDIR" - Expected "cache miss" to be 1, got 0 cache directory /home/ishikawa/repos/ccache/testdir.16250/.ccache primary config /home/ishikawa/repos/ccache/testdir.16250/.ccache/ccache.conf secondary config (readonly) /usr/local/etc/ccache.conf cache hit (direct) 0 cache hit (preprocessed) 0 cache miss 0 preprocessor error 1 files in cache 0 cache size 0.0 kB max cache size 5.0 GB TEST FAILED Test data and log file have been left in testdir.16250 make: *** [test] Error 1 I investigated. The error is due to the preprocessor error from the left ccache.log in the temporary created test.NNNNN directory. >From the log: [2013-09-02T21:37:52.836170 27715] Command line: /home/ishikawa/repos/ccache/ccache gcc -I/home/ishikawa/repos/ccache/testdir.16250/dir1/include -c src/test.c [2013-09-02T21:37:52.836268 27715] Hostname: debian-vbox-ci [2013-09-02T21:37:52.836297 27715] Working directory: /home/ishikawa/repos/ccache/testdir.16250/dir1 [2013-09-02T21:37:52.836512 27715] Source file: src/test.c [2013-09-02T21:37:52.836533 27715] Object file: test.o [2013-09-02T21:37:52.836624 27715] Running preprocessor [2013-09-02T21:37:52.836643 27715] Executing /usr/bin/gcc -I../../../../../../new-hd1/extra/ishikawa/download/repos/ccache/testdir.16250/dir1/include -E src/test.c [2013-09-02T21:37:52.836661 27715] Unlink /home/ishikawa/repos/ccache/testdir.16250/.ccache/tmp/test.tmp.debian-vbox-ci.27715.i [2013-09-02T21:37:52.836702 27715] Unlink /home/ishikawa/repos/ccache/testdir.16250/.ccache/tmp/tmp.cpp_stderr.debian-vbox-ci.27715 [2013-09-02T21:37:52.875022 27715] Unlink /home/ishikawa/repos/ccache/testdir.16250/.ccache/tmp/test.tmp.debian-vbox-ci.27715.i [2013-09-02T21:37:52.875144 27715] Unlink /home/ishikawa/repos/ccache/testdir.16250/.ccache/tmp/tmp.cpp_stderr.debian-vbox-ci.27715 [2013-09-02T21:37:52.875186 27715] Preprocessor gave exit status 1 [2013-09-02T21:37:52.875215 27715] Failed; falling back to running the real compiler [2013-09-02T21:37:52.875233 27715] Executing /usr/bin/gcc -I/home/ishikawa/repos/ccache/testdir.16250/dir1/include -c src/test.c [2013-09-02T21:37:52.875270 27715] Acquired lock /home/ishikawa/repos/ccache/testdir.16250/.ccache/c/stats.lock [2013-09-02T21:37:52.875513 27715] Releasing lock /home/ishikawa/repos/ccache/testdir.16250/.ccache/c/stats.lock [2013-09-02T21:37:52.875534 27715] Unlink /home/ishikawa/repos/ccache/testdir.16250/.ccache/c/stats.lock [2013-09-02T21:37:52.875560 27715] Result: preprocessor error So why did preprocessor failed? Executing /usr/bin/gcc -I../../../../../../new-hd1/extra/ishikawa/download/repos/ccache/testdir.7413/dir1/include -E src/test.c I ran the compiler command manually. "cd" to testdir.16250/dir1/ /usr/bin/gcc -I/home/ishikawa/repos/ccache/testdir.16250/dir1/include -c src/test.c Funny, this worked manually!? Now why on earth, does the original Command line: /home/ishikawa/repos/ccache/ccache gcc -I/home/ishikawa/repos/ccache/testdir.7413/dir1/include -c src/test.c gets transformed to the preprocessor command line: Executing /usr/bin/gcc -I../../../../../../new-hd1/extra/ishikawa/download/repos/ccache/testdir.7413/dir1/include -E src/test.c with relative pathname? What is the content of path_stderr file when the cpp failed. if (status != 0) { if (!direct_i_file) { tmp_unlink(path_stdout); } tmp_unlink(path_stderr); <--- deleted after failure... cc_log("Preprocessor gave exit status %d", status); stats_update(STATS_PREPROCESSOR); failed(); } I patched the ccache.c so that this temporary file is left behind after a preprocessor failure, and found in another test run (note the different testdir.NNNN directory name), the following file was left behind after the failure. /home/ishikawa/repos/ccache/testdir.7413/.ccache/tmp/test.tmp.debian-vbox-ci.18985.i It contained the following. --- begin quote src/test.c:1:18: fatal error: test.h: No such file or directory compilation terminated. --- So after invoking /usr/bin/gcc -I../../../../../../new-hd1/extra/ishikawa/download/repos/ccache/testdir.7413/dir1/include -E src/test.c Working directory: /home/ishikawa/repos/ccache/testdir.7413/dir1 it fails. Again, why? On a hunch, I checked for the include directory. it DOES NOT EXIST!!! The preceding number of "../" to climb up the directory tree is WRONG! >From "../dir1" directory, I needed the following path name to properly enumerate the include files (i.e., test.h): ls ../../../../../../../../new-hd1/extra/ishikawa/download/repos/ccache/testdir.7413/dir1/include test.h Note the required number of "../" is 8 while somehow test.sh produced only six "../". I am not sure if - there is something wrong in the test.sh logic, - there is something wrong in ccache source code which tries to rewrite the include path name into a relative one. On my PC, /extra is symlink: ls -l /extra lrwxrwxrwx 1 root root 14 Oct 3 2011 /extra -> /new-hd1/extra/ other components of the paths are ordinary directories. --- TIA PS: As I noted above, the test runs successfully with the tree without symbolic name component on another computer, the translation process that tries to base the pathname against the basename of the ccache seems to buggy in the presence of symbolic link in the pathname component. TIA
This looks like the same issue as bug #9801. Could you try if the attached patch works for you?
(In reply to comment #1) > This looks like the same issue as bug #9801. Could you try if the attached > patch works for you? Joel, thank you for your comment. Things has gotten a little complicated. With your patch ccache-3.1.9 source now runs its test fine: make test finishes successfully (!) But the current ccache git repository source still fails with the same error. Then I realize that the current git repository has many more tests than the current release: current release has > PASSED: 176 assertions, 64 tests, 9 suites ************************** whereas the current git repository has > PASSED: 439 assertions, 86 tests, 10 suites *************************** so I assume that one of the new tests is failing in the current git repository source. Sorry, I should have mentioned this in my original post. But your patch gives me a clue about where to look for potential problems. I will sprinkle the source file with printf statements and see what is transforming the header include directory with a series of "../", and a couple of them missing at that. TIA
Created attachment 9190 [details] current working directory needs to be stored in canonical form. I found the root cause. The relative path generation routine assumes that both pathnames given to it are in canonical forms (removing unnecessary a/../b, or resolve symlinks). In util.c: /* * Compute a relative path from from (an absolute path to a directory) to to (a * path). Assumes that both from and to are well-formed and canonical. Caller * frees. */ char * get_relative_path(const char *from, const char *to) However, one important variable that contains a directory path is not necessarily in canonical form. That is, current_working_dir. The attached patch fixes this issue. (There are two places where current_working_dir is set after calling get_cwd().) One of the places got it right (I think it is called after ccache reset, but the initialization routine forgets to canonicalize current_working_dir.) After the fix, the |make test| passes on the PC where the source tree of ccache is stored in a directory which is symlinked to another one: I always use the directory path that contains the symbolic link in its path component for historical reasons and got bitten. TIA PS: Maybe get_relative_path (in a debug build for testing purposes) can check that the passed pathnames are indeed in canonical form by calling x_realpath(...) : which is expensive, and that is why only for debug build for testing purposes.
(In reply to comment #3) > I found the root cause. > [...] > However, one important variable that contains a directory path is not > necessarily in canonical form. That is, current_working_dir. It looks like I made a mistake when merging maint to master at some point - the setting of current_working_dir was intended to be done lazily in make_relative_path(), not in initialize(). I committed a fix and a test to master in c9f60ad112ad8e254880d78cd1e5d16da5c98c53. Thank you very much for finding the root cause!
Included in v3.2.