After searching more I found that I can't build some packages with ccache or without it. Traces leading to libtool, as if I rebuild libtool without ccache error vanishes.

Rebuilding libtool without ccache doesn't fix the problem I described though.

It looks like configure thinks that -fPIC doesn't work because the test produces stderr output ("warning: argument unused during compilation: '-D PIC'"). This is because clang warns for things that GCC doesn't warn about, in this case that "-D PIC" have no effect when compiling preprocessed code. See bug #8118.

ccache doesn't support clang yet, so it's in a sense just pure luck if it works at the moment. However, until clang is officially supported, a workaround should be to set CCACHE_CPP2 as this will avoid compiling preprocessed code.

Well, it's not that simple. The problem is not in -fPIC detection code.

Notice the command that autoconf tries to execute. The same command completes successfully on the same source file. But it seems that ccache modifies command before running it. clang is always noisy and strict and it says there's another '-D PIC' in the command, but command contains only '-DPIC' without space. It seems that ccache does here some gcc magic which clang doesn't understand.

I'll try to retest with CCACHE_CPP2 though.

Please also set CCACHE_LOGFILE to get a debug log. That way, you will get more information on what ccache does, which commmands it runs, etc.

Created attachment 6936 [details]
compilation logs

I've tried again autoconfiguration for -fPIC. I've also added CCACHE_RECACHE to prevent cache lookup.

In both cases autoconf sees this:

configure:11180: checking for clang++ option to produce PIC
configure:11432: result: -fPIC
configure:11440: checking if clang++ PIC flag -fPIC works
configure:11458: clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fPIC -DPIC conftest.cc >&5
clang++: warning: argument unused during compilation: '-I /usr/local/include'
clang++: warning: argument unused during compilation: '-I /usr/local/include'
clang++: warning: argument unused during compilation: '-D PIC'
configure:11462: $? = 0
configure:11473: result: no

I'm totally puzzled how -I option was specified more then one time (or even triple?).

Tried this script:

#!/bin/sh -

echo $* >> /tmp/full.log
ccache /usr/bin/clang $*

Nothing intresting except now autoconf result is positive.

(In reply to comment #6)
> Tried this script:
> 
> #!/bin/sh -
> 
> echo $* >> /tmp/full.log
> ccache /usr/bin/clang $*
> 
> Nothing intresting except now autoconf result is positive.

Positive but only for clang++...

(In reply to comment #7)
> > Nothing intresting except now autoconf result is positive.
> Positive but only for clang++...

Sorry, that was only because of incorrect link to ccache.

(In reply to comment #5)
> In both cases autoconf sees this: [...]

It doesn't make sense that clang++ emits those warnings when you use CCACHE_CPP2. As can be seen from the ccache log, ccache runs this commands (output name tweaked):

  /usr/bin/clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fPIC -DPIC -E conftest.cc
  /usr/bin/clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fPIC -DPIC -o file.o.foo conftest.cc

If you run those commands manually, does clang++ emit the same warnings?

> I'm totally puzzled how -I option was specified more then one time (or even
> triple?).

What's strange about that?

(In reply to comment #9)
> (In reply to comment #5)
> > In both cases autoconf sees this: [...]
> 
> It doesn't make sense that clang++ emits those warnings when you use
> CCACHE_CPP2. As can be seen from the ccache log, ccache runs this commands
> (output name tweaked):
> 
>   /usr/bin/clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing
> -I/usr/local/include -fPIC -DPIC -E conftest.cc
>   /usr/bin/clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing
> -I/usr/local/include -fPIC -DPIC -o file.o.foo conftest.cc
> 
> If you run those commands manually, does clang++ emit the same warnings?

I'll include full list of my tests, just in case. Mentioned file is:

int some_variable = 0;

Trying to run first command without ccache:

[] /usr/bin/clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -fPIC -DPIC -E conftest.cc
# 1 "conftest.cc"
# 1 "conftest.cc" 1
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 137 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "conftest.cc" 2
int some_variable = 0;

Same with ccache:

[] clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -fPIC -DPIC -E conftest.cc
# 1 "conftest.cc"
# 1 "conftest.cc" 1
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 137 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "conftest.cc" 2
int some_variable = 0;

Second command without ccache:

[] /usr/bin/clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fPIC -DPIC conftest.cc

Same with ccache:

[] clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fPIC -DPIC conftest.cc
clang++: warning: argument unused during compilation: '-I /usr/local/include'
clang++: warning: argument unused during compilation: '-I /usr/local/include'
clang++: warning: argument unused during compilation: '-D PIC' 

> > I'm totally puzzled how -I option was specified more then one time (or even
> > triple?).
> 
> What's strange about that?

I dunno.

Running second command with ccache but only one -I parameter results in:

[] clang++ -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -fPIC -DPIC conftest.cc

And no errors...

But I think I found my poison. Here is a piece of code:

#include <stdio.h>
#include <unistd.h>

int main(int argc, char** argv) {
        const char* cmd = "/usr/bin/clang++";

        for(int i = 0; i < argc; i++) {
                printf("%s ", argv[i]);
        }

        printf("\n");

        execvp(cmd, argv);
};

Nothing hard to explain - prints command line and executes another binary.

And here is the sample output:

[] ./a.out -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fPIC -DPIC conftest.cc
./a.out -c -O2 -pipe -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fPIC -DPIC conftest.cc
/home/arcade/a.out -cc1 -triple x86_64-unknown-freebsd9.0 -emit-obj -disable-free -main-file-name conftest.cc -pic-level 2 -mdisable-fp-elim -relaxed-aliasing -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64 -momit-leaf-frame-pointer -coverage-file conftest.o -resource-dir /home/arcade/../lib/clang/3.0 -D PIC -I /usr/local/include -I /usr/local/include -O2 -fdeprecated-macro -ferror-limit 19 -fmessage-length 142 -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o conftest.o -x c++ conftest.cc

So there is another call for the binary and now I perfectly see the changes made to the command line: '-DPIC' => '-D PIC'. And this second call is complaining about something.

I'll try to retest all of this with 'fork'.

clang should not be complaining about unused -I/-D flags when compiling (-c).  if it is, that's a bug to report to the clang authors.

This one is not about how clang complains but how clang executes.

I rerun my test on current version of clang/ccache and they keep giving me the same result:

No ccache:

arcade@green\~/work/test> clang -c -O2 -pipe -march=core2 -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fno-rtti -fno-exceptions test1.c
./clang -c -O2 -pipe -march=core2 -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fno-rtti -fno-exceptions test1.c 
/home/arcade/work/test/clang -cc1 -triple x86_64-unknown-freebsd9.0 -emit-obj -disable-free -main-file-name test1.c -mrelocation-model static -mdisable-fp-elim -relaxed-aliasing -masm-verbose -mconstructor-aliases -munwind-tables -target-cpu core2 -momit-leaf-frame-pointer -coverage-file test1.o -resource-dir /home/arcade/work/test/../lib/clang/3.1 -I /usr/local/include -I /usr/local/include -fmodule-cache-path /var/tmp/clang-module-cache -O2 -fdebug-compilation-dir /home/arcade/work/test -ferror-limit 19 -fmessage-length 186 -mstackrealign -fno-rtti -fgnu-runtime -fobjc-runtime-has-arc -fobjc-runtime-has-weak -fobjc-dispatch-method=non-legacy -fdiagnostics-show-option -fcolor-diagnostics -o test1.o -x c test1.c 

Ccache:

arcade@green\~/work/test> clang -c -O2 -pipe -march=core2 -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fno-rtti -fno-exceptions test1.c
./clang -c -O2 -pipe -march=core2 -I/usr/local/include -fno-strict-aliasing -I/usr/local/include -fno-rtti -fno-exceptions test1.c

The problem may also be in file naming differences. GCC consists of many different executables and clang is the only one. So when clang re-executes itself with it's own parameters those parameters are filtered through ccache too.

Anyway I'll try to bug clang developers.

from ccache's perspective, the internal behavior of clang doesn't matter.  if clang is incorrectly warning about -D/-I flags when it is compiling something, then it's a bug to take up with the clang developers.  ccache should not be working around broken behavior in clang.

(In reply to comment #12)
> This one is not about how clang complains but how clang executes.

I'm still not sure about that. I haven't seen anything that makes me suspect anything other than that the problem is that clang complains about -D/-I and friends when run with -E.

(The fact that clang calls itself a second time shouldn't matter: clang can't see that it's masqueraded by ccache since ccache sets up argv[0] (passed to execvp when executing the compiler) to be the compiler, unlike your test program which has itself in argv[0].)

Please try this script:

===[cut here]=========================================================
#!/bin/sh

export CCACHE_DIR=ccachedir.$$
command="/usr/bin/clang -I/foo -c ccachetest-$$.c"

run() {
    rm -rf $CCACHE_DIR
    echo "int variable = 0;" >ccachetest-$$.c
    echo "--- $*"
    $*
    rm -rf $CCACHE_DIR ccachetest-$$.c
}

run $command
run ccache $command
run env CCACHE_CPP2=1 ccache $command
===[cut here]=========================================================

When run with clang version "3.0-6ubuntu3", it gives me this output:

===[cut here]=========================================================
--- /usr/bin/clang -I/foo -c ccachetest-7495.c
--- ccache /usr/bin/clang -I/foo -c ccachetest-7495.c
clang: warning: argument unused during compilation: '-I /foo'
--- env CCACHE_CPP2=1 ccache /usr/bin/clang -I/foo -c ccachetest-7495.c
===[cut here]=========================================================

That is, when ccache doesn't run clang with -E (the CCACHE_CPP2=1 case), clang doesn't complain. The tricky thing here is that once you've built without CCACHE_CPP2=1, then the "argument unused during compilation" warnings have also been stored in the cache, and then it doesn't matter if you later run with CCACHE_CPP=1 since ccache will give you the previous result from the cache. That's why my script above makes sure to run with a separate clean ccache directory each time.

> [...] So when clang re-executes itself with it's own parameters those
> parameters are filtered through ccache too.

If this were the case, you would see the re-execution in ccache's log, which I guess you haven't?

(In reply to comment #13)
> ccache should not be working around broken behavior in clang.

I'm not against adding special cases for clang or other compilers as long as it doesn't affect other compilers negatively. ccache already has compiler-specific workarounds: there is code for handling several GCC bugs, there's special treatment for a broken distcc version shipped by Apple, etc.

Yes CCACHE_CPP2 works for me and behaves correctly. I think I was totally wrong about the cause.

The root of the problem is that clang correctly points out that none of preprocessing flags affects compilation result. GCC just skips them.

There are other opinions on fixing this.

http://llvm.org/bugs/show_bug.cgi?id=13282

How about stripping preprocessing flags when running over preprocessed source? As Richard Smith says this perfectly works for gcc too and this seems quite logical for me.

(In reply to comment #16)
> How about stripping preprocessing flags when running over preprocessed source?

Sure (as commented in http://www.mail-archive.com/ccache@lists.samba.org/msg00823.html, but you're perhaps not reading the ccache list?). One problem is finding a good list of options to strip. Suggestions are welcome.

> As Richard Smith says this perfectly works for gcc too and this seems quite
> logical for me.

There are other compilers than GCC and clang (and many older GCC versions) to consider as well, but I think it's likely that there will be no problem.

Various clang-related improvements have been made on the master branch.

Volodymyr, could you try if the current ccache master branch works for you?

(In reply to comment #18)
> Various clang-related improvements have been made on the master branch.
> 
> Volodymyr, could you try if the current ccache master branch works for you?

I've tried with master on d1b2219297 and this is still an issue with clang. Defining CCACHE_CCP2=1 is still required.

Example call that works without ccache or with CCACHE_CPP2=1:

/usr/local/libexec/ccache/world/cc  -O2 -pipe -march=pentium4 -I/usr/src/lib/libelf -I. -DLIBELF_TEST_HOOKS -std=gnu99 -Qunused-arguments -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls -Wold-style-definition -Wmissing-variable-declarations -Wno-pointer-sign -Wno-empty-body -Wno-string-plus-int -c /usr/src/lib/libelf/elf_scn.c -o elf_scn.o
/usr/src/lib/libelf/elf_scn.c:198:41: error: equality comparison with extraneous parentheses [-Werror,-Wparentheses-equality]
 if (((&e->e_u.e_elf.e_scn)->stqh_first == ((void *)0))) {
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~
/usr/src/lib/libelf/elf_scn.c:198:41: note: remove extraneous parentheses around the comparison to silence this warning
 if (((&e->e_u.e_elf.e_scn)->stqh_first == ((void *)0))) {
     ~                                  ^             ~
/usr/src/lib/libelf/elf_scn.c:198:41: note: use '=' to turn this equality comparison into an assignment
 if (((&e->e_u.e_elf.e_scn)->stqh_first == ((void *)0))) {
                                        ^~
                                        =
1 error generated.


Line of code:

    if (STAILQ_EMPTY(&e->e_u.e_elf.e_scn)) {

The macro:

    #define STAILQ_EMPTY(head)      ((head)->stqh_first == NULL)

@Bryan: I'd say that what you do describe with "#define" is totally different bug that would take place with any other compiler. It's the result of:

    #define MACRO(x) (x == NULL)

    if (MACRO(y)) {

Yielding this after preprocessing:

    if (MACRO((y == NULL)) {
    ^--------^           ^-^ - original code
              ^---------^    - macro substitute

Thus this would give us extraneous parentheses.

This problem can be mitigated by not doing full prepossession pass but reverting to unwind only #include's.

However I can't reproduce this bug with this sample:

    #define MACRO(x) ((int)x == (2 - 1))

    int main(void) {
      if (MACRO(1)) {
        return(0);
      } else {
        return(1);
      }
    }
    
This file perfectly compiles with ccache and without CCACHE_CPP2. My current compiler is clang3.3.

Isn't this simply a duplicate of bug #8118?

> Isn't this simply a duplicate of bug #8118?

Since Bryan Drewery tested d1b22192 which contains 1ea5033e (fix of #8118), I guess not?

Has anybody been able to provide a reduced test case that demonstrates the problem?

(In reply to comment #22)

> Has anybody been able to provide a reduced test case that demonstrates the
> problem?

A variant of this problem is logged here:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192158

I've tried to reduce it to a testcase, but so far that didn't work. CPPFLAGS is definitely the trigger, removing it also solved other ports suffering from this problem.

Closing this now since I get believe that the originally reported bug has been resolved.