Bug 8081 - ccache 3.1+ doesn't handle -MTtarget and -MQtarget like GCC 4.x does
Summary: ccache 3.1+ doesn't handle -MTtarget and -MQtarget like GCC 4.x does
Status: CLOSED FIXED
Alias: None
Product: ccache
Classification: Unclassified
Component: ccache (show other bugs)
Version: 3.1
Hardware: All All
: P5 normal
Target Milestone: 3.1.5
Assignee: Joel Rosdahl
QA Contact: Joel Rosdahl
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-04-12 12:39 UTC by john.smith
Modified: 2011-05-29 15:51 UTC (History)
0 users

See Also:


Attachments
log file after make (2.98 KB, text/plain)
2011-04-13 11:16 UTC, john.smith
no flags Details
compared logs from ccache 3.0.1 and 3.1.4 (9.13 KB, text/plain)
2011-04-13 11:43 UTC, john.smith
no flags Details
verbose log (6.05 KB, text/plain)
2011-04-14 09:43 UTC, john.smith
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description john.smith 2011-04-12 12:39:19 UTC
This seems related to gcc precompiled headers (despite I have not enabled them, only using gcc 4.5.2 defaults).

Since ccache 3.1.4, when I modify a single *.h header file in eclipse CDT (no special compiler options added, only defaults) the make command does not detects the change. Thus, the code is not recompiled, and the library crashes when used from other projects. Adding the "-fpch-preprocess" to the compiler flags as ccache man page recomends, it works properly, but there is no speed gain compared to entirely disabling ccache.

The environment variable CCACHE_SLOPPINESS doesn't makes it work (only -fpch-preprocess seems effective).

However, downgrading to ccache-3.0.1, there are no problems detecting modifications in C header files. And ccache really does greatly speeds up the compilation under gcc 4.5.2. Intermediate ccache versions (3.1.1 to 3.1.3) couldn't be tested, because the compiler returned the following error: "cc1plus: error: to generate dependencies you must specify either -M or -MM".

Notes: I have fully erased the .ccache directory before testing each ccache version.
Comment 1 Joel Rosdahl 2011-04-13 06:34:36 UTC
(In reply to comment #0)
> This seems related to gcc precompiled headers (despite I have not enabled
> them, only using gcc 4.5.2 defaults).
>
> Since ccache 3.1.4, when I modify a single *.h header file in eclipse CDT (no
> special compiler options added, only defaults) the make command does not
> detects the change.

That sounds odd. Unless Eclipse CDT does something I don't know (which
certainly is likely), it's make's job to detect whether a recompilation is
needed; ccache is only used when make has decided that something needs to be
compiled.

Can you enable debug logging (set CCACHE_LOGFILE to a suitable log file path)
and catch a log of what ccache thinks happens when you have edited a header
file and then build?

> Thus, the code is not recompiled, and the library crashes when used from
> other projects. Adding the "-fpch-preprocess" to the compiler flags as ccache
> man page recomends, it works properly, but there is no speed gain compared to
> entirely disabling ccache.

Please create a ccache log for this scenario as well.

> However, downgrading to ccache-3.0.1, there are no problems detecting
> modifications in C header files. And ccache really does greatly speeds up the
> compilation under gcc 4.5.2. Intermediate ccache versions (3.1.1 to 3.1.3)
> couldn't be tested, because the compiler returned the following error:
> "cc1plus: error: to generate dependencies you must specify either -M or -MM".

If you want, you can probably test older versions if you apply this patch:

http://git.samba.org/?p=ccache.git;a=blobdiff_plain;f=ccache.c;h=5e36706afceb2a13258b50b4680bffa22236f134;hp=900146d6dc412a9d880031a5a291ed3e74333c6a;hb=c69f951ecd6a9fe5c087f8836f7c2de6a7159896;hpb=e113d8a816f79398db55b7dbd5df5a9bffdb71ec
Comment 2 john.smith 2011-04-13 11:16:16 UTC
Created attachment 6403 [details]
log file after make
Comment 3 john.smith 2011-04-13 11:18:47 UTC
The ccache log file obviously doesn't records anything when make does nothing. However, I have finally discovered what is causing make to not compile anything!. 

Until ccache 3.0.1, the "main.d" generated after the initial compilation, started with the following lines:

src/main.d src/main.o: ../src/main.cpp \
...
  (full list of project header files)
...

With the latest ccache versions, this first line is slightly different:

src/main.d: ../src/main.cpp \
...
  (full list of project header files)
...

Note the missing object file!. If I manually add it, make correctly takes into account the header files modifications. Something has changed in ccache modifying the previous (correct) behaviour.

I attached above the ccache log file generated just after the make command which generated the wrong main.d
Comment 4 john.smith 2011-04-13 11:43:12 UTC
Created attachment 6404 [details]
compared logs from ccache 3.0.1 and 3.1.4

This attachment has the log output generated by the different ccache versions: 3.0.1 vs 3.1.4. The old ccache version included "-MT src/main.o" in the first two gcc invocations, while the latest ccache omits it. This causes the main.d file to not being generated equally, by just such object file, and header file dependencies are not taken into account. My versions: gcc 4.5.2 and make 3.81.
Comment 5 Joel Rosdahl 2011-04-13 19:17:14 UTC
Thanks for the analysis!
Comment 6 Joel Rosdahl 2011-04-13 20:22:41 UTC
Unfortunately, I can't reproduce the problem. Darn. Can you please apply this patch and capture a new log?

--- a/ccache.c
+++ b/ccache.c
@@ -1793,6 +1793,7 @@ ccache(int argc, char *argv[])
 
        sloppiness = parse_sloppiness(getenv("CCACHE_SLOPPINESS"));
 
+       cc_log_argv("Command line: ", argv);
        cc_log("Hostname: %s", get_hostname());
        cc_log("Working directory: %s", current_working_dir);
Comment 7 john.smith 2011-04-14 09:43:53 UTC
Created attachment 6406 [details]
verbose log

The patch rejected my source code (to version 3.1.4) but I manually applied it, provided that only consisted on the cc_log_argv() call (I'm not a master with diff output). The log has been generated with an empty .ccache directory on a cleaned project.
Comment 8 Joel Rosdahl 2011-04-17 16:29:53 UTC
Thanks! I now know what the problem is.

GCC's man page says that "-MT target" changes the target of the rule emitted by dependency generation. So, if you pass "-MT foo" when you compile bar.c, then the created .d file will contain a "foo: bar.c" rule instead of the "bar.o: bar.c" rule that it would otherwise create. The man page also says that you can use multiple -MT options to specify multiple targets.

However, if you use "-MTtarget" instead of "-MT target", then GCC apparently doesn't change the target but instead just adds the specified target while keeping the default target. That it's possible to use the "-MTtarget" form and that it's different from the "-MT target" form is undocumented. (This is true for at least GCC 4.3, 4.4 and 4.5. GCC 3.4 makes no difference for "-MT target" and "-MTtarget", so I'm not sure if it's a regression in 4.x or if it's intended but undocumented behaviour.)

ccache needs to understand and possibly modify the flags related to dependency generation since ccache runs the preprocessor on the source code, and the semantics of those flags is in some cases different from that of normal compiler invocation. However, ccache 3.1.2 and below don't recognize the "-MTtarget" form, so they act as if no -MT option was specified, which means that the default dependency target value will be used. That's why it happens to work with ccache 3.0.1 in your case. ccache 3.1.3 and above knows about the "-MTtarget" form, but not that "-MT target" and "-MTtarget" have different semantics.
Comment 9 Joel Rosdahl 2011-04-17 16:47:06 UTC
Fixed in 8c3039d676d71564e98d50f1ca8ba029d1efa46c.
Comment 10 john.smith 2011-04-20 17:02:14 UTC
Ccache itself is as reliable as I suspected, despite we need to keep an eye over compiler updates. Joel: many thanks for your great work!
Comment 11 Joel Rosdahl 2011-05-29 15:51:56 UTC
Included in v3.1.5.