Bug 11519 - ccache fails to create directories on QNX
ccache fails to create directories on QNX
Status: CLOSED FIXED
Product: ccache
Classification: Unclassified
Component: ccache
3.2.3
All Other
: P5 major
: 3.2.5
Assigned To: Joel Rosdahl
Joel Rosdahl
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-14 10:14 UTC by Igor Rondarev
Modified: 2016-04-17 14:49 UTC (History)
0 users

See Also:


Attachments
ccache log file (23.60 KB, text/x-log)
2015-09-14 10:14 UTC, Igor Rondarev
no flags Details
test .c file (48 bytes, text/x-csrc)
2015-09-14 10:14 UTC, Igor Rondarev
no flags Details
CORRECT ccache log file (6.10 KB, text/x-log)
2015-09-14 10:17 UTC, Igor Rondarev
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Igor Rondarev 2015-09-14 10:14:36 UTC
Created attachment 11436 [details]
ccache log file

>>> System description:
ccache version: 3.2.3 (built from sources with no errors: 
# configure --prefix=/opt/ccache; make; make install)

OS: QNX Neutrino 6.5.0 (x86)

>>> Problem description: 

ccache fails with error message:

---------- begin ------------
ccache: error: Failed to create file /root/.ccache/tmp: No such file or directory
---------- end ------------

>>> Steps to reproduce:

1) Create test.c file (attached)

2) Create compiler symlink in /opt/ccache/tools (i486-pc-nto-qnx6.5.0-gcc -> /opt/ccache/bin/ccache)

3) Run ccache:
# CCACHE_LOGFILE=/tmp/ccache.log CCACHE_VERBOSE=true PATH=/opt/ccache/tools/:$PATH  i486-pc-nto-qnx6.5.0-gcc -c test.c

ccache: error: Failed to create file /root/.ccache/tmp: No such file or directory

4) mkdir /root/.ccache/tmp (trying as a workaround)

5) Run ccache:
# CCACHE_LOGFILE=/tmp/ccache.log CCACHE_VERBOSE=true PATH=/opt/ccache/tools/:$PATH  i486-pc-nto-qnx6.5.0-gcc -c test.c

ccache: error: Failed to create file /root/.ccache/c/6: No such file or directory

Log file attached (ccache.log)
Comment 1 Igor Rondarev 2015-09-14 10:14:58 UTC
Created attachment 11437 [details]
test .c file
Comment 2 Igor Rondarev 2015-09-14 10:17:47 UTC
Created attachment 11438 [details]
CORRECT ccache log file

proper ccache log file
Comment 3 Joel Rosdahl 2015-09-14 18:14:37 UTC
That's strange. It's most likely related to unexpected behavior of syscalls/libc on QNX.

Would it be possible for you to run a tool similar to strace (syscall tracer) to see which syscalls ccache makes and what the OS gives back as results?

If you're interested in debugging ccache, it seems that the create_parent_dirs function in util.c for some reason doesn't work on QNX.

Some by-the-way comments:

1. CCACHE_VERBOSE has no effect for ccache 3.*.
2. Setting PATH=/opt/ccache/tools/:$PATH on the same commandline as the compiler has no effect (unless QNX's /bin/sh is a very strange shell).
Comment 4 Igor Rondarev 2015-09-15 13:42:43 UTC
I've added some debug in create_parent_dirs() function, but today this error has gone for some reason, and 3.2.3 works well. I'll keep to watch on it.

As for PATH before compiler, that's common trick both for QNX and Linux. I usually do this way instead of export'ing variables in case they are needed just for one time/few times. 
# PATH=/opt/ccache/tools/:$PATH i486-pc-nto-qnx6.5.0-gcc - will run ccache thru symlink
# i486-pc-nto-qnx6.5.0-gcc - will run native compiler (/usr/qnx650/host/...)
Comment 5 Joel Rosdahl 2015-09-15 20:16:16 UTC
> As for PATH before compiler, that's common trick both for QNX and Linux.

Ah, my bad.

> I've added some debug in create_parent_dirs() function, but today this
> error has gone for some reason, and 3.2.3 works well. I'll keep to watch
> on it.

Great, that would be appreciated. I'm not able to test it on QNX myself.
Comment 6 Joel Rosdahl 2015-10-08 18:10:04 UTC
Any success in debugging this issue?
Comment 7 Igor Rondarev 2015-10-08 18:26:37 UTC
(In reply to Joel Rosdahl from comment #6)
Unfortunately, the problem has not taken place since first time. In fact, i don't use ccache under QNX very often (generally preferring cross-compiling, so using it under Linux).
Comment 8 Igor Rondarev 2015-11-13 12:39:27 UTC
Finally, found the cause for this. It't due to QNX's mkstemp(char *template) function's behavior (unit.c:1181): it may change contents of template char array if it fails with ENOENT trying to create temporary file in non-existent directory (what it actually does - it leaves only path to that non-existent directory in 'template' array, cutting the rest). Linux and, probably, some other OSes' mkstemp() leave template unchanged even in case of failure. Seems it is not covered by any standards like POSIX or its XSI extensions, so we cannot rely on template's contents after mkstemp() failure. Probably it would be better to use a copy of template, someting like:

util.c:1181
--- cut ---
    char *tempdup = strdup(template);
    int fd = mkstemp(tempdup);
    free(tempdup);
    if (fd == -1 && errno == ENOENT) {
--- cut ---
Comment 9 Igor Rondarev 2015-11-13 14:50:32 UTC
Or probably better like this (to avoid unnecessary alloc/free in case of success):

---cut---
int
create_tmp_fd(char **fname)
{
        char *tmpstr = tmp_string();
        char *template = format("%s.%s", *fname, tmpstr);
        int fd = mkstemp(template);
        if (fd == -1 && errno == ENOENT) {
                reformat(&template, "%s.%s", *fname, tmpstr);
                if (create_parent_dirs(template) != 0) {
                        fatal("Failed to create directory %s: %s",
                              dirname(template), strerror(errno));
                }
                fd = mkstemp(template);
        }
        if (fd == -1) {
                reformat(&template, "%s.%s", *fname, tmpstr);
                fatal("Failed to create file %s: %s", template, strerror(errno));
        }
--- cut ---
Comment 10 Joel Rosdahl 2015-11-13 20:36:03 UTC
Thanks for the investigation!

What do you think about this fix?

--- a/util.c
+++ b/util.c
@@ -1180,15 +1180,16 @@ create_tmp_fd(char **fname)
        char *template = format("%s.%s", *fname, tmp_string());
        int fd = mkstemp(template);
        if (fd == -1 && errno == ENOENT) {
-               if (create_parent_dirs(template) != 0) {
+               if (create_parent_dirs(*fname) != 0) {
                        fatal("Failed to create directory %s: %s",
-                             dirname(template), strerror(errno));
+                             dirname(*fname), strerror(errno));
                }
                reformat(&template, "%s.%s", *fname, tmp_string());
                fd = mkstemp(template);
        }
        if (fd == -1) {
-               fatal("Failed to create file %s: %s", template, strerror(errno));
+               fatal("Failed to create temporary file for %s: %s",
+                     *fname, strerror(errno));
        }
 
 #ifndef _WIN32
Comment 11 Joel Rosdahl 2015-12-06 19:30:25 UTC
Igor Rondarev: Have you had time to try out my proposal?
Comment 12 Igor Rondarev 2015-12-07 08:12:01 UTC
Sorry, i've been extremely busy. I guess it should work perfectly, but this part of code

        if (fd == -1) {
-               fatal("Failed to create file %s: %s", template, strerror(errno));
+               fatal("Failed to create temporary file for %s: %s",
+                     *fname, strerror(errno))

will print directory name in case of failure, not a file name. So probably it make sense to patch only this part:
--- a/util.c
+++ b/util.c
@@ -1180,15 +1180,16 @@ create_tmp_fd(char **fname)
        char *template = format("%s.%s", *fname, tmp_string());
        int fd = mkstemp(template);
        if (fd == -1 && errno == ENOENT) {
-               if (create_parent_dirs(template) != 0) {
+               if (create_parent_dirs(*fname) != 0) {
                        fatal("Failed to create directory %s: %s",
-                             dirname(template), strerror(errno));
+                             dirname(*fname), strerror(errno));
                }
                reformat(&template, "%s.%s", *fname, tmp_string());
                fd = mkstemp(template);
Comment 13 Joel Rosdahl 2015-12-07 19:58:39 UTC
> I guess it should work perfectly, but this part of code [...] will print
> directory name in case of failure, not a file name. So probably it make
> sense to patch only this part: [...]

Nope, it will print a filename (*fname). I did that intentionally after I made the observation that the content of the template is undefined for other errnos as well (for instance EEXIST) after reading my system's man page.

Fixed in 07a529dfeb452cf6bb77b5a317539eeda54b40fd.
Comment 14 Joel Rosdahl 2016-04-17 14:49:12 UTC
Included in 3.2.5.