Bug 9270 - While meet the max cache size, some task kill the folder and some task put the file on the deleted folder
While meet the max cache size, some task kill the folder and some task put th...
Status: CLOSED FIXED
Product: ccache
Classification: Unclassified
Component: ccache
3.1.8
All All
: P5 normal
: 3.1.9
Assigned To: Joel Rosdahl
Joel Rosdahl
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-09 09:46 UTC by brianpc.huang
Modified: 2013-01-06 17:04 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description brianpc.huang 2012-10-09 09:46:03 UTC
Due to slow build time of android, we turn on ccache and make parallel build (make -j 24) to speed up the 2nd build. However, ccache storage would occupy almost 5G data (No compress). If the configuration of ccache size is less than 5G, then some task would clear the folder but other task would like to generate the dep/obj files on ccache, it produce the FATAL Error message as following:
ccache: FATAL: Could not create /proj/user/.ccache/d/9/0d33b843b730a552f8ea38387a3a02-1515162.o.tmp.stdout.my_server.3448 (permission denied?)

Our work around solution is to add retry mechanism while error happened. Do it have smart solution ?

diff -du ccache-3.1.8/ccache.c original_code/ccache-3.1.8/ccache.c
--- ccache-3.1.8/ccache.c       2012-10-09 17:11:47.280689370 +0800
+++ original_code/ccache-3.1.8/ccache.c 2012-08-11 17:03:17.000000000 +0800
@@ -519,11 +519,9 @@
 static void
 to_cache(struct args *args)
 {
-       unsigned to_sleep = 1000; /* Microseconds. */
        char *tmp_stdout, *tmp_stderr, *tmp_obj;
        struct stat st;
        int status;
-       int retry_times;
        size_t added_bytes = 0;
        unsigned added_files = 0;

@@ -550,30 +548,11 @@

        cc_log("Running real compiler");
        status = execute(args->argv, tmp_stdout, tmp_stderr);
-       if (stat(tmp_stdout, &st) != 0) {
-               retry_times++;
-               cc_log("FATAL_ERROR %d:build argv = %s, tmp_stdout=%s, tmp_stderr=%s,error_string=%s",retry_times,*args->argv, tmp_stdout, tmp_stderr,strerror(errno));
-               /* Retry after sleep to_sleep, we guest it is because of the ccache size over the config and recyline issues*/
-               usleep(to_sleep);
-               status = execute(args->argv, tmp_stdout, tmp_stderr);
-               if (stat(tmp_stdout, &st) != 0) {
-                       retry_times++;
-                       cc_log("FATAL_ERROR %d:build argv = %s, tmp_stdout=%s, tmp_stderr=%s,error_string=%s",retry_times,*args->argv, tmp_stdout, tmp_stderr,strerror(errno));
-                       usleep(to_sleep);
-                       status = execute(args->argv, tmp_stdout, tmp_stderr);
-                       if (stat(tmp_stdout, &st) != 0) {
-                               retry_times++;
-                               cc_log("FATAL_ERROR %d:build argv = %s, tmp_stdout=%s, tmp_stderr=%s,error_string=%s",retry_times,*args->argv, tmp_stdout, tmp_stderr,strerror(errno));
-                               usleep(to_sleep);
-                               status = execute(args->argv, tmp_stdout, tmp_stderr);
-                               if (stat(tmp_stdout, &st) != 0) {
-                                       fatal("Could not create %s (permission denied?)", tmp_stdout);
-                               }
-                       }
-               }
-       }
        args_pop(args, 3);

+       if (stat(tmp_stdout, &st) != 0) {
+               fatal("Could not create %s (permission denied?)", tmp_stdout);
+       }
        if (st.st_size != 0) {
                cc_log("Compiler produced stdout");
                stats_update(STATS_STDOUT);
Comment 1 Joel Rosdahl 2012-11-04 13:50:14 UTC
Could you try if this patch fixes the problem?

--- a/cleanup.c
+++ b/cleanup.c
@@ -74,8 +74,8 @@ traverse_fn(const char *fname, struct stat *st)
                /* delete any tmp files older than 1 hour */
                if (st->st_mtime + 3600 < time(NULL)) {
                        x_unlink(fname);
-                       goto out;
                }
+               goto out;
        }
 
        if (num_files == allocated) {
Comment 2 Joel Rosdahl 2013-01-06 16:45:07 UTC
Fixed in 83943e2522f8ea261c616adcc901620af80abb06 and 17791e1e460aec9c0efc1edbd031e975b81c1855 on the maint branch.
Comment 3 Joel Rosdahl 2013-01-06 17:04:16 UTC
Fix included in v3.1.9.