Bug 2581 - stat cache grows unlimited
stat cache grows unlimited
Status: CLOSED FIXED
Product: Samba 3.0
Classification: Unclassified
Component: File Services
3.0.11
PPC AIX
: P3 normal
: none
Assigned To: Samba Bugzilla Account
Samba QA Contact
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2005-04-07 03:37 UTC by Heinrich Mislik
Modified: 2005-08-24 10:19 UTC (History)
0 users

See Also:


Attachments
Proposed patch. (2.45 KB, patch)
2005-04-07 13:39 UTC, Jeremy Allison
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Heinrich Mislik 2005-04-07 03:37:54 UTC
config has:
stat cache = yes
case sensitive = no

When a client does open or stat lots of files (virus scan or find on a share
with 1.5e6 files) all the full pathnames go into the statcache until the process
reaches its ulimit (128MB in my case). Then I get lots of:

[2005/04/03 09:39:21, 0, pid=946416] smbd/statcache.c:stat_cache_add(132)
  stat_cache_add: Error storing entry PROFILE/FIEDERM7/SICHERUNG
C/WETTERDATEN/KLIMA_SEIT_1971/KLIMA2000/DATEN/KLIMADATEN ->
profile/fiederm7/Sicherung C/WetterDaten/Klima_Seit_1971/klima2000/daten/klimadaten

The process seems to still work OK but slow, although I wonder how memory
corruption is avoided finding this in tdb/tdb.c:

835         if (tdb->flags & TDB_INTERNAL)
836                 tdb->map_ptr = realloc(tdb->map_ptr, tdb->map_size);
837         else

There is no error checking!

I suggest two things:

1: impose some limit on the size of the stat cache and remove older (or all)
   entries, when the limit is reached.

2: tdb shouldn't use realloc without checking for NULL.

setting "stat cache = no" removes the problem for now.
Comment 1 Jeremy Allison 2005-04-07 13:39:57 UTC
Created attachment 1144 [details]
Proposed patch.
Comment 2 Jeremy Allison 2005-04-07 13:40:42 UTC
I fixed the tdb problem, and here is a rate parameter :

"max stat cache size" (in kb) that will limit growth.

Please test and let me know.

Thanks,

Jeremy.
Comment 3 Heinrich Mislik 2005-04-08 03:37:52 UTC
(In reply to comment #2)

> I fixed the tdb problem, and here is a rate parameter :
> "max stat cache size" (in kb) that will limit growth.

	if (sc_size && (sc_size*1024 > tdb_stat_cache->map_size)) {
		reset_stat_cache();
	}

Shouldn't this be the other way round:

	if (sc_size && (tdb_stat_cache->map_size > sc_size*1024)) {
		reset_stat_cache();
	}

Am I missing something?
Comment 4 Jeremy Allison 2005-04-08 10:05:12 UTC
No, you're completely right, I'm an idiot :-).
Thanks.
Jeremy.
Comment 5 Heinrich Mislik 2005-04-12 06:22:20 UTC
(In reply to comment #2)
> I fixed the tdb problem, and here is a rate parameter :
> 
> "max stat cache size" (in kb) that will limit growth.
> 
> Please test and let me know.

I tested the corrected patch. The new parameter works as expected.

Thanks
Comment 6 Jeremy Allison 2005-04-12 12:35:19 UTC
New parameter fixes the bug.
Jeremy
Comment 7 Gerald (Jerry) Carter 2005-08-24 10:19:51 UTC
sorry for the same, cleaning up the database to prevent unecessary reopens of bugs.