Bug 8051 - smbtad uses lots of CPU when idle
smbtad uses lots of CPU when idle
Status: RESOLVED FIXED
Product: smbta
Classification: Unclassified
Component: smbtad
unspecified
All All
: P5 normal
: ---
Assigned To: Holger Hetterich
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2011-03-30 17:09 UTC by Robert Piasek
Modified: 2011-04-20 08:40 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Piasek 2011-03-30 17:09:05 UTC
I noticed that smbtad uses a lot (subjective opinion) of cpu when idle. In my case samba vfs module wasn't even loaded.

Quick strace provided:

restart_syscall(<... resuming interrupted call ...>) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_IGN, [CHLD], SA_RESTORER|SA_RESTART, 0x7f95a02181f0}, 8) = 0
nanosleep({5, 0}, 0x7f95a01e4b10)       = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_IGN, [CHLD], SA_RESTORER|SA_RESTART, 0x7f95a02181f0}, 8) = 0
nanosleep({5, 0}, 0x7f95a01e4b10)       = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_IGN, [CHLD], SA_RESTORER|SA_RESTART, 0x7f95a02181f0}, 8) = 0
nanosleep({5, 0}, 0x7f95a01e4b10)       = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_IGN, [CHLD], SA_RESTORER|SA_RESTART, 0x7f95a02181f0}, 8) = 0
nanosleep({5, 0}, 0x7f95a01e4b10)       = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_IGN, [CHLD], SA_RESTORER|SA_RESTART, 0x7f95a02181f0}, 8) = 0
nanosleep({5, 0}, ^C <unfinished ...>
Process 2401 detached

quick grep and we've got:

grep nanosleep *  -R
src/sqlite/sqlite3.c:  nanosleep(&sp, NULL);

grep 5000 *  -R
src/sqlite/sqlite3.c:# define SQLITE_MAX_VDBE_OP 25000
src/sqlite/sqlite3.c:# define SQLITE_MAX_LIKE_PATTERN_LENGTH 50000
src/sqlite/sqlite3.c:        usleep(500000); /* wait 0.5 sec and try the lock again*/
src/sqlite/sqlite3.c:  pInfo->estimatedCost = 500000;
src/sqlite/sqlite3.c:# define SQLITE_MAX_LIKE_PATTERN_LENGTH 50000
src/cache.c:            usleep(5000);

should we pool it every 5 sec or can we use better way to do it - like update only when data is present?
Comment 1 Holger Hetterich 2011-03-30 22:42:12 UTC
sqlite functions don't have an influence on the CPU time in an idle smbtad, as they are only called when it comes to actually work on data.

The main CPU time an idle smbtad creates is by running a thread while the main select call:

/* 
 * sqlite >= 3.7.0 allows WAL. We will use this feature.
 * Run as a thread and run any query that is waiting.
 */
void cache_query_thread(struct configuration_data *config)
{
        pthread_detach(pthread_self());
        sqlite3 *database = config->dbhandle;
        /* run a query and add the result to the sendlist */
        while (1 == 1) {
                usleep(5000);
                int res_len;
                int res_socket;
                int monitorid = 0;
                char *res = query_list_run_query(database,
                        &res_len, &res_socket, &monitorid);
                if (res != NULL && monitorid ==0) network_send_data(res,res_socket,res_len);
                if (res != NULL && monitorid !=0) monitor_list_set_init_result(res, monitorid);
        }
}

It is checking if new queries have to be run (for example from smbtaquery or smbtamonitor (<- on initialization)), as we have to do this at the same time new data will be received by the vfs module, it is a thread. We can do this because newer versions of sqlite3 support WAL. query_list_run_query returns NULL immediately if no queries are in the queue.

Moving to a different (networked) database can remove this thread completely, as in this case smbtaquery and smbtamonitor can employ the database directly, not depending on smbtad to do this, and make the idle state CPU usage of smbtad lower.

I would prefer creating a general database layer that allows to use a database like mysql to access the data. Then, smbtad can concentrate on writing, and doesn't need to fulfill queries for clients.
Comment 2 Holger Hetterich 2011-04-19 09:46:56 UTC
With DBI in place, is the issue described here still occuring? Can we resolve this bug?
Comment 3 Robert Piasek 2011-04-20 08:40:15 UTC
Yep, with current devel branch it's all gone.