[SLL] limit on number of files in a directory and hashed dir vs. flat dir file access time

[Ana]

On Sat, Nov 03, 2007 at 05:19:04PM -0700, Andrew Sweger wrote:
> On Sat, 3 Nov 2007, Ana wrote:
> 
> > I'm attempting a new test, using a simple C program.
> > 
> > It looks like there is a small increase with every 10x increase of the
> > file count, until the 1,000,000 file count attempt.  There's a 3x
> > increase in system time spent between 100,000 and 1,000,000 file-count
> > tests.  At least that's how I'm reading the result.  I'm open to other
> > interpretations and to all suggestions about how to improve the test.
> > If you want more information (about my system or whatnot) just ask.
> 
> I think you mean a more than 30-fold increase on the last test.  Any
> chance your program would block on disk-io from another process?  Running
> in single-user mode?

30-fold?  oh yeah.  silly me.  ;)

I didn't use my computer while the test was running, and I don't run any
servers to speak of...  but maybe I should shutdown those that are
running.

The daily cron jobs start at 6:25am.  All the tests up until the million
file test took about 10 minutes.  According to the output, the million
file test started at 6:07.  If the pattern were to hold, it should have
finished by around 6:17, and probably at least before sometime near
6:20.  If it didn't finish before the daily cron jobs started (which is
obviously the case), then updatedb could have slowed things down quite a
bit.  The last test took almost 7 hours though.  Do you think updatedb
could gum things up for that long?

I'll re-run the million file test a couple times and see if things
change.  I may also run 2, 4, 6, and 800,000 file tests to try to spot
some kind of curve.  I should also run the entire test a few times
anyway to find out if the numbers are consistent.

- Ana