[kforge-dev] access control role authorisation sequence

[John Bywater]

Rufus Pollock wrote:

> Great work John. Any idea on how much speedup? Maybe we could use the 
> profiler ...


Thanks. I described a rough "big O" analysis in the other email. 
Basically, now its O(1) with respect to the time taken by the underlying 
database system to do a SELECT against a table of N records. Before it 
was something like O(N) with respect to the same thing. This means it 
works at about the same speed if there is only one record, but 100 times 
faster for 100 records.

The test suite on my laptop takes:

dm: Ran 126 tests in 8.687s

kforge: Ran 151 tests in 95.396s

So, in short it's a lot, lot quicker. (Particularly when one loads up 
the database with lots of objects... :-))

One interesting fact is that the tests run about 50% slower when the log 
level is DEBUG rather than INFO.

> Great work.


Thanks! I am happy to have worked this out. The follow through would be 
to bring the permission granting code into the access controller. The 
cluster of concepts pertaining protectedObject, protectedName, 
protectionObject needs clarifying.

> Given the number of protection objects this should result in  some 
> very dramatic speed improvements. As I said above might be useful (if 
> we have the time) to actually profile this bit of code and see if we 
> can get some before and after data as well as more info on where the 
> performance bottlenecks are.

That's what I did Fri-Sat....

Just run the kforge- or dm-profiletest script, piping STDOUT to a file.

$ ./bin/kforge-profiletest > profile.log

Then look for methods that take lots of time:

$ less profile.log

You won't find any at the moment because I went through it this weekend 
and eliminated them all. (Ha!)

>
> After writing this you seem to have fixed most of the failing tests :) 
> However from the latest run of the automated builder I see there are 
> still a few issues. I'll look into this and see if I can fix any 
> remaining problems.


Yes, I'm not sure about my database. I can't remake it because all my 
admin scripts are broken, I think, because of recent changes you made? 
For example:

$ ./bin/kforge-admin db rebuild
Password:
Traceback (most recent call last):
  File "./bin/kforge-admin", line 5, in ?
    kforge.cli.admin.UtilityRunner()
  File "/home/john/kforged/lib/python/dm/cli/admin.py", line 276, in 
__init__
    status = self.runUtilityOnce(self.line)
  File "/home/john/kforged/lib/python/dm/cli/admin.py", line 343, in 
runUtilityOnce
    status = utility.onecmd(line)
  File "/usr/lib/python2.3/cmd.py", line 210, in onecmd
    return func(arg)
  File "/home/john/kforged/lib/python/dm/cli/admin.py", line 95, in do_db
    self.takeDatabaseAction(actionName)
  File "/home/john/kforged/lib/python/kforge/cli/admin.py", line 24, in 
takeDatabaseAction
    actionMethod()
  File "/home/john/kforged/lib/python/dm/util/db.py", line 69, in rebuild
    expectedError = 'database "%s" does not exist' % self.dbname
AttributeError: 'Database' object has no attribute 'dbname'


Could we fix this asap? (Can we idenitify a test for this error?)

Perhaps I'll call you tomorrow.

Regards,

John.