[kforge-dev] domainmodel API design suggestions

[John Bywater]

Jo Walsh wrote:

>dear Rufus, John, thankyou both for indulging me in pair- and triad-
>programming sessions last week and enduring my barrage of stupid
>questions and answering many of them.
>  
>

Jo, your questions weren't at all stupid. It's fantastic to have you 
involved. But there were a number of things that happened during this 
iteration which were quite stupid, things I either don't want to 
continue or don't want to continue with. Things which I found 
"challenging" during this iteration include:

1. Starting the iteration unable to plan, due to broken development 
systems: due to our policy of "eating our own dog food" combined with 
inadequate acceptance testing by the Product Owner of the product before 
upgrading the production service on which we depend. "Eating your own 
dog food" in a way that means that when the product is broken the 
process is also broken is an element of instability which must be and 
can only be stablised by rigourous acceptance testing of the product 
before it is considered ready for, and put into, production. Product 
Owners who don't have time to do rigourous acceptance testing don't have 
time to double up as software developers on the same project. Writing 
acceptance tests is the best way to take the work out of this repetitive 
task. But nevertheless, it is quite stupid to have things "poisoned" in 
this way.

2. Degrading the quality of the system for no good reason: it is 
regressive to worsen the names of the domain model instead of installing 
and using a stable Postgres version. This sort of thing is a double 
failure, because not only was an opportunity missed to progess the 
system, other time will be consumed fixing up things afterwards.

3. Making undisciplined changes to the code: it isn't acceptable to make 
code changes without runing the test suite before, and most importantly 
after changes have been made. Not being aware that the test suite has 
been broken simply because it wasn't run, but committing changes 
regardless is a terrible thing to impose on an agile software develop 
project.

In general, I have no desire to work on an agile software development 
project that doesn't plan at the start of an iteration, doesn't care 
much for good namings, can't be bothered to run a regression test suite 
before committing changes.

>I was just looking over my notes on the domainmodel interface and
>wondering about possible API reworking. I find the current API more
>*seems* overcomplex because it talks the language of the pattern
>process so literally. Well illustration is always best; the following
>is what my notes say now about how to query for instances of an object
>in the domain model:
>
>    from ckan.soleInstance import application
>    register = application.registry.getDomainClassRegister()
>    domainClasses = register.keys() 
>    domainClass = domainClasses['PackageName']
>    objectRegister = domainClass.createRegister()
>    objectRegister.findDomainObjects(filter)
>
>^^^^- note my four-space tabs, Rufus :P 
>
>I would like to be able to address domainmodel more idiomatically thus:
>
>    ...
>    classes = application.classes()
>    package = classes['Package']
>    package.list(filter) or package.search(filter)
>  
>

Which ideom is this? Do you have a name for it? Can you explain in which 
context it works, which forces it resolves, what are the limits of its 
applicability, why it pertains to the domainmodel package, and what the 
other choices are available?

Does it make sense to have the type of an instance be also responsible 
for collecting the set of instances of that type? I think it can make 
sense, but doesn't in our case:

We have a need to support two distinct responsibilities, and have 
written two distinct system objects for this: the DomainObject class, 
and the DomainObjectRegister class. Having one fall back onto the other 
means that a DomainObject instance can't instantiate a collection of 
aggregate domain objects without somehow producing an objective 
confusion (which Spinoza defines as mixing two distinct concepts, such 
as 'triangle' and 'square'; in our case 'type' and 'collection').

Of course, a very simple model, which only has "top level" collections 
with no notion of aggregates wouldn't suffer from such a confusion, 
because there would be no aggregates (subsets of the set of all 
instances of a type) to support. The requirements of the KnowledgeForge 
service (with things such as members of a project) led to the 
distinction I describe.

Of course, we could make the type (the DomainClass object) support 
delegation of collection-shaped responsibilites to the registry's 
register, which is a collection of all instances of a particular type.

Summary: the domainmodel code makes a distinction between type and 
collection, so that instances can hold subsets of instances without need 
for further conceptual complexity. But we didn't implement delegation by 
types to the collections of all instances of that type. But we can do that.

>And have the stuff below abstracted away a bit more. It is a beautiful
>if baroque design underneath, that works really well, but as an
>application developer i don't think i need to be so exposed to the
>inner workings and to the raw pattern language as the current interface
>necessitates.
>
>I wish i had the focus to accompany this with more constructive
>illustrations but it probably wants to be talked over a bit more
>before code written anyway, the latter is relatively unimportant?
>  
>

I would encourage you, at least at first, to understand the software as 
the product of adding tests and tested features to adequate to user 
stories selected by the Product Owner, closely followed by refactoring 
to improve the design of existing code. At no time have we tried to make 
it "correct" in some sense.

I wouldn't necessarily say that there was an API at the moment. There is 
a collection classes, each having responsibilities and collaborations, 
and therefore holding several attributes and methods. And so there are a 
number of de facto interfaces, but no API as such has been designed or 
documented. There are many voids in dm, kforge, ckan, all of which call 
out for further development work. But as neither KForge nor CKAN is a 
liquid-containing vessel, that doesn't consitute an inadequacy, but 
quite the opposite: the possible ways forward are more or less obvious.

That having been said, the notes you made on "how to query for instances 
of an object in the domain model" were based on my answering your 
question of how to access what you referred to as "meta" data. That's 
why I was steering you to the register of domain classes, each of which 
hold a meta object that describes a type of domain object.

As you really want the model data as such, rather than the model meta 
data, I would suggest taking a slightly different approach: use the 
domain object registers that have been instantiated on the registry 
directly (e.g. registry.projects).

But the real way to go is to write down what you really want as a test 
case. Then we can satisfy your test case. Then we can refactor to make a 
better interface, probably first making a FACADE (not shouting: patterns 
are often written in capitals). Creating a "better interface" first 
isn't the way to do agile development, and therefore isn't the way to 
work with the established grain of this project.

I have do desire to discuss improvements to software systems without 
reference to what that improvement is to adequate to, expressed normally 
as a unit test, because the process doesn't complete, and is therefore 
bound to be exhausting. I would like this to apply the idea of 
adequation to discussion of the qualities of a software system: you say 
it's baroque (ie overly complex?) but you don't say what you measure and 
judge its complexity against.

Certainly, I find your API simplifications to be oversimplifications 
because: for example an "application's classes" 
("application.classes()") aren't simply the registered domain object 
classes, but a great deal more. I would be happier with something like 
"application.domainClasses()").

Further, I don't think that there is a "pattern process". There are 
patterns of process, and of design. But the process of development 
itself isn't a pattern, and patterns don't consitute a process. So I 
wish to make a bref remark about the reason for having patterns in 
software systems and development processes:

If (as Ward Cunningham proposes) a program is a set of rules for solving 
a problem in the future, and programming is a discussion about such 
rules, then the first point of discussion may be that the near future 
will be more similar to the near past than the distant future will be to 
the distant past. Therefore it would make sense to shortcut understands 
in the future with understandings from the past, and carry forward 
things that work in the past (in a way that works) into the future, such 
that we can give most of our attention to what is *different* about the 
future ("embrace change").

To begin, if you can *see* the patterns, then I'm happy. Just to 
clarify, the "raw pattern language" is in the black and red Martin 
Fowler book, that you said you haven't read so far.

Perhaps you could find time to expose yourself to the real "raw pattern 
language" of enterprise applications before we next meet? Then you will 
be able to understand the application of enterprise applications that 
has been made on this project over the last 18+ months.

>I am probably working fulltime for the next couple of weeks (!) but
>would love to come back to Cambridge in 3 or 4 weeks time and have a
>more proper sprinting session then, and patch the docs up.
>  
>

Well done for getting the fulltime work! Good luck with it.

I look forward to continuing the f2f discussion again soon. I'm far from 
convinced that email has the bandwidth for a discussion of software 
design that hasn't been dramatically compressed by the application of 
design patterns.

With all best wishes, and lots of love!

John.

>love,
>
>
>jo
>  
>