Rufus Pollock wrote:
> I'll try to come back and discuss the rest of your email later but for
> the time being I'll just focus on this part. While I'm sorry that
> writing this email has taken you so long I don't think your time has
> been wasted. You tend to write very detailed responses -- and that is
> appreciated. (If it is any comfort my email also took a while to
> compose though probably not as long as yours. I think we should also
> remember that writing things down, though more time-consuming than
> direct f2f communication, is often more valuable).
In general, face-to-face communication provides higher bandwidth and
more reliable and productive communication. In general, remote
communication, and writing everything down, kills time and kills
projects. If restricting bandwidth by writing everything down helps,
then that must be because we can't handle a higher bandwidth. That's
fine for now, but it does suggest that we have got stuck on the "way to
language", that there is a working poverty of conception, that we have
not yet begun to think.
But I'm happy to write emails. :-)
However I will feel that my time hasn't been well spent if we don't get
our discussion running along the OO, patterned, agile discourse. But let
me expand....
> But most importantly I feel that much of what you've said isn't just a
> repetition of what you have said before or 'obvious'.
It might not be obvious, but, honestly :-) it really is repetition
(albeit with differences)....
> For example you've made clear that you don't think trying to use an
> existing 'framework' such turbogears more heavily is a good idea.
Turbogears isn't a framework, it's a toolkit.
Regarding the pros and cons of using frameworks, I posted this almost
exactly one year ago to kforge-dev, regarding whether KForge is a web
application or an enterprise application, how pattern language can play
a role in our development, and so on:
http://lists.okfn.org/pipermail/kforge-dev/2005-October/000074.html
That message also took me a while to write. I think I explains the
concerns rather very well. :-)
I find it sad that we didn't grok this yet....
> You've also made clear that us having that you think we should
> definitely have our own domain model layer (as opposed to using
> sqlobject in a dual role as ORM and domain model)[^1].
Yes, but this is something I have never stopped saying. It's more than
that. The very first thing I did on this project was to create a domain
model using sqlobject (feel the name...) to convert domain objects to
sql database, and back.
I am repeating myself. ;-)
"Dual roles" is probably a code smell. If we had a single class acting
as object-relational mapper and as domain model, we would generate at
least these four bad code smells [Beck]: Large Class, Inappropriate
Intimacy, and Refused Bequest, all leading to Comments. We could
continue this process by trying to collapse the presentation on the
domain-object-mapper large class. Then the logical next step would be to
factor out the application layer altogether by writing everything in
stored procedures. Either that, or writing database transactions in
javascript, and sending them over the HTTP request. But that would be
nonsense. The mistake would be not having everything doing one thing
well. Do you understand the mistake?
To be absolutely clear: I require that we have our own domain model
layer. This may help:
http://domaindrivendesign.org/books/foreword.html (by Fowler, who
follows Evans)
http://domaindrivendesign.org/articles/blog/evans_eric_ddd_and_mdd.html
>
> There are also, of course, some things I'm not sure we do agree about
> totally. For example, that the history of recent software development
> has shown that code reuse has failed (after all we make extensive use
> of various libraries including the python standard library) but I
> don't think that is always a cause for exasperation -- it is possible
> for reasonable men (and women) to disagree without one of them being a
> fool or a charlatan! Often it is from constructive discussion of
> disagreements -- and their resolution -- that one learns most.
Firstly, Nobody is saying anybody is a fool, or a charlatan. Likewise,
nobody is claiming to be a know-all, or in control. Software development
is hard. Making mistakes with software development doesn't mean
anything. But I am saying there are a few inadequacies of conception on
this project. I am trying to be objective.
There's a choice: one either follows others' thinkings that seem to work
and experiences both less mistakes and no pride to be offended, or "one
thinks for oneself" and suffers both from making lots of mistaken
propositions and from painfully identifying oneself with them. But that
opposition is false: really there is just the extent to which "one's
thinking for oneself" rejects one's own fancy-little-ideas and joins in
with a more general intellectual flow; or the extent to which one
rejects more common thoughts and instead involves oneself in one's own
ideation.
I think and say "let's follow Fowler" not to be conformist, but rather
to benefit from the largest flow of objective, affective consideration I
have discovered so far within software development. You propose to
collapse one or two of his distinctions, and without further explanation
(although you do give a code example to repeat your proposition, you
don't discuss the advantages and the disadvantages of the proposition)!
But it's strange that you want to reuse as much code as possible, but
you don't seem nearly as interested in reusing as much conception as
possible. Given Fred Brooks' comment about "concepts take up the time" I
think this is an objective mistake: it would be more advantageous to
invest in a higher leverage position, to remove the most constraining
things first: the time required to achieve adequate concepts.
I don't suppose I'm much interested even in discussing whether or not to
accept what Fowler says. But I would urge you not to question Fowler,
but instead to find something more productive to question (such as the
coverage of unit tests, the well-factoredness of the code, acceptance
tests, more logical database migration support, etc. -- our real and
pressing problems).
Secondly, everybody knows that software reuse fails for many reasons.
This isn't to say that there is never any reuse. There is a lot of
reuse! From a developer copying little functions from one project to
another, perhaps through third-party blackbox components, and towards
large open source projects that are object-oriented, and well patterned.
But reuse fails. One failure mode happens when the authors of a piece of
code don't understand OO design (which was designed to enable reuse!
...amongst other things), or the patterns in their domain. There are
lots of way it fails (both inadequate technical skills and inadequate
organisational skills can lead to reuse failures). The failure modes of
software reuse are fairly numerous, and fairly well known. The IEEE has
been writing about them for years (maybe 30?).
Regarding Python: we are users of Python, not reusers. We don't reuse
Python, but quite simply use it. Same with the Linux kernel. We don't
reuse the Linux kernel, but quite simply use it. Django users
(particularly at the start) were reusers of code that was written "for a
newsroom in a year and a half". Because Django wasn't internally
object-oriented, despite being object based, lots of things couldn't be
reused easily, and I was forced to copy and paste one or two things,
like the form wrapper class... But that's reuse too. Django internals
were very disappointing. Trac was the same.
I do hope this will change in time, that the adequacy of design
conception in open source software development rises, but if it takes
smart people like you so long to accept, then I really wonder where the
problem lies. Do you have any ideas on this matter?
>
> Thank you once again for taking the time to reply.
Thank you for writing initially. I do realise it does also take you time
to write, and to think about. But that is even more reason why I wish
you were thinking about other things, because there are lots of better
things to do, and there is still lots of progress to be made. But
thankyou for having the patience to read my reply.
>
> Regards,
>
> Rufus
>
> [^1] For example using sqlobject.inheritance we could do things such as:
>
> from sqlobject import *
> from sqlobject.inheritance import InheritableSQLObject
>
> sqlhub.processConnection = connectionForURI('sqlite:/:memory:')
>
> class State(SQLObject):
> name = StringCol()
>
> class StatefulObject(InheritableSQLObject):
> state = ForeignKey('State')
>
> class Project(StatefulObject):
> name = StringCol()
> services = MultipleJoin('Service')
>
> class Service(StatefulObject):
> name = StringCol()
Quite! Its all about columns and keys. We don't want a domain model
written in terms of columns, keys, and joins, but attributes, attribute
associations of different kinds, aggregations, compositions, different
kinds of times, particular types of strings, custom numbers types, other
sorts of attributes such as objectified associations, and more.
I repeat: the purpose of the domain model is to support and develop an
abstracted conception of the problem domain. If you don't understand
this, and want to reject it, and to cloud the domain model concern with
relational database semantics, then I need us to separate our roles, so
that you write user stories, and I write software. I don't really want
to follow your software architectural inclinations at the moment,
because I don't think you understand software development as well as I
do at the moment, and I don't feel that you are really following me very
closely, or paying too much attention to my suggestions. If you want to
develop software with me, then please let me set the technical agenda,
please give me confidence that you are following what I'm saying, please
let me make the technical decisions, and please focus on refining and
extending what I'm doing rather than resisting and forming oppositions.
I don't want to shut you up, but I do need to change the manner of the
interaction.
I suppose the concern runs a little deeper: if we can't in short time
get on the same page with this line of thought, then I'm going to have
to change one or two things, because I'm finding it really exhautsting
(and compromising of other things) to have all this stuff called into
question every other month. I would prefer utterances to be confined to
the OO, patterned, agile discourse of the cautious engineering school of
software development (a la Booch, Cunningham, Fowler, Gabriel, Beck,
etc, etc) that has given us such wonderful concepts as OO pattens of
design, agile approach to development, and so on.
Please note that I have no problem coding against the sqlobject API.
It's a great idea. In fact, I like it so much that I wrote code to
construct sqlobject classes automatically from our domain model meta
data. If you want to know something more of what I did there, then I can
simply report that I implemented Concrete Table Inheritance. If somebody
doesn't know want that means, they can do what everybody else who knows
has done: go and read Martin Fowler for 15 minutes and then come back.
But overall, the position is that domain model delegates to data mapper,
and doesn't inherit. If it was the other way round, you would be hearing
lots of things from me about the need to replace inheritance with
delegation. But the forces are resolved, and I don't think there is an
unresolved concern.
If you think there is an unresolved concern, I would be very interested
to hear about it.
With best regards,
John.