Five-sided Square

MyDMS Document Management System

2006-07-18T17:20:00.000+01:00

In case you've been wondering why there isn't much activity here (since I'm talking to myself, there is very little in the way of wondering as such -- after all I'm asking myself the question. This blog is a metaphysical mirror rendered in UTF-8: discuss), well it's because of this:

MyDMS

It's an open-source document management system that I have started to contribute code to. Actually, I'm also pretty much maintaining the software and the web site and answering questions from users and so on. It's a lot of work.

Having resolved a fairly serious performance deficit and improving the search times by a lot (from 4 minutes down to under 2 seconds), I'm now working on all kinds of new features. Most importantly, I'm writing a document approval work-flow so that documents can be reviewed and approved by selected peers.

I'm also rewriting the user interface in an effort to improve the layout and general usability.

Conversing with a 2 Year Old

2006-03-29T19:19:00.000+01:00

"How are you doing?"

"No."

"No?"

"Yes."

"...?"

"Chin."

Complete the Circuit

2006-02-01T11:47:00.000Z

Redirection entry in my company's blog. Bit of a rant really...

Blugh.

Little Apple Dolls

2006-01-22T20:57:00.000Z

Little Apple Dolls

Saw these dolls today in a local store and thought that they were, well, interesting. And yes, I am a grown man and sufficiently heterosexual to understand that I should not utter this out loud. However I'm writing this on a blog that no-one reads. What's the worst that can happen?

I thought that Eldest daughter would like one of these (and it is her I was thinking of when I saw them). Wifey thinks they are creepy and that eldest daughter would agree with this sentiment. She's probably right (Wifey is often right, and is always opinionated; see blog for details) . The nails are a nice touch. It makes me wonder if Clive Barker maybe missed something: Geisha's in Hellraiser...

Or Dead Or Alive 4 Extra-super-ninja Geisha with nail-apple attack!

Five Corners

2006-01-22T20:45:00.000Z

The Five Corners Quintet

So these guys are a fantastic Jazz whadjamacallem, quintet that have a thing about the corners rather than the edges. That's fine, I can cope. Their debut album is also available on vinyl which is a rare treat these days, and is presented as a double album (although each side is actually quite short, meaning that you are getting a very high quality package). Beautiful music, beautifully packaged.

Use Fu! And Fight!

2006-01-22T20:19:00.000Z

We got a flier through the mail this weekend... Or it may have come with the newspaper. Anyways it was on the floor and it looks like this:

And I thought "Cool! We can teach the kids Kung Fu!". Both practical and exotic in the same packaged bundle. That's one up on the neighbor's spawn of Tae Kwon Do. Then I read the reverse:

Now this brings a whole new perspective into play: now I can learn Kung Fu too! In fact, the whole family can train in this most distinguished of the oriental martial arts. We will become the Fu Family! The Family Fu! We'll have a name that is like the Foo Fighters, but we will not fight only Foo nor will we be Foo that fights. We can use Fu to fight.

Naturally, we follow a strict doctrine of peace and will meditate long in order to attain balance. But I'm here to tell you, if there's trouble I will unleash three-fingered fist while eldest daughter invigorates the debate with ultimate malleable bamboo, striking panda claw.

Most of the family already have the proper satin jammies. Eldest and youngest daughter have traditional Chinese costumes, beautifully embroidered, while Wifey is not short of appropriate attire for nocturnal combat. For myself, I fear that I must begin my new career in Kirkland Signature's finest flannel PJs (kitting out the family in satin attire is not without its financial burden you know).

Perhaps by Father's Day I will have gained enough respect with my new skills to warrant a gift of shiny garments.

Not the Good Kind of Redundant

2006-01-22T19:42:00.000Z

So basically, this blog is, in its current form, redundant. Not only that, but my ideas now appear to be quite primitive and certainly incomplete. All thanks to the already fascinating SkyOS project. I draw your particular attention to the following flash movie:

SkyOS Beta Demonstration

There doesn't appear to be any dialogue, but I recommend sitting in the dark and playing something a bit ambient. Eno perhaps, or for the more adventurous, how about some Husker Du? (Yes Robin, I have finally acquired one of Husker Du's good albums).

So there you have it. After scrabbling around in the dark ages for so long, perhaps the renaissance of modern desktop computing is finally upon us. Apple users: no, MacOS X doesn't count, although I do like the peripherals (I have a nice usb keyboard and I quite like playing with that remote that ships with the iMac -- something to do in PC World while wifey searches for a product to combat RSI).

I shall now revert to type. The future of this blog is uncertain, but is likely to contain a heightened sense of rambling.

One More Thing

2005-07-21T22:40:00.000+01:00

I just found this in my bookmarks:

Metadata for the desktop

It's been in there for ages. Wow. I'm going to have to rename this blog "My Life in a Sound-proof Bubble".

Bubble Boy

2005-07-21T22:24:00.000+01:00

Apparently, I've been living in a bubble. A good read through the material on BeFS was very enlightening. And then I went over to The Man Also Known As "I'm Batman!"'s blog on the back of this slashdot post.

This is just incredible stuff! I genuinely thought I was on to something, but look: it's been a core part of the BeOS experience for years. And AKA I'm Batman has pretty much nailed a lot of the lower level issues. It's an amazing blog, and I am in awe.

Personally, I think it would be fantastic boost to Linux (and BSD, Solaris) to adopt BeFS as a reference file system. This has the advantage of perhaps being quicker to implement than Batman's BDB+FUSE cocktail, since there exists a linux project (somewhat dormant) that implements a BeFS driver. On the other hand, the advantage of FUSE is the potential for extensibility as it resides outside of the kernel.

Command Shell

2005-07-21T22:14:00.000+01:00

It is important that the command shell be able to interact with documents with the same access to attributes as the desktop shell. It should be possible for the desktop and command line to be used interchangably.

This probably requires a supporting command structure. For example, to interact with the label attributes, one could imagine the following sort of syntax:

Logical operations for the label attributes:

UNION: llist <label1> UNION <label2>
Show documents with either <label1> or <label2>
INTERSECTION: llist <label1> INTERSECTION <label2>
Show documents that have <label1> and <label2>
NOT IN: llist <label1> NOTIN <label2>
Show documents with <label1> but not <label2>
XOR: llist (<label1> UNION <label2>) NOTIN (<label1> INTERSECTION <label2>) llist <label1> XOR <label2>
Show documents that have either <label1> or <label2> but not both

Queries can also be stored as Virtual/Pseudo labels.

Attributes Again

2005-07-21T22:00:00.000+01:00

I was thinking about representing shared documents. Is it shared over NFS, SMB, BitTorrent? Is it uploaded to an FTP or web server?

Some of these attributes are tricky to categorise... Is it really appropriate to put all of these attributions into the file, or is there a secondary set of attributes that are better stored as part of a user's desktop. Some properties are intrinsic to the document, some are to do with an individual's application of that document.

I guess I'm just looking to find a way to extract information about a document in a uniform, intuitive way that is not constrained by having to identify a particular application or shell that contains the information I am looking for. And of course, I need an effective means to ascribe attributes to a document.

New desktop features should improve the effectiveness and productivity of the user. Improvements to the graphics enhances the experience and is certainly exciting, but these are ancillary to the need for a robust system that enables productive work.

Document Structure -- 2

2005-07-20T19:48:00.000+01:00

Treat the document as an information provider. Don't look to other systems to manage a document -- interrogate the document itself. Look at the Neuromancer project for parallels (see Jim Waldo's Contrarian Mind for information on Neuromancer), where the edge device is an information provider. In a similar way the document becomes an information provider to the shell, not just to the user who opens it and consumes the "actual" content.

Every document has a database, or perhaps it is more appropriate to say that each document is a database. An LDAP or XML tree structure would be ideal conceptually, if somewhat cumbersome.


<document>
  <data> [raw document content goes here]
  </data>
  <transactions>
    <log id='1000000' datestamp='1121886227' name='create' owner='malk@null.com'></log>
    <log id='1000001' datestamp='1121886227' name='email' owner='malk@null.com'>bloggs@recipient.net</log>
  </transactions>
  <attributes>
    <attr type='note' id='2000000' title='Additional Info'>Should I change the colour?</attr>
    <attr type='todo' id='2000001' title='Email Brian'>brian@nowhere.net
    <attr type='alarm' id='2000002' title='Deadline' trigger='1121889000'>Don't Forget</attr>
    <attr type='emblem' id='2000002' title='Warning' uri='file:///home/malk/pandafireman.svg'></attr>
  </attributes>
  <applications>
    <app id='3000000' name='vim' datestamp='1121886227'/>
  </applications>
</document>

Try to ignore the fact that I'm using XML to represent this information -- I am very well aware of its short-comings, particularly when applied to this problem. But it is a commonly understood format and therefore an excellent tool for articulating concepts to a wide audience. I'd also be grateful if you glossed over the details of the elements and attributes. This is not a design, it's an impression.

Other attributes:

Attachments
Links
Labels (cf. GMail, BeFS)

As an aside, the technologies to support XML are quite mature, so it is not inconceivable to see tools such as Berkeley DB XML being used here. Not to mention the powerful XPath, XQuery and XSLT tools.

It would appear that BeFS provides a good foundation upon which to build an effective user experience with document-orientation.

Labels replace folders

Use labels to replace traditional folders functionality. Each label created is visible as a folder-like object on the desktop or from the command line. Provide a command line shell that also supports this model. Create a consistent metaphor between
GUI shell and command shell.

Preferred Application

Keep a history of the applications used to manage this document. Mime association is managed by the document itself, not by external systems. This means that the operating environment is only
used to suggest a default in the absence of any other information. Must be able to flatten the document for export to other file systems.

Attachments

Why not? I suppose it is a rather extreme extended attribute. It is more likely that one would embed a URL or some other linkage but it might be useful as a way of associating related material in a direct way so that the document overall is autonomous. Look at how OpenOffice / MS Office documents handle images, charts and so on.

Links

Special annotations that refer to other documents or material available on the network. Rather like the list of references printed at the end of any research paper or similar material. Any legal URI will do.

Notes

Post-its attached to the document. Or speaker notes. Or unfiled material. Or anything, really. Just a short item of text that has relevance within the context of the document.

To Do

A list of open/completed actions against the document.

Alarms

A reminder to do something on or near a given time. A deadline for completion, or a reminder to send the document to another department or print it out in order to send it in the morning's post.

Emblems

Nothing new here. A visual cue associated with the document. Could follow the document as an attachment. Could also be the document's icon.

History

I find that transaction history is an increasingly important aspect of document management and maintenance, both in my professional life and at home. The idea of being able to track changes is not new; CVS, ClearCase, Subversion, Arch et al. are testament to that fact. I use CVS every day as part of my system for organising almost every document that I maintain, regardless of its type or purpose. It's a little cumbersome to have to remember to checkin changes every now and again, but it's worth it for the extra flexibility and power that tagged files and history deliver.

It's a bit much to expect that any file system is going to automatically record all changes to a file and also maintain that history within its own internal structure. But at the least there should be some sense of when changes were made and by whom. What software was used to make those changes?

Another aspect of document history will be who sent the document to whom. To whom did I email this file? From whom did I receive it?

Privacy

(This does not represent a thorough examination of the right to privacy and how it affects the design of a document-centric operating environment. It is a simple introduction of the topic.)

Privacy is an important right and one that requires careful consideration. It will be necessary to provide auditable facilities to manage the contextual information. Users must have the right to erase history, attachments and all other metadata. We have to be able to flatten the file, which in this system would mean presenting only the contents of the data element. A flattened or exported document is presented and stored in exactly the same way as any other document currently in existence -- without metadata. We need the Anonymiser. Or Flattener. Or... You decide.

As a corollary, it should also be possible for users to lock certain attributes that they have contributed. The transaction that records the original author should be immutable, for example.

BeFS

2005-07-20T08:02:00.000+01:00

During some additional reading last night, rediscovered the BeOS file system, BeFS, which has support for extended attributes. I haven't used BeOS for a long time, so I will have to investigate this further.

Document Structure

2005-07-19T21:36:00.000+01:00

There are two aspects of the document that are important. There is the content, the document itself. And then there is context -- the [current] name of the document, when was it last changed, who owns it, it's place in the file system.

The contextual elements of a document help us to identify, filter and categorise the information stored on the computer. Existing systems don't give us very much help in managing these files and folders; it can hard to establish relationships between documents, make associations outside of the file system (e.g. web sites, people) or review transactions (the history of the document).

It is the purpose of this blog to document a method for capturing and presenting a wider variety of contextual information, and from there design a document-centric system for managing our electronic information.

Maintaining Context with Tree Structures

I propose three main elements for any document:

The Data (or content) element
The Transaction (or log) element
The Attributes element

The data element is nothing special -- it is just the document as it exists today, more or less.

The transaction element contains a series of log entries that describe the history of the document. As well as tracking how the data has changed over time, one could also examine past ownership (where the file originated, how it was received by the current owner) and with whom or by what means you have shared the document (sent on by email, stored on a shared folder, uploaded to a web site).

The attribute element is somewhat free-form. Here the user is invited to ascribe arbitrary context to the document. Here you can assign emblems, attach notes, reminders and alarms, perhaps even a To-Do list. Associate a URL to a web site that has a related theme. More significantly, instead of assigning documents to folders, one categorises a document according to a set of labels (cf. GMail). So a document can be associated with the categories "White paper" and "Networking" as part of a natural expression of what the document means to the person who manages it. And if you change your mind, it is a simple matter of editing the attribute list and removing that label which is no longer relevant. No shuffling files around or creating and removing directories or adding links. Directories and folders are not used within this system.

Another top-level element that is worth considering is that of preferred application. Rather than having the operating environment determine the associations between file extension / mime type, place this information in the document context. For example, one could maintain a history of the applications used to manage the document and pick the most frequently used application as the default. Or you could just embed a single attribute. If no such attribute exists, or if the application listed is not installed on the platform, then the operating environment makes an informed suggestion based on existing methods.

Document-Oriented Desktop [Introduction]

2005-07-17T21:51:00.000+01:00

A general dissatisfaction with desktop computing has led me to start thinking about what it is I want from my computer, and to see what it would be like to evaluate the requirements of a computing system that manages documents rather than processes or applications.

I do not intend to tackle the problems of user interface design or graphic design as such. The system described is as relevant to the command line shell (e.g. bash) as it is to the graphical shell (X11, Windows, or that shiny MacOS stuff). What I want to encapsulate is the concept of a document-centric user-interface that presents information to the user that is logical and intuitive, removing the emphasis from the desktop as a suite of applications.

I do not wish to critique existing desktops. Suffice to say that they are all very pretty and, once properly configured, deliver some reasonable conveniences. However, they provide limited facilities for the management of documents.

I find file management to be a chore; it is quite often difficult to assign a document to a single category and subsequently remember how I originally categorised that document (what directory should I file this document under? Where is the document that I filed away?). Unix links help, but have limitations and cannot be used on non-Unix platforms.

This is one of the reasons I use "find" so much:

find /home/malk -type f -exec grep -i "something pertinent hopefully" {} . \;

Crude interface, but effective execution. And yet, just searching and categorising isn't the whole solution. For a given document, how can I know what I have done with it? Where did it come from? To whom have I sent it? What is the history of this document?

For a while, I was using my GMail account as a document manager. It's not the most efficient way to store documents, but I'm not paying for the storage. It does the basics very well, but I found some of the things I was doing were a bit of a hack -- sending messages to myself in order to provide annotations to a document (these notes show up as messages in what GMail terms a "conversation"). The labels, filtering and searching are fantastic. GMail is not integrated into the desktop and applying a hack (albeit a very clever networking hack) such as GmailFS doesn't do anything to advance the user's interface to their information.

I'm not sure I'm making a very good case here... Instead of my rambling on, let me just present the features I would like to see in the operating environment of my computer as a series of articles in this blog.