My first thought after digesting the conversation was, “Hey, wait, I’m not even talking about native XML databases at code4libcon!” My proposal is about using XQuery in the digital library realm. True, we are using a native XML database here, but just because one uses XQuery doesn’t mean s/he is using a native XML database. You can use XQuery just as easily with DB2 or Oracle’s database (or files on the file system).

The one thing that native XML databases and XQuery do have in common, though, is that they let you interact with your data directly — it doesn’t have to be deconstructed into another structure and then reconstructed when you want the whole thing back out again (in the case of XQuery being used over a relational database, that (de|re)construction takes place invisibly in the database layer).

But, is this a good thing? Ed kept saying he didn’t see any data modeling going on with native XML databases.

There is data modeling going on with native XML databases, I’d suggest, but it happens on the metadata side of things. Andrew Nagy made this observation recently on the code4lib mailing list when he noted how poorly just putting MARCXML into a native XML database performs.

This is because putting MARCXML into a native XML database makes MARCXML the data model. MARC was intended for concise transfer, not for working with the data… it was assumed by the architects of MARC (I believe and hope) that MARC would be reconstructed into something else before anyone tried to do anything meaningful with it (for what it is worth, this only partially happens in the library world).

XQuery allows the developer to work more nimbly with the data models s/he is given (instead of mapping them into other data models that match the database s/he has chosen to use). So, what are these data models? They are the XML metadata standards being created by the different knowledge communities. Unsurprisingly, to use these data models, the developer needs to know them (i.e., having programmers in our libraries is a better idea than contracting out to people not in the profession).

Are the people creating these (meta)data models working to make accessing the data easier? That could be one critique leveled at native XML databases (from the perspective of the developers)… if you aren’t doing the data modeling, can you trust the people who are?

It’s not that bad though (put down that gun you cynical library developers); keep in mind that XQuery isn’t a fulltext query language. Think of it more as a database query language (even though there doesn’t have to be a database).

To use XQuery in the digital library world, in my opinion, you still need to use a fulltext indexer (like Lucene or the type built into many XML-enabled databases). Indices may be used through proprietary extensions to the XQuery language (indicated by a different namespace) or through separate processes which feed the XQuery engine (as in a pre-processing stage).

For what it is worth, there is a fulltext extension to the XQuery spec that is being written to take advantage of these external indices, but it is not really out there in the world yet. In the meantime, even if our metadata models (e.g., MARCXML) aren’t the best, we can still create and use indices that provide an intelligent view of the data.

One nice thing about working directly with the data/data models you receive is that there are less “moving parts” to fix when things change (in other words, less things to get in the way) — because we all know digital library standards don’t change, right?

Rather than go through the process of re-mapping to the database’s structure, you only need to modify the parts of your code that deal with the parts of the metadata that have changed. You’d have to do this with the other option too… just because you have a standard way of normalizing data doesn’t mean that all data is structured in the same way (in terms of how you get at the pieces you want).

I could mention, I guess, some reasons why I like native XML databases in my presentation, but I’m not sure this is a good idea. I think it may distract from the beauty that is XQuery. I’m also hoping Andrew Nagy will cover this territory in his presentation comparing different native XML databases. One of XQuery’s strengths is that it is database agnostic; I shouldn’t stray from that.

For the record, the Ed Summers that appears in this post is not the real Ed (despite the conversation actually happening), but one of my own conception for rhetorical purposes only.

The talk started out with what they see as their business model for developing their suite of tools (which will contain a freely available (as in beer) cross-browser plugin that fully supports the upcoming SVG 1.2 standard). In short, they see “software as a service” as an effective business plan — one that will enable them to release the plugin for free (the plugin should be okay with any valid SVG so won’t have to be used with their non-free products). The presentation then moved into the description of the product (which sounds really great). It ended with a long question and answer period from the SVG experts in audience.

I think it was the longest and most amusing question and answer period I have ever heard at a conference. Unlike many of the presentations, this was really a vendor presentation — not so much of a presentation on a particular project. As a result, there were many claims about what their product will do and, since it will not be released until the end of the year, the experts in the audience were (probably rightfully) a bit skeptical (after all the product claims to have done what many in the XML community have been wanting for awhile now). The questions from the SVG experts ranged from the obvious to some really interesting clarifications of EvolGrafix’s claims.

It was also amusing because it seems that EvolGrafix has not been involved in the W3C and has been vocal about what could be done if the standard was moved along a little quicker than it has been. There was, of course, an invitation from one of the SVG working group members for EvolGrafix to join the group and help move the standard along more quickly (so that it can better compete with Flash — which would be good for any and all implementors). The whole question and answer period had the audience tittering. Quite amusing…

Another interesting snippet from the presentation was that EvolGrafix does not plan to implement any form of XForms (interesting to us in particular because towards XForms is the direction in which we’ve been wanting to move). They see sXBL as the future direction for forms and data binding. I don’t know much about sXBL. From what I gather, instead of having a compound document that might contain an XForm and an SVG image (along with other XHTML), sXBL would allow the binding of data to exist in the SVG component.

Also interesting… Renesis will support interacting with the DOM of the XHTML of the page; it will not, though, if I understand correctly, allow interaction between the SVG and JavaScript in the HTML. This is because Renesis does all its ECMAScript and C# work in its own interpretter (one for each of those languages). Yes, they have implemented their own C# interpretter and JavaScript interpretters (the whole SVG app could be compiled to native code if one wanted to do that — and that is one of the options they see as a possibility further down the line).

This sounds like a huge project and I cannot wait til the end of the year to see if a product is released. If it is, and it really supports the 1.2 spec (and all they have promised) without requiring any proprietary dependencies, this could really benefit SVG’s growth (especially since Adobe will probably be moving more towards Flash with their purchase of Macromedia). One has to take a wait and see approach I think. If it happens, though, this could help push us towards SVG as a viable way to add more interactivity into our web applications.

We are thinking about adding these additional features as an optional bonus… users could still use the existing, more accessible, site or could choose the more interactive one (that would require a plugin — at least for the time being). We looked at the possibility of a Java applet, a Flash script, or something with SVG. SVG is the most appealing because it is XML and would fit with our current architecture (and because it is an open standard and not owned by any one company (as a side note: it will be interesting to see what happens with Adobe now that they own Flash and the leading SVG plugin)).

SVG, not currently supported natively in browsers (though Opera and Firefox/Mozilla have committed to building in native support), has seen a lot of growth lately in the cell phone market (and in bleeding edge desktops, like Gnome, which support SVG for images/icons). There are also new SVG plugins waiting in the wings to be announced (supposedly at the SVG Open conference). Still, SVG is far from a widely implemented technology.

What we would like from it is some sort of traditional page turning animation where turning the pages results in a page curl. That is not the key feature though. Being able to select regions of interest dynamically (instead of relying on predefined tiles) and being able to zoom in an out more fluidly would be nice (we can do this with our own interface now, but not so interactively as we would like). I think going to the conference will be a fact finding mission. Does SVG do what we think it can? Who is doing interesting work with SVG (from which we might be able to benefit).

There are some digital libraries out there doing interesting work with Flash and Java applets, but having everything in XML is just too convenient (and future proof(?)) What we really need, though, is for a tool to support loading JPEG2000 images into an SVG document (Batik currently doesn’t do this). We’re thinking about getting rid of our archival TIFF images (and just keeping a backed-up JP2 image); it would also be nice to not have to go from JP2 to JPEG to SVG (going, instead, just from JP2 to SVG). I don’t know… I’m surfing the Web trying to learn more about this. I expect the conference will also be of great benefit.