XQuery - rare used but ramp-up gets easier

A very nice overview of what XQuery is, what are the relation to other XML based standards and why it is still very rare used is summarized here http://grtjn.blogspot.com/2011/10/xquery-novelties-revisited.html.

Once you want to dive into XQuery a bit deeper, get a feeling if the approach is sufficient for your use case you should try out BaseX. A XML database, open source. The main advantage for tryouts which are not available on alternative solution is a lightweight but very useful UI fronted for the content stored in the database.

Adding XML documents, try out queries and see how / what they match and last but not least a noticeable view of how information in a XML store looks like (from a conceptional point of view) comes aligned with easy install and ramp-up costs.

Alternatives (commercial and open source) are collected here: http://trent-intovalue.blogspot.com/2010/08/xquery-design-patterns.html

HTML5 and XML

HTML5 will be the main syntax for the Internet in the next few years and will replace the today most frequently used HTML 4.01. Main driver for this shift was Google and is now adapted by all major browser / OS vendors and organizations.

The main advantage of HTML5 are the new amount of build in features which reflect most of the todays common requirements for web based applications.


Why XHTML 1.0/1.1 failed so far? It mainly was much to strict for the web community - the web and also the world is a non perfect place and therefore HTML5 is much more suitable to fit into this world than the XML based approach of XHTML can provide.

Does this mean XML for the web has loose and does not make sense at all. No there is still space for the XML based standards beside HTML5:

• XHTML5
  XML serialization of HTML 5 with stricter parsing rules
  mime-type: application/xhtml+xml
• Polyglot XHTML (see http://dev.w3.org/html5/html-author/#polyglot-documents)

Major advantage of using XML to express the content on the web is a much more easier way to integrate the resulting content into XML processing chains using regular XML transformation tool chains.

• Easier Reuse content for different channels using XSLT / XQuery
• Retrieve content as XHTML and extract only dedicated parts (views) required for different use-case
• Store and request the content using XQuery based infrastructure

The drawback is that today only the newest browser support the mime-type "application/xhtml" therefore for a while Polyglot XHTML might be a good opportunity to deliver the mass and keep processing use-cases doable.

A good summary of Polyglot XHTML and related XML based alternatives for HTML5 can be found here: http://www.xmlplease.com/xhtml/xhtml5polyglot/

XQuery Design Patterns

Nice summary of XQuery Design Patterns (see http://patterns.28msec.com/). This side also contains a link to a online XQuery engine based on Zorba (see http://try.zorba-xquery.com/).

The real power of XQuery comes with a database hosting the data for dynamic retrieval and processing. Today there are several databases out there supporting XQuery processing, even as open source.

Open Source

• eXist (http://exist.sourceforge.net/)
  XQuery sandbox here http://demo.exist-db.org/exist/sandbox/sandbox.xql
• sedna (http://modis.ispras.ru/sedna/)
  XQuery sandbox here http://wikixmldb.org/xq/
• BaseX (http://www.inf.uni-konstanz.de/dbis/basex/)
  XQuery sandbox here http://phobos101.inf.uni-konstanz.de/basex/demo
• brandnew TNTBase (http://tntbase.org/)
  XQuery sandbox here http://alpha.tntbase.mathweb.org:8080/tntbase/ores/Try.html
  uses Subversion and the Berkley DB XML. This implementation adds more XML processing infrastructure which intends to be a XML Pipelining infrastructure including a XML database.
• Oracle Berkeley DB XML (http://www.oracle.com/us/products/database/berkeley-db/index-066571.html)
  There is a commercial version available as well.

Commercial

• The big 3 relational db vendors (MS, Oracle, IBM) provide XQuery support. This approach combines relational storage of xml fragments and is therefore only suitable in certain use-cases
• MarkLogic (http://www.marklogic.com/)
  the most powerful XML DB currently available in my point of view.
  no online sandbox available, but a free developer edition can be downloaded from http://developer.marklogic.com/products
• ....

EXtensions...xml processing

you always faced with "missing features" if you working with existing standards.

two reasons for that:

1. the feature is specific to your domain or solution
2. the feature is common but for some reasons (and there might be good ones) didn't get it in the existing standard

for the second case common EXtensions might be the area you looking for. in the xml domain the following projects trying to define "standard extensions". this means that extending an existing standard out of the scope of the standard process itself but without loosing the portability of applicaton using the extensions.

projects

• XPath
  => EXPath
• XSLT 1.0
  => EXSLT
• XSLT 2.0
  => see discussion about EXSLT 2.0
  
• XQuery
  => EXQuery
• XProc
  => EXProc

beside EXSLT which is well-known and already produced a sufficient amount of output all other EX communities are brand new and therefore doesn't provide any output so far.

watch and contribute......

identify content: powerful but tricky

regular expression and xpath are two approaches to identify a matching subset of content within a given amount of content for further usage. the first one based on plain text the second one based on xml.

both are powerful but if you not using it on a day by day basis are tricky and error prone. error prone means that you have to avoid all edge cases where a given expression shouldn't match but it does or the other way around where a given expression should match but it doesn't.

if you work in XML related world you often need both approaches e.g. using XSLT or XQuery.

there are few commercial IDE's which helping out to develop the right expression for the required use case but there are two which i'm often use dedicated to help you dealing either with regular expressions or xpath

SketchPath

powerful standalone tool to create and verify xpath expression. support for xpath 2.0 and most features you ask for in this context. this tool is based on well-known Saxon for xpath 2.0 and .net subsystem for xpath 1.0

home: http://pgfearo.googlepages.com/

Regulator

powerful standalone tool to create and verify regular expression. with buildin regex analyser and on-the-fly validation, code generation (for .net and vb only), ....

download: http://downloads.sourceforge.net/regulator/Regulator20Bin.zip?modtime=1189740236&big_mirror=0

content & service composition: small and simple showcase

if you want to see and learn how easy a information aggregation use case incl. corresponding presentation can be solved take a look at "Make dashboards with XQuery".

this sample is all about composition, from content and service (functional) point of view.

most of the concepts required in the file of information processing are involved. even if the implementation has drawbacks and limitation in several points you see how information centric requirements can be solved.

Webinar: Building RESTful Services with XQuery and XRX

O'Reilly Webinar "Building RESTful Services with XQuery and XRX" happens on 28.01.09 (10:00 a.m. PST).
if you want to know how the mentioned technologies can work together.....

additional semantic for existing information sources

how information enrichment works in real world showing two public use-cases based on different technologies:

• freebase
  "Freebase is an open, shared database that contains structured information on millions of topics in hundreds of categories. This information is compiled from open datasets like Wikipedia, MusicBrainz, the Securities and Exchange Commission, and the CIA World Fact Book, as well as contributions from our user community."
  
  this means that applications like freebase using already existing information and trying to add additional semantic to them based on combining and extracting information and context or in this case let user add additional semantics without modification to the source of the information.
  
  the tool thinkbase using freebase to provide a visual graph of information and corresponding link dependencies.
  
• MailMark using a xml database (Mark Logic) as backbone for building the application just on XQuery
  
  the semantic comes from information aggregation and combination. in this application no additional user interaction is possible
  

both examples are very useful and showing real world application which you might transform into your own information landscape.....