cmis: Apache Chemistry

good news on CMIS front. Apache incubating a new project called "Chemistry".

project description

Apache Chemistry is a generic Java language implementation of the upcoming OASIS CMIS specification.

Proposal

Apache Chemistry is an effort to provide a Java (and possibly others, like JavaScript) implementation of an upcoming CMIS specification, consisting of a high-level API for developers wanting to manipulate documents, a low-level SPI close to the CMIS protocol for developers wanting to implement a client or a server, and default implementations for all of the above. Chemistry aims to cover both the AtomPub and SOAP bindings defined by the CMIS specifications.

with success of this project there might be an open source CMIS reference implementation not coupled with a existing CMS and therefore has the change to on the one hand provide a potential good starting point for vendor specific integration work and on the other hand a reference makes the specification "real" and touchable.

see also:

• http://trent-intovalue.blogspot.com/2009/03/just-another-standard-to-manage-content.html
• http://xml.coverpages.org/ni2009-04-30-a.html
  

win32 memory management

i believe that one year from now 64bit operating systems are mainstream in server infrastructure. by the way currently there are still 32bit windows server / software out there. therefore it's time to understand limitation you leave behind using 64bit OS. so don't miss

• Memory Management 101
• Memory Management - Demystifying /3GB

EXtensions...xml processing

you always faced with "missing features" if you working with existing standards.

two reasons for that:

1. the feature is specific to your domain or solution
2. the feature is common but for some reasons (and there might be good ones) didn't get it in the existing standard

for the second case common EXtensions might be the area you looking for. in the xml domain the following projects trying to define "standard extensions". this means that extending an existing standard out of the scope of the standard process itself but without loosing the portability of applicaton using the extensions.

projects

• XPath
  => EXPath
• XSLT 1.0
  => EXSLT
• XSLT 2.0
  => see discussion about EXSLT 2.0
  
• XQuery
  => EXQuery
• XProc
  => EXProc

beside EXSLT which is well-known and already produced a sufficient amount of output all other EX communities are brand new and therefore doesn't provide any output so far.

watch and contribute......

Programming is Fun But Shipping is Your Job

pretty old but always valid post on common shortcoming of developers.
one of the most critical part of each software project is to get the last 2 % of the job done. regardless which development process you choses (waterfall, agile, rup, ...) you are always faced to get the team focused until the stakeholders of the project are really happy (and not only the developers creating the assets).

identify content: powerful but tricky

regular expression and xpath are two approaches to identify a matching subset of content within a given amount of content for further usage. the first one based on plain text the second one based on xml.

both are powerful but if you not using it on a day by day basis are tricky and error prone. error prone means that you have to avoid all edge cases where a given expression shouldn't match but it does or the other way around where a given expression should match but it doesn't.

if you work in XML related world you often need both approaches e.g. using XSLT or XQuery.

there are few commercial IDE's which helping out to develop the right expression for the required use case but there are two which i'm often use dedicated to help you dealing either with regular expressions or xpath

SketchPath

powerful standalone tool to create and verify xpath expression. support for xpath 2.0 and most features you ask for in this context. this tool is based on well-known Saxon for xpath 2.0 and .net subsystem for xpath 1.0

home: http://pgfearo.googlepages.com/

Regulator

powerful standalone tool to create and verify regular expression. with buildin regex analyser and on-the-fly validation, code generation (for .net and vb only), ....

download: http://downloads.sourceforge.net/regulator/Regulator20Bin.zip?modtime=1189740236&big_mirror=0

... too long.

if you read

http://www.sharepointreviews.com/sharepoint-articles/104-sharepoint-workflows/480-how-long-is-a-sharepoint-project.html

i would say, too long. because i really do not have any statement of goal and expected improvements in the mentioned list.

last statement is at least strange:

If you add up the weeks in brackets you will see that it comes to 26 weeks which is six months. This is purely an estimate and project length really depends on the workload of the Project Team as well as the amount of data involved.

if only those two constrains are responsible for project length is do not expect any measurable outcome beside "making IT people and consultants happy"....

thats also a good example for "IT centric projects likely to fail"

save money means save money

the following blog post from Mark Woeppel "Is Your Continuous Improvement Organization a Profit Center?" summarize one of the major misunderstanding of continuous improvement in many companies. not too much to add, beside that i can agree to the suggestion within the post based on personal experience.

same statements must be made for any (IT) project claim to improve existing processes. while working with the stakeholders to identify the scope, requirements and more important the rational for those i often hear the statement that a particular requirement / process change / feature improves the process and therefore saves money.

after the project is finished and in operation, the same amount of people with at least the same amount of budget working on the "improved process" and the upfront investment already gone. based on simple calculation, the company just loose money with such a project (even if the cause - see below - might be different).
that leads to continues and increasing management distrust in changes stated as "money saver" and subsequent projects are more likely to rejected from senior management.

why?

1. project scope or requirement's simple wasn't target oriented. means they do not really help to improve the overall process they more or less leads to "doing the same in a different way".
   that is a common project failure and can be avoided if project / requirement / system management skills are improved or more simple by doing a better job. the project teams fails to address the goals.
   
2. other process parameters are changed as well. for example the process throughput increase, additional process outputs are created or budget / people now assigned to other activities within the same department to resolve bottlenecks in this areas, ....
   

the second one is the more important one, because this means that the change has an impact but the company isn't able to express / see this in terms of cash. means budget is simple moved from one service item to others.

how to avoid this?

divide the individual rationals for a project in measurable and independent topics and for each of them assign responsible person from senior management. means if a certain amount of requirement state to save money, the budget schedule for the period the result of the project is in operation must be reduced. if on the other hand upcoming new demands must be addressed those must be implemented in budget schedule as dedicated position.

sounds trivial, but believe me that does not happen very often in real world situations. the reason for this is simple -- it is a pretty hard work and sometimes not easy to define the saved money. This not only helps to get project agreed, provide traceability for senior management but also tries to get the bottom of the stated effect (which sometimes simple is not right which leads to effect described in 1. reason).

goal

the different goals and results can be traced and measured and if a project or few parts of a improvement projects intent to save money, you have to save money means you have to have reduced budget if the project succeed.

summary

save money must mean saved money at the end. All other "statements" and explanations are not valid especially in todays business context....

cash is kind

"cash is king" is a principle that many companies today have to follow. this simple means all decision must follow to increase cash flow.

for all activities and departments of a company with no direct impact to the core business process this means that they are forced to immediate reduce their budget to a minimum / essential value.

HR, IT, TechDoc, .... are examples for affected units.

the current situation is not designed for mid or long term investments in the mentioned buisness areas which "only" improves long term operation. what you need is short term benefit without too much investment.

good opportunity for outsourcing, isn't it? i expect significant amount of companies try to reduce their supporting activities through outsourcing services even if the services doesn't fit 100% to their specific needs and the missing parts isn't essential for daily operation.

what do you need for service outsourcing:

1. service function must be fully defined (in technical context this means that the interface must be fully defined)
2. service availability must be defined
3. service security must be defined
   =>what happens with the data maintained by a different organization?
4. service termination must be defined
   =>what happens if service is moved in-house again or moved to different service provider?
5. price must be defined
   =>pay per use / per time period / ?

having this in mind i expect increasing request to ASP / PaaS / SaaS / PRaaS buisness models and companies can provide such services in certain domains.

ASP
provide IT infrastructure (software & hardware) for daily operation for a dedicated amount of money.
=>IT assets based

PaaS
provide a dedicated platform to implement and host a application for a dedicated amount of money.
=>development asset based

SaaS
provide a dedicated set of software features for a dedicated amount of money.
=>software application / function based

PRaaS
provide a dedicated Process / Business function incl. the corresponding IT asserts for a dedicated amount of money.
=>business process based

the amount of semantic involved is increasing from top to down. therefore the potential value but also the potential risk for the requesting company increase. todays knowledge and IT infrastructure make usage of those business models not only possible for big, enterprise companies but also for mid and small size ones.

but beside few other points you still have to trust in trust if you want to go this direction.....

model a DITA compliant model

DITA today is more and more adopted at least in the techdoc domain. the success and adoption rate is based on two major advantages over other existing standards:

• adaptability
  the DITA data model can be adapted based on defined rules to specific needs and domains. this concept is called specialization
• modularization
  content creation is not based on document paradigm but on module paradigm. authors no more create documents they create topics which represent some artifact of the system they describe.
  

problem for an information architect

since i started working with DITA i reviewed several data models from coworkers, from customers and last but not least created by myself. All of them claimed to be DITA compliant.
Using them with also DITA complaint tools often fails for certain features or in some cases completely.

reason

1. the created models wasn't valid according to the released specification
   creating a complaint DITA model (DTD or W3C Schema) requires to know all rules and requirements the DITA specification provides without having any tool support goes beyond the support of creating regular DTD's and W3C Schema's. this means the information architect is leaving alone with the underlying complexity.
   
   i do not know exactly but based on data models i'm faced with i expect that at least 50% of customized DITA models out there are not complaint to the DITA spec.
   
   
   
   few years ago i already posted this here: http://dita.xml.org/how-verify-certain-datamodel-dita-complaint
   
2. the used tools requires additional, tools specific configuration or semantic or simple doesn't implement a feature according to the specification

the second reason is getting better over time and today you find a bunch of tools which are production ready even for enterprise usage.

the first reason still not resolved.

you might argue, why not use the DITA model out of the box?

• if you have specific requirements in your business process and you require additional semantic to support those
• if you have to simplify the usage of content creation for the authors and users to get better and more consistent content outcome.
  note: i consider DITA subsetting / configuration as just another way for the generic concept specialization
  
• if you introduce new business domain / taxonomy into your content maintenance strategy.

issues if your model doesn't really confirms to the specification?

• interoperability is no more guaranteed
  this is mainly a problem for the enterprise and in most cases not at the beginning of the usage of DITA
  but if you once look into this problem you forced to fix all content created against your model or adopt the processing chain your content is delivered to
• process chain does not work
  you add new tools to your environment and certain features doesn't work. you again can fix all your existing content and the data model and tools or adopt the tool with certain workarounds

this week i stumbled over DITAworks modeling module which is the first tool I'm aware of addresses the modeling use-case in the DITA domain.

I'm not yet verified it to see to until which extend this tool supports the modeling / validation process and how much manual work is still involved and how complete the implementation currently is. because this tool is still beta and is brand new i expect much outstanding work but i hope this tool opens the door for speed up DITA related development work and the more important part of the story improve quality for DITA related information models.

i know that the amount of money / amount of customer can be found in this domain is pretty small and the complexity of the problem isn't small enough therefore i do not expect too much competitors in near future, but i might be wrong.

time will show....

cloud toys

essence of this blog created using http://www.wordle.net/

Correlation

lets see: http://xkcd.com/552/

content & service composition: small and simple showcase

if you want to see and learn how easy a information aggregation use case incl. corresponding presentation can be solved take a look at "Make dashboards with XQuery".

this sample is all about composition, from content and service (functional) point of view.

most of the concepts required in the file of information processing are involved. even if the implementation has drawbacks and limitation in several points you see how information centric requirements can be solved.

distributed ping

just ping....your server infrastructure.

just another standard to manage content

CMIS is another try to provide a vendor independent content exchange api. major ECM vendors, like MS, Oracle, IBM and many others are joined the corresponding OASIS TC to release the first version of this new standard.

good overview of the current state can be found in "CMIS meeting notes".

I'd like the idea to have a common accepted and wide adopted standard to maintain resources and all related concepts.

major advantages

• decoupling of client implementation dealing with standard related concepts from server implementation
  =>decoupling
  
• usage of several server implementations through single client implementation
  =>integration
• adaption of common and rich infrastructure for specific needs
  =>enrichment

this leads to the core vision i have in mind talking about such a standard.

Vision

core infrastructure of a content management is provided by regular OS client infrastructure and specific needs are implemented on top of those core layer. a wide range of server side infrastructure can be used and integrated without adding IT complexity.

WebDAV

we already heard about those vision, thats what WebDAV claim to provide....
why not working stronger on WebDAV instead of re-inventing the wheel? looking at the world of WebDAV shows the issues in this area:

• many incomplete implementation out there
  
• integration in regular OS infrastructure is half-assed and error prone
• WebDAV is implemented as an additional API with limited maturity and attention of CMS vendors.
• DeltaV extension only supported by a very rare and not widely used implementations
  
• ....

of course, WebDAV standard is still not feature complete. especially search interface and concept of typed links are not yet provided by the WebDAV standard. but adding this as additional extension is not a big deal (see also http://www.webdav.org/specs/rfc5323.html).

the major issue with WebDAV is simple the lack of robust and complete implementation. the main reason: it is easier to define a standard than fully implement those standard and resolve existing issues within the standard.

the major members in CMIS TC are the same as those working on WebDAV standard. i personal hope that the success and sustainability of implementation will be improved. otherwise there is just another standard no one takes care of once it is 80% complete.

i do not compare both approaches because this is another story but in terms of simplicity WebDAV is currently in pole position. more to come......

Windows Unlimited?

i often have to deal with application server design and deployments based on 32bit Windows server OS.

if you have to maintain more than one application server on a single host and for some reasons have to have more virtual address space for one application server you have to deal with /3GB switch enterprise server OS provides.

the expected load -> request to each running application server -> amount of threads on each application server must be defined carefully and requires more OS in depth know than I'm willing to have.

you have to understanding how the OS deals with system resources and in this context i stumbled over a good blog post

Pushing the Limits of Windows: Paged and Nonpaged Pool

which describe the limits of Windows pretty clear and understandable even for guys like me ;-)

not often read such comprehensive and focused information for this particular topic. thanks Mark for that.....

open usage of sequence of data points

lot of important data out there are simple "Time series" data, means a sequence of one or more data points change over time.

a service to share and use such data is Timetric. currently the amount of user and useful time series are small but in general such pretty platform can deploy common time series from many different domains.

problem

who takes care that the shared data is correct and therefore is valuable to use? all services based on public contribution and usage are faced with the same issue. do you trust the data you see? do you trust wikipedia? in general you should not. you have to double check at least 2 different sources before you use the provided data.

in addition once you double checked your data you have to make sure that the quality of data is guaranteed over time. that is much more difficult.

solution?

if the data is mission critical you should not use data before validating them. in case of non static data you have to validate the data each time they change. this means that you either need more than one data source as service which are not based on same data source or you have to look and buy commercial services takes care of the provided data or you request the service from the organization owns / collecting the data. each of those solution requires special handling for the particular domain.

summary

availability of public data service are promising but i currently do not see a available model to trust in. therefore usage is pretty limited only for some kind of "outline view"

data processing workflows

if you are faced with data processing workflow which requires to process / transform a huge amount of data in a limited amount of time this can end up in pretty complex implementations. if you have enough hardware to do the job you need a infrastructure makes use of the hardware.

using hardware for a limited time today isn't a big issue. the cloud infrastructure out there (e.g Amazon ec2) is perfect if you have to process a huge amount of data in limited amount of time for a limited duration. you are able to scale the usage of the required hardware for the time they are required and just pay for the required duration.

now you also need a ready to use software infrastructure to implement the processing workflow. MapReduce is a software infrastructure for such kind of problems.

Apache hadoop implement this MapReduce but it lacks of easy of use means is a pretty low level infrastructure and of course lacks of higher level workflows which is not defined by MapReduce.

Cascading closes the gap. based on "stream processing" the MapReduce pattern are applied and used. it is not too complicated within one day i was able to create a simple applicaton which convert 20 TB of svg data to jpg and doing some transformation in between using batik and 20 concurrent hardware nodes.

for some use cases the power of cloud is easy to tell....

eat your own dog food

if you in a position to hire a consultant you would ask which one to choose? in each business domain the amount of consultant are huge in each domain.

two things you should consider (after looking into the slides they present):

• don't hire someone who leaves before the suggestions are implemented. so each project tries to involve external consults should make sure that the involved people are not only responsible to provide some kind of "best practices" but also to implement their usage in the particular case they are hired
• don't hire someone who do not eat his own dog food. means ask you're potential consultant how comparable cases are solved by his own company or his or her own daily business.
  each problem can be traced down to some analogue problem which requires the same approaches you faced with. if you talk about the daily work and solution with your consultant before hire him you get a good feeling if the external person really knows what he talking about

its often not the slides which are important its more often the individual experience and knowledge make the difference between good and excellence help in you particular problem domain.

link: Download Free Office 2007 Software from Microsoft

good summary of MS Office 2007 add-ons available for free can be found here:

http://bhandler.spaces.live.com/blog/cns!70F64BC910C9F7F3!1822.entry

Information Dynamics

i stumbled over a dissertation which focus on "Information, Their Effects and Management in Supply Chains". unfortunately the paper is only available in German. quotation from the summary:

I therefore suggest focusing SCM initiatives on information processing and information efficiency in order to enhance overall system behaviour and efficiency.

take the time and read the paper, it provides a interesting view on effect and impact of information in SCM.