if you want to see and learn how easy a information aggregation use case incl. corresponding presentation can be solved take a look at "Make dashboards with XQuery".
this sample is all about composition, from content and service (functional) point of view.
most of the concepts required in the file of information processing are involved. even if the implementation has drawbacks and limitation in several points you see how information centric requirements can be solved.
Tuesday, March 31, 2009
Monday, March 30, 2009
Sunday, March 29, 2009
just another standard to manage content
CMIS is another try to provide a vendor independent content exchange api. major ECM vendors, like MS, Oracle, IBM and many others are joined the corresponding OASIS TC to release the first version of this new standard.
good overview of the current state can be found in "CMIS meeting notes".
I'd like the idea to have a common accepted and wide adopted standard to maintain resources and all related concepts.
major advantages
Vision
core infrastructure of a content management is provided by regular OS client infrastructure and specific needs are implemented on top of those core layer. a wide range of server side infrastructure can be used and integrated without adding IT complexity.
WebDAV
we already heard about those vision, thats what WebDAV claim to provide....
why not working stronger on WebDAV instead of re-inventing the wheel? looking at the world of WebDAV shows the issues in this area:
the major issue with WebDAV is simple the lack of robust and complete implementation. the main reason: it is easier to define a standard than fully implement those standard and resolve existing issues within the standard.
the major members in CMIS TC are the same as those working on WebDAV standard. i personal hope that the success and sustainability of implementation will be improved. otherwise there is just another standard no one takes care of once it is 80% complete.
i do not compare both approaches because this is another story but in terms of simplicity WebDAV is currently in pole position. more to come......
good overview of the current state can be found in "CMIS meeting notes".
I'd like the idea to have a common accepted and wide adopted standard to maintain resources and all related concepts.
major advantages
- decoupling of client implementation dealing with standard related concepts from server implementation
=>decoupling - usage of several server implementations through single client implementation
=>integration - adaption of common and rich infrastructure for specific needs
=>enrichment
Vision
core infrastructure of a content management is provided by regular OS client infrastructure and specific needs are implemented on top of those core layer. a wide range of server side infrastructure can be used and integrated without adding IT complexity.
WebDAV
we already heard about those vision, thats what WebDAV claim to provide....
why not working stronger on WebDAV instead of re-inventing the wheel? looking at the world of WebDAV shows the issues in this area:
- many incomplete implementation out there
- integration in regular OS infrastructure is half-assed and error prone
- WebDAV is implemented as an additional API with limited maturity and attention of CMS vendors.
- DeltaV extension only supported by a very rare and not widely used implementations
- ....
the major issue with WebDAV is simple the lack of robust and complete implementation. the main reason: it is easier to define a standard than fully implement those standard and resolve existing issues within the standard.
the major members in CMIS TC are the same as those working on WebDAV standard. i personal hope that the success and sustainability of implementation will be improved. otherwise there is just another standard no one takes care of once it is 80% complete.
i do not compare both approaches because this is another story but in terms of simplicity WebDAV is currently in pole position. more to come......
Friday, March 27, 2009
Windows Unlimited?
i often have to deal with application server design and deployments based on 32bit Windows server OS.
if you have to maintain more than one application server on a single host and for some reasons have to have more virtual address space for one application server you have to deal with /3GB switch enterprise server OS provides.
the expected load -> request to each running application server -> amount of threads on each application server must be defined carefully and requires more OS in depth know than I'm willing to have.
you have to understanding how the OS deals with system resources and in this context i stumbled over a good blog post
not often read such comprehensive and focused information for this particular topic. thanks Mark for that.....
if you have to maintain more than one application server on a single host and for some reasons have to have more virtual address space for one application server you have to deal with /3GB switch enterprise server OS provides.
the expected load -> request to each running application server -> amount of threads on each application server must be defined carefully and requires more OS in depth know than I'm willing to have.
you have to understanding how the OS deals with system resources and in this context i stumbled over a good blog post
Pushing the Limits of Windows: Paged and Nonpaged Poolwhich describe the limits of Windows pretty clear and understandable even for guys like me ;-)
not often read such comprehensive and focused information for this particular topic. thanks Mark for that.....
Wednesday, March 18, 2009
open usage of sequence of data points
lot of important data out there are simple "Time series" data, means a sequence of one or more data points change over time.
a service to share and use such data is Timetric. currently the amount of user and useful time series are small but in general such pretty platform can deploy common time series from many different domains.
problem
who takes care that the shared data is correct and therefore is valuable to use? all services based on public contribution and usage are faced with the same issue. do you trust the data you see? do you trust wikipedia? in general you should not. you have to double check at least 2 different sources before you use the provided data.
in addition once you double checked your data you have to make sure that the quality of data is guaranteed over time. that is much more difficult.
solution?
if the data is mission critical you should not use data before validating them. in case of non static data you have to validate the data each time they change. this means that you either need more than one data source as service which are not based on same data source or you have to look and buy commercial services takes care of the provided data or you request the service from the organization owns / collecting the data. each of those solution requires special handling for the particular domain.
summary
availability of public data service are promising but i currently do not see a available model to trust in. therefore usage is pretty limited only for some kind of "outline view"
a service to share and use such data is Timetric. currently the amount of user and useful time series are small but in general such pretty platform can deploy common time series from many different domains.
problem
who takes care that the shared data is correct and therefore is valuable to use? all services based on public contribution and usage are faced with the same issue. do you trust the data you see? do you trust wikipedia? in general you should not. you have to double check at least 2 different sources before you use the provided data.
in addition once you double checked your data you have to make sure that the quality of data is guaranteed over time. that is much more difficult.
solution?
if the data is mission critical you should not use data before validating them. in case of non static data you have to validate the data each time they change. this means that you either need more than one data source as service which are not based on same data source or you have to look and buy commercial services takes care of the provided data or you request the service from the organization owns / collecting the data. each of those solution requires special handling for the particular domain.
summary
availability of public data service are promising but i currently do not see a available model to trust in. therefore usage is pretty limited only for some kind of "outline view"
Sunday, March 15, 2009
data processing workflows
if you are faced with data processing workflow which requires to process / transform a huge amount of data in a limited amount of time this can end up in pretty complex implementations. if you have enough hardware to do the job you need a infrastructure makes use of the hardware.
using hardware for a limited time today isn't a big issue. the cloud infrastructure out there (e.g Amazon ec2) is perfect if you have to process a huge amount of data in limited amount of time for a limited duration. you are able to scale the usage of the required hardware for the time they are required and just pay for the required duration.
now you also need a ready to use software infrastructure to implement the processing workflow. MapReduce is a software infrastructure for such kind of problems.
Apache hadoop implement this MapReduce but it lacks of easy of use means is a pretty low level infrastructure and of course lacks of higher level workflows which is not defined by MapReduce.
Cascading closes the gap. based on "stream processing" the MapReduce pattern are applied and used. it is not too complicated within one day i was able to create a simple applicaton which convert 20 TB of svg data to jpg and doing some transformation in between using batik and 20 concurrent hardware nodes.
for some use cases the power of cloud is easy to tell....
using hardware for a limited time today isn't a big issue. the cloud infrastructure out there (e.g Amazon ec2) is perfect if you have to process a huge amount of data in limited amount of time for a limited duration. you are able to scale the usage of the required hardware for the time they are required and just pay for the required duration.
now you also need a ready to use software infrastructure to implement the processing workflow. MapReduce is a software infrastructure for such kind of problems.
Apache hadoop implement this MapReduce but it lacks of easy of use means is a pretty low level infrastructure and of course lacks of higher level workflows which is not defined by MapReduce.
Cascading closes the gap. based on "stream processing" the MapReduce pattern are applied and used. it is not too complicated within one day i was able to create a simple applicaton which convert 20 TB of svg data to jpg and doing some transformation in between using batik and 20 concurrent hardware nodes.
for some use cases the power of cloud is easy to tell....
eat your own dog food
if you in a position to hire a consultant you would ask which one to choose? in each business domain the amount of consultant are huge in each domain.
two things you should consider (after looking into the slides they present):
two things you should consider (after looking into the slides they present):
- don't hire someone who leaves before the suggestions are implemented. so each project tries to involve external consults should make sure that the involved people are not only responsible to provide some kind of "best practices" but also to implement their usage in the particular case they are hired
- don't hire someone who do not eat his own dog food. means ask you're potential consultant how comparable cases are solved by his own company or his or her own daily business.
each problem can be traced down to some analogue problem which requires the same approaches you faced with. if you talk about the daily work and solution with your consultant before hire him you get a good feeling if the external person really knows what he talking about
Monday, March 02, 2009
link: Download Free Office 2007 Software from Microsoft
good summary of MS Office 2007 add-ons available for free can be found here:
http://bhandler.spaces.live.com/blog/cns!70F64BC910C9F7F3!1822.entry
http://bhandler.spaces.live.com/blog/cns!70F64BC910C9F7F3!1822.entry
Sunday, March 01, 2009
Information Dynamics
i stumbled over a dissertation which focus on "Information, Their Effects and Management in Supply Chains". unfortunately the paper is only available in German. quotation from the summary:
take the time and read the paper, it provides a interesting view on effect and impact of information in SCM.
I therefore suggest focusing SCM initiatives on information processing and information efficiency in order to enhance overall system behaviour and efficiency.
take the time and read the paper, it provides a interesting view on effect and impact of information in SCM.
ODF and OOXML: interoperability issues
as already mentioned here using a standard does not guarantee to be vendor / implementation independent.
the following paper "Lost in Translation: Interoperability Issues for Open Standards -- ODF and OOXML as Examples" shows that complexity is one major show stopper in this area.
why?
OOXML and ODF try to define office documents this means that content, content structure and layout (and application semantic) must be standardized.
because the complexity of both standards is high only limited amount of vendors are able to implement 100% coverage of the standard and even if they try they are not able to prevent errors in implementation.
lessons learned?
standards should carefully consider good old "Everything should be made as simple as possible, but no simpler." (see http://en.wikiquote.org/wiki/Albert_Einstein) principle. in our context this means that those standards require certain, atomic level of conformance which makes it possible for each vendor to implement a certain and complete subset if the complete set is not possible and useful for a particular application.
the following paper "Lost in Translation: Interoperability Issues for Open Standards -- ODF and OOXML as Examples" shows that complexity is one major show stopper in this area.
why?
OOXML and ODF try to define office documents this means that content, content structure and layout (and application semantic) must be standardized.
because the complexity of both standards is high only limited amount of vendors are able to implement 100% coverage of the standard and even if they try they are not able to prevent errors in implementation.
lessons learned?
standards should carefully consider good old "Everything should be made as simple as possible, but no simpler." (see http://en.wikiquote.org/wiki/Albert_Einstein) principle. in our context this means that those standards require certain, atomic level of conformance which makes it possible for each vendor to implement a certain and complete subset if the complete set is not possible and useful for a particular application.
Sunday, February 22, 2009
IT centric projects likely to fail
during the last two years many projects around trends like "social networking" / Web 2.0 comes to IT departments. the IT guys reading stories from well known vendors like Microsoft that collaboration and information sharing is one of the most important trends for next years.
they take this input and run to their sponsors (business departments) and ask them if they have trouble maintaining information around in their daily business and surprise, surprise they received a "yes we have problems".
why?
most of todays problems are caused by incufficient information lifecycle. the core buisness assets are more or less maintained by information supporting and guiding the core buisness assets are still not really under control. project teams are not able to share a common view of project / work related information, information get lost from one human interfact to the next. supporting documents are not findable even if they exist somewhere.....
IT trys to fix this
they hire few consultants train them installing the product of interests providing wiki, blog, chat, document managment functionality. if they are smart enough they ask the buisness department for their requirements and now they try to setup a pilot using their product of choice.
great everything works after few days of development.....business departments start to use the good new world....solution gets adapted and released.....
one year later looking back and surprise, surprise the problems still the same just in another layout.
this story happens several times during the last year. Look at the most MS Sharepoint related projects out their -- most of them a great experience for the IT / consultants and developers but few or zero benefit for the business.
why?
its simple and everybody knows the answer. IT systems doesn't solve a problem and on the other hand are not the cause of the problem. the problem is caused and must be solved within the business process itself. IT systems can only provide sufficient support for certain steps in the process if the process and connected process itself is healthy.
example
if business people claim they always faced with outdated information the cause is manifold. there might be no process to update the corresponding information at a certain step in the process. there might be no time to update the known information source, there might be no information which are the relevant information source for other people in the team, there might be redundant information source and the wrong one is used.......
sounds trivial and obvious but....
why we still faced with IT centric projects?
and therefore both parties believe they solve a existing problem but they simple implement a solution to decant the problem into another IT solution.
summary
never try to solve a existing problem related to information management never introduce a IT solution first. enforce the business to solve and describe their problems using the existing tool chain and try to identify the CAUSE of the problem. if this is done and works successful the areas a IT solution can support is easy to identify and now its much easier to identify which IT solutions is the right one to choose....
p.s.
i don't say that the mentioned product MS Sharepoint is good or bad but it is a product IT tend to play with and therefore a good example many working people faced with....
they take this input and run to their sponsors (business departments) and ask them if they have trouble maintaining information around in their daily business and surprise, surprise they received a "yes we have problems".
why?
most of todays problems are caused by incufficient information lifecycle. the core buisness assets are more or less maintained by information supporting and guiding the core buisness assets are still not really under control. project teams are not able to share a common view of project / work related information, information get lost from one human interfact to the next. supporting documents are not findable even if they exist somewhere.....
IT trys to fix this
they hire few consultants train them installing the product of interests providing wiki, blog, chat, document managment functionality. if they are smart enough they ask the buisness department for their requirements and now they try to setup a pilot using their product of choice.
great everything works after few days of development.....business departments start to use the good new world....solution gets adapted and released.....
one year later looking back and surprise, surprise the problems still the same just in another layout.
this story happens several times during the last year. Look at the most MS Sharepoint related projects out their -- most of them a great experience for the IT / consultants and developers but few or zero benefit for the business.
why?
its simple and everybody knows the answer. IT systems doesn't solve a problem and on the other hand are not the cause of the problem. the problem is caused and must be solved within the business process itself. IT systems can only provide sufficient support for certain steps in the process if the process and connected process itself is healthy.
example
if business people claim they always faced with outdated information the cause is manifold. there might be no process to update the corresponding information at a certain step in the process. there might be no time to update the known information source, there might be no information which are the relevant information source for other people in the team, there might be redundant information source and the wrong one is used.......
sounds trivial and obvious but....
why we still faced with IT centric projects?
- IT people are happy to develop new solutions
+ - business people are happy to find someone guilty for existing problems.
+ - IT people don't like to identify the real cause they are mainly focus on the solution.
+ - business people cannot image what a certain IT solution mean for their daily business.
+ - IT people don't understand the CAUSE they only understand the PROBLEM itself (expressed in "requirements")
and therefore both parties believe they solve a existing problem but they simple implement a solution to decant the problem into another IT solution.
summary
never try to solve a existing problem related to information management never introduce a IT solution first. enforce the business to solve and describe their problems using the existing tool chain and try to identify the CAUSE of the problem. if this is done and works successful the areas a IT solution can support is easy to identify and now its much easier to identify which IT solutions is the right one to choose....
p.s.
i don't say that the mentioned product MS Sharepoint is good or bad but it is a product IT tend to play with and therefore a good example many working people faced with....
Saturday, February 21, 2009
classified as INTJ
stumbled over Typelyzer which tries to define Myers Briggs Type Indicator. i immediate tried it out using my own blog http://trent-intovalue.blogspot.com/
viola: this blog is classified as INTJ
does this fit? well, it at least fits to the content of my blog.
how does it work?
they using free http://www.uclassify.com/ engine for text classification based on content similarity. they trained their classification with many reference material created by persons known and classified as one of the different types.
to identify psychological preferences based on blog entries is another great experiment to use available information and draw conclusions.
viola: this blog is classified as INTJ
does this fit? well, it at least fits to the content of my blog.
how does it work?
they using free http://www.uclassify.com/ engine for text classification based on content similarity. they trained their classification with many reference material created by persons known and classified as one of the different types.
to identify psychological preferences based on blog entries is another great experiment to use available information and draw conclusions.
Monday, February 16, 2009
maps in svg
if you search for maps in svg you can have a look at blank maps from wikipedia: http://en.wikipedia.org/wiki/Wikipedia:Blank_maps
svg is pretty smart if you require automatic creation of dynamic content like charts and visual reports based on dynamic data.
by the way svg standard has still many limitation in terms of interoperability and tool support. but in case svg cannot be published as-is Apache Batik is there to transform the svg source into format you require (pdf, eps, png, tif).
svg is pretty smart if you require automatic creation of dynamic content like charts and visual reports based on dynamic data.
by the way svg standard has still many limitation in terms of interoperability and tool support. but in case svg cannot be published as-is Apache Batik is there to transform the svg source into format you require (pdf, eps, png, tif).
Sunday, February 15, 2009
RDF support using yahoo's BOSS
yahoo added rdf/microformat support to their public search api called BOSS.
to understand how it works read http://developer.yahoo.com/search/boss/structureddata.html
this makes semantic sources if available usable in searches and therefore BOSS might be one real alternative to what the bg (big google) provides......
to understand how it works read http://developer.yahoo.com/search/boss/structureddata.html
this makes semantic sources if available usable in searches and therefore BOSS might be one real alternative to what the bg (big google) provides......
change is todays baseline
stumbled over the following short video sequence which shows the pace of change
http://www.youtube.com/v/jpEnFwiqdx8&rel=1&fs=1&showsearch=0
hold on and pay attention. thanks tohttp://leadonpurposeblog.com/2008/11/25/the-pace-of-change/
for pointing me to this link.
open source search engine: apache lucense & solr
Apache Lucene is one of the most interesting search api's in open source marketplace. it provides a powerful and pluggable interface to provide full test feature set to many java based application.
but Lucene is "only" an api not an application.
Apache Solr is on the other hand is an enterprise search server based on Apache Lucene. the feature set behind that solution is worth to look at, means Solr introducing features some commercial systems still missing. for example using standard enterprise search provided by MS within their Sharepoint product you wonder how less features are available there.....(maybe adding FAST into next generation of Sharepoint change that but thats another story),
since few month professional services for the mentioned open source products can be used from Lucid Imagination. providing support and integration work for open source products makes the usage for the products more useful for the enterprise.
if you have to provide advance "full text" feature double check the open source community before you invest in commercial variations....
but Lucene is "only" an api not an application.
Apache Solr is on the other hand is an enterprise search server based on Apache Lucene. the feature set behind that solution is worth to look at, means Solr introducing features some commercial systems still missing. for example using standard enterprise search provided by MS within their Sharepoint product you wonder how less features are available there.....(maybe adding FAST into next generation of Sharepoint change that but thats another story),
since few month professional services for the mentioned open source products can be used from Lucid Imagination. providing support and integration work for open source products makes the usage for the products more useful for the enterprise.
if you have to provide advance "full text" feature double check the open source community before you invest in commercial variations....
Sunday, February 08, 2009
Generate DITA Java API reference documentation using DITADoclet and DITA API specialization
one example to show a step to close the gap between developers who creating the artifacts and outgoing and enriched documentation created from their source without the need of additional redundancy.
it is not too far away from approach introduced by javadoc / doxygen beside the datamodel used behind this approach has much more value for additional information integration as provided by the classical approach.
not all yet done by the "out-of-the box" solution provided here: http://www.ibm.com/developerworks/xml/library/x-DITAdoclet/
but you can imagine the possibilities using this kind of approach:
it is not too far away from approach introduced by javadoc / doxygen beside the datamodel used behind this approach has much more value for additional information integration as provided by the classical approach.
not all yet done by the "out-of-the box" solution provided here: http://www.ibm.com/developerworks/xml/library/x-DITAdoclet/
but you can imagine the possibilities using this kind of approach:
- integration of additional information provided by marketing or tech-doc groups or service groups without any media break in information usage
- usage of other deployment processes for api documentation available within companies infrastructure
- .....
improve your full text search experience
Cloudlet a free firefox extension improves your google search experience and provide "cloud" based access to additional keywords might help your to access the relevant information faster. there is no magic behind that (and the results are not perfect in all cases) just using the information provided by google itself.....full text search as its best.
link: Project Euler
this project
Project Euler is a series of challenging mathematical/computer programming problems that will require more than just mathematical insights to solve. Although mathematics will help you arrive at elegant and efficient methods, the use of a computer and programming skills will be required to solve most problems.is out there for a while but i just stumbled over it. check it out.....
The motivation for starting Project Euler, and its continuation, is to provide a platform for the inquiring mind to delve into unfamiliar areas and learn new concepts in a fun and recreational context.
http://projecteuler.net/
Sunday, January 25, 2009
Rocks Into Gold - Helping Programmers THRIVE through the Credit Crunch
summary of "Rock Into Gold" from Clarke Ching can be downloaded at http://silkandspinach.net/2009/01/18/rocks-into-gold/.
just read.....
just read.....
Subscribe to:
Posts (Atom)