Figment Engine

so as part of the process of learning XQuery I wanted to use a unit test approach to document my understanding. Doing some research quickly shows theres not much going on in this area, bumblebee is the most mentioned in http://jimfuller.blogspot.com/2007/09/poor-mans-unit-testing-with-xquery.html however there does not appear to be any information on the referenced site so I'm not sure it exists anymore.

Another option is mentioned at http://jimfuller.blogspot.com/2007/09/poor-mans-unit-testing-with-xquery.html which is a kind of roll-your-own solution. I think that this may be the way to go, but I want to add a few more things:

• create the test methods on the basis on unit test best practice (if there is such a thing) this would involve some research to see if there is a meta-process for creating unit test systems - maybe xUnit will provide a template for this. I'm guessing a lot of this is premised on the testability of the harness itself (better watch out for a Gödel brick wall on this http://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems)
• Make the functions more useful in a XQuery world, specifically being able to test if an XPath expression is true, this would allow me to test that a query result contains expected data - maybe this could be a set of expressions. This would be more powerful and flexible than just having "expected XML" output.

Anyway, some research first and then I can post up my first attempt (which will of course be pants)

with oil near $115/barrel its looking good for a short now, so unless something odd happens in the next day or so I'm going with this option...

why? well because I think the world-wide economy is due a period of slow growth (optimistically) or is about to go into at least a mild recession... and who needs oil in a recession?

I don't buy the "emerging countries will make up for the US drop in demand" story, we are all more connected then ever, if anything we are leveraged against each other.

so thats my thoughts today, I will review on Monday and place the short if I'm still happy with the idea...

The irony of Gordon Brown telling British
http://www.ft.com/cms/s/0/39805ecc-0a41-11dd-b5b1-0000779fd2ac.html

and American bankers
http://news.bbc.co.uk/1/hi/uk_politics/7347995.stm

to tighten their belts, come clean on their hidden debts and pass on the interest rates reductions to account holders is amazing to see....

considering that the bank that Gordon now owns has not passed on these rates
http://news.bbc.co.uk/1/hi/business/7346081.stm

that his government wanted to keep this debt off the books
http://www.guardian.co.uk/business/2008/feb/07/northernrock.economics

and that the regulatory environment that he created failed to cope with the situation on a fundamental level:
http://www.economist.com/world/britain/displaystory.cfm?story_id=10978416

actually maybe he is the best person qualified to talk about these issues!

I'm in the process of teaching myself XQuery and native XML databases. A simple way of understanding this areas is to draw parallels with the relational database world where XQuery=SQL and Native XML Database=RDBMS, XQuery Modules=Stored Procedures.

http://www.xml.com/pub/a/2002/10/16/xquery.html

The major difference being that instead of tables, columns and rows we have collections of XML documents. The significant advantage here being that an XML document contains all the data that is need to satisfy most queries. There may be need to join to other documents or external data but in general the object selected in an XQuery world is less likely to need a join than in a relational world where data has been normalised out.

The term "native" generally refers to the database being designed from the ground up to hold XML, rather than as something that has been added afterwards (thinks SQL XML). This means that the database has been designed to handle XML in an optimal fashion.

You could of course roll-your-own XML database system, but these products offer some key functionality that would take some time to implement:

• Support for a w3c standard XQuery
• holding large datasets (think about DOM models and holding terabytes of data)
• Xml Schema support, especially in terms of query validation and optimization, (think sorting of date fields)
• Indexes built on XML structures (using schemas to allow better understanding)and you can see that its a non-trivial product to recreate.

My background means that I used to be SQL mad, modelling everything in SQL and using it to store everything, then about the time of XML (about 10 years ago) I became disillusioned with the whole RDBMS solution in the context of XML aware applications.

If you have a C# application that uses XML serialisation in memento patterns to store state and support undo/command patterns you start to think that putting in an RDBMS is not adding anything to the equation. In fact you have to create data models, write stored procedures and data access layers. Whilst MS have made strides in making this process simpler the whole process is questionable when you know your code already has a XML representation for its classes.

So imagine being able to put those serialised objects as XML into a database that supported it out of the box, that only required you to supply a schema (optional for simple apps) and gave you a general purpose query tool (XQuery) that could do all the things that SQL could do but understood XML.

Now don't get me wrong RDBMS are not dead, if you have fact-type data that looks like you could easily put it in Excel then its probably best in an RDBMS. However if your data looks like an XML document (tree like) or is mostly narrative XML rather than data XML then maybe, just maybe you should look at a native database:

http://www.rpbourret.com/xml/XMLAndDatabases.htm

I'm in the process of creating a 5 year plan, and I identified the need to have a web site and blog as part of this.

So the first step was to get a hosting service with Server 2008 (as I did not want to start creating stuff based on older technologies) and to create a mini-web site using Visual Studio 2008. All pretty simple stuff.

However pretty quickly I ran into a problem when trying to publish - namely I could not get FTP publishing to work. Obviously it must be the new box and the firewall settings - checked that, nope I can happily FTP and put files on the server from a command prompt.

So I started looking at the usual suspects and searching the web for the error:
"An error occured trying to enumerate the contents of folder"
this was a bit of a wierd error, maybe I had not given list permissions on the file system? nope once again I could list the diretories in a command prompt FTP session.

After a bit of searching I found that someone else had the same problem:
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=321202

which had been closed as apparently it was by design. WTF, this was a bit wierd - surely specifing Active or Passive should have some effect - from looking at the FTP log on the server it appears as if VS2008 ignores this setting either way.

In the end the work around was to remove the FTP server (the one you get out of the box is actually the old IIS6 FTP server, which requires the yucky metabase) an install the IIS7 FTP server that did not make RTM in time - FTP7:
http://learn.iis.net/page.aspx/309/configuring-ftp-firewall-settings/

and then 2008 specific firewall steps
http://blogs.iis.net/jaroslad/archive/2007/09/29/windows-firewall-setup-for-microsoft-ftp-publishing-service-for-iis-7-0.aspx

after which it all worked fine - now I can get back to actually writing code rather than fighting firewalls and errors that don't actually help you with the problem...