Storm Alert

What's Wrong with Presentation Formats

Fri, 29 Jan 2010 19:05:28 +0100

Quite an interesting scam appeared on Facebook. It was just a matter of time when something like that will pop up, yet I was quite surprised when I have actually seen it. The scam works like this: There is a simple HTML page that promises to provide nude photos in zip file if you click on the button. However, if you click on the button you will see no butts and tits. A link to the tricky page will be posted to your facebook profile instead. If you want to try it go to http://homeslices.org/f2.html (if the page is still around). But you have been warned.

The trick is simple. The page creates an iframe containing pretty standard facebook form to share a link. However the frame is almost invisible, therefore you cannot see it. But the browser still think you can see it and is processing it. The tricky page has a "View" button on the same location as is "Share" button on the invisible facebook page. You think you are clicking on the "View" button but instead you are clicking on the "Share" button on facebook. The iframe is fetched by your browser, therefore it is your identity that is used on facebook to post the link.

This page is pretty innocent. All it does is a bit of humiliation for the victims, amusement for experts and undoubtedly a lot of fun for the author. But imagine that this very same method is used to subvert your Internet banking. I guess that the method could be adapted to subvert many of current Internet banking applications. It won't be that funny any more.

This is the price we pay for flexible presentation formats. There are two basic principles of the trick:

Mix the content from two sites in one window. Content from facebook is displayed in a page where you do not expect it, it a wrong context, with a wrong URL in the URL bar.
Create ambiguous display of information. The browser thinks you can see the "Share" button. If has 1% opacity, therefore it is still somehow opaque and ergo visible. Therefore it thinks that if you click to the place where "Share" button is you want to submit information to facebook. But in fact you do not see the "Share" button because if has only 1% of opacity and therefore is almost invisible. You are clicking to that area because you see "View" button that is behind it.

The first problem is a specific problem of HTML. It can be fixed quite easily, if there would be enough "political will" to do it. But the second problem is the problem. How much opaque something should be to be considered opaque enough? Should 1% grey text on white background be considered visible? Or can a 2pt big font be considered readable?

Probably the most serious implication of this problem is a bit independent from a Web. Presentation formats are very dangerous when used it a legally binding way. For example if you sign a document with a digital signature. If you sign a contract and it contains a paragraph written in light grey text on a white background, should such a text be considered part of the contract or not? Some devices may display that text as well readable while on some devices it cannot be seen. This opens up a huge door to a scam of all sizes.

This problem applies universally to any data format that includes rich presentation features: HTML, Microsoft Word documents, RTF, OpenDocument and many more. But maybe the worst aspect of all of this is that our government as well as many other governments in Europe explicitly allows such data formats for legally binding documents signed by "guaranteed digital signature". I'm really lucky that I have no qualified certificate to create such a signature.

What's Wrong with SOAP

Tue, 26 Jan 2010 09:34:27 +0100

SOAP is one of the prominent protocols for remote procedure invocation (RPC). It can do more than that, but it is used almost exclusively for RPC. More specifically it is used for RPC across the Web, both internally and externally. It is used on the Web so frequently that most people working with SOAP do not even realize that it can be used without HTTP and in non-RPC way.

SOAP by itself is quite simple XML-based message format. However it is accompanied by army of profiles, recommendations and especially a set of so called WS-* specifications. That creates a "SOAP stack" that is quite complex. This labyrinth of specifications is an attempt to solve the qualities of SOAP such as security, reliability, addressing, distribution of policies, etc. It makes SOAP quite a flexible mechanism. But ...

Flexibility does not come for free. Until very recently the price was paid by suffering all kinds of interoperability problems. It was so severe that a special organization was established to improve interoperability. Now basic SOAP implementations work together acceptably well, but the situation is not that good for various WS-* extensions. It will take a lot of effort to make implementations fully interoperable. But that is not a problem of SOAP itself. It is an inherent cost of complexity and distribution.

SOAP is not the first "fabric" of distributed systems. There was CORBA before SOAP and Sun-RPC before CORBA to name just a few of many existing mechanisms. However, the designers of SOAP failed to learn from the past. The intent of SOAP was to simplify things. But SOAP stack is now almost the same complexity as CORBA was ten years ago. SOAP is XML-based and with HTTP it can pass easily through firewalls (that are broken anyway). But that's almost the only advantage over CORBA. And now about the drawbacks ...

The most serious failure of SOAP design is the lack of support for object orientation. SOAP is not about invoking methods on objects, it is about invoking operations of static services. Objects cannot be arguments in SOAP messages, cannot be returned from operations and there is no support for object references. All of that was a fundamental part of CORBA, yet there is no concept of objects in SOAP. In fact it is an odd joke to call it Simple Object Access Protocol - as it is definitely not about objects and either not simple or not a protocol (depends on your point of view).

SOAP is also not outright compatible with World Wide Web architecture. Web is based on REST style that defines few basic operations that should be common for all services. SOAP services can use arbitrary operations without any link to the operations of REST. REST architecture also naturally assumes object orientation - web resources are (almost) objects. SOAP does not deal with objects at all. Therefore applicability of SOAP on Internet scale is quite a controversial topic.

SOAP is good in the enterprise and in quite closed environments where interoperability can be assured by testing. SOAP with WSDL has quite a strong interface definition mechanism. It is a rare trait for a technology born on the Internet and it is a necessary condition for composing complex systems. SOAP is also almost the only option for integration, as CORBA is dead and asynchronous mechanisms are seen as too complex and unnecessary by buzzword-driven integrators. If we will be lucky enough, SOAP may eventually get to the state where CORBA was a decade ago ... I can almost hear the melody ... just little bits of history repeating.

What's Wrong with RESTful Web Services

Mon, 25 Jan 2010 00:43:37 +0100

RESTful web services are seen by many (especially young) developers with almost religious awe. Such services are built using standard HTTP protocol with usual HTTP methods as operations. RESTful web services have no arguments, they GET, PUT, POST and DELETE resource representations. The resources are identified by URLs that are also used for links among resources. Such an approach requires a fundamental change of mindset when compared to a more traditional RPC-style of building services. But that is not really a problem: most simple services can be acceptably well modeled using the RESTful approach. The problem is not in the functional aspect.

The problem is, as usual, in the tricky non-functional aspect. Web services are mechanism for communication between computers, but the Web was designed for human-to-computer interactions. Many issues appear from the blue if the Web is used for something that it was not really designed for. Let's have a look at security aspect of RESTful web services as an example.

It is difficult to authenticate invoker of the service to the provider. There are two authentication mechanisms for HTTP (basic and digest), but these are design for interactive human-to-computer authentication. HTTPS in mutual authentication mode provides another solution. This can be non-interactive, but is quite hardcoded to X.509. Under normal circumstances it can authenticate two sites to each other. What would a service need is to authenticate user to the site. If you want to authenticate user on the client side to server, you can still do that with somehow non-typical use of X.509. In that case each client site must be a certificate authority. However as certificate constraints are not well supported, root certificate authorities are not likely to issue certificates that allow creating subordinate certificate authorities to clients.

But even if HTTPS/SSL/X.509 can be fixed, it will most likely not solve the problem. I doubt that X.509 can be flexible enough to support broad variety of security schemes that Internet-wide technology requires. And the flexibility comes with a cost: interoperability. The people working with enterprise PKI know how difficult is to achieve interoperability of different X.509 implementations, and that is miles away from Internet scale. There was only a slightly improvement in two decades of X.509 existence therefore there is little hope that X.509 will be the right solution for the Internet.

There are (relatively) new security mechanism out there, but these apply more to the RPC-style web services. WS-Security and SAML are good examples. WS-Security specifies a header to SOAP request that contains security credentials. SAML specifies protocols and security token applicable for various scenarios, including Internet-scale single-sign-on and federation. However it is difficult to use SAML with RESTful web services. SAML tokens are usually many lines of XML code. In SOAP there is a place for the token in message header, but there is no such place in HTTP. I don't think that placing few kilobytes of XML data in custom HTTP request header is ideal solution. If that would work at all it will be a non-standard hack. And there is no other place in HTTP GET request for such data. There is a way how to shorten SAML token into a few bytes of SAML artifact. But artifact resolution requires additional round trip. In fact several round trips as a new TCP connection (and most likely also SSL handshake) is usually required. It also requires active client being able to listen for connections and maintenance of state on client side. There is also a question how to pass the artifact to the server. The usual way of putting that in the query string is a violation of REST principles, therefore the result will likely be non-standard solution or broken architecture.

The situation is quite similar for many other non-functional aspects. It is difficult to guarantee consistency, atomicity and coordination of RESTful web services (e.g. make them part of a transaction). As URLs are both service endpoints and object identifiers, it is difficult to move service without breaking compatibility. There is no practical interface definition language and interoperability guidelines. Each definition of RESTful service is a free-form text for humans to read and implement with very limited possibility for code generation ...

I'm not trying to tell that all that is RESTful is useless. Both REST and RESTful web services can be very useful, especially with services that shoot for Internet scale. RESTful web services undoubtedly have many advantages but also many limitations. Standard RESTful web services are not yet ready for anything but very simple public services - for that RESTful solution could be ideal. However RESTful approach fails if service quality is important. Custom non-standard solutions can help a bit, but these have their own dangers, especially if the goal is to create interoperable Internet-scale services.

Engineering is not religion and technologies should be assessed with sceptic eye. An engineer that designs anything RESTful should be well aware of the limitations of REST and Web instead of blindly following the hype.

Garbage In, Garbage Out

Mon, 18 Jan 2010 13:29:33 +0100

The world is not an objective place. There seems to be no single point of view, no absolute truth. There is only a little piece of information that could be regarded as reliable - an information that is well summarized by the famous cogito, ergo sum. All the rest is, more or less, speculation.

Consider some quite distant land, for example Antarctica. Have you been there? Have you observed it personally? Most probably you haven't. All you know about Antarctica is second-hand information. They say that strange birds that cannot fly live in Antarctica. Penguins, that's how they are called. Would you believe that? Yes, you probably would. Have you heard that Yeti recently moved to Antarctica? Would you believe that? You probably wouldn't. Both "here be penguins" and "here be Yeti" are information. These are not facts, but mere information. It is the belief that makes them into facts.

But even things and phenomena that you personally observe cannot be automatically regarded as true. Think of David Copperfield and Statue of Liberty. People have witnessed how the statue disappeared. Yet, if you were one of them, would you believe that huge steel-and-copper statue has really ceased to exist for those few moments? Probably not. How many times have you seen pretty ladies sawn in half, disassembled into pieces and reconnected again or levitating freely in the air? What we see may not be what it appears to be. I'm sure you will be amazed by this excellent performance by my favorite duo Penn and Teller.

Think about your date of birth. Do you think that the date of your birth is an unquestionable fact? Not really. You were there when you were born, but you probably do not remember it. And you was quite incapable of checking the date for yourself at that time. Therefore you date of birth is just an information. It comes from quite a trusted source but, strictly speaking, it is not unquestionable.

Any information must be regarded with an appropriate level of confidence. You will probably not really doubt your date of birth, therefore the level of confidence is very high. You believe in that information. But you will probably doubt that Yeti lives in Antarctica (everybody knows that Yeti lives in Himalayas). Therefore a level of confidence for that information is low. You do not believe in that. However, you may slowly increase the level of confidence as more and more expeditions will report encounters with Yeti in Antarctica. As it goes beyond a certain threshold you may start believing that. And once the popular press brings a convincing evidence that what was considered to be Yeti was in fact a mutated giant rat from Mars transported to Antarctica for the sole amusement of penguins by the four headed hyper-intelligent lizards of Sirius IV, you may quite stop believing in Yeti.

Seems pretty obvious, isn't it? Now how is it related to software?

Software is all about information. However, overwhelming majority of software systems have no ability to be "somehow inclined to believe in Yeti" or "quite doubt that Yeti has moved recently". Most software systems have only one level of confidence: fact. That was not a problem when the information systems were small and disconnected. A user working with a specific system was somehow aware what is the reliability of information coming from that system. The user either knew how the system worked or slowly learned how reliable the information is by confronting it with reality. The user as a thinking human being is correcting inability of computer system to deal with uncertainty.

But such a simple approach will fail in case of global distributed hyper-connected information super-highway such as the Web or Semantic Web. Users don't know how the displayed information was acquired and processed and usually have no time spending few years confronting the information with reality. Users of the Web have no way how to asses reliability of information they see. The simple binary model of true-false will not work in this environment. Any system using such binary model that includes computer-to-computer communication on a large scale is doomed to failure.

I quite believe that the future is not really bright for Internet-scale web services and semantic web. Unless they can learn how to doubt.

Mysterious Abstractions

Wed, 13 Jan 2010 15:57:49 +0100

I was surprised to find out that not many people can create good abstraction. Many people are good in thinking about concrete objects and problems, but only a few of these can think abstractly. We in the software industry are forced to think abstractly from the very beginning, as software itself is somehow abstract. However when it comes to creating higher-level software abstractions, people often fail.

Interfaces are probably the most significant abstractions in software. Interfaces are formed from programming languages constructs, network protocol messages, states and sequences, signals, file formats, XML tags and many other elements. Interfaces provide a basic mechanism that an architect can use to exercise control over the system. Interfaces are powerful tool to contain change, to enhance reusability, to make the system more understandable and manageable. Yet too many interface definitions are weak, imprecise, incomplete or outright misleading.

During the course of several years I found myself gradually compiling a list of items that need to be included in a good interface definition. Recently I have found the time to put it into a document, add some explanation and examples. The result is here:

Interface Definition, Guidelines and Recommendations

I have decided to publish it under Creative Commons Attribution license (CC-BY) so you can freely use it in your project as long as you give me a proper credit. I recommend you to copy and paste parts of the document to create a guidelines suitable for your project. I hope that this helps many people to improve the skill of creating abstractions.

Thou Shalt Not Reassign Identifiers

Fri, 8 Jan 2010 18:42:19 +0100

The Web and especially OpenID has yet to learn important lesson: nothing is permanent. Will Norris mentions it in his post. To make his long story short, the problem is that OpenID relies on DNS and DNS names can be reassigned. With change of control of DNS name the control of associated OpenID identifier is changed as well. Therefore a user may be required to pay for a domain that he does not want any longer just to avoid losing control over the OpenID identifier. The root of the problem is that DNS is not really an identification mechanism, but rather an addressing mechanism. OpenID design does not account for that.

The purpose of address is to locate an object, therefore it contains information about object's location - directly or indirectly. Address must change if the location of the object changes. DNS is using a level of indirection to reduce the number of changes needed if object location changes, but it does not reduce them to zero. You may be forced to pay for a domain forever if you want to make DNS name a permanent identifier - assuming you can do that at all. For example the rules for sk top-level domain will force you to yield your domain in case someone registers a trademark that is the same as your existing domain name. Therefore making DNS name persistent may be quite costly. DNS domain is an address. Get over it.

The purpose of identifier is to distinguish the object from other similar objects. Well-designed identifiers does not need to change. The identifier may identify an object that does not exist any longer, but it should never identify a different object. Think of ANS.1 OIDs, ISBNs or similar identifiers. For identifiers to be efficient their assignment should be very cheap and maintenance must be extremely cheap or entirely free.

It is not wrong per se to use address in your system. But it is a mistake to use an address and assume that it has properties of identifier. It is a failure to assume that address will not change - almost as serious a mistake as assumption that identifier can always be resolved.

Litmus Test

Fri, 11 Dec 2009 16:31:37 +0100

There are lots of lots of sites that horde and gather content on the web. Sites that offer you to maintain a photo album, video collection, bookmarks and whatnots. Each and every such site tries to gather a community of its own. How could you tell apart the sites that are worth your attention and the sites that would mean just plain waste of time? How you could see whether there is healthy community or just a bunch of uninteresting loosers?

I have figured out a three-seconds test that seems to work quite universally. Just go to the site and use the search input field to search for some controversial topic. I usually search for "nude". If the search results are just porn or a horde of flame-infested discussions, the site is uncontrolled wilderness. Avoid that site. If nothing relevant turns up or you can see just some carefully censured bikini shots, the site is too conservative to be useful or entertaining. Avoid such site as well. If the search results show decent selection of artistic nudes or some good texts on nudity, it is worth the time to explore the site further.

What is Resource?

Tue, 1 Dec 2009 14:41:30 +0100

World Wide Web Architecture, and the REST architectural style as well, deal with resources. Resource is one of the central concepts in the web. Web pages are just representations of resources, resources are identified by URIs, the web is all about resources. But what is a resource? Now, that's a mystery.

The World Wide Web Architecture document provides quite vague and indirect definition:

By design a URI identifies one resource. We do not limit the scope of what might be a resource. The term "resource" is used in a general sense for whatever might be identified by a URI. It is conventional on the hypertext Web to describe Web pages, images, product catalogs, etc. as “resources”. The distinguishing characteristic of these resources is that all of their essential characteristics can be conveyed in a message. We identify this set as “information resources.” [...] However, our use of the term resource is intentionally more broad. Other things, such as cars and dogs (and, if you've printed this document on physical sheets of paper, the artifact that you are holding in your hand), are resources too. They are not information resources, however, because their essence is not information. Although it is possible to describe a great many things about a car or a dog in a sequence of bits, the sum of those things will invariably be an approximation of the essential character of the resource.

That means that anything can be a resource. Dogs, houses, books, specific version of a book, specific paper-based copy of a book, photograph of the book, files containing data scanned from that book in pixmap format, data containing content of that book in ASCII format, HTML-formatted content of that book, the web page that contains the HTML formatted content of that book and even web page describing that book in an electronic shop - all that could be resources. But wait, isn't a web page containing HTML-formatted content of the book in fact a resource representation? Yes, it is. And many of the objects and concepts mention above may be resource representations. And they may, at the same time, be themselves a resources. In fact there seems to be no difference between representation and a resource (maybe except for non-information resources). The world of web in not black-and-white with abstract resources and concrete representations (as it seems to be at least partially assumed by REST). There are many shades of gray between abstract and concrete. And maybe the pure abstractness and pure concreteness are just theoretical extremes that cannot be reached in practice. Such a fuzziness of meaning is one the most difficult parts of Web architecture to understand.

However, allowing real-world things to be resources make a awful lot of problems. The panorama of these issues starts with the problem of who is authorized to assign URI to star known as "Sirius" (as it obviously can be a resource and it should have a single URI). Then it goes through a problem of completeness, as it is quite difficult to imagine that an "information resource" would capture all aspects, characteristics, feature and (potentially conflicting) viewpoints that concern a specific real-world thing. Many more problems follow and I'm sure we do not yet see most of them. I've tried to capture the obvious problems in my paper. Semantic web activity is trying to address some of these issues, but so far it seems that the result is to make the problems machine-processable and efficiently distributable to Internet scale. I have seen no real solution so far.

Therefore I have proposed to limit the definition of resource to only include so called "information resources". The information resources may indirectly refer to the real-world things and concepts, but Web in fact does not need (and cannot) deal with the real world directly.

Fortune Favors the Bold

Fri, 27 Nov 2009 16:56:47 +0100

The Web, created more that 15 years ago, has changed everything. Many people cannot imagine how they possibly could work efficiently without Web, how they would kill the time or how to find they way around. The Web, looking from a user perspective, is a huge success. However, looking under the hood uncovers the surprising truth: Web is just a technology, with all the drawbacks. The deeper one goes into understanding Web concepts, the more he is surprised that Web works.

Current web architecture seems to be reflected in two documents:

Architecture of the World Wide Web, Volume One is a W3C recommendation that tries to put all the basic principles of Web architecture into a single document. It is a must-read for any technologist that tries to create anything on the Web.
Architectural Styles and the Design of Network-based Software Architectures is a dissertation of Roy Fielding, one of the authors of HTTP 1.1 specification. REST architectural style is defined in this document with an intent to guide the development of Web architecture and protocols (although I would bet that most of the web developers have no idea that REST is an architectural style or what an architectural style is).

Both of these documents were created in a retrospective fashion: to document and formalize what was already created and working. Both documents describe ideal state and does not entirely match the reality, but that is to be expected from a high-level architectural work. However, what I would also expect from an architectural work is consistency and ability to evolve. While the Fielding's work is consistent and exact, the WWW Architecture document does not feature such qualities. The definitions are vague, there are unexplained concepts and although there is no obvious major internal inconsistency, the document contradicts other documents published by W3C Technical Architecture Group (TAG). See this paper for all the details.

It looks like the very foundation of the Web is quite weak and confusing. No wonder that Web developers create all kinds of abominations, no wonder they are confused. They don't know what is good and what is wrong, what will bring the web closer to the ideal and what will be just another nail in its coffin. It is a natural outcome that most developers choose to ignore web architecture as it is mostly useless anyway. And that is going to hurt is badly in a long run.

However, there is an important lesson to be learned. Fifteen years ago the Web was just a simple hypertext system that slowly spread across the small Internet. It had only a minimal design, almost no documentation and no protocol specifications. If the Web should have to pass an architectural review or any major project scrutiny, it would probably be canceled. The web architecture team haven't managed to get the architecture right in all these 15 years. Yet the web is successful, maybe one the most successful technologies ever. Does it mean that quality and technological merit are far less important than popularity, courage and luck?

Caveat Emptor

Fri, 16 Oct 2009 12:01:16 +0200

Now I will disclose one of the most secret of secrets of software business: All software is bad. Except maybe for very rare specimens that are even worse. It does not matter whether it is commercial or open-source, young or mature, big or small, it is bad. The quality is universally low. I'm not talking just about the bugs, but about all software qualities: performance, scalability, understandability, flexibility, visibility, reliability and security.

I cannot remember if I have seen good software in my entire career. I mean a software that was appropriate for the purpose. Software that worked as expected. Worked not only on the day one, but even 10 years later. Software that could be evolved without undermining its basic architectural principles. Software that was intuitive and easy to use, well documented, secure, ...

The reason for this situation is not technological. It is not that we software engineers are ... ehm ... idiots. We are not. We are doing our work well, considering the circumstances. The reasons are purely economical. It is just not profitable to create good software. Bad software can be very successful on the market. Quality is not high priority when making software purchasing decisions, but features are. Quality is difficult to understand and it is usually not directly visible. However features are outright visible and can be presented in in an impressive way. Quality can be usually seen only after the system survives first few years under production load. That's the point where the defects will manifest themselves, usually in a spectacular way. But at that point the software is already purchased and strongly hardwired in place.

From this point of view it is just a plain waste of money to invest in quality. Increased quality will not increase software sales and therefore not increase profits of software companies. In fact it may even harm their business: higher quality means lower motivation to purchase support services. Quality is not a competitive advantage. Spending more money on quality is a competitive disadvantage.

Therefore, caveat emptor.

Merchant's Reputation

Wed, 23 Sep 2009 23:19:52 +0200

I do shopping on-line quite frequently. It's convenient, time-saving and often cheaper. I used to be quite an non-trusting customer, opting for personal pick-up of the items and paying in cash or card on pick-up. Everything worked well, until quite recently. In last two months or so I had some problems. One local e-shop haven't even bothered to respond to my order. Another shop seemed fine, but the delivery was delayed obviously due attitude of one of the shop's employees. The problems were sorted out quickly and I hope that this specific employee is not employed any more. But the third case was the most interesting. A shop in the Luxembourg asked for a payment in advance but promised to deliver in few days. After I've paid they become silent. After a couple of weeks they sent a mail to inform me that they do not have the item I've ordered and that it could be deliver, but they need few more weeks. Unacceptable. I don't understand this. I would expect that in the times of economic crisis the shops will be struggling to attract the customers and respect them. But what I see is exact opposite. I would guess that the reason is a low entry barrier: almost anyone can open an electronic shop and pretend to be a merchant in good standing.

What I've learned is that it is better not to trust any shop too much (and not to shop in Luxembourg any more). But learning to distrust everybody is bad for business and for overall cooperation. I tend to believe that some companies should not live to see tomorrow, while others should receive much more attention that they have now. The question is which companies should die, which should flourish and how to make sure that the collateral damage is minimized during that sorting process. There are so many shops, so many customers and so many products that it is unlikely that a "word of mouth" will spread fast enough to influence future customers. Just consider for a moment what are my options to act against a dishonest merchant In my Luxembourg Case? Just walk away does not solve the problem. I was shopping there for the first time and It is not likely I will shop there again even if their service is excellent. The item I was buying was quite special - as are most of the items people buy on the Internet. I could sue that dishonest company or I could notify Luxembourg regulatory authorities, but that would cost me time and money. While I could still do it to calm my anger, it is not efficient from a rational point of view. Providing feedback should be much easier and cheaper.

We live in the information age (or at least we think so). What we need is an automated, Internet-based mechanism how to express an opinion about shops, vendors, products and similar things. I would like to express my dissatisfaction with the Luxembourg shop so my friends (and fiends of their friends) can avoid it. And I would like to express satisfaction with a shop in Slovakia that can keep up their promises, so my friends (and fiends of their friends) may prefer that shop over other similar shops. This could help to kill the weed and let the crop flourish. Especially in a hard times like these. What we need is a decent, easy-to-use and reasonably universal reputation system.

Metasystem and the Network Effect

Thu, 11 Jun 2009 12:54:22 +0200

Kim Cameron recently posted a paper "Proposal for a Common Identity Framework: A User-Centric Identity Metasystem". Although this paper is hard to read at places, it brings up some interesting points. It somehow formalizes the "Identity Metasystem" in form of a set of abstract services, which I understand as (possibly unconscious) attempt at creating a abstract architectural layer for identity services, one of the shearing layers in software architecture.

The paper suffers from inconsistent terminology and its use through the paper. If frequently fails to distinguish cyberspace entities and realspace entities. It also seems to assume a binary view of trust: something is either "in doubt" (in claims) or becomes a "fact". I consider this binary view to be one of the worst fallacies of most current identity architectures and systems.

However I believe that the worst drawback of the proposed architecture is that it does not reflect one of the most important requirement for Internet-scale distributed systems: the requirement to support positive Network Effect. If the network node can communicate with each other directly, the value of the network is proportional to the square of the number of nodes. However, if communication is limited to channels, the value of the network is significantly limited by the number of channels. The value of such channelized network grows much slower. In the identity space you can see an example of this in the PKI for qualified digital signatures. This PKI is being constructed for almost 10 years, received substantial investments both for research and implementation, it was frequently stated that there is a need for such a mechanism. And still the technology acceptance is very low. May the limitation to channel identification through the accredited certificate authority channels be one of the reasons for PKI failure?

The paper proposes or assumes similar channelization of the Identity Metasystem: the contractual agreements between claims providers. I don't believe it is feasible to consider contractual agreements between network nodes in Internet-scale systems. There are just too many connections, combinations of interactions. How many claims providers there could be on the Internet? 10? That's clear monopolization. 100? That's approximately one per country. I think that there will be much more of them. Probably one per an organization. That means millions. And that means millions of millions of contractual agreements. That's clearly unfeasible. There would need to be trusted third parties that will "bridge" gaps between organizational claims providers. And there we are back in the PKI world which haven't demonstrated any substantial success on the Internet during a decade of well-funded efforts.

I think that the primary problem is in the binary view of the "trust". The information in the cyberspace cannot be considered to be "facts", but rather an opinion of its author. No information is absolutely reliable and all the information (at least in cyberspace) is subjective. Therefore there may not be a strict need for a contract between parties, but rather a method of a proper risk management and information reliability evaluation. Any "identity" system needs to be firmly based on a "trust" infrastructure between nodes, between resources, cyberspace entities. And I don't think that a "trust" infrastructure based on a binary view would be feasible on the Internet scale.

Good Vibrations

Tue, 2 Jun 2009 19:31:02 +0200

Just a few days ago Google launched Wave. The demo is a fun to watch. The technology seems quite impressive, even in this early stage. I went through all the documents and here are my impressions.

First of all it is obvious that Google Wave is still in early development stage. The most obvious signs of that are in the area of architecture and its documentation. The terminology is inconsistent and often quite confusing:

It is not clear what is the difference between wavelet and wavelet copy. For example there is a statement "... local wavelets are those created at that provider ...". local wavelet are in fact wavelet copies, can they be "created" at a provider? Or only a wavelet can be "created" and the copy is just a side-effect of that?
Local wavelet: is it local to the client? local to provider? And it is a wavelet copy after all. It should be called "Local wavelet copy" or "provider-local wavelet copy". Or is "local wavelet" really a "wavelet" and only a remote wavelet is a wavelet copy?
What means "processing a wavelet operation"? It is changing state of wavelet? Or wavelet copy?
The "frontend" component that is mentioned in federation description is not mentioned in the "cient-server protocol" document, alhough the fereration is referencing the other document for more details on that.
It is not defined what "WSP" means (in protocol specification)
The developer's guide mentions conceptual hierarchy: wave-wavelet-blip-document. Other documents does not mention blips (almost) at all. This terminology or consistency problem needs to be cleaned up.

Some documentation sections are far from being clear. For example the sentence "In the same way a user can submit operations to a remote wavelet, namely by letting the federation proxy connect to the remote federation proxy and submit the operation to its wave server." should obviously be "... letting the federation proxy connect to the remote federation gateway ...". Or not? In previous text it is stated that proxy connects to the gateway, there is no mention of proxy connecting to proxy or gateway connecting to gateway. However the next paragraph also describes gateways connecting to each other. This needs to be cleaned up or clarified. Conceptual sequence diagrams would help a lot.

The protocol and architecture description needs more pictures. Much more pictures. I would suggest creating figures to illustrate at least following concepts:

Architecture big picture, showing all the high-level system components, illustrating their roles and interactions. This is almost mandatory to any architectural description. I'm surprised it is missing.
Some kind of deployment diagram: How wavelet store, wave server, federation proxy and gateway relate to each other? Are they under the same organizational control (site) or not? It is important to understand that federation is really a federation.
Sequence diagrams that illustrate basic communication exchanges. The single attempt on sequence diagram that I've found is not really sufficient to describe massively parallel, distributed, federated, real-time, open and insert-you-favorite-buzzword-here system.

After reading all the documents I could understand how the system works. But I was working on a system that was somehow similar and this whole concept is not new to me. However, the description is far from being easy to read. If Google's intent is to gain community support, the readability and understandability of the architecture has to significantly improve.

Apart from formal inconsistencies and difficulties to understand the architecture, there are deeper concerns. The architecture seems to be problematic in few aspects.

Google Wave architecture does not adhere to architectural best practice. It is not minimal. The robots are described to communicate with Wave by HTTP/JSONRPC (robot is server), Client apparently communicates by HTTP (as AJAX application?) , while the wave federation protocol is described as XMPP-based. Why do we need so many protocols? Is there any reason why robot protocol and client-server protocol needs to be different? The non-minimalistic approach can be seen in the OT operations as well. The antidocumentelementstart and endantidocumentelementstart operations seems redundant to me. If they are not redundant, their existence should be explained in the architectural documents.

I'm a bit afraid of Google Wave scalability. Persistent queues are used in federation gateways. This may mean too much state to maintain, too much I/O operations, too much context switches in implementation. It may scale to several hundreds of interconnected nodes, but the scalability to an Internet scale is questionable. Similar concern may apply to use of digital signatures for authenticating wavelet operations may be too expensive. Even though hash trees are used, I wonder how this could scale with millions of users writing in real time. If would be nice to have empirical data on scalability of these mechanisms before going on with the prototype, especially considering that these mechanisms determine some of the basic properties of the system and the protocols.

The documents does not mention failure cases. While designing an distributed system of this scale, the failure cases are as important as positive use cases. How will the system be affected if one of the wavelet-hosting servers will not be available? What happens if master server for a wave goes down? And can the system reliably work on Internet links with quite high latencies and low reliability?

There is a question of trust infrastructure. The trust infrastructure is not considered in the Google Documents or in the paper draft by Kissner and Laurie. The XMPP specification (RFC 3920) also pushes the trust infrastructure outside of the specification scope. I can feel that TLS/X.509/DNS combination is somehow (almost silently) assumed. But for Wave to be used as an ubiquitous system, such infrastructure must exists and be universally available. Will CAs offer Wave (XMPP) certificates? What CAs will Google accept? Cannot that lead to monopolization? How much will such certificate cost? Will not that be kind of a ransom that a site must pay to be able to participate?

Wave is changing paradigms. People can no longer take back what is released. Even if someone deletes part of the document, the deleted part can be seen in playback. While this "permanent memory" was there almost since the beginning of the Internet, it was never before real-time. How could we take back an information from a Wave? Imagine you have misplaced your password to the wave instead of password input box. It will always be visible. OK, I could change my password, but what about unfortunate copy&paste event with a credit card number?

But the worst architectural deficiencies of Google Wave go even deeper: Wave is not aligned with WWW architecture and the specific nature of user identities is not considered.

Let's for a while abstract from all the deficiencies of WWW architecture itself and let's agree that, for better or worse, the WWW architecture is still useful. According the the WWW architecture agents should provide URIs as identifiers for resources. Waves, wavelets, blips and documents can definitely be regarded as resources, however Wave architecture does not assign URIs to them. Wave specification uses QNames (XML namespaces) a lot, however it does not provide QName to URI mapping as it should. Some problems of Wave architecture are caused by XMPP, such as violation of URI opacity and URI reuse. The very nature of Wave goes against REST. REST assumes stateless interactions, while Wave is inherently stateful. I don't blame Wave for all those problems. WWW architecture, XMPP and REST can be guilty as well (as they are). But I would expect discussion of these problems in the Wave architectural description and reasoning behind the decision that were taken while designing Wave.

Wave does very little to consider user identities. The demo seems to use only a simple drag&drop from contact list. But how will these contact lists get constructed and maintained? All the documents seem to assume that the use of email-like identifier for users. Will this be global? With all the unfortunate consequences? Could Wave avoid linking user activities at different sites? Does it support pseudonyms, pair-wise identifiers, user privacy controls, anonymous groups, or anything that can support user control over their personal data? How does Wave plans to defend against spam and phishing? Or do they expect that this will not apply to Wave? That would be really naïve.

My last concern is about Wave maintainability. It is only half of the success to create a system - the second half is to keep it operational and efficient. How could Wave handle change of domain names of participants? Would they loose all old waves and drop off their friend's contact lists? Domain names are human-readable, and they do change occasionally. Can Wave handle changes in network topology? For example moving wave servers here and there without service interruptions? Merging two servers or splitting a server to several boxes? To several organizations? Mergers, acquisitions and re-orgs happen all the time ...

Even though I must be making an impression that I do not like Wave, that's not true. Quite the contrary. I was having fun watching the demo, I like the idea and I think that the overall concept is good. However creating useful, reliable, real-time and Internet-scale system is really a major challenge. The Wave team is obviously up to that challenge. But there is still a long way to go. I would conclude that it is critical for basic architectural concepts of Wave to be sorted out as soon as possible, especially the alignment with Web architecture and concerns related to user identities. Wave technology is undoubtedly attractive and it will be most probably very successful. However it can be really harmful for the whole Internet if the Wave would be deployed in this form, with all the architectural deficiencies. If that would happen, the whole system will fall apart in few years or decades and it will block further innovation in this area.

Machine-Centric Identity

Mon, 18 May 2009 18:22:49 +0200

Users do not authenticate themselves to web applications. Users pass their password to their workstation (or any terminal device) that will pass it to the browser which will in turn pass it to the web application. The application does not interact with the user, it interacts with a browser software installed on a machine that is (maybe) used by human being. I have described that long ago in a persona model. Now Fulup Ar Foll mentions similar thing in this quite interesting interview. That's good. At least one more person is having the same idea. Maybe we are getting somewhere.

You may ask what is the difference? Authenticating user or user's computer or whatever? The user cannot be sure that his computer or mobile device is operating as expected. And well, let's admit that many people have no idea what that damed machine is doing. It is easy to make a mistake and send your password to a wrong site (phishing). It is difficult to defend against viruses. And we are damned lucky that vast majority of the viruses are pretty harmless things. Computer or mobile device can be stolen, ale your persona may be stolen as well. Just admit it, the device you are using to interact with the cyberspace is just not secure.

More and more important services are getting on-line. They assume that the entity that is providing credentials is really a human. But it may not be the case. It is much safer to think in terms of "digital persona" than "living person" when it comes to the design of authentication systems.

And that also means that the entire field that proudly calls itself "user-centric identity" is in fact a bunch of machine-centric solution.

Phish Pharm

Mon, 6 Apr 2009 18:29:54 +0200

During last few months there were quite simple but very effective phishing attempts at twitter, deviantART and maybe also other social sites. The phishers spoofed login page of twitter (or deviantART). They sent out a catchy messages along the lines of "funny blog about you" or "I have seen your photos on this blog". These messages contained link to the spoofed login page that phished user passwords. Phished credentials were used to spread the message further, creating an avalanche effect.

This approach was efficient because the messages were apparently coming from trusted friends. Why would a good of friend of mine send me to a phishing page? Because he was phished just a minute before! Simple, but efficient. My estimate is that a significant portion of users that were on-line fell in this trap. I was on-line on deviantART when this happened. As far as I know nobody of the people that I watch detected that the login page was spoofed. They have only detected the consequences: that someone is sending out strange messages using their account.

I have immediately noticed that the login page was spoofed. The spoofed page was slightly different than the normal login page. But the primary reason that I was alerted was that the page was out of context. I should be seeing a blog that stole my photos, not a login page. I haven't logged out and the session should not expire in such a short time. An immediate look at the URL bar confirmed my suspicion.

This specific phishing attempt was not very sophisticated. But if the phisher tries a more subtle tactics next time, I may become a victim as well. Such a tactic will probably display a login page in the correct context, where it is expected. Even the most cautious users may automatically enter the credentials without any suspicion (you just cannot watch URL bar all the time). Now it is quite difficult to construct a phishing message that would direct you to a login page in the right context. The correct context for entering a password is at the start of interaction with the site. But the world is changing ...

Using OAuth it is usual to enter your username and password almost any time during the interaction with a site. OAuth will normally redirect user to your trusted site that stores his data - to request an authorization to use the data. But that usually results in asking user to log in. It may be quite easy for a phisher to simulate that flow and to spoof the login page of the legitimate page. OAuth in fact trains users how to get phished easily by making the normal use case similar to the phishing case.

This problem is amplified when OAuth is combined with OpenID. It is not a big deal if a twitter password gets phished. But if a key to the OpenID kingdom is phished, that may be entirely different story. OAuth-OpenID combination is increasing the exposed surface as well as impact: there are more places that user may be phished and the phished credentials are more valuable.

I can understand that neither OAuth nor OpenID created the primary problem. But the use of OAuth and OpenID is amplifying existing problems. Therefore instead of increasing the privacy and security of users the use of these mechanisms may if fact have exactly the opposite effect.