Storm Alert

Open Source Identity Management Systems

Tue, 10 Jan 2012 12:08:08 +0100

I have compiled a list and an evaluation of open source identity management systems. This document will be continually updated as the systems evolve.

Sapienter resistere

Fri, 9 Dec 2011 23:32:59 +0100

R.I.P.

Interview

Thu, 1 Dec 2011 10:59:22 +0100

There's an interview with me regarding midPoint at Luca Mayer's blog.

MidPoint

Mon, 24 Oct 2011 09:58:13 +0200

It is time to publicly announce a project that I was working on recently. It is an open-source identity management system named midPoint. It is based on the OpenIDM version 1 code developed by the our part of the team. MidPoint aims to be a usable, pragmatic IDM product. We have based it on many years of experience deploying other IDM products. We have learned from what worked and what have been failing during real deployments. The bad thing is that too many things failed - and that's something we want to improve with midPoint.

MidPoint is a user provisioning tool. It can do basic provisioning as well as provisioning driven be expressions. It is using Identity Connector Framework (ICF) to connect to other systems. It has a live synchronization capability (similar to Sun ActiveSync) and other synchronization methods are under development. There is a basic RBAC support that is continually improving. Lot of time and effort was invested into diagnostics and support for deployment such as good error reporting and logging. We know where are the pain points of IDM deployments and we are working hard to improve what we can.

MidPoint version 1.9 was released few days ago. It is a third version developed under the Evolveum brand name. This version is worth checking out as a preview of the final product that is planned as version 2.0 for early next year.

Temperaments

Fri, 8 Jul 2011 22:28:40 +0200

I was a very young student when I came across a book named Programátorské poklesky (Programmer's Misdemeanours) by Ivan Kopeček and Jan Kučera. The authors describe in a humorous way what are the results of programming errors. It was probably my very first book about programming that was not a programming language manual. It was a year after our country woke up from the communist era and programming books were difficult to come by. I think the book had influenced me more than I have anticipated or was willing to admit at the time.

One of the parts that I particularly remember was the software "psychology". Authors observed four temperaments of programs:

Sanguine programs provide readable and helpful error messages, have useful help texts, try to recover from errors and try to communicate reasonably in general. Yet, user interaction is maybe the only useful part of such programs.
Choleric programs does its job well. Such programs do not crash, but the error messages are very dense and cryptic. They do not provide any additional information and there is no help text. It does not try to recover from errors - it expects that the user will know what to do. Experts find these programs easy to use, but all other people hate it.
Melancholic programs get very sad when they encounter the smallest of problems. The program just crashes, does not provide any message or description. They refuse to communicate about the problem any further and usually does not even provide a way to resolve it.
Phlegmatic programs ignore any errors. They just carry no matter the cost. No error message, no indication, it just works on. Of course they may provide wrong results from time to time, but they run. That's the most important thing.

All of that came to my mind as I was discussing the error handling approach in mainstream programming languages (mostly Java). It usually boils down to handling exceptions.

The original approach in Java was to use checked exceptions. Programmer has to either catch them or declare them to be thrown. The authors of Java hoped that it will lead to a better error handling. But it looks like there is a glitch: error handling is very difficult to do right. It takes a lot of time and the error handling code may well be a significant part of the system. This leads to sanguine programs: they provide good information about errors, but they do little else. There is just not enough time and resources to do everything right.

Laziness is one of the three great virtues of the programmer. Therefore programmers soon stared to focus on the "meat" and simplified the exception handling. The easiest way at hand was to ignore all the exceptions. Catch all exceptions and handle them with empty code block. This obviously leads to a phlegmatic program. It will run no matter what happens. But the results may not be the best.

The current trend is to switch all the exceptions to the runtime exceptions. These do not enforce checking and handling. The usual outcome is that nobody checks or handles them. Any exception will bubble up through the call stack to the upper layers until is is caught by the framework. That may be an application server that will display a nicely formatted error message that essentially says "something somewhere went wrong" and terminate the request. The user has no idea what went wrong and where or how to recover from the problem. This is a melancholic program.

Luckily, some programmers display at least the exception type and message to the user. But what will the user do if presented with the message "ConsistencyException: Consistency constraint violated"? It is not really helpful. Most programmers also display or log a complete stack trace. But that won't help the user a bit. Even members of a core programming team have problems understanding that, user does not stand a chance. That gives us a choleric program.

Obviously, one size does not fit all. There is no single right way to do it. If a good error handling is required then a sanguine approach is needed. But there is a cost to pay: either reduced functionality or much more effort to do the "same" thing. Robust system asks for somehow phlegmatic approach while cheap code is best done melancholic. However, the usual approach is choleric code. Errors are reported, but nobody really understands them. You just can't always win.

Cycles

Fri, 3 Jun 2011 23:21:44 +0200

I'm still quite young and my "professional memory" does not even count two decades. But I just cannot help to see some recurring patterns. Quite a scary patterns.

I was a student when Sun RPC was the cool thing. It has all that a C programmer needed at that time to create a distributed system. But obviously it was too simple.

CORBA was taking the place of the "cool thing" as I was finishing university. It had all that a C++ programmer may wish for to create a distributed system. Interfaces, object-orientation, "interoperable" references, ... But it was obviously too complicated to use.

XML took over during the dot-com bubble. Or better to say it was XML-RPC as a mechanism for Internet-scale distributed systems. It has all that PHP programmer would want. It had the "feature" of seamlessly passing firewalls. It was the cool thing for the Internet. But obviously, it was too simple.

SOAP came shortly after that. The mechanism by which Java and .NET architectures promised to bridge enterprise and the Internet. Originally designed as simple thing to do something with objects. It ended up as a maze of WS-WhatEver specifications that are far from being simple and actually have nothing to do with objects. This is obviously too complex to use.

RESTful religion is the current trend with JSON as its holy prophet, worshiped by the scripting crowd. It is based on an idealistic and internally inconsistent principles of Web Architecture with a loud promise of simplicity. But obviously, this is too simple to be practical.

Now we see JSON schema, namespaces, security and actually all the things that we have already seen in SOAP/WS-* and CORBA. I expect we will see a formal RESTful interface defintions soon. Will this be too complex to use, again?

What we see are cycles. Each new generation of engineers is re-inventing what the previous generation has invented, making all the mistakes all over again. Can this eventually converge? How long are the customers going to tolerate this? And what we really know about distributed systems?

Sorry guys. I just refuse to participate in this insanity.

Why SOA Fails?

Tue, 26 Apr 2011 17:15:25 +0200

Service Oriented Architecture is not a bad idea. Quite the contrary. What is bad about it is its way of implementation and an unbelievable hype.

Usual SOA implementation starts with purchase of an "infrastructure", which is usually some combination of ESB and process manager (usually BPEL-based). The next step is an attempt to connect existing services to ESB and "orchestrate" them using BPEL. While attempting to connect the very first real service it becomes quite clear that this is much harder that one would expect from product datasheets. Existing services - even though they are based on "web services" - are not quite ready for SOA. They need to be tweaked, schemas need to be modified to conform to the requirements of the ESB, new headers need to be added and so on. This usually result in wrapping existing services with yet another service layer to connect them to "SOA". But the ridiculous part just begins here. Now the services needs to be "orchestrated", e.g. the result of one service needs to be passed as an input to another service. But they have incompatible data formats. Now the deployment usually takes two different courses. First is an attempt to abuse BPEL to transform data formats. The result is unreadable, complex and unmaintainable "integration" mess expressed in BPEL. Second usual course is to create a special "conversion" service that just transforms data formats. This results in explosion of services: for each pair of cooperating services that is a new "adaption" service. Complex and unmaintainable integration mess again. Even if this task miraculously succeeds, there is another major problem ahead. The original services were neither independent nor idempotent, they usually have lot of (undocumented) side effects. Therefore they just cannot be freely reused and recombined into business processes. So the next necessary step is to re-engineer all the services (while maintaining backwards compatibility, of course). This is very costly and slow process, but until it is done the benefits of SOA are more than questionable.

What is the real benefit of such "SOA" deployment? I can't think of any. The mess is still there. It may seem that it is just in one place and therefore is easier to maintain. But that's just an illusion. There are service wrappers an transformation services that are usually not in one "place". And even if it is, it is so difficult to navigate that any real benefit of centralization is lost.

What are the downsides of such "SOA" deployment? First of all there is an investment to purchase SOA "infrastructure" and to set it up. Secondly, there is an investment to convert existing non-SOA integration mess to a SOA integration mess. Thirdly, the "infrastructure" is yet another moving part that can fail. It becomes a business-critical piece and needs additional (substantial) cost to set up high availability and resilience. It also takes a lot of energy and attention that could be invested in much better way, it disrupts usual business and builds a false of hope of better future. And I haven't even mentioned "advanced" problems such as synchronicity and consistency. Clearly and plainly, such SOA deployment is a waste of time. Waste of huge amount of time.

How to make it better? Just remember what SOA is: Service-Oriented Architecture. The focus should be on services not the infrastructure. The better way of SOA deployment is to start from there. Try to assess what services are already there, how well they are defined, whether they can be reused and whether they actually need to be reused at all. Every experienced developer knows that reusability does not come for free. It is actually quite costly quality. Therefore focus on services that need to be reused and re-engineer them to be reusable. This is best done as part of natural system upgrades and replacements. Try to gradually develop a common data model for your business so there will be less requirements to convert data from one service dialect to another. Some services may need more than one "upgrade" to get into a reusable form. This is not a fast process, so the number of services in SOA will grow very slowly. At the beginning of such SOA initiative the easiest way of "orchestrating" the services is to use any way that you are familiar with, e.g. a simple Java web application. Just make sure you can modify the orchestration code quite easily, as many adjustments will be needed as the things slowly settle down. Having an internal employee or a very flexible partner to do that job would be probably a good choice. As the number of services will be initially low and ability to reuse them will be quite limited, such "orchestration" code will be acceptably maintainable even if some things are hardcoded. This may be a good solution for first few years. Once the number of services grows then it is a good time to think about ESB and BPEL (or alternative technologies that most likely will be available in the future). At that time there will already be a considerable number of services therefore the cost of infrastructure could be easily justified. This service-oriented process to SOA deployment will be less expensive, less disruptive and will continually bring benefits proportional to investments.

Service-Oriented Architecture is nothing new. It is just an ordinary architecture. The architect works with systems instead of components. The architect works with services instead of component interfaces. Apart from that, it is still just an software architecture. Vast majority of principles and experiences applicable to intra-system architecture are reusable also for extra-system SOA. And that is probably the most important part of Service-Oriented Architecture.

The Big Picture

Tue, 19 Apr 2011 11:11:35 +0200

I'm frequently exploring and evaluating new products. It is part of my job. The information overload is huge and it is important to know where not to look, how to quickly rule out products that are not worth looking at. I'm also reviewing architectures and design, consulting, commenting, advising and maintaining architectures. During the years I've noticed that it is quite easy to roughly evaluate a system by just looking at few places.

The first place to look is a web page section called Architecture or System Overview. If there is no such page or document, you can be pretty sure that the system is using a popular and well-established big ball of mud architectural pattern. Looking anywhere else is just a waste of time. The option you have is to try the system for yourself or look at the sources. But that's very time-consuming and usually the result is not worth the effort. Scratch such system.

If you have found the architectural description, it is usually not worth start reading yet. Just skip ahead and look at the first figure in the document. It usually looks like the "diagram" below. What does this creative depiction tells about the system? Well, it is a three-layer system. Or at least that's what the architect meant. There is a bunch of stuff in the middle layer, but the picture does not provide any information about the structure inside. Dependencies are not shown so system maintainability is unknown. Interfaces are not even hinted, so there is probably no interface at all or there is a jungle of competing and redundant interfaces. The system structure may not be set yet. Or maybe the architect does not understand the system to draw a better picture. Or maybe the team is afraid to show the structure to the public. None of that is a good sign. The arguments that "we don't want to clutter the picture with too much details" is usually just an excuse. Reading through the text is mostly a waste of time as it will most likely be just a marketing-oriented nonsense (MON). If there is no better figure anywhere nearby the best strategy is to ... run away.

If there is a picture similar to the following one, you are almost there. The system is a good candidate for further exploration. Such picture gives a reasonable level of details and indicates that the architect has quite a clear idea about the system structure. It is not just the form itself, it is the content of the diagram that you should focus on. Look at the figure below. You can see that there is a Repository component. Two data stores below indicates that the component is supposed to act as an abstraction for various data storage mechanisms. Although it is not shown in the picture, you can pretty much expect an interface on top of the Repository component (and other components as well). Similarly with Integration component. You can also see that similar approach is used for two subsystems (Repository and Integration) unified by a common Model component. User interface is placed on top of Model, which clearly isolates it from the low-level details. The most important dependencies are shown in the diagram which provides some hints about maintainability of such architecture. You can get quite a good idea how such system works by just looking at the diagram.

Now it is the right time to read through the text and other documents. Look especially for explanation of motives, not just the structure. For example, look for an explanation of reasons that the Repository and Integration components are not unified into a single component. That gives confidence that the team actually though over several alternatives before committing to current architecture. Look for links to papers, books and other sources. That gives a hint that the team spent some time "in the library" instead of trying to save time by blindly re-inventing things in the laboratory. Especially look for buzzwords. If find a buzzword used without a deeper explanation of motives it is a serious warning sign that the architecture may be buzzword-driven and therefore not sound. Look also at hints that the architecture was done all the way down. That means that the architect thought about deployment and usage of the system. Presence of an deployment diagram or a typical usage scenario is a good sign that this happened.

It is a curious thing how much can be learned from a single picture. The picture is really worth a thousand words.

Hub and Spoke Myth

Wed, 13 Apr 2011 23:26:04 +0200

This diagram can be found in almost any marketing document dealing with integration problems. It illustrates a concept that is known as hub and spoke. The difficult O(n²) problem of full mesh is reduced to a simpler O(n) problem by introducing a central hub. That's what the marketing guys say (although they don't usually use the O() notation).

What the marketing guys don't say is that this approach usually does not work. The most hubs are just simple message routers. This does not simplify anything. The hub just passes the message from sender to receiver. Oh yes, the hub may use abstract addresses instead of concrete ones, but that's yet another indirection thing that can be easily done without hub. Oh yes, the hub may do some basic protocol adaptation, but a well-chosen data representation library will do essentially the same thing. That won't justify the cost of the hub. And what's really the cost? Except for pretty big pile of money to pay there is an impact on systemic qualities. The hub is inherently a single point of failure. The hub introduces latencies. Hub can cause additional problems as it is yet another moving part in the system. And the communication with the hub is making the code more difficult to understand (just have a look at JMS).

This is a typical anti-pattern for many SOA-motivated deployments. Enterprise Service Bus (ESB) is usually the first component that gets deployed in SOA initiative. But it initially brings no substantial benefit, as there are no services to put on the bus. And if there are services, they are not independent and definitely not idempotent. Such services just cannot be efficiently reused in any other way that it was before the hub came into a play. In fact, the ESB should be among the last components that get deployed in SOA initiative, not the first. SOA is about the services, not about the bus.

Yet, there are few cases when the hub-and-spoke approach works:

Asynchronous system: The hub is used to break the timing dependencies of the communicating systems. Such as in well-designed systems based on Messaging-Oriented Middleware, sending system does not need to wait for receiver to process the message. But such system needs to be designed for asynchronous operation, which is usually much more difficult to do than simple synchronous system. Also, the hub needs to be up all the time.
Visibility: The hub is used to audit messages passing among the systems. But the hub needs to understand the protocol quite deeply to be of any real use.
Common data model: Used to merge data from disparate data sources, converting each of them to and from a common format. The hub forms a "common denominator" communication interface. But, the interface needs to be designed very carefully, as it needs to satisfy the requirements of many communicating parties. Maintenance of such an interface may be in itself a much more difficult problem than the original full mesh thing.

Yet, most of these are very difficult to do right. And if done wrong, they may bring much more pain than benefits. Handle with care. The hub is a dragon egg and the spikes are venomous snakes.

OpenIDM Design Summit 2011

Sun, 16 Jan 2011 12:37:49 +0100

The OpenIDM project is getting up to speed pretty fast. We have done a lot of work recently, but there is even more work to do. Although OpenIDM is led by ForgeRock it is an open-source project as it should be. The project is open to the community from the very beginning, but now it is a time to encourage broader community to participate.

The OpenIDM Design Summit 2011 is a place for a design discussions, place to ask questions and get answers. It a place to participate. I will be glad to see all of you in Oslo in few weeks (January 26-27).

The Misery of Distributed Systems

Wed, 12 Jan 2011 23:16:39 +0100

The more I learn about distributed systems, the more I start to think that nobody has any idea how to make them well. It is unbelievable how little we know about distributed systems design and development. And how many failures do we still need to suffer to learn at least something?

As far back as I remember, there was Sun RPC, CORBA, DCOM, RMI, ... And now there are Web Services and REST. This is at least 20 years of development, yet all of that approaches seems to manifest the same fallacy: they try to hide the network from the developer. I will use Java JAX-WS as an example, but this is in no way specific to Java. The JAX-WS provides a runtime for a web service and generates a web service client. Both are designed to use local Java calls, efficiently hiding the network boundary. It goes a great deal for a programmer to feel comfortable. For example it hides the network exceptions from the programmer (by making them runtime exceptions). This may seem like a good approach, but it is a bad think in the very principle.

Most people would be surprised how applicable is the theory of relativity to a design of software systems. And most developers would be really surprised how slow the light is. The light will travel only approx. 300km in one millisecond. If the system needs a response under 1 millisecond, it just cannot communicate with anything that is more distant than 150km. Add a TCP three-way handshake and you are pretty much down to a size of a city. Assuming Einstein was right there is nothing, absolutely nothing, that could be done about this. No amount of technology or money can speed it up.

One millisecond is unbelievably long time for a computer system. Even a cheap computer can execute more than a million instructions in 1 millisecond. Local memory access is a bit slower, but even with that slow-down the local call is incomparably faster than a call over a fast network. Faster by several orders of magnitude. How could anyone hope to hide such a difference?

Reliability is another problem. While local call cannot (reasonably) fail, network call can and often does. Hiding the errors from the engineer does a disservice. It usually means that the network problems are ignored or handled at the top-level of an application. It means that any serious error in network communication means a sudden death of the application. Applications that are written a little bit better still manifest serious usability problems in presence of network failures. I can see that well enough on my iPhone.

Network is a significant boundary. A boundary that just cannot be swept under the carpet of a software development framework. Anyone trying to do that will most likely fail. Fail miserably.

Giant Leap

Wed, 27 Oct 2010 22:02:41 +0200

A giant leap have happened today, even that it may look like a small step from the outside. OpenIDM Snapshot 1.5 was released today. OpenIDM is an open-source provisioning system. The source code was fully available from the very beginning, but this is the first real release of the product, although it is only considered a snapshot, a preview release. I'm very proud to have a chance to participate on that.

OpenIDM does not do much if seen from the outside. It does very basic provisioning and few other minor things. The real goodness is under the hood. OpenIDM is based on component architecture. It is in fact just a bunch of components that can be combined in multiple ways. The components can be modified, inserted or replaced with custom solutions. OpenIDM is based on ESB/JBI principles, therefore the components may take a various forms, including Java, BPEL, XSLT and others. Yes, BPEL is a native part of OpenIDM from the very beginning rather than being some kind of afterthought. The inter-component interfaces are defined using WSDL, so the implementation language does not matter that much. And it is "contract-first" WSDL, manually written and carefully designed. It is not generated by some kind of tool that almost works. OpenIDM takes the architecture seriously. Take a look yourself.

OpenIDM has Identity Connector Framework as a workhorse. So even though OpenIDM is a new thing, there is already a decent set of connectors that can be used with it. The tooling is provided by Netbeans SOA plugins now. There's composite application editor, graphical BPEL editor and debugger. And much more than that - a complete environment that a serious developer would expect. And honestly - IDM projects are heavily customized, good development tools are really critical in IDM field.

I believe that OpenIDM is starting a new generation of IDM systems. That's definitely one of the project goals. We have made a first leap today. I would like to thank a lot everybody that was a part of it. And I'm already looking forward to the next steps. It will happen ... soon!

These are just my personal opinions as an engineer. It is not an official statement of ForgeRock, nLight or any other entity except myself.

Facedown

Fri, 24 Sep 2010 10:21:33 +0200

During last few days Facebook went down and up and down and up again. It was not nice, but it was also not unexpected.

It was not nice because I suddenly haven't had a place where I could post my usual sarcastic remark (Twitter and all the rest just don't matter any more). That was painful.

It was not unexpected because Facebook is still just a technology (even though it sometimes looks like magic). Technology fails.

Facebook changed the communication paradigm. Except for private(*) point-to-point communication there are options for semi-private "multicast" status updates, groups, events, etc. I believe that the communication paradigm that Facebook popularized is a good one. However, Facebook failure disrupted communication of too many people at the same time. That failure is a good demonstration that Facebook architeture is wrong. Totally wrong. Because it is centralized, both from technological and from organizational point of view.

Centralization means that one party has a control all over the place. They can see everything, change anything, even rewrite history (that actually almost happened with terms of service and privacy settings). Think nineteen eighty-four.

Centralization means single point of failure. Good engineers know that this should be avoided, especially in Internet scale systems.

Centralization has an economic impact as well. Who will pay for all these boxes (and electricity, cooling, space, staff, ...) that are needed to handle a communication of more than a billion people?

The future of Facebook in its current form is more than questionable. Yet very few people can see it. I would not recommend to buy Facebook shares.

*) If a communication that must be shared with a strange multi-national corporation can be considered private.

Role Explosion

Tue, 21 Sep 2010 22:06:49 +0200

Everybody knows RBAC, the Role-Based Access Control. The principle is simple, the theory behind it seems to be sound and it is very popular. The problem is that is does not scale. RBAC is fine for few tens of roles. But as the number of connected systems grow, the number of roles grows as well. Organization with thousand employees can easily end up with few of thousands of roles. The difficult problem of managing thousand employees will be transformed to even more difficult problem of managing few thousands of roles.

The reasons are quite understandable but they are far from being obvious:

Cartesian product: Let's have roles for bank teller, supervisor and branch director. Let's operate in London, Berlin and Bratislava. We need a role for "teller in London", "teller in Berlin", ... 9 roles total. The number of roles grows very quickly.
Atomization: This can be sometimes avoided by decomposing roles to re-usable re-combinable units. Combine them to a "higher-order" roles using role hierarchy. This gives us 6 roles instead of 9. Unless the teller shares part of the access rights with janitor. Then we need "basic branch office access" role, "teller's IT access" role and a "teller" role that combines these two. Now we have 8 roles. The number of roles still grows quite quickly.
Uniqueness: Many employees have one-of-a-kind set of access rights. Any attempt to fit them into the roles will inevitably result in a number of roles that is equal or greater than the number of such employees. This is quite pointless.

This is known as role explosion. It is fatal for IDM projects. A project that started with good intention to simplify user management will end with a role structure that is much more difficult to maintain then before.

Static RBAC model usually cannot be used to efficiently handle role explosion. There are some solutions, but none of it is a panacea:

ABAC - Attribute-Based Access Control. This seems to be a new buzzword. The principle is that instead of having roles there is a expression that computes access rights. The expression takes user attributes as parameters. It seems to be very flexible. But it is a new concept and has yet to be proven. Also, it replaces formal structure of roles with logic. That smells of programming instead of configuration. ABAC may solve the problem of role management, but does it also solve the generic problem of policy complexity and management?
RBAC, but forget about least privilege. Create "bigger" roles than what would the principle of least privilege dictate. Roles that allow much more than they should. This will improve role re-use and keep the number of roles manageable. It is less secure, but may be good for some environments.
Spice up roles with some logic. That usually means adding rules to make roles more generic without sacrificing security. E.g. instead of creating "teller in London" create a parametric role "teller" that takes locality from user profile as a parameter and computes the path to the home directory. This is somewhere between ABAC and static RBAC. It has good features from both approaches and seems to be the most popular choice for provisioning systems. But does it really solves the problems? Nobody seems to know for sure.
Manage the chaos. Let user's request access rights, let others to approve it. The obvious problem is how to revoke the unneeded rights. There needs to be yet another process to review and revoke them, usually called attestation. It is a good for CYA strategy, as there is always someone who have approved what he shouldn't. But is it really secure? The approvals are often done by "I just look at it and if it seems OK I approve". Especially if someone has to approve hundreds of requests each day.

Practical enterprise IDM solution will most likely need all of these mechanisms, not just one. And especially a good, experienced team of people using these mechanism. Because every IDM deployment is different and one size does not fits all.

Identity Garbage

Mon, 23 Aug 2010 23:09:33 +0200

Recent discussions are about whether push or pull is the right model for future identity management. Unpractical standards are being revived. Everybody discussing the technology, the future, the visions. There is almost no discussion about the most difficult current problem of identity management: data.

There is (at least) one critical problem with implementation of single sign-on, identity federation, provisioning to the cloud and other fancy buzzwords. The problem is user database. It is not that difficult to deliver the information that someone should have access somewhere using whatever push, pull, standard or proprietary method - as long as you have that information. The reality is that enterprises does not have that information in a usable form. It is almost always distributed in several data stores, usually provided in incompatible formats, it is often inconsistent and sometimes even contradictory. And it is far from complete. Many provisioning cases are solved by non-algorithmic methods, e.g. manager or security officer deciding whether the request "looks valid enough" to be approved. The current situation is best described as semi-organized chaos.

How could anyone build an automated, Internet-scale, cloud-enabled and standards-based identity management mechanism on top of that? Hardly. Such project will most likely fail. But it will waste a lot of time and money before it fails.

The first step is to consolidate the data. Build a consistent user database, align policies, design business processes that can support 80% of cases with 20% of effort. It is naïve to expect that everything could be automated, therefore prepare for a reasonable amount of exceptions and human interactions from the very beginning. Single sign-on, identity federation, support for the cloud (whether push or pull) and even the standards will not provide any considerable help in that. It is mostly manual work of security staff, business people and engineers that is needed.

What can help is a well-designed and well-deployed provisioning system. In contrary to the popular beliefs the provisioning system is not really about provisioning. Yes, provisioning is a important part of the system, but other aspects are in fact much more important. Provisioning system can take data from several sources, covert them to a common format and merge them. Therefore it can create a unified database. Provisioning system can compare data among several system, correlating them, therefore detecting the inconsistencies. Provisioning system supports workflow and human interaction to clean up the data and supplement missing information. Both during initial migration and (most importantly) during the day-to-day operation.

Reasonable identity consolidation project including a decent provisioning system is a necessary pre-requisite for any other identity-related activity. It is a shame that engineers forget the Garbage In, Garbage Out phrase that was popular few decades ago. If the data are bad, any system built on top of such data can only be worse.