I've considered the specific topic for gl alot of times before and with io usermodules its actually a possibility. One of the major problems with the distributed approach is how to resolve collisions and ordering. Ie, does the order of updates matter? This is central cause if they do, you need numbered transactions. And each server numbering their own transactions syncronization becomes a real issue and prevents lazy updates which are essential for efficiency.
So how important is it for a users account to be totally updated on all servers all the time? Is there an acceptable error margin here?
For situations where complete accuracy is required I've thought of two approaches. Both build on transactions and ordering (and thus, numbering) them.
The first is to basically emulate token ring or the more advanced version, fddi. To simplify greatly, every part is connected in a ring. In this ring you pass a token around and around and around. When you have the token and it is empty, you may pass data with it. When its not, you simply forward it.
Despite the relative simplicity of token ring, it works and its quite speedy. The disadvantage is when the ring breaks or the token is lost. I wont delve more into it as I think a ring-based solution will be too complicated.
The second, and viable solution I think is to combine the push/pull solution.
At any given time, one server in the ring is the update server. Each change is sent to the update server, who packages it and numbers it. That change is then either pushed to other servers or pulled from the update server. If/when the update servers go down, an election is forced from some criteria and the winning server takes over as update server. One of the criteria could be the server with the highest update number.
Disadvantage? For an election to be forced, the servers need to be aware of eachother.
For efficiency, each site prolly needs its own local db. No problem since mysql or any free db would work well.
These are just some ideas, please dissect them and flame them.
