The reason why Microsoft became a successful empire is not because their software was superior. Neither MS-DOS nor the x86 processors from Intel were better than comparable products. The x86 processor architecture is indeed often considered as ugly. But they were cheap and widespread. Compatibility was the key. PCs with MS-DOS were business standard. They were good enough to run simple word processing and spreadsheet software. Software written for MS-DOS would run on any MS-DOS computer. A lock-in effect with a positive feedback loop set it: people wrote software for PCs because PC sold well and were widely distributed in the business world, and people bought in turn PCs because there were at lot of software available for them. Soon everybody in the business world was using PCs, and the old DEC empire started to crumble. Microsoft used the new market power to gain a competitive advantage in the world of windows systems. Again compatibility was the key. How many people remember the OS/2 operating system from IBM or VAX/VMS from DEC today? All commercial competitors disappeared until only Microsoft was left with Windows. Linux was able to survive in the open-source corner, a niche that is hard to tackle even for large corporations. But it was no serious opponent in the world of window systems.

This has changed. There are 750 million Android devices today. Times in the IT industry change fast. Now apparently the Microsoft empire starts to decay (or at best to stagnate). The very pillars which made Microsoft successful begin to crumble. The new Windows 8 system is no longer compatible to the classic world of Microsoft Windows software. There is no longer a central desktop where Windows applications would run. There is a desktop, but it is hidden behind a new interface. As you know Windows 8 comes with a new colorful surface named “Metro”, which is intended to replace the desktop. Microsoft wants people to use the new “Metro” interface instead of the classic desktop, and wants to people to download apps from their app store, similar to Apple’s app store, or Google Play (the former Android Market). Apparently Microsoft tries to keep pace with their competitors. Unfortunately they seem to damage the very pillar they are built on: compatibility.

Using old Windows software on a new Windows 8 system is a hassle. Older versions of windows programs for instance use often a help in the Windows Help format. This format is no longer supported in Windows 8. Just try to enable the legacy windows help system winhlp32 on windows 8. It is annoying. If you start an old applications which uses Windows Help, then you might get the following message:  “The Help for this program was created in Windows Help format, which depends on a feature that isn’t included in this version of Windows. However, you can download a program that will allow you to view Help created in the Windows Help format.” If you do this, and follow the official links, then you will get a link to an update of the help system, and if you try to install this update, then an error message occurs which claims “the update is not applicable to this computer”. Great. It is possible to get it working, it is just difficult. There is in fact a non-functional stub of WinHlp32.exe in Windows 8, which shows the above message that the help does not work. It is possible to replace the WinHlp32 file, but the “TrustedInstaller” prevents you from doing it. Obviously Microsoft does not mind or does not care if older programs (for their own platform) do not work.

From my humble point of view, Microsoft needs to fixed two things: they need to ensure compatibility as much as they can (for example by fixing things like the WinHlp32 problem, even if it is a minor issue), and they must win the hearts of business customers back. These are the pillars their empire is built on.

• Microsoft successfully managed to alienate many of their loyal developers and now even their main customers, i.e. small and large businesses. Their main software is called Office, and it is used in offices: in most offices I know there are PCs running Microsoft Windows. If MSFT continues to alienate these customers, then they should have a problem. These users do not have touch screen devices, and they are used to classic graphical user interface with desktop and mouse input. They want to use the Office software they know (Word, Excel and Powerpoint) in the way they always used it. The new Metro interface is not useful at all for classic computers with keyboard and mouse. By hiding the old desktop behing the new Metro UI, the multi-dimensional Window UI is essentially being replaced by a 2-dimensional UI made of rectangular colorful tiles. Like the ones we had in the age of DOS. The new Metro UI and the flat colored “live tiles” feel like a step back to the age of DOS. A finger is always less precies than a mouse pointer, just because it is much wider. It is maybe useful to point to pictures or icons, but it is not useful to use office software. A real step forward would have been a 3D UI (as they can be found in games today), where the traditional desktop could be accessed through windows. That would have been revolutionary.
• Apparently they neglected the compatibility of existing Windows software. This was always an advantage of Windows. Now traditional Windows software does not run as good it always did, and the new Microsoft App Store offers only a few apps. If Microsoft’s app store will offer as many good apps as the stores from Apple and Google remains doubtful. Developers tend to develop software for widely distributed systems, but most of the new devices run Android (i.e. a Linux derivative). Users increasingly use and buy computers without Microsoft OS, either smartphones (iPhones and Android phones) or tablets (iPads or Android tablets). Whether Windows phones will be successful is an open question.Any UI rises and falls with the number of good apps available for it. A total replacement of the old desktop in the medium term would render all existing applications useless. And when it comes devices with touchscreens, iPad and Android devices are at least as good as the new Windows 8, but wider distributed.

This means Microsoft loses all traditional advantages at once by the radical switch to a new UI. We will see how it turns out. I have a feeling that it will not turn out well. Too much change and too late. Is this the beginning from the end of the Microsoft empire? Will they end like IBM, a pale shadow of their former self? People increasingly buy smartphones and tablet PCs, but they are not from Microsoft: they are mainly from Apple (iPhone & iPad), or equipped with Android. We have seen in the Microcomputer revolution what happens to older, larger systems if they are increasingly replaced by newer, smaller systems with a new operating system. I am curious how it will turn out this time.

( Photo Credit: Pedro Vezini via Compfight cc )

And then there are times when things look desperate, when nothing works at all, and you do not know why, and can not figure it out. An exception has been raised, an error occurs, or something does not work, and you have no idea why. Plugins for instance are wonderful if they work out of the box, autmatically. But if they do not work, then it becomes cumbersome. The more automated a plugin or component is, the more annoying is it when it stops to work, because in this case you have no other options than examining it in detail, which means to drill down through the simple shell into the complex core where you understanding nothing at first.

Version conflicts and dependency hells can be very time-consuming and annoying, too. Ruby-on-Rails programs for example need the right combination of Ruby Version (for example Ruby 1.8.7 or 1.9.2), the right Ruby-On-Rails Version (2.3.8 or 3.2), and the right RubyGems Version (say 1.3.5). The gems or plugins have their own versions, too. The whole system only works if everything fits together. In the beginning this is no problem, for a new system usually everything is up-to-date. But then time goes on, and you have to update the Linux version, or the Ruby version, or the RubyGems Version. And suddenly the other versions no longer fit. It can be very frustrating to get the system working again in this case.

Software programs usually are not fault-tolerant systems at a basic level, there is no graceful degradation in machine language. On the lowest level in machine language or assembly the program works only if there is no error. A single error can be the system to a full stop. Either the computer program runs, which means you have to get every instruction right, or it hangs, throws an exception and stops completely. It is of course usually possible to figure the problem out, if you have enough time, but sometimes it takes a long time to understand what is going on in the various stages of debugging.

Photo Credit: tim caynes via Compfight cc

This is in fact what programmers do quite often: their identify themselves with their code. After all, they have written and created every line and every character. They have invented the names, the functions, and the structures. Nobody else knows their code as good as they do. They own their “precious” code. Programmers are like little gods who like to rule their own universe.

The advantage is obvious: if the software is succesful and you identify with it, it is your success. The drawback: if the software is not succesful and you identify with it, it is your failure. This is similar to a sports team: if a sports team wins, then everybody wants to take part in the success. If the team continues to lose, then everybody starts to blame each other: the president the trainer, the trainer the players, the players each other, etc.

It often works to claim the ownership of something because people have a lot of cognitive biases. One of these biases is the fundamental attribution error in Psychology: we have a tendency to over-emphasize personality-based explanations and ignore the role of other influences (for instance situational ones). We also tend to attribute great events to great men, know as great man theory.

While it is debatable if this is a good thing or not, a developer of a modern web application can hardly claim he is the only author of it. In the early days of PCs, it was only the programmer and the CPU that mattered, at least if you did machine programming in assembly language directly. Then we had the first high-level programming language to program systems with disk-operating systems like CP/M or various forms of DOS. Together with graphical user interfaces object-oriented programming languages arrived, and for the web comfortable high-level languages like Java, Ruby or Python with garbage collection appeared. Today we have 4 or 5 layers between the programmer and the CPU: for example for Ruby programs the programs are written in Ruby, Ruby is written in C, C is written in Assembly, and Assembly boils down to machine code.

And this is only the language itself. A modern web application is like an iceberg, the stuff above the surface is written by you and your team, the stuff below by countless others. It is not only the language and the tools for editing and debugging, a web application is based on a lot of different servers and systems

• the operating system like MacOS or Linux
• the web server like Apache or Nginx
• the web server modules like Phusion Passenger
• the database server like MySQL or PostgreSQL
• the caching server like Memcached or Redis
• the mail server and mail transfer agents like Postfix or Sendmail
• the message queue processing server like ActiveMQ, RabbitMQ or ZeroMQ

Then there are also the languages and version management systems, frameworks and libraries,
gems and plugins, written by countless other developers:

• languages like C, Ruby, Python or Javascript
• version management systems like SVM, Git, RVM or rbenv
• frameworks like Rails or Django
• libraries like Prototype or jQuery
• gems and plugins for pagination, authentication, etc.

In order to build a modern application, you setup different servers and configure them, choose a language, a framework and suitable libraries, and finally you select different plugins and gems and stick them together in a unique way. If you have done all this you can hardly claim you have created the system. And yet we tend to do it..

Therefore if you are a Ruby developer and you have produced more than others, it is not because you are taller or smarter. It is probaby because you are standing on the shoulders of many others.

(The sourcecode photo is from Flickr user nyuhuhuu)

After my 8-year old laptop refused to work this year, I looked for a while to buy a new one. The Lenovo ThinkPads looked good, they are quite popular among Linux fans. Sony and Apple make good machines as well. Finally I decided to buy a new Samsung Series 7 “chronos” laptop, and tried to create a dual boot system for Windows 8 and Ubuntu 12.10. This turned out to be more difficult than expected.

By default the machine has Windows 8 installed, uses UEFI and has “Secure Boot” switched on in the BIOS by default. After I switched “Secure Boot” off in the BIOS (and set it to “UEFI and CSM OS”) I was able to install Ubuntu, by booting from CD with Settings/Change PC Settings/General/Advanced Startup in Windows 8. The installation was cumbersome, because after the installation and the restart of the machine somehow ignored Ubuntu and booted only Windows 8. With the help of Boot Repair it finally worked.

So now I have got a new Samsung Series 7 laptop with dual boot setup for Windows 8 and Ubuntu 12.10. Or so I thought. Windows 8 starts fine, but if I wanted to start Ubuntu regularly the following Machine Check Exception error occured:

[Hardware Error] CPU 1: Machine Check Exception: 5 Bank 6
[Hardware Error] RIP !inexact! 33
[Hardware Error] TSC 95b623464c ADDR fe400 MISC 3880000086
.. [similar messages for CPU 2,3 and 0] ..
[Hardware Error] Machine Check: Processor context corrupt
Kernel panic - not syncing: Fatal Machine Check
Rebooting in 30 seconds


As you know kernel panic is the Linux equivalent of the Windows Blue Screen of Death. Something which you don’t want to see too often. It certainly does not sound good. The laptop started to reboot every time after the Kernel panic. The second boot trial often worked, but the Kernel Panic errors were of course annoying. I wondered if it is a Kernel or a driver problem. I deactivated Hyperthreading in the BIOS and also disabled the Execute Disable Bit (EDB) flag in the BIOS. EDB is an Intel hardware-based security feature that can help reduce system exposure to viruses and malicious code. Then the error did occur less frequently, but it still appeared occasionally.

Finally I found a Kernel bug report 47121 where someone reported that it maybe helps to set the “OS Mode Selection” in the BIOS to “UEFI OS”, instead to “UEFI and CSM OS”. The packages and libraries that are loaded seem to be different. I needed to switch to “UEFI and CSM OS” to install Ubuntu in the first place. Now I had to switch it off again. But after I switched it back to “UEFI OS” the Grub boot meanu now seems to have a higher resolution and – it booted without errors. It looks like UEFI was the root cause for all the major troubles.

Thus if you get a Kernel Panic error on a Samsung Series 7 and Series 9 laptop like the above one, then have look at the BIOS settings. Deactivate all advanced settings to increase performance like Hyperthreading and EDB Bit, and set “OS Mode Selection” to “UEFI OS”. Using the right BIOS settings the laptop from Samsung works really well, with both Windows 8 and Ubuntu 12.10. It is a nice machine, high quality, good equipment, comparable in every aspect to a Macbook Pro (just like the Samsung Galaxy S2/3 is like the iPhone 4/5, and the Samsung Galaxy Tab is like the iPad).

Local Port Forwarding (Outgoing Tunnel):

• Principle: Local host forwards/displays content of remote host. Local host acts as proxy. Tunneling opens a listening socket on localhost and transfers content to remote server
• Command: ssh -L local_port:remote_host:remote_port login@servername
• Tunnel: local host -(SSH tunnel)→ remote host -(SSH tunnel)→ local host
• Example: check remote host behind load-balancer or firewall on localhost

Remote Port Forwarding (Incoming Tunnel):

• Principle: remote host forwards content of localhost. Remote host acts as proxy. Tunneling opens a listening socket on the remote server host and transfers the content to the local host
• Command: ssh -R remote_port:local_host:local_port login@servername
• Tunnel: remote host -(SSH tunnel)→ local host -(SSH tunnel)→ remote host
• Example: make localhost visible in the internet or giving access to a service on your home machine to people at work

You know death march projects: in the software development and software engineering industries, a death march is a name for a project that is destined to fail. Some software projects are Sisyphus projects: you build it up only to break it down a bit later. An example are Facebook applications: you build them up in a cumbersome process, and if you are finally ready, you can break it down and start rolling up the boulder again, because Facebook has changed its API again. The old API is now deprecated, and the application is no longer compatible with the new one. Congratulations! One feels a bit like Sisyphus who rolls the boulder up the hill. Remember in Greek mythology Sisyphus was a king punished by being compelled to roll an immense boulder up a hill, only to watch it roll back down, and to repeat this action forever.

Optimizing web applications for different browsers can be cumbersome, too: if you finally have optimized you application for IE6-9, and you have fixed the 3 Pixel Jog Bug and other nice relatives, there is a new browser IE 10, followed rapidly by IE 11-19, which behave completely different and introduce bugs you never dreamed of. You can start all again rolling the boulder up the hill. Sometimes software development can be exciting and fascinating, but sometimes it is just frustrating and exhausting.

From the inside social networks are like expanding universes, they grow bigger each day by adding more members, more relationships, and more informations. From the outside they are quite the opposite: they are acting like big black holes who suck in information and make it inaccessible for everyone outside. Robert Scoble compared large closed corporate social networks like Facebook to black holes and coined the term “data black holes”. Data vanishes inside these black holes, and never comes out again. Can we save the common, open web or is it too late?

Currently, Facebook behaves indeed much like a data black hole who sucks in a larger part of the web and private information every day. Facebook is much more closed in this respect than Google+, which has like Twitter a public feed, better private/public sharing options through the “circle” feature, and it allows the export of data by “Data Liberation”. After the IPO, Facebook wants to grow and expand its own intergalactic advertising universe even further. Whether this will be successful or not will be seen. Will Facebook be able to beat Google AdWords with their own Advertising programm by sucking in the majority of information and traffic of the web? What do you think?

(The image of a quasar or growing black hole is from Wikipeda and can be found here)

To be more precise, software engineers or software architects who work together are a bit like architects who live together in the house they are building. It works best if everybody agrees on a central plan (if there is any) or if everyone works and “lives” in his own space, i.e. everyone has his own room(s), owned only by himself, where the others are not allowed to change things. If you are lucky, then the others may show you their rooms occasionally. Some shared regions like the main entrance, can be public, though. Basically, programming well with others is difficult, just like living together, because likes, tastes and preferences vary.

Since programmers are like little gods, almighty and all knowing entities in the world they have created, the clash begins already if two or more programmers with equal rights must work together. One wants a red room, the other a green. One wants wooden furniture, the other plastic. To find a compromise means to communicate and talk with each other. And this is an area were most programmers do not excel. I am not good at it, either. I think it is justified to say many have chosen to become a programmer in order to avoid too much idle talk and communication. This is were the difficulties begin..

(The picture is from Flickr user Annette Bouvain and shows a flat share)

Good software engineers make code which have a minimum number of lines. They try to design a simple minimum-code program, because in software design a minimal number of lines means to minimize the complexity. Less code means less effort to understand it, less trouble while maintaining it, and less hassle in changing it.

Thus in general, less items in engineering means less cost and less complexity. A minimum number of items, whether chips or lines of code, is important to achieve simplicity and the KISS principle. The KISS principle states that most systems work best if they are kept simple rather than made complex, therefore simplicity should be a key goal in design and unnecessary complexity should be avoided.

(the image from Flickr User Fellowship of the Rich shows an Apple motherboard)

Social networks like Google+, Twitter, or Facebook are similar. They do not only contain billions and billions of users and pages, they are also getting bigger all the time. Each day, more users join, and existing users add more content: they add more photos, share more links, and write more posts.

This is the major challenge in the engineering of social networks: to handle a system which is getting bigger all the time. An engineer of such a network is like a cook whose pots are constantly boiling over. He must add more pots each day, and keep the old still cooking.

Facebook was founded in 2004, and it was expanding ever since, gaining more users and more friends each day. Since it does not allow users to quit, and does not delete old content, it grows more and more each day. Twitter was founded in march 2006, and it was expanding ever since that time, too. Each day more users join, existing users gain more followers, and create more tweets. The latest and most advanced social network is Google+, which was started in July 2011 and combines the best features of Facebook and Twitter. It is growing at an ever accelerating rate, too.

The fundamental problem for the engineer is clear: a system which is getting bigger all the time is hard to handle, because it requires nearly unlimited scalability. If the engineers do not consider this from the start, it will be hard to build it later. Apparently, Google has the best prerequisites to master this challenge, it knows how to build scalable systems better than any other company. Some websites say Google is the King of scalability.

(The picture of the accelerating universe is from Wikipedia)

Software engineers tend to own their creations. It goes well in two cases: if everybody knows and owns his own part, i.e. if everybody has his own territory, or – if if the territory belongs to the team. This means everybody knows and can change everything, everything is well documented and the team agrees on certain coding standards, policies and guidelines. Unfortunately, working in a team is not easy, because it is difficult to create useful documentation, and programmers do not like giving up their freedom. They like their superpowers. Developers want autonomy, mastery and purpose. Working in a team is difficult, because it means to have a team ego instead of an individual ego. Agreeing on standards does not mean that everyone in the team has to accept tools, libraries and plugins you prefer, rather it means to accept things others prefer more than you, and to abandon personal habits and likes. This is not easy, because programmers are often very opinionated. So the two extremes that work are:

• The individual rules: team has no strong ego, territory belongs to the individual
• The team rules: individuals give up their ego, territory belongs to the team

Between those extremes it gets complicated. Code changes can lead easily to personal conflicts if code is owned by people. Therefore developers tend to avoid changing the code from others. Programming well with others is difficult. It certainly requires good communication. Most programmers know how to communicate, but they don’t want to. They want to do it themselves, and they see others as competitors. It takes indeed a bit the fun out of programming if you have to justify and explain every decision to others. To find the right balance between cooperation and competition in a team is not easy. In general, the golden rule helps: One should treat others as one would like others to treat oneself. Do not ignore them, ask others for their opinion, show them your respect, include them in important decisions, explain your architecture, show them your achievements etc. Communication is the glue that keeps groups together. This is the reason why most programmers have difficulties when it comes to programming well with others. They lack social skills, and the only languages they really like are programming languages. They are used to work with machines and not with people.

Ben Collins-Sussman and Brian Fitzpatrick give us some hints how to program well with others. Documentation is important: document your design decisions, your architecture, your goals, your failures, for example on an easy editable Wiki. The ability to communicate is also an important aspect. They argue in this video (and their forthcoming book with the same title) that collaborating and communicating with others is at least as important as having great technical skills. According to Ben and Fitz, the right attitude is important in a team. You should be thinking about “how do I work with a team” and not “how do I present myself as a genius”

I’ll always be for the poor engineers. They have so much skill and experience but get paid very little. They design the products that change the world but those who write stories about them get paid more. But the thrill of making things yourself is just too strong for real engineers to give up.

How true!

Normal applications are usually protected by authentication modules or plugins. Like warships they protect the application against hostile intruders and regulate who is allowed to get in. In Ruby on Rails one would use RestfulAuthentication or AuthLogic for this purpose. They are powerful and easy to configure, and they provide enough protection for a whole application. Once they are adapted to the application and are able to communicate with it, they work fine.

Big Single Sign-On (SSO) Servers are different. Instead of adapting them to your application, you must adapt your application to them. If normal authentication plugins are like warships, then Single Sign-On (SSO) Servers like Shibboleth and OpenAM are like aircraft-carriers of authentication. They are very powerful, and can protect multiple applications, but they are also very huge, heavy and expensive, because they are complicated to configure and to maintain. They are accompanied by multiple other units, for instance web policy agents (which correspond to airplanes). They operate with highest security measures like SAML. They are powerful weapons once they are configured, delivered and deployed. Unfortunately, it often takes a while until they are ready.

(The picture is from Wikipedia and shows the USS Nimitz)