(The picture is from Flickr user Annette Bouvain and shows a flat share)

Good software engineers make code which have a minimum number of lines. They try to design a simple minimum-code program, because in software design a minimal number of lines means to minimize the complexity. Less code means less effort to understand it, less trouble while maintaining it, and less hassle in changing it.

Thus in general, less items in engineering means less cost and less complexity. A minimum number of items, whether chips or lines of code, is important to achieve simplicity and the KISS principle. The KISS principle states that most systems work best if they are kept simple rather than made complex, therefore simplicity should be a key goal in design and unnecessary complexity should be avoided.

(the image from Flickr User Fellowship of the Rich shows an Apple motherboard)

Social networks like Google+, Twitter, or Facebook are similar. They do not only contain billions and billions of users and pages, they are also getting bigger all the time. Each day, more users join, and existing users add more content: they add more photos, share more links, and write more posts.

This is the major challenge in the engineering of social networks: to handle a system which is getting bigger all the time. An engineer of such a network is like a cook whose pots are constantly boiling over. He must add more pots each day, and keep the old still cooking.

Facebook was founded in 2004, and it was expanding ever since, gaining more users and more friends each day. Since it does not allow users to quit, and does not delete old content, it grows more and more each day. Twitter was founded in march 2006, and it was expanding ever since that time, too. Each day more users join, existing users gain more followers, and create more tweets. The latest and most advanced social network is Google+, which was started in July 2011 and combines the best features of Facebook and Twitter. It is growing at an ever accelerating rate, too.

The fundamental problem for the engineer is clear: a system which is getting bigger all the time is hard to handle, because it requires nearly unlimited scalability. If the engineers do not consider this from the start, it will be hard to build it later. Apparently, Google has the best prerequisites to master this challenge, it knows how to build scalable systems better than any other company. Some websites say Google is the King of scalability.

(The picture of the accelerating universe is from Wikipedia)

Software engineers tend to own their creations. It goes well in two cases: if everybody knows and owns his own part, i.e. if everybody has his own territory, or – if if the territory belongs to the team. This means everybody knows and can change everything, everything is well documented and the team agrees on certain coding standards, policies and guidelines. Unfortunately, working in a team is not easy, because it is difficult to create useful documentation, and programmers do not like giving up their freedom. They like their superpowers. Developers want autonomy, mastery and purpose. Working in a team is difficult, because it means to have a team ego instead of an individual ego. Agreeing on standards does not mean that everyone in the team has to accept tools, libraries and plugins you prefer, rather it means to accept things others prefer more than you, and to abandon personal habits and likes. This is not easy, because programmers are often very opinionated. So the two extremes that work are:

• The individual rules: team has no strong ego, territory belongs to the individual
• The team rules: individuals give up their ego, territory belongs to the team

Between those extremes it gets complicated. Code changes can lead easily to personal conflicts if code is owned by people. Therefore developers tend to avoid changing the code from others. Programming well with others is difficult. It certainly requires good communication. Most programmers know how to communicate, but they don’t want to. They want to do it themselves, and they see others as competitors. It takes indeed a bit the fun out of programming if you have to justify and explain every decision to others. To find the right balance between cooperation and competition in a team is not easy. In general, the golden rule helps: One should treat others as one would like others to treat oneself. Do not ignore them, ask others for their opinion, show them your respect, include them in important decisions, explain your architecture, show them your achievements etc. Communication is the glue that keeps groups together. This is the reason why most programmers have difficulties when it comes to programming well with others. They lack social skills, and the only languages they really like are programming languages. They are used to work with machines and not with people.

Ben Collins-Sussman and Brian Fitzpatrick give us some hints how to program well with others. Documentation is important: document your design decisions, your architecture, your goals, your failures, for example on an easy editable Wiki. The ability to communicate is also an important aspect. They argue in this video (and their forthcoming book with the same title) that collaborating and communicating with others is at least as important as having great technical skills. According to Ben and Fitz, the right attitude is important in a team. You should be thinking about “how do I work with a team” and not “how do I present myself as a genius”

I’ll always be for the poor engineers. They have so much skill and experience but get paid very little. They design the products that change the world but those who write stories about them get paid more. But the thrill of making things yourself is just too strong for real engineers to give up.

How true!

Normal applications are usually protected by authentication modules or plugins. Like warships they protect the application against hostile intruders and regulate who is allowed to get in. In Ruby on Rails one would use RestfulAuthentication or AuthLogic for this purpose. They are powerful and easy to configure, and they provide enough protection for a whole application. Once they are adapted to the application and are able to communicate with it, they work fine.

Big Single Sign-On (SSO) Servers are different. Instead of adapting them to your application, you must adapt your application to them. If normal authentication plugins are like warships, then Single Sign-On (SSO) Servers like Shibboleth and OpenAM are like aircraft-carriers of authentication. They are very powerful, and can protect multiple applications, but they are also very huge, heavy and expensive, because they are complicated to configure and to maintain. They are accompanied by multiple other units, for instance web policy agents (which correspond to airplanes). They operate with highest security measures like SAML. They are powerful weapons once they are configured, delivered and deployed. Unfortunately, it often takes a while until they are ready.

(The picture is from Wikipedia and shows the USS Nimitz)

Marketing is social, it is about convincing people. Nelson Mandela said: “Don’t talk to their minds; talk to their hearts.” You convince people by arousing their interests and emotions. You need psychological knowledge to persuade and convince people. Often you need to distort the facts and the truth to convince people to buy something.

Engineering is not very social, it is technical. It is about convincing machines to do the right thing. Machines and people are different, you control machines by giving them exactly the right instructions and commands. You need technological knowledge to do this. You need to stick to the facts and the truth if you want to find out what happens in a machine.

In Marketing …

• markets rule the world
• appearance matters, beautiful people are common
• arousing emotions is important
• psychological knowledge is required
• persuasion and manipulation of people is necessary
• everything is based on untruth, illusions, distortion of facts
• names (and “brands”) are invented, but now new things

In Engineering …

• machines rules the world
• appearance does not matter, nerds and geeks are common
• suppressing emotions and precise execution of instructions is important
• technical knowledge is required
• construction and instruction of machines is necessary
• everything is based on correctness, truth, honesty
• things are invented, but not new names

(The have_tag or have_selector methods are not part of RSpec and are defined in Webrat and/or Capybara.)

$ ps ax | grep "tomcat"    # returns all instances of tomcat runnning on your system
$ find . -name *xml        # find all XML files in the current directory
$ grep -r keyword *        # find all Files which match a certain keyword recursively

If you want to inspect log files, the following commands are useful. The grep command is useful to get all log file entries for a certain expression with a short description (-Ax means list x lines after the match) e.g. to list all logins, all registrations, or all exceptions of a certain kind

$ tail -f error.log        # view live access to log file
$ grep -A1 keyword *.log   # find all lines around the match for a certain keyword, 
                           # i.e. all logins for the keyword="SessionsController#create"

Various commands are available to add and count line numbers..

$ ls | nl                  # Numerates files in a directory 
$ ls | wc -l               # Count the number of files in a directory 
$ cat README | wc -l       # Count the number of lines in a file
$ nl -ba README            # Show file content with line numbers

Some essential file system operations:

$ ls -lsrt                            # file list sorted by (reverse) timestamp.
$ scp file.txt username@server:~      # copy file from local machine to remote server
$ scp username@server:~/file.txt .    # copy file from remote server to local machine

$ ssh user@hostname
user@hostname:~$ mysqldump -u db_user -p dbname_production > db_backup
user@hostname:~$ exit
$ scp user@hostname:~/db_backup .
$ mysql -u db_user -p dbname_production < db_backup

Google has noticed what Facebook overlooked. You probably have multiple circles of friends, and these friends belong to different areas: family, job, hobby, sports, etc. People are members in multiple groups, for example you are a member of your family, you might be an employee in a company, but also in other groups, like the Stanford Alumni Association or the parrot breeder group of Valley Springs in South Dakota, for example. Google+ gives users an easy way to organize their social contacts into different “circles” — friends, relatives, colleagues, etc. — with which they can share things: thoughts, links, photos, etc.

In Facebook, users receive information by everyone by default, and they broadcast their information to everyone by default. Facebook allows you only to have one circle of friends and one single identity, your physical identity characterized by your real name and your real photo. Since the identity forms the core of a person, it reduces you to one circle, to your mere physical appearance. At its core, Facebook is a dating site for immature teens. If you are not good looking or if you have no friends, then Facebook classifies you as a loser. It denies you to be what you want to be, but the declaration of independence says: “all Men are created equal, they are endowed [..] with certain unalienable Rights, among these are Life, Liberty and the pursuit of Happiness.” It does not mention the right to have a Facebook profile.

Google+ is much better in every aspect. Right now it is the cutting edge of social networks. It makes Facebook look arrogant and awkward. Google shows how you can do it right. It produces great services and products because it has some of the best software developers of the world. And most of all, it shows that you can make money without being evil. Good work Google!

javascript_include_tag 'ckeditor/ckeditor.js'

..and write a bit JavaScript code which replaces textareas by CKEditor instances:

  $(document).ready(function() {
     if ($('textarea').length > 0) {       
       var data = $('textarea');
       $.each(data, function(i) {
         CKEDITOR.replace(data[i].id);
       });     
     }  
  });

That’s all! Quite simple, isn’t it?

Well, first you need to install them. Installing Paperclip in your app is simple, just add the gem or plugin and follow the README. Adding a CKEditor to your Rails app without a plugin is easy, too: download it, place all the files of the Zip file in the javascripts/ckeditor sub directory, and write a bit JavaScript code which replaces textareas by CKEditor instances. Connecting CKeditor (at the frontend) to Paperclip (at the backend) is a bit tricky. There are several possibilities:

1. You can upload the image files traditionally with file upload fields, and paste the links in the CKEditor yourself, manually
2. You can upload the image files traditionally with file upload fields and offer only a custom filebrowser url for the CKEditor. Then the CKEditor only needs to paste a link to an existing image.
3. The CKEditor starts the upload. You can offer a custom fileupload url for CKEditor, if you have defined a special function to handle the upload. You can configure CKEditor so that it sends the image file to your application. Your app must store it and give an URL back. CKEditor then uses this URL to paste a link in the text.

The connection between CKEditor and your app are two urls, a custom fileupload url (POST) and a custom filebrowser URL (GET). The first is used to upload an image, the second to browse already available images. These custom browser/uploader urls must be registered at the CKEditor when you instantiate it. You can pass the right URLs by specifying options for the editor, for instance by calling CKEDITOR.replace(elementOrIdOrName, config) with appropriate configuration options that specify the urls. An example would be

load_ckeditor({filebrowserUploadUrl:UploadUrl,filebrowserBrowseUrl:BrowseUrl});

if load_ckeditor is the Javascript functions which replaces the elements with a CKEditor instance:

   function load_ckeditor(options){
     if ($('textarea').length > 0)
     {       
       var data = $('textarea');
       $.each(data, function(i)
       {
         CKEDITOR.replace(data[i].id,options);
       }
       );     
     }
    }      

If you have a class Image < ActiveRecord::Base which uses Paperclip and has_attached_file :data, a suitable controller action for a fileupload url would be

      def upload_image
        @func_num = params["CKEditorFuncNum"]
        @ck_editor = params["CKEditor"]
        if params.include?(:upload)
          data = params.delete(:upload)
          @image = Image.create({:data => data}) if data.present?
        end
        render :layout => false
      end

Your action will receive a parameter CKEditorFuncName which specifies the callback function of the CKEditor.
The corresponding HAML page `upload_image.html.haml` displays the image and calls the CKEditor callback function specified by the parameter:

    - if @image
      = image_tag @image.data.url
      :javascript
        CKEditorFuncNum = #{@func_num};
        function SetUrl( fileUrl )
        {
          window.parent.CKEDITOR.tools.callFunction(CKEditorFuncNum, fileUrl);
        }
        SetUrl("#{ @image.data.url }");

The code for the custom file browser is similar, you only have to display multiple images and call window.opener.CKEDITOR.. instead of window.parent.CKEDITOR.. if an image is selected.