Vikram and Neha

Project Euler in R: Problem 25

2012-01-24T19:19:00.000-08:00

Solutions in R to Project Euler problems are generally hard to find, so I've recently started posting R solutions on this site. Here are the previous problems: problem 22, problem 23 and problem 24

Now let's look at Problem 25 from Project Euler. Here is the problem statement:

The Fibonacci sequence is defined by the recurrence relation: Fn = Fn1 + Fn2, where F1 = 1 and F2 = 1.
Hence the first 12 terms will be:
F1 = 1
F2 = 1
...
F11 = 89
F12 = 144
The 12th term, F12, is the first term to contain three digits. What is the first term in the Fibonacci sequence to contain 1000 digits?

This is an easy problem to solve for a small number of digits. It gets interesting once we increase the required number of digits to 1000. The R integer type can hold 10 digits at most, and the type double can hold 309 digits. There is no Big Integer class in R that can hold larger numbers, so to compute the 1000 digit Fibonacci number we will have to be creative.

Computing Fibonacci numbers iteratively

Here is a basic iterative Fibonacci function, that works for Fibonacci numbers with 309 or fewer digits.

fib <- function(n) {
  a = 0
  b = 1
  for (i in 1:n) {
    tmp = b
    b = a
    a = a + tmp
  }
  return (a)
}

The version that works with larger numbers is similar, with the following changes

Integer vectors are used to store each Fibonacci number.
Each element of the vector is at most nine digits long.
Once all the nine digits are used, a new element is added to the vector and the Fibonacci number computation is carried over to the new element.
The variable numberDigits counts the number of digits in the latest Fibonacci number. When numberDigits exceeds 999, the function exits, returning the Fibonacci number index.

fib.bigint <- function(lim) {
  a = c(0)
  b = c(1)
  n = 0           # Fibonacci number index
  integerSize <- 1000000000  ## How big is each integer
  while (1) {
    n = n + 1     # Compute next Fibonacci number
    tmp = c(rep(0,length(a)-length(b)), b)  # Pad beginning with zero
    b = a
 
    # Add a and tmp
    carry = 0
    for (i in length(a):1) {
      sum.i = tmp[i] + a[i] + carry
      a[i] = sum.i %% integerSize
      carry = floor (sum.i / integerSize)
    }
    if (carry > 0) {a <- c(carry, a)}
    numberDigits <- (length(a) - 1) * 9 + countDigits(a[1])
    if (numberDigits > (lim - 1)) { return (n) }
  }
}

countDigits <- function(n) {
  count <- 1
  for (i in 1:9){
    if ((n/10) > 1) {
      count <- count + 1
      n = n / 10
    } else {
      return (count)
    }
  }
}

On my Intel Core 2 Duo 1.6Ghz laptop running Linux, this function less than 4 seconds to find the Fibonacci number with 1000 digits.

The analytical solution
Fibonacci numbers have a closed form solution, which leads to a simpler approximation by rounding (from Wikipedia),

F_n = the closest integer to (golden ratio)^n / sqrt(5)

We need the first Fibonacci number with 1000 digits. This number will be greater than or equal to 10^999.

F_n = the closest integer to (golden ratio)^n / sqrt(5) >= 10^999

We can drop the closest integer function because if F_n rounds off to the floor of the ratio, the resulting Fibonacci number will have less than the required digits. So we need smallest n satisfying

(golden ratio)^n / sqrt(5) >= 10^999

Or, n >= (999 log(10) + log(sqrt(5)))/log(golden ratio)

Or, n >= 4781.86

So we need the 4782nd Fibonacci number to get the required 1000 digits.

This problem was particularly interesting to R users because it runs into an R-specific limitation. Many other languages don't have this problem and can just use a simple Fibonacci generator to get the answer. For example, in Scheme (or other Lisp variants), a naive Fibonacci implementation produces all the digits.

Aside: Having started writing about R, I have found an excellent online resource, R-bloggers - an aggregator of content from various blogs about R. You can search for other R Project Euler solutions here. Be sure to check out the top articles box on R-bloggers landing page if you're looking for new and interesting applications of R.

Project Euler in R: Problem 24

2012-01-19T19:52:00.000-08:00

I had previously posted solutions in R to Project Euler problem 23 and problem 22. This is the next problem from Project Euler. The statement of problem 24 is as follows.

A permutation is an ordered arrangement of objects. For example, 3124 is one possible permutation of the digits 1, 2, 3 and 4. If all of the permutations are listed numerically or alphabetically, we call it lexicographic order. The lexicographic permutations of 0, 1 and 2 are: 012 021 102 120 201 210
What is the millionth lexicographic permutation of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9?

This is a very interesting problem and it can be solved analytically. The key insight is that n distinct digits can be arranged in n! permutations.

Here's the logic you could use. If the permutations of the numbers 0 through 9 are arranged in lexicographic order,

The first 9! permutations, #1 to #362,880 are numbers starting with 0, in order from 0123456789 to 0987654321
The next 9! permutations, #362,881 to #725,760, are numbers from 1023456789 to 1987654320
The next 9! permutations, #725,761 to #1,088,640 are numbers from 2013456789 to 2987654310. The millionth permutation is contained in this set. Thus we know that the millionth number must start with a 2. In fact, it is the 274,240th = (1000000 - 725760) number contained in this set.
Our problem has now reduced to finding the 274,240th lexicographic permutation of the numbers 0, 1, 3, 4, 5, 6, 7, 8 and 9 - the nine digits excluding 2. We can repeat the above process to get each successive digit.

Essentially we're writing one million as the following sum

1000000 = (a1 x 9!) + (a2 x 8!) + ... + (a10 x 1)

As shown above, the index a1 is 2. That is, only two times 9! numbers fit perfectly within 1,000,000. So the first digit of the millionth number is the third digit in 0123456789. The index a2 is 6, so the second digit of the millionth number is the seventh digit of the remaining numbers 013456789, which is 7. The remaining digits can be found similarly. While finding the digits you have to be careful to ignore the digits that have been previously used. This is a permutation, so each digit appears exactly once.

The process is tedious, so here is some R code that will make it easier. This generalizes the solution to find the nth permutation of the numbers 0 to 9.

findperm <- function (n) {
  # Find the indices a1 - a10
  indices = c()
  for (i in 9:0) {
    index = 0
    while (factorial (i) < n) {
      n = n - factorial (i)
      index = index + 1
    }
    indices = c(indices, index)
  }
 
  # Use the indices to find the nth permutation
  input = 0:9                         # The list of numbers to permute
  output = c()
  for (j in 1:10) {
    output[j] = input[indices[j] + 1] # Move digit to output
    input = input[-(indices[j] + 1)]  # Remove the assigned digits from input
  }
  return (output)
}

That was straight-forward and a fun illustration of R's capabilities.

But you can do better. The strength of R lies in the wealth of user defined libraries and functions that can ease the implementation of tricky computation. For a simpler solution, try the lexicographicPermutation function in the library R.basic. Here is the entire solution.

library(R.basic)
lexicographicPermutation(c(0,1,2,3,4,5,6,7,8,9),999999)

The first solution illustrates that even tricky computation can be coded easily in R and the code is pretty readable. The second solution suggests that before writing your own code, it is a good idea to look for existing libraries that can help. Statistical computations can get complex and tricky. You can write your own k-means clustering algorithm but it is difficult getting all the edge cases correct. Rather than writing your own routine, look around and see if it has been written already. An expert does a better job writing such routines. And you can leverage their code and their expertise.

Project Euler in R: Problem 23

2012-01-15T15:56:00.000-08:00

I was just looking through the programming language statistics on Project Euler. It shows that only 7% of the problems have been solved in R, whereas 8% have been solved on any kind of spreadsheet. This is outrageous!

Let's look at the solution of problem 23 in R. Here is the statement of Project Euler's problem 23:

A perfect number is a number for which the sum of its proper divisors is exactly equal to the number. For example, the sum of the proper divisors of 28 would be 1 + 2 + 4 + 7 + 14 = 28, which means that 28 is a perfect number.
A number n is called deficient if the sum of its proper divisors is less than n and it is called abundant if this sum exceeds n.
As 12 is the smallest abundant number, 1 + 2 + 3 + 4 + 6 = 16, the smallest number that can be written as the sum of two abundant numbers is 24. By mathematical analysis, it can be shown that all integers greater than 28123 can be written as the sum of two abundant numbers. However, this upper limit cannot be reduced any further by analysis even though it is known that the greatest number that cannot be expressed as the sum of two abundant numbers is less than this limit.
Find the sum of all the positive integers which cannot be written as the sum of two abundant numbers.

Find proper divisors
First we write a function to find the proper divisors (that is divisors less than the number itself) of a number n.

divisor <- function (n) {
    divs <- c(1)
    i = 2
    lim = floor (sqrt(n))
 
    while (i <= lim) {
        if (n %% i == 0) {
          divs <- c(divs, i)
          if (n/i != i) { divs <- c(divs, n/i) }
        }
        i = i + 1
      }
    return (divs)
  }

Check if number is abundant
Next we create a simple function which checks whether a number n is abundant.

abundant <- function (n) {
    if (sum (divisor(n)) > n) return (TRUE) else return (FALSE)
  }

The solution
With the above two functions we have the tools we need to solve the main problem: finding the sum of all positive integers which can not be written as the sum of two abundant numbers.

All the numbers greater than 28,123 can be written as sum of two abundant numbers, so we only need to check numbers smaller than 28,123 for this property. The vector abunlist in the code below holds all abundant numbers starting from 12 to 28,123. With every new abundant number than we find, we create all possible two abundant number sums and store them in the vector sumno. The answer we're looking for is the sum of all numbers 1 to 28,123 except the ones in the vector sumno.

abunlist <- c(12)            # The smallest abundant number = 12
sumno <- c(24)               # The smallest abundant number sum = 12 + 12 = 24
 
for (i in 13:28123) {
  if (abundant(i)) {
    abunlist <- c(abunlist, i)                   # Latest abundant number = i
    tmp <- abunlist[(abunlist + i) <= 28123] + i   # New abundant sums <= 28123 using i
    sumno <-unique(c(sumno, tmp))                # Add to existing list
  }
}

This is certainly not the most efficient solution, but runs in approximately 25 seconds - well within the stated 1 minute goal.

Note: The code for this post was formatted in Pretty R at inside-R.org.

Generate UML Diagrams From Java Source

2012-01-11T20:31:00.000-08:00

I was navigating through a large Java project today, and I needed some source tools to help me visualize the massive class hierarchy. Looking on Google didn't produce anything obvious: people pointed to eUML2, which was difficult to install, and some tools didn't generate UML from existing code.

After a while of searching, I remembered the Doxygen utility which writes documentation from source code. It works on C,C++, and Java, among many other languages. While its diagrams might not be perfect, they were very helpful for C++ projects in the past.

So I decided to give it a shot. It worked out beautifully.

Here are the steps involved in generating UML diagrams.
First, you install doxygen (to parse the files) and graphviz (to write out PNG of graphs)

$ sudo apt-get install doxygen graphviz

Then, run doxygen to generate a config file.

$ cd <path-to-source>
$ doxygen -g

Now you have a config file called Doxyfile. Edit this to allow recursive searching, to look for java source, to generate call graphs and to generate pictures, and to generate UML style diagrams. Here are all the changes I made to this file.

FILE_PATTERNS     = *.java
RECURSIVE       = YES
HAVE_DOT        = YES
CALL_GRAPH       = YES
CALLER_GRAPH      = YES
DOT_GRAPH_MAX_NODES  = 500
UML_LOOK            = YES

Now, generate documentation with this command:

$ doxygen Doxyfile

To test this out for a blog post, I downloaded the source code to Azureus, a large Java-based project. Here is a graph from one of the files. The top right shows a close-up view of a tiny section of the picture. Even in this simple example, you can see how a zoomed-out view allows you to understand the complex structure.

Doxygen reads code written in C, C++, Java, and many other languages. So if you are in need of UML style diagrams to better understand a class hierarchy, give it a try. It is freely available, easy to install, and easy to use. In addition to class graphs, it can also make call graphs and caller graphs.

Entertainment comes from many activities

2012-01-09T00:00:00.000-08:00

I came across an interesting post by Roger Ebert about why movie revenue is dropping. He raises important points about high ticket prices and the poor theater experience. I agree with most of the post. Many years ago, I had written about the abysmal theater experience and it is nice to see that my concern is shared.

Movies are losing revenue because they are chasing ghosts. The days of mass theater watching is history. Even in India, people are comfortable renting or buying DVDs and watching the movie at home. The theater experience deserves much blame. Theaters function in a world of limited supply. Their glory days were when few movies were released and there was little alternate entertainment. Both assumptions are false today. Thanks to streaming video and increased access to foreign movies, I can watch Korean movies from 2011 if Hollywood produced poor movies. And I don't need to watch movies. I can play video games, watch short videos online, or spend my evening looking at cute kittens online.

Our definition of entertainment is changing. Movies are just a way of spending time having fun. We have many more ways of having fun. Traditional media likes their definition of "entertainment", which is "consuming content". This is a very narrow definition. It has never held true. Children's idea of enjoyment ranges from drawing to making a mess out of newspaper. Adults are similar.

Computers on the Internet are the ultimate tool. I write articles (like this one) and it entertains me. Others use Facebook or Google Plus to keep in touch with their friends and family. Still others contribute to Wikipedia or online forums. Each of these activities provides enjoyment for the person. Each of these are entertainment.

Some of the terminology from the movie industry indicates this bias. "Entertainment Online" and "Entertainment Weekly" are focused entirely on the movie and television industry. Entertainment on MSN is all about movie stars.

Even in the absence of online streaming videos and DVDs, the number of people watching movies would drop. There is a lot more stuff you can do on a computer. And sitting in an uncomfortable chair for ninety minutes staring at a screen is not always entertainment.

Learning Languages as a puzzle game

2012-01-08T00:00:00.000-08:00

I am interested in languages. I tried learning some Japanese when I was young. I gave up rather quickly. It might have been due to the lack of good books, or the lack of teachers. It might have been the lack of native Japanese speakers with whom I could practice my skills. My experience with Japanese instilled a love for learning new languages, and understanding a culture from their perspective, through their language.

A few months ago I started learning Mandarin Chinese. We found ourselves in Beijing, and we came across some great Chinese books lesson books for English speakers. These were useful in teaching characters and sentence construction in a very academic setting. Learning a language in this way is difficult. For one: your pronunciation is all wrong. For another, it is difficult to apply such academic learning to everyday speech.

Next, I tried using audio lessons only. These were a bit more helpful. They had correct pronunciation and useful everyday phrases. I did find that the lessons were geared more towards scoring women rather than real everyday phrases. Early lessons started with, "Where do you want to go to drink?", "Would you like to come to my place?" Useful stuff I'm sure, but not for a boring family man like me. A person like me needs directions to the hospital and the rest room. Further, the audio lessons made it difficult to distinguish between close syllables like 'ga' and 'ka'. You could easily mispronounce words and not realize it. Finally, you have no idea of written Chinese. This is a real limitation. Chinese is written in a strange and confusing way. Starting out with characters is much better than trying to learn them later. Building an initial comfort level helps if you travel.

Finally, we settled on Rosetta Stone (RS for short). RS teaches language in an interesting way. You don't see any of your native language written down. Concepts are introduced entirely through pictures, and you see Chinese to go along with the concepts. It is a strange feeling. You see a big ball and a small ball, and there are strange words. With time, you form the connection between the word and the concept. All without relying on the equivalent word in your native language. I was skeptical of this idea when a friend explained it to me. And RS costs an arm and a leg, which made me even more skeptical.

Luckily for me, I seem to learn Chinese with RS. I use RS as a puzzle game rather than language learning. It is fun to try to understand the game and get better. Now I'm on the final unit in Level One, I am surprised at how much Chinese I can understand. We have a few Chinese friends and I can understand bits of their conversation with their children. It is a great feeling.

Chinese is a devilishly hard language to learn. Right at the beginning, you start out learning pictographs (characters), tones in pronunciation, vocabulary, and grammar. If RS works for Chinese, I suspect it will work for Japanese too. Once I go through RS Mandarin lessons, I might just look at the RS Japanese lessons.

Fun fact for the day: The original rosetta stone was a stone which had three languages inscribed on it. The three languages had the same message. The discovery of this stone led to a breakthrough in deciphering Ancient Egyptian. It is ironic that the Rosetta Stone software teaches you a language by only showing you the language you wish to learn. Rosetta Stone never shows you a rosetta stone.

Project Euler in R: Problem 22

2012-01-07T15:55:00.000-08:00

I solve Project Euler problems for recreation. I am using the Statistical Language R to solve these. R is free for use and download, so I would recommend downloading it if you are interested in Statistical computation.
This is problem 22 from Project Euler:

Using names.txt, a 46K text file containing over five-thousand first names, begin by sorting it into alphabetical order. Then working out the alphabetical value for each name, multiply this value by its alphabetical position in the list to obtain a name score.
For example, when the list is sorted into alphabetical order, COLIN, which is worth 3 + 15 + 12 + 9 + 14 = 53, is the 938th name in the list. So, COLIN would obtain a score of 938 x 53 = 49714.
What is the total of all the name scores in the file?

While this is an easy problem, its solution involves some nice R concepts. Let's go through solving this in R.

Reading free form text using scan()
The first order of business is to read the text file. The file names.txt is a single line of text, separated by commas:

"MARY","PATRICIA","LINDA","BARBARA", ...

The scan function reads free form text and stores each element in a vector. We specify that the input is character data separated by commas. That should do the trick, right?

nlist <- scan("names.txt", what= "", sep=",",)

Unfortunately, this does not work. After many minutes of scratching your head you'll find that one of the names in the file is 'NA'. This confuses the scan function since it is interpreted as the constant NA, or "Not Available" in R. This can be fixed by setting an na.strings argument in the scan function.

nlist <- scan("names.txt", what= "", sep=",", na.strings="")

Dictionaries / hashtables in R
Getting the alphabetical value of each character is easiest if it is stored in a hashtable somewhere. In R, named vectors can be used as hashtables. The traditional definition would go as follows:
dict = c("A" = 1, "B" = 2, ...)
While you can sit and write the entire dictionary by hand, you should avoid such repeated work. Here's a niftier definition using the names() function:

dict = 1:26
names(dict) = unlist(strsplit("ABCDEFGHIJKLMNOPQRSTUVWXYZ", ""))

This should be a trivial method to write in a C-style language where you can turn a character into an integer. R, however, has no internal ASCII table that I could find. If there is a simpler solution, please let me know.

Name scoring function
The scoring function gets the name as an input, pulls the score for each letter in the name and then adds up the letter scores.

score <- function (name) {
    return (sum(dict[unlist(strsplit(name, ""))]))
}

Calculating weighted scores
The vector of name scores is obtained by applying the score function defined above to each name in nlist. We use the sapply function, which gives a simplified vector output by default. The vector of weights is a sequence of numbers from 1 to the length of nlist. Multiply the two vectors to get the vector of weighted scores.

weighted.score = (1:length(nlist))*(sapply(nlist, score))

The full solution

nlist <- scan("names.txt", what= "", sep=",", na.strings="")
dict = 1:26
names(dict) = unlist(strsplit("ABCDEFGHIJKLMNOPQRSTUVWXYZ", ""))

score <- function (name) {
    return (sum(dict[unlist(strsplit(name, ""))]))
}

nlist <- sort(nlist)
weighted.score = (1:length(nlist))*(sapply(nlist, score)) 
sum(weighted.score)

This was an interesting problem to solve in R. It demonstrates the flexibility of R in reading and processing free form text. Trying to do a similar task in SAS would be hard.

Book Review: Foundation Trilogy by Asimov

2012-01-06T00:00:00.000-08:00

Want to start reading some good science fiction? Pick up the Foundation series by Isaac Asimov.

Asimov was a prolific writer. The foundation series comprises many books. The first three are called "Foundation", "Foundation and Empire" and "Second Foundation".

The books have an interesting premise: the behavior of individual humans is unpredictable, but the behavior of a large number of people can be collectively predicted. This is similar to the behavior of molecules of a gas, or movements of electrons. It is an interesting idea, and Asimov takes this concept and writes a kind of history for the Galactic Empire. In the introduction, Asimov says that he wanted to write a science fiction book, modeled on Gibbon's History of the Roman Empire.

The first three books are very interesting, though I did find my interest flagging in the third book. As expected, the first is the strongest. It talks of how the first Foundation was established, and how they made it through their turbulent years. The second book talks of how the Foundation faced very determined opposition, and how it escaped through it. The third talks of the second Foundation, whether it was indeed established, and what happened to it.

I don't read too much science fiction, and I am impatient when reading fiction. The foundation series were a great introduction to the genre. They are old now, but they are short and punchy. After reading the first book you'll have a pretty good idea whether you like the genre.

Science fiction is a popular genre among computer engineers and programmers. Imagining the future gives the author a lot of room to imagine a better future. The first Foundation books were published in 1952. Fifty years in the future, I was wondering how much of the world has changed. Looking back at old science fiction gives you an idea of what the older generations were hoping for.

The first three books are a great trilogy to start reading science fiction.

Book Review: The Elements of Computing Systems

2012-01-05T00:00:00.000-08:00

Want to really, truly understand how computers work? Read "The Elements of Computing Systems", by Noam Nisan and Shimon Schocken.

I studied Computer Science through graduate school. I love the field, and I own the texts that make this field fun. So I was surprised when this book came from nowhere. It promised to teach Computer Science completely through building an entire computer from the most basic pieces. After reading through it, and doing most of the exercises, I am convinced. This is the way to teach Computer Science. This way doesn't come easy, a high school student could go through it, but he would need a lot of exposure to Computing concepts in advance: flip-flops, how a VHDL-like language works, and programming. An undergraduate student could start on this book in this third year. It forms the textbook for a course at MIT. Lucky students.

The most important concept in Computer Science is abstraction: how a layer of abstraction can hide the underlying complexity. This is a difficult concept to grasp in practice, though you see it everywhere. A CPU is an abstraction layer: you program it in Assembly, and assume that it is a magical box that "understands" assembly. The CPU itself is built using transistors and logic gates. Computer Science is built on many abstractions: a memory, a processing unit, a lower-level language, a compiler, an operating system.

The Elements of Computing Systems is written in a very interesting way. It starts out with logic gates and a flip-flop as the basic unit. Using these, you create all the complexity of an adder, a register, a simple arithmetic unit. Using your creation, you create a full CPU. Using your CPU, you use an assembly language to program it. For your assembly language, you write a compiler, and then an operating system. This shows you the entire stack for a make-believe computer. The computer made is a simpler version of your laptop or desktop. Knowing one allows you to understand the more complex version.

This method of learning by doing is very powerful. This is the only way this understanding sticks in the minds of students. I've seen many of these topics covered in separate Computer Science coursework. But a single book tying the topics together has more impact.

This book isn't for everyone. You will need some experience with a C or Java like programming language, and perseverance. A smart high school student could tackle this with some help from his parents. I cannot think of a better way to learn computing.

Book Review: Three Cups of Deceit

2012-01-04T00:00:00.000-08:00

Want to know the true story behind Greg Mortenson? Read Jon Krakauer's short book, "Three Cups of Deceit", to find how Greg was able to fool millions of people.

Greg Mortenson is a person who runs the Central Asia Institute (CAI), where they collect charity money to construct schools in Afghanistan and Pakistan. His book, "Three Cups of Tea", has sold many copies, and is popular in the West. Greg claims that he stumbled into the town of Korphe, and was so moved by the poverty that he made a solemn vow to return and construct a school there. In his books, he recounts this journey, and the difficulty in building a school in such a remote place. He claims to have been apprehended by Taliban, and describes how he was released on the promise of building a school. His story is a miraculous tale of transforming hotbeds of fundamentalist religion by building schools.

It would be fantastic, if it were true. Jon Krakauer shows how much of Greg's story is a fabrication. Jon supports his claims with real proof. The most damning evidence is Greg's photograph alongside people whom Greg claimed were Taliban. In reality, they were his bodyguards through dangerous territory. Greg is shown holding an AK-47, and happily posing for the photograph with his tormentors. Jon also shows how other claims of success are completely hollow. Not only is the CAI building far fewer schools than it claims, but also the schools are mere buildings. There are no students because there are no teachers.

Jon Krakauer writes beautifully. From his book about Everest, "Into Thin Air", to the insightful account of Mormons, "Under the Banner of Heaven". Get this book now. It is a quick and captivating read.

We're done with the review. But there is a lingering feeling in my mind that something has been left unsaid.

A passage in Jon's book made me think deeply. On Page 68, Jon says how Greg was able to fool his audience.

It soothed the national conscience. Greg may have used smoke and mirrors to generate the hope he offered, but the illusion made people feel good about themselves, so nobody was in a hurry to look behind the curtain. Although it doesn't excuse his dishonesty, Mortenson was merely selling what the public was eager to buy.

This brilliant passage made me think of more than just Greg Morentson's fraud. People have an inherent need to believe in a feel-good story. We want to believe happy stories from hopeless areas because we are so eager for hope. This is as true in yogi-infested India as it is in the West. Many babas and yogis originate in rural India. We are eager for stories that show India in a good light. We want to believe the lies that India has a stronger spiritual connection with God than the rest of the world. We want to believe that India has all the mystic and spiritual men, the land that gave birth to five religions. In the absence of good news about the economy, or education, or political progress, we'll believe in far-fetched stories about how God favors Indians. In our eagerness to believe such stories, we are fooled by crooks posing as saints.

I am glad that Jon wrote a book exposing Greg Mortenson. But the real people I applaud are his audience. People realized that they were fooled, and they admitted this. Jon himself was a supporter of Greg's work, and had donated more than $77,000 of his own money to support the effort. It takes a lot of courage to investigate someone's fraud. But it takes a log more courage to admit that you have been fooled. I applaud the people for realizing their deceit, and having the strength to end the charade.

Indie Gaming

2012-01-03T00:00:00.001-08:00

I was going to bemoan the loss of innovative ideas in gaming. I had seen too many first person shooters, too many car racing games, and the same old Japanese Role Playing Games.

And then I met Indie Gaming, and I was blown away.

If you're new to the concept, as I was, Indie Gaming are games made by independent studios. Sometimes, these are very small teams of developers producing a single game. The large game studios want to chase millions of installs and the big blockbuster game. They do not deviate much from formulaic games, massive story-lines, insanely detailed graphics and big budgets. Independent games are made by smaller teams, so they have a limited story, small budgets. This limits what they can do: they cannot put millions of dollars into graphics. And as a result, they need something to attract audience away from well established games. They need to be innovative.

My introduction to Indie gaming was through the Humble Indie Bundle. The concept of an Indie bundle is as innovative as the games. Independent developers cannot get their games stocked in Walmart, and they don't have the traffic that can sustain Internet-only sales. As a result, they suffer from poor visibility and lackluster sales. So some independent game developers tried to offer a bundle of independent games at a price that the customer sets. Everyone chooses their own price. If you pay more than the mean donation amount, you also get some freebie games. The games are available for Windows, Mac, and Linux. The games have no Digital Rights Management, so you can confidently run the game on your computers in the future. They have no copy protection, so you can install the games on all your home computers. The independent games bundle concept has taken off. The sales are bigger and more prominent each year. The Humble Indie Bundle was the first bundle, and now there are many different types of bundles offered on the Internet.

This year, I purchased the Humble Indie Bundle #4, and it included the Indie Bundle #3, since my donation amount was larger than the average. I paid $10. I get eleven games with no DRM, and they all work on Linux: it is a bargain.

The games are refreshingly innovative. Night Sky is a 2D physics platformer, where you have to roll a ball around to progress through levels. Hammerfight is a game in which you battle with a floating hammer. Hammerfight uses only a mouse, and no buttons. Bit Trip Runner is a rhythm-based platformer, where your character has to jump and navigate through a level and your moves have to be synchronized to the background music.

Independent gaming are bringing fun back into gaming with their innovative games.

Book Review: Binary, by Michael Crichton

2012-01-02T00:00:00.000-08:00

Want to read good fiction? Read "Binary", by Michael Crichton, one of his best works that you've never heard of. Yes, that Michael Crichton.

The first book I read by Michael Crichton was an unknown book called "Binary" that none of my friends had heard about. I was in high school, and my father's friend had dropped this short book at our place. The author was John Lange, whom nobody had heard of. Inside, the book mentioned that the author was now revealed to be the great Michael Crichton, but I hadn't heard of Crichton either, so I wasn't expecting much.

The book starts with a gripping description of a heist. It grabs you, and pulls you in. The book is a thriller, of a plot involving dangerous weapons, and one man's race to stop it all. It is a short book, written very well.

I had harbored fond memories of this book. I had recommended it to many friends, knowing full well how difficult it was to obtain a copy. Recently, I received a copy of this book, and I sat down to read it. A short evening passed, and I was done. Despite having read it over a decade ago, I remembered most of the plot and many of the scenes. It was a thrilling book made punchier by the fact that I knew the story. I marveled at the plot and the excellent writing.

A fantastic novel, a thrilling plot. The ending isn't blockbuster by today's standards, and that is part of the lure. No superfluous nonsense, and no extra pages. A snappy story beautifully told.

Book Review: The Argumentative Indian

2012-01-01T00:00:00.000-08:00

Suffering from insomnia? Read "The Argumentative Indian", by Amartya Sen.

"The Argumentative Indian" is supposed to be observations about Indian culture and society. It might be that, but the language was so convoluted that any beauty or meaning was lost in the maze of words. Here is a sample passage, after which I decided to drop the book.

Yet, as is discussed in Essays 7, 8 and 15, general statements about India and Indians can be found throughout history, from the ancient days of Alexander the Great, of Megasthenes (author of the Indika, in the third century BCE), and of Apollonius of Tyana (an India-expert in the first century CE) to the 'medieval' days of Arab and Iranian visitors (who, like Alberuni, wrote so much about the land and the people of India), all the way to the Enlightenment and post-Enlightenment Europe (with heroic generalizations about India presented by Herder, Schelling, Schlegel and Schopenhauer, among many others).

Say what? If you were counting, that contained one sentence.

I'm sure the book contains some useful insight but the unconnected ideas and the lack of flow made the book unreadable. I would recommend V.S. Naipaul instead. Not only does he have insight into Indian culture and society, but the language is beautifully readable.

Book Review: "Secrets" by Daniel Ellsberg

2011-12-31T00:00:00.000-08:00

Do you always do the "right" thing, or do you take the path of least resistance? "Secrets", by Daniel Ellsberg reveals the life of a person who did the right thing.

Daniel Ellsberg is the person who leaked The Pentagon Papers, a secret study about the lessons learned during the Vietnam war. Leaking the study required a lot of personal courage. He was tried for acts of treason, though he was later acquitted. I find the lives of people like Daniel fascinating. They knew the path of least resistance, and yet they actively worked against it because of their conviction about the "right" thing to do. I wonder if I will have the strength that people like Daniel did.

Secrets starts of with the Gulf of Tonkin incident, and Daniel talks about his observations when he was posted in Vietnam. This section of the book was very riveting: Daniel has incisive observations about the war that he saw. Later, Daniel was a scholar for the RAND corporation, studying the Vietnam war. The Pentagon Papers are the result of this study. Daniel talks about his experience with the secrecy in RAND and the Department of Defense.

I found Daniel's observation about the effect of access to secret clearance beautifully insightful. He warned Kissinger what the access to secret information will do. (Page 237 in the paperback version). I'll paraphrase Daniel's commentary:

At first, you'll feel exhilarated by the volume and extent of information. Then, you'll feel like a fool for having analyzed these topics without even knowing that these secret documents existed. Then, you'll be aware of others who don't have this information, and you'll think that these people are fools. Finally, you'll become incapable of learning from people, because most people don't have access to these secret documents. No matter how great their experience compared to yours, you'll be incapable of learning from them.

This passage should be read in the original. It is a beautifully crisp understanding of the impact of secret clearances, and power in general.

Daniel talks about his thought-process, and why he decided to release the confidential study. He talks about the impact of the study, and the impact to his freedom. It makes a very interesting read.

It reads like a war thriller, but it carries deep lessons about how difficult it is to do the right thing.

Book Review: Autobiography of a Yogi

2011-12-30T00:00:00.001-08:00

Why not read some third-grade fiction this weekend? Pick up a copy of "Autobiography of a Yoga" by Paramhansa Yogananda.

The book is about a "yogi", a person who has attained enlightenment of some sort. However, the book reads like a supernatural fairy tale. The very first chapter started off with this hoot of a passage.

The helpless humiliations of infancy are not banished from my mind. I was resentfully conscious of not being able to walk or express myself freely.... Psychological ferment and my unresponsive body brought me to many obstinate crying-spells.

The author claims that he was fully conscious of his infancy, and that he cried because he couldn't express himself. This is pretty damn hard to believe. Of course, the author is gone now, and no supporting evidence can be obtained. From what we know about brain development, it is difficult to believe that any child could have such capability. A few sentences before that, he claims that in his previous birth he was a Himalayan yogi. You would think such a highly perceptive creature would wait for the infancy days to get over before trying to talk.

A few pages later, we get another passage tailor-made for the gullible audience:

An immense flash of light at once manifested to my inward gaze. Diving shapes of saints, sitting in meditating posture in mountain caves, formed like miniature cinema pictures on the large screen of radiance within my forehead.
"Who are you", I spoke aloud
"We are the Himalayan yogis." The celestial response is difficult to describe; my heart was thrilled.
"Ah, I long to go to the Himalayas and become like you!" The vision vanished, but silvery beams expanded in ever-widening circles to infinity.
"What is this wondrous glow"
"I am Iswara, I am Light." The voice was as murmuring clouds.
"I want to be one with Thee!"

It is as poorly written as it is imagined. In the first chapter, he had recognized himself as a Himalayan yogi from a previous birth, and now that he sees a few people more, he needs them to positively identify themselves. The book is filled with extravagant recounting of supernatural events. Looks like the author lived in a charmed world, only dimly aware of laws of Physics. The miracles themselves are difficult to take seriously. Extraordinary claims require extraordinary proof, none of which is provided by the book. And they aren't even written in a funny or engaging manner. They're stated with a nonchalance that suggests that this is commonplace. Even if this were a work of fiction I'd pass it by.

The lesson from the book is that you are a yogi if unnatural stuff happens to you. If it doesn't, tough luck. Maybe next birth pal. Wait this life out. Even as a spiritual text the book is abysmal. It has no lesson about how to live a good life, or how to achieve peace. A much more concise and beautiful lesson is the one-liner from Mae West, "You only live once, but if you do it right, once is enough."

I am amused by "Autobiography of a Yogi". It comes highly recommended from seemingly sane people. People swear by it, and say that it changed their life. You have to be very desperate for meaning if this book gives you any. It is like finding deep meaning in "Baa Baa Black Sheep".

Due to the poorly imagined plot, and the haphazard story-line, I didn't read too far. If someone could point out passages that they thought were spiritually enlightening, I'd be happy to revise my opinion.

(In case you are offended by this: Calm down. If this guy is half as divine as he claims, he won't mind it.)

Book Review: The Golden Gate by Vikram Seth

2011-12-29T00:00:00.000-08:00

Want to read a moving novel in sonnets? Grab a copy of "The Golden Gate", by Vikram Seth.

I am prejudiced against "popular" Indian authors. Past experiences have made me critical towards them, especially if their fiction is popular in the West. I don't know what I was thinking when I picked up "The Golden Gate" by Vikram Seth. I had heard of Vikram Seth, but I had not read anything by him.

The Golden Gate is a story about a few friends, and a few years of their life. The book is written entirely in verse, which took me by surprise. It has been a few years since I read verse. I need not have worried, the Golden Gate is a great way to start reading verse again. It is easy to read, and many passages in it are moving, and thought provoking. The book starts out with a person who should be happy at his success, but he feels lonely. It follows his journey, and tells you about his friends, and their lives. Written during the turbulent period of the Vietnam era, it raises interesting questions about life, ambition, following one's dreams, and what it takes to be happy and at peace.

The locations are all from the San Francisco Bay Area: San Francisco, San Jose, Marin county, and introduce the reader to the joy of living in this wonderful part of the world. It makes the reader appreciate the world we live in, and our friends.

I started out skeptical, and ended up enjoying the book immensely. I was sad when it ended, Vikram had done such a fine job of introducing the characters that they felt like friends.

Get the book, and enter the charming world of Vikram Seth's verse.

Image courtesy Wikipedia.

Book Review: Spice Route

2011-12-28T00:00:00.000-08:00

Want to know why bay leaves were once more expensive than saffron? Read "The Spice Route", by John Keay.

The book is a history of the Spice Route, and the general quest of Europeans to find exotic aromas and flavors. It goes through time, showing how the trade route was a long chain of sea and overland routes through the Middle Ages. Various spices were harvested in South East Asia: India, Indonesia, and other countries. These were traded by Arabian traders, who brought them to the Middle East. Here, they were traded to others who took them to the Mediterranean.

The history of this route is fascinating: spices were a big motivation for the European powers to expand East and West. Many campaigns were waged: trying all the routes, trying all ways to gain more access. The merchants from Holland successfully bypassed the Mediterranean, and the Spanish and Portuguese managed to go in different directions: the Spanish went towards the New World, while the Portuguese went East towards Goa and Japan. It is difficult to transport a person to that time, where travel was wretched and dangerous, but John does a fine job in giving the reader a flavor of that world.

The spice route led to a colonization of the world, as the European powers carved up the commercial world among themselves. It is instructive to read this history, and learn about how a new improvement fundamentally changes the field.

This book will shed a light into the colonial era, and will take the reader back through time. It made me look at my spice cabinet very differently, and treat my bay leaves with a lot more respect.

Book Review: H20 A Biography

2011-12-27T00:00:00.000-08:00

Looking for some good science writing about a common topic? Pick up Philip Ball's, "H20, A Biography".

Philip writes about water from a few different angles. It talks about how water originated in the Universe. The book talks about the origin of the Universe (Big Bang, not 6000 years), and how planets were formed. Then it goes into how Earth might have got its water. We take water for granted, but many planets have little or no water at all. If we believe that intelligent life requires water, we need to understand how water could have come around in the first place. It is a very strange set of circumstances that caused Earth to get water and retain it. Many planets might have had water, but they lost it over time. Astronomers and Physicists have looked at Venus and Mars and tried to reason about how Earth might have had water. The Science behind this is quite beautiful and involved, and it helps that Philip explains it so well.

Next, Philip explains how humans discovered water. This is the most interesting part of the book, the history of Science. Ancients thought that water was an element, and they did not know that ice, water and steam are the same compound. This led to a long set of mistakes and it shaped the thinking of early chemists, leading them astray. I found this section remarkable. How does one realize what water contains? Philip lists all the major Physics and Chemistry milestones that allowed humans to recognize that water is a compound of hydrogen and oxygen. The book also talks of how humans realized that ice, water and steam are the same compound. The mis-steps are very instructive. They lead to a better understanding of how science proceeds and how mistakes are corrected. Early chemists thought that when something is burned, it gives up phlogiston (rather than taking up oxygen). This was an easy mistake, given the instruments of the time. Early chemistry made quick advances when people showed that burning was a process of oxidation, combining chemically with oxygen. (Similar mistakes were made in early experiments with electricity. Electricity was through to flow from the positive lead to the negative lead, when the actual flow is in the opposite direction. Thus, electrons now have a negative charge by convention.)

In the third section, the book talks about the behavior of water in cells, and the anomalous behavior of water under various circumstances: low temperature, high pressure, low pressure, presence of solid bodies. I didn't know much of it, though I confess that I wasn't able to retain much of this information. It was just too dense for me in parts, and I had to skim through. The book ends with drinking water, and explains why the lack of drinking water is one of the biggest problems facing humanity.

The book is a beautiful history of early chemistry and various aspects of water. It does get technical at times explaining in quite some detail how certain things work. It is best to skim such sections and return to them later. A majority of the book is an easy and insightful read.

Book Review: Seeing Voices

2011-12-26T08:05:00.000-08:00

Oliver Sacks is a neurologist of some renown. He has written some fascinating books on the working of the mind: the most famous are "The Man Who Mistook His Wife for a Hat", and "Musicophilia". "Seeing Voices" is a short book about the deaf.

When I first picked up the book, I wondered if brain development for deaf people is very different from that of hearing people. Oliver's book lists the various ways in which language development suffers if a deaf person isn't allowed to sign naturally, or if he doesn't have access to other people who sign. I didn't know much about sign language before reading the book. Sign language isn't a simple translation of English or Hindi into gestures. Rather, it has its own idioms, its own grammar. So sign has a notion of poetry and of puns.

Oliver lists various cases of deaf people whose parents did not sign. When these children grew up and finally encountered signing, a world of language opened up to them. This process is beautifully described, and provides valuable insights into the working of the language part of the brain. For example, without language, there was no way to describe time: today versus tomorrow blend in if there is no way to describe them.

There are also examples of a few hearing individuals whose language development has been artificially suppressed. These examples showed how both hearing and non-hearing individuals pick up language in similar manner.

"Seeing Voices" is a great book to read if you have a small child in the house. Children learn sign language faster than spoken language. Spoken language requires a complicated co-ordination of the vocal chords, and these develop rather late in children. Sign language requires arm motion which children acquire rather early. Based on this, many hearing parents teach their children sign language. From an early age, the child learns how to express himself: pain, hunger, boredom, and a need for mommy, daddy, or sibling. This allows for a limited communication between parent and child, removing frustrations on both ends.

After reading this book, I could think of many friends who would love to read this. You don't need to be interested in neuroscience, language development, child development, or the world of the deaf to love this book. This short book makes you aware of language, and gives you much to consider about.

Book Review: The Man Who Knew Infinity

2011-12-25T18:21:00.000-08:00

Srinivas Ramanujan was a brilliant Indian mathematician who made seminal contributions to Number Theory and the theory of continued fractions. Robert Kanigel's book, "The Man Who Knew Infinity" is a biography of Ramanujan.

The book talks about Ramanujan's life in quite some detail. Robert has provided a lot of color by traveling to the places involved, and deeply researching Ramanujan's life. The book has photographs of the places and maps of areas, making it easy to identify with the story. The biography is well balanced and impartial. If this was all the book contained, it would already be a worthy read.

What I found most interesting was the associated commentary on Indian society, values and the education system. Ramanujan was ignored by the Indian education system, largely because he refused to conform to its requirements. Ramanujan showed an early brilliance in Mathematics. But the system didn't care for exceptional performance in one field. Due to the inflexibility of the system, the talent of a brilliant mathematician was wasted. Having suffered through the Indian education system, I found the passages revealing. Even back in Ramanujan's day, the system was inflexible and idiotic. Many people recognized the inflexibility of the system, its arbitrary outcome, and the ill effect on genuine talent.

How much part did Indian society and customs play in Ramanujan's downfall? Ramanujan refused to alter his diet in cold, cloudy England. As a result, he got very little vitamin D, and suffered poor health. This is a problem that persists to this day: Indians who firmly adhere to a restricted diet suffer from problems in countries where an Indian diet is unsuitable. Ramanujan's wife was poorly treated by his mother, and this poor treatment led to misunderstanding and stress in Ramanujan's life. Would the outcome have been different if the Indian arranged marriage was not as stifling?

Ramanujan's life is full of questions. It makes me sad at the outcome. But more importantly, it makes me wonder.

Would a Ramanujan be possible today? Could a lower middle class boy with no talent for anything other than Mathematics be recognized as a genius? Could he even reach a point where he could seek collaboration or patronage from Western mathematicians?

It is a very sad tale, though one I think every Indian student should know. Indian students yearn for exposure to foreign education and worldwide recognition. This is one example of a person who got both, and yet he had a sad and lonely life. The outcome could have been so much different. Ramanujan was as good as Euler or a Gauss, according to people who worked with him. And yet his talent was squandered away. An early death, a lonely life full of struggle. And Ramanujan was lucky. Today, he probably would have no hope of success.

Image courtesy Wikipedia.

Book Review: The Male Brain & The Female Brain

2011-12-19T00:00:00.000-08:00

Rather than waste your time with pop-culture books about men and women, how about read books with real science and insight this holiday season?

This is a review of two delightful books called "The Female Brain" and "The Male Brain" written by Louann Brizendine. Dr. Brizendine is a professor at UC San Francisco. Both books talk about the peculiarities of each brain from a scientific viewpoint. The books are scientifically accurate, and are accessible to a layperson with no background in neurology.

The books contain a lot of insight about the behavior of both men and women. For example, The Female Brain talks about how women's brain is highly geared towards social connections. It provides many examples to demonstrate this, and talks about the development of a female brain from a newborn to adult and to a mature brain. Along the way, you see the various changes in the brain. A lot of behavior changes accompany the development of the brain. This book made me understand the motivation behind the baffling behavior of friends and relatives.

The Female Brain was the earlier book. Recently, Dr. Brizendine wrote The Male Brain. The second book is as interesting and as revealing as the first. It talks of the development of the male brain through the years. There are many ways in which the adult male brain is different from the teenage male brain. Reading about the male brain allowed me to better understand how I will change as I grow older. It also allowed me to understand male toddlers behavior.

Both books are a quick read, they make cutting-edge scientific research accessible to everyone. This is scientific writing at its best.

Book Review: Kashmir, The Wounded Valley

2011-10-12T09:02:00.000-07:00

Many years ago, Neha and I were walking around Santa Rosa in Northern California. We came across a used bookstore where the owner was listening to some good blues, and improvising with a harmonica. It was a perfect setting. We entered the store and found some very interesting books. Some of these books have been impossible to find elsewhere.

Today's review is about one book from that store. The book is "Kashmir, The Wounded Valley", by Ajit Bhattacharjea.

The book is a condensed history of the Kashmir valley. It starts around 700AD, when authoritative written record begins, and continues on till 1992. It is an insightful read. Written in an easy style, Ajit takes the reader through time in Kashmir's history. Entirely factual, and entirely gripping, the book has the pull of a crime thriller rather than the yawn of history books. It is even more chilling when you realize that all the events occurred. And when you turn to the evening news, you realize that we are still living with the repercussions.

Two things struck me when reading the book.

The valley has had mismanagement and cruel monarchs. Few rulers have cared for the well-being of their people. Dynasties follow a predictable pattern. The first monarch in a dynasty is an aggressive conqueror, or a cunning statesman. The second is a benevolent and able leader. Successive leaders are selfish, greedy, and insensitive to the needs of the people. Few monarchs have focused on improving education or living conditions. As a result, the people have suffered greatly and their lot has worsened over the years. Even before the British Raj, the condition of the people was miserable.
The people are surprisingly resilient, and a little gullible. Kashmiris have a liberal outlook towards life. Most Kashmiris converted from Hinduism and Buddhism to Islam willingly. The benevolent influence of liberal sufis struck a chord among the liberal Kashmiris. Culture played a bigger role than religion in their life. All through the turmoil of Mughal India, Kashmir had peaceful co-existence among Hindus and Muslims. The gullibility of the people leads to their willingness to be ruled and oppressed. And it shows in the lack of popular uprising to demand better rights and better living conditions. They supported outsiders to overthrow the existing monarch, only to find the new boss was same as the old boss. Even when democracy was at hand, they did not demand their rights aggressively. As a result, leaders turned into tyrants, ignoring the needs of the people and holding Kashmir back from fully awakening.

The book also explains how Kashmir became a contentious issue between India and Pakistan, and the mistakes that led us there. The book is impartial: it points out mistakes on all sides. The ruler of Kashmir, Hari Singh, refused to make a concrete decision on joining either side. He had hopes of staying independent despite the impossibility of such an arrangement for Kashmir. Rulers of India and Pakistan refused to give the Kashmir issue the importance it deserved. And as we have seen countless times before, administrative incompetence and widespread corruption made a bad situation worse. Heavy-handed actions by the central government, corruption by well-known people, and a lack of respect for the people's needs alienated an entire state. The people started out amicable to India, eager to strengthen ties with the land of Nehru and Gandhi. Over years, all good will was squandered, culminating in the excesses of the armed forces. Warm friends turned into bitter enemies.

A small bruise festered into an open sore, leading to much pain and hardship for both countries for over fifty years. Over time, that pain has spread through both countries. It has led to war and terrorism.

This book should be a required reading for every Indian and Pakistani student. It has valuable lessons for future statesmen. Due to mortal follies, a place with heavenly beauty has been turned into a dark hell. Having deteriorated so far, it is debatable if the situation will ever improve. If a workable remedy exists, knowing how we got into this mess is the first step towards it.

The State of Free Software Today: What We Need to Do

2011-10-10T22:37:00.000-07:00

Richard Stallman (popularly called RMS) wrote an eulogy to Steve Jobs recently. It was widely reported due to its angry tone. In summary, RMS was happy that Steve Jobs is dead because Jobs personifies the "walled garden" approach to computing. In the Free Software camp, nothing is more abhorrent than the walled garden approach. Free software is all about giving people a lot of freedom: including the full source code of the software, and the freedom to modify it and copy it. The walled garden approach is all about tight control: not just of the source code but also the development and the overall experience.

It is easy for me to pick sides: I have been a free software enthusiast since 1995, when my friend and I chanced upon something called Linux. I spent many hours playing with Linux, and have used it as my primary system for well over a decade. In addition to Linux, I spend all my time in free application software: Firefox, Chrome, Blender, GIMP, R, Eclipse, Emacs, Arduino, ... The list goes on.

I abhor the walled garden approach. It is damaging to my livelihood as a software engineer if I cannot learn from a system or modify it. I enjoy programming and have tinkered with most systems I own. My tinkering was lousy and inconsequential, but I enjoyed it. And I learned a lot by poking inside them.

It is easy to identify with RMS' criticism of Steve Jobs. It is easy to rally people when you demonize someone. But free software doesn't need this kind of rallying.

I think RMS is wrong.

The Lure of Free Software
Free software is compelling. Many programs like Blender, Eclipse, Emacs are the best in their class. They are exceptional tools. I write better in Emacs than any other editor. Emacs is an extension of my mind: it is a supremely capable editor. My other editor, vim, is equally good. It is beautiful to watch an expert use emacs or vi: the software makes them insanely productive. Free software has done exceptionally well in technical areas like Statistics (R), 3D modelling (Blender), web serving (Apache), microcontroller hackery (Arduino), development tools (Eclipse, GCC, emacs/vi, ..) In these areas, non-free tools are at a significant disadvantage. Who would bother using some souped-up microcontroller tool when Arduino works so beautifully? Who would bother with a strange new Statistical language, now that you can download, install and use R right away? And if you have a newbie question with R, the wizard Peter Dalgaard himself replies to questions! That is how awesome the community is. The software is great, the community is knowledgeable and helpful, and you can read the source code to learn more. It doesn't get better than that.

With software like that, you don't need to yell about the virtues of free software and the evils of the walled garden. The software stands on its own merit. And it continues to attract people like me who care about the freedom.

The free software world has produced some exceptional non-technical software: Firefox and Chrome as browsers, VLC as a media player. But in the non-technical area, free software has lagged behind. Ubuntu is making a gorgeous operating system that my mother can use. But companies like Canonical are the exception rather than the norm.

The Lure of the Walled Garden
In recent past, Apple has gained a large following. Their products like the iPhone and the iPad are phenomenally popular. People are buying these products because of the superior user interface, and the ease of use. The user interface really is the killer feature. Free software advocates might dislike the walled garden approach, but we cannot deny that Apple devotes extraordinary resources to getting an excellent user interface. The first bite of the Apple is bewitchingly delicious. The interface sells.

For a long time, the free software world has been blind to the needs of the "common" user. We have aimed at the technical audience, and have ignored the needs of the vast majority of computer users: our parents, our grandparents, our friends. We justified this stance because there were limited developers, and we needed to get our own house in order. We needed development tools, drivers for technical products, technical software. Developers were more interested in scratching their own itches, writing programs that satisfied their needs. Nobody can be blamed for the situation we are in. Every free software developer has been working hard, quite often without pay. Asking a Statistics programmer to write a cute game would be a disaster.

In the meanwhile, a walled garden has appeared with exceptional usability, even if it has a limited set of features.

It is a testimony to good usability that people are willing to enter the walled garden. Many people don't care about the freedom. People who do care continue to enter the walled garden despite their loss of freedom. It is a question of convenience, and people choose their battles carefully.

The way forward
We have many glimmers of hope. Look at Firefox or Chrome. Both are free software, and both wrested browser market share from Internet Explorer (IE). IE was never an explicit walled garden, but it did curtail freedoms: it ran on specific platforms, it fragmented the web with its own extensions. For a long time, the lack of a good browser was the chief hurdle for free software users. We couldn't use IE, and nothing else was good enough. Navigator was barely usable.

Then came Firefox: it was a compelling browser. Firefox was fast, it ran on every system, it was more secure. Most importantly, it was beautiful. Compared to Netscape Navigator and IE: man, was it gorgeous! Put the navigation buttons and URL bar along the menu bar, and you had one pretty interface. Chrome went one step ahead: it is an exceptional browser. It is fast, it is pretty, it is secure. Oh, and the source is available. People who don't care about freedom of software use it because it is good. And people who care about freedom can download its source code.

Something similar is happening with Android now. Android's source code is available. But that's not why millions flock to buy the Nexus-S or the Droid. They want the system because it is excellent. With Gingerbread, the Android system looks beautiful. With the next release, I'm sure it will look even better. Now that it is comparable on features, it is polishing its usability.

That is the way forward: to make exceptional software. Make the best damn system in the world. Make it gorgeous, fast, cheap, reliable, rugged, stable. And then release the source code. Find opportunities like the Mozilla Foundation did with Firefox. Like Canonical did with Ubuntu. Or like Google did with Chrome, Chrome OS and Android.

Users are being charmed by the the usability of the software that Steve Jobs provided. Provide the same usability with the freedom. The walled garden will disappear by itself.

Indian Universities are no longer world class

2011-10-09T16:28:00.000-07:00

A recent survey carried out by the Times Higher Education magazine ranked all the universities in the world. Sadly, Indian universities don't figure too highly. The best Indian university in that list is IIT Bombay. Its ranking is 301-350: no specific rank is given. One thing is for sure, IIT Bombay is worse than many small universities few people have heard about, like places in Turkey and Brazil.

Many years ago, I had written about how the IITs are not world class. When you venture out of India, you realize how bad they are. That article got a lot of pageviews, and lots of comments. Many IIT students wrote to say how deluded I was, in their own special way. Eat humble pie, my friends.

The Times Higher Education survey is quite comprehensive. You can look at the methodology details here. In short, they asked industry and education leaders for their views of universities. Universities were graded along a variety of factors: the number of citations for their papers, the amount of research funding, the impact of its research on industry, the teaching environment. The study looks solid: all income has been adjusted for purchasing power parity, so poor countries are not penalized for their lower cost of living. Also, there is a normalization across disciplines, so sciences where overall publication frequency is lower are not penalized for having lesser research output. The weight for research output is 30%, which is perhaps too low. They have an iPhone application which allows you to change the weights and see the impact on the ranking.

I don't have access to their raw data, but from reading about the study, their method looks sound. Ranking an institution is a difficult task, and this study is probably the most precise answer we have right now. A better study could look at the impact of graduates from the universities, or the difference between the capability of students with and without the education: thereby measuring the impact of the institution itself. Alas, such controlled studies are difficult to come by.

So where does this leave us? This leaves us in the sorry state of admitting that Indian educational institutions are really not that good. The Chinese are far ahead of us. The prestigious Tsinghua University in China is at position 71, ranking near well known universities Rice and Vanderbilt, both in the US. Among Asian countries, Japan, Hong Kong, and Singapore all fare very well. In fact, the top Asian universities are dominated by Japan, Hong Kong, China, Singapore, and Korea, and now the reasons are somewhat clearer. . These countries have made investments in education, and the results speak for themselves. Even between Brazil, Russia, and China, India spends the least amount of money on education, per student.

Admitting that our educational institutions have fallen behind is the first step. Only after we admit the problem can we find ways to rectify the situation.

Learning a new software: Blender

2011-10-08T13:00:00.000-07:00

Blender is a 3D design software. It is a complete tool, allowing you to create entire worlds in 3D, modelling, meshing, applying texture, and animating the world.

Here is an example of what Blender can do. This movie was created completely in Blender.

Isn't that impressive? What is more impressive is that Blender is available for free, you can download it and install it right now (Windows/Mac/Linux). And if you want to make your own movie based on the characters behind Sintel, you can download all the art and characters at the Sintel website. This level of sharing is refreshing: it allows new graphics designers to learn a complex tool, and be able to look behind the scenes of a very complex project.

Blender has a beautiful UI, though it looks very complicated initially. When you load it up the first time, the number of buttons and controls is overwhelming. This is what the default Blender 2.5 UI looks like. There is a small cube placed at the center of the viewport, and many properties and control widgets are open. Once you learn a few Blender principles, you find that the UI is clearly arranged, and every control is in the perfect place. As with any good tool, the tool itself vanishes pretty quickly. Within hours of learning Blender, you focus on the 3D modeling activity rather than the Blender tool.

Learning Blender
I tried learning how to use Blender using a book. It is hard to explain the various UI elements using written words. It is even harder to describe the 3D world you are creating using just words. The best book I have found till now is called Blender for Dummies. It requires no previous knowledge, and is an excellent resource to begin modelling simple meshes in Blender. It covers Blender 2.5.

In case you want to start while your book is on its way, here are a list of online resources. I list Blender 2.5 resources only. The Blender interface changed a lot between 2.4 and 2.5. If you are starting out, it is best to start with 2.5 directly.

Getting Started with Blender: This is a set of seven video tutorials. They are short and cover the very basics of using Blender. You can watch all seven within an hour and you will have a great idea of the Blender UI and basic features.
Neal Hirsig's blender learning course: More than a collection of isolated tutorials, Neal has created a comprehensive learning course. You can download all videos to your machine and follow along with Neal as he patiently explains every feature of Blender. Neal's explanatory style is impeccable: it is a pleasure to follow along and learn. You can downloads all these videos from sagefans.net.
A collection of tutorials at Blenderlinks: Finally, when you have mastered the UI, you can focus on a specific topic by looking through the links at Blenderlinks. The videos are created by many contributors, so the videos differ in sound quality and expository style. But if you want to learn a single topic, the videos at cgcookie linked from Blenderlinks often provide the perfect recipe.

Blender is a powerful software, and it compares well against software which costs thousands of dollars. With a little bit of effort, you can learn how to create an entire 3D world inside your computer. It is scriptable in Python, which allows you to use Blender 3D models from an automated system. Give Blender a try, and see what you think!