You can't get there from here

I like looking at Reddit because it brings the distance between the page I'm viewing and an interesting page somewhere on the Web that I wouldn't otherwise know about down to one. Wikipedia has many interesting pages, most of which I don't know about. To all intents and purposes, a random page on Wikipedia is interesting. But I haven't seen it yet, and I don't know what it's about. How long will it take me to get there, in number of clicked links, if I start at the Wikipedia home page?

This is a measure of the connectedness of Wikipedia. But not the general symmetric connectedness in the sense of "Six Degrees of Kevin Bacon". There's already Omnipelagos for that. It will tell you that "David Beckham" is connected to "Bend It Like Beckham" which is connected to "Box office" which is connected to "Pierce Brosnan". And it's true that David Beckham" links to "Bend It Like Beckham" which links to "Box office", and that "Pierce Brosnan" links to "Box office"...but that doesn't help me get to "Pierce Brosnan" (the random page which Reddit would otherwise have linked to) if I'm physically looking at the "Box office" page.

No, if I start from David Beckham, I have to click like crazy: Los Angeles Galaxy -> Major League Soccer -> Major League Baseball -> List of highest paid baseball players -> Derek Jeter -> Mariah Carey -> Sean Puffy Combs -> Broadway -> Hugh Jackman -> Pierce Brosnan. This is basically PageRank: "the likelihood that a person randomly clicking on links will arrive at any particular page.

You may wonder why I am going on about something that Google have already calculated for every page on the Web. Indeed, the PageRank of the Pierce Brosnan page is minutely related to its distance from the David Beckham page. But all the pages that (transitively) link to Pierce Brosnan are included in his page's score, even if many of those pages can't be reached from David Beckham. PageRank typically assumes you can start from any page; I only want to start from one. So my measure is simpler than PageRank, but more useful because it is personalised. Personalised to a specific and very important subset of the Web: what I can see from where I am. That matters, because in Wikipedia, you don't search for interesting pages, top down; you find them, bottom up. (This may not be a effective use of your time, of course :-)

In the end, there will be an average distance between pages A and B in an undirected graph, and an average distance between the same two pages in a directed graph. The difference between the two averages represents two things: 1) the current lack of connectedness of Wikipedia, and 2) a small fraction of the worldwide cost that HTML has imposed by only having unidirectional links.

Now it's just a matter of downloading Wikipedia's HTML and coding some graph algorithms...

Films you rarely see

I went to the ICA on Friday to see "Everyday Except Christmas" (1957; a day in the life of the Covent Garden fruit&veg market) and "The London Nobody Knows" (1967; a tour of various points of non-interest in a city before Starbucks and steel-and-glass). I was very pleased to recognise Pump Court in the Middle Temple (where the last London gas-lighter worked) and found my way afterwards to Carting Lane (off the Strand) to inspect the everlasting lantern powered by sewer gas rising up the hollow lamppost!

One of the places in the film was Bankside, a pre-Tate power station looming in the background. Among the working wharves, littered with cranes, the presenter points out a house whose facade dates from the 1800s when the house is 300 years older. I wondered if it was the same house I'd seen a book about earlier in the week, and indeed it is! (The white one.) Cardinal Cap Alley and Clink Street complete the charm of the area, though the Borough area was a lot less desirable to walk around in 1867 and 1967 than 2007.

PS If you have five minutes, listen to Richard Schiff and Armando Iannucci discuss US v. UK political drama.

Science studies

Reading University are closing their physics department because, oh well, no-one wants to study science and, guess what, you can't make people study something they don't want to study. I find this response in particular very annoying:

Liberal Democrat education spokeswoman Sarah Teather said what had happened highlighted the vital importance of igniting an interest in science early on in education.

"It's a terrible cycle that must be broken. We need more science graduates as expert teachers in school to fuel demand in the next generation for science courses at university," she said.

What? We need more people studying science at university to ensure that more people want to study science at university? This is the worst sales pitch for the intellectual challenge of a 'hard science' degree, ever.

The irony is that in the next 20 years, Britain does need more science graduates, but not because we need more scientists. Instead, we need more business analysts, and the secret of physics courses at top universities is that they are recruiting grounds for management consultancies. (Sarah Teather, a NatSci graduate, was a consultant before becoming an MP.)

Secret passageways

Some that I have come across in London:

• Cockpit Steps/Birdcage Walk, Westminster - charming
• Victoria Road/Kynance Mews, South Kensington - pretty
• Ennismore Street/Rutland Street, Knightsbridge - surprising
• Vineyard Mews/Houblon Road, Richmond - useful (to me :-)
  

However, the City really has them all beat; take this outstanding walk to see why.

Update - The guide to alleys, courts, passages and yards of central London!

Cannot Compute

A letter in Metro yesterday (30/8/06):

I write in response to your piece saying the future of the NHS's £12.4billion Electronic Health Record is in doubt. According to experts, the IT project 'may never work because of the sheer size of the NHS'. Yes, and Google may never work because of the sheer size of the Internet. The sad truth is that if, instead of designing software by committee and hiring expensive underperformers to implement it, they hired some smarter people, the project would have cost a fraction of what it has and would have been completed in a lot less time than ten years. But governments and City corporations alike will never learn when it comes to IT.

Well said, Bas Scheffers of London SW12!

Closures and currying

By Tennent's Principle of Correspondence: A closure with body C and a free variable x whose closed value is v, is equivalent to a (curried) function with body C and a formal parameter x which was bound to an actual parameter of v in a partial invocation. (Up to the evaluation strategy for x.)

The Mythical Man Month

Idly flipping through, I discovered some apposite quotes. The famous chapter "No Silver Bullet" has this to say on whether OO is a revolutionary advance or an incremental one:

Such advances [classes,subtyping,encapsulation] can do no more than remove all the accidental difficulties from the expression of the design. The complexity of the design itself is essential...an order-of-magnitude gain can be made by object-oriented programming only if the unnecessary underbrush of type specification remaining today in our programming language is itself responsible for nine-tenths of the work involved in designing a program product. I doubt it.

Who can doubt that Brooks today would be for dynamic typing, and that he would add Python and Ruby to the list of "Exciting products" on page 203, to complement Unix and Smalltalk?

On the limitations of high-level languages as silver bullets:

At some point, the elaboration of a high-level language becomes a burden that increases, not reduces, the intellectual task of the user who rarely uses the esoteric constructs.

On the problem of screen size in graphical programming:

The so-called 'desktop' metaphor of today's workstation is instead an 'airplane-seat' metaphor. Anyone who has shuffled a lapful of papers while seated in coach between two portly passengers will recognize the difference.

On specifications:

Formal definitions are precise. They tend to be complete; gaps show more conspicuously, so they are filled sooner. What they lack is comprehensibility. With English prose, once can ... delineate structure in stages or levels, and gave examples ... most importantly, one can explain why. The formal definitions put forward so far have inspired wonder at their elegance and confidence in their precision. But they have demanded prose explanations to make their content easy to learn ... I think we will see future specifications to consist of both a formal definition and a prose definition.

I also discovered that chapter 4 ("Aristocracy, Democracy, and System Design", on the need for conceptual integrity) is where I learned the difference between "simple" and "straightforward". Most of computer science is simple ("as measured by the number of distinct elementary concepts") but is not straightforward ("The expression of the things one wants to do often requires involuted and unexpected combinations of the basic facilities. It is not enough to learn the elements and rules of combinations; one must also learn the idiomatic usage, a whole lore or how the elements are combined in practice.") An important lesson is that "Simplicity and straightforwardness proceed from conceptual integrity".

Now, you don't often see it noted, but Brooks believed in modularity but not encapsulation. He thought every programmer should have ready access to details about any part of the system, through a "project workbook". He commented:

D. L. Parnas has proposed ... that the programmer is most effective if shielded from, rather than exposed to the details of construction of system parts other than his own. This presupposes that all interfaces are completely and precisely defined. While that is definitely sound design, dependence upon its perfect accomplishment is a recipe for disaster. A good information system both exposes interface errors and stimulates their correction.

In retrospect, Brooks decided that giving everyone access to each other's code is wrong: "I am now convinced that information hiding, today often embodied in object-oriented programming, is the only way of raising the level of software design." The irony is that Brooks' original belief was because:

Harlan Mills has argued persuasively that 'Programming should be a public process', that exposing all the work to everybody's gaze helps quality control, both by peer pressure to do things well and by peers actually spotting flaws and bugs.

which has a certain familiarity! Of course, in a good open source project, this gaze is so strong that it trumps a bad programmer's ability to check in "smelly" code that relies on implementation details elsewhere in the program. Open source projects also frequently have a "benevolent dictator", which is just Mills' Chief Programmer. Consequently, it would be fascinating to know what Brooks makes of the development practices used in Linux, Python or Apache.