A Few Words...

A Few Words...

Mark Pruett  //  programmer / writer

Jul 11 / 4:46pm

Posterous now supports Github/Gist code in posts (geeks only)

A new post over at Posterous announces that they now support embedding of GitHub Gist code snippets. You just put a Gist URL on a line by itself, and Posterous automatically expands it.

Here's a Perl example I created at GitHub, saved at a URL at http:// gist.github.com/145433:


I wasn't aware of GitHub or Gist before this, but I can certainly put it to good use. Gist supports a lot of programming languages, and can syntax-highlight C, Perl, Python, XML, Java, and a bunch of others.

Comments (0)

Jul 5 / 4:38pm

Visualizing Twitter

twitter_viz01

Twitter let's you post a 140-character opus out into the ether. Blog entries don't have this 140-character limit -- the end result is that tweets outnumber blogs over the same period of time.
 
Another key difference is the interconnectedness of twitter users. You follow people and they follow you. This creates an interesting web of users, with messages passed through these connections ("retweets"), and often-rapid conversations taking place.
 
On top of all this, the Twitter folks provide an extensive API (Application Programming Interface) that lets us write programs to
harvest this vast field of data.
 
But how do you visualize this ocean of data? A lot of people are developing applications that try to make sense from this seeming chaos. The Twitter API lets you get a list of any user's followers and friends (though the term "friends" has now been replaced with "following").
 
I decided to see if I could graphically display all my friends and followers.

The Twitter API

This is a RESTful API, so all you need to get the data is a well-formed URL and a tool that can talk HTTP. My scripting language of choice is Perl, so I picked the LWP::Simple Perl module. It comes installed with most recent Linux distributions (Linux is my OS of choice).
 
First I needed to construct two API URLs: one to get all my friends ("following"), and the other to get my followers. Here's what I used
for me (twitter user @afewwords):

http://twitter.com/friends/ids.xml?screen_name=afewwords
http://twitter.com/followers/ids.xml?screen_name=afewwords

I wrote some Perl code to make these requests. The data I get back is in XML format, so I used the XML::Simple Perl module (available at http://search.cpan.org/ ) to parse the XML and return a list of Twitter IDs. These numeric IDs aren't the twitter user's screen name, you'll have to use yet another API call to get that.

Visualizing Friends and Followers

So now I've got two lists of IDs: friends and followers. Put another way, I have people who I point to (people in my "following" list), and
people who point to me (people in my "followers" list). Graphviz, an open source visualization program ( http://www.graphviz.org/ ) makes it easy to visualize networks (data where nodes connect to each other via arrows).
 
The Graphviz tools read data in their own programming language, called DOT. Here's a simple DOT program:

digraph HelloWorld {
 "me" -> "you";
}

I can use the dot program to convert this program to a graph. Here's the Graphviz command-line to do this:

dot -Tps -o simple.ps simple.dot

This command takes the program, saved in the file simple.dot, and outputs the file simple.ps, a graph image in postscript format. [Graphviz supports many output image formats, including PNG, SVG, and Postscript.] Here's the graph: 

dot_simple

I added a Perl subroutine to create a DOT program from my friends and followers lists. The Perl program then feeds the DOT program into a Graphviz program for conversion. Graphviz includes separate command-line programs to render different types of graphs. I experimented with all these programs, but settled on neato. Neato is designed to draw pictures of undirected graphs.
 
Nodes on the graph, my friends and followers, are represented in different colors. My node, labeled "afewwords", is at the center.

Other twitterers are connected to me by arrows. Arrows move outward from "afewwords" to the people I follower -- these nodes are green. Twitterers who follow me are orange, and these nodes have arrows pointing inward toward "afewwords". Nodes in red are twitterers who are both following and followed by me.

twitter_viz01
 
Showing the Twitter IDs isn't as useful as the actual twitter names. To do this, you need to make a Twitter API call for each ID, and you quickly run into the dreaded Twitter API rate limit. An option to return the screen names of friends and followers would be helpful.
 
Now I'm curious what other information can be gleaned from Twitter. My first attempt only looks at my immediate vicinity -- I could map out another level, showing friends of friends, and so on. Or I might try to weight the graph, attempting to divine the "rank" of each node by how many followers that user has. People are already doing interesting things with the timeline and search
functions in the Twitter API. These are ripe for visualization. I'm open to suggestions.

Filed under  //  graphviz   linux   perl   twitter   visualization  

Comments (0)

Jun 28 / 12:34pm

Embedding a Google Map in Posterous

Adding a Google Map in Posterous is easy now too. Below is a map showing the route I take to work when I ride my scooter.
 

View Larger Map
To do this in Posterous, first go to http://maps.google.com and create your map. Then click on Link near the top right of Google Maps. At this point you can either just copy and paste the HTML contents (the one that starts "<iframe...") or click "Customize and preview embedded link" to refine the embedded map's appearance.

Comments (0)

Jun 28 / 9:58am

Posterous Enables IFRAMEs

Okay, this is incredible responsiveness from the folks at Posterous.  It's Sunday, and I'm playing with my new Posterous blog.  I'd like to embed content using IFRAMEs, but Posterous filters out IFRAMEs tags. 

So I send a short message to them, suggesting IFRAMEs would be A Good Thing.

Within a couple of hours, I get an e-mail from Gary Tan:

I've just enabled iframes. Please give it a try and let me know how it works for you.

Now that's fast.  On a Sunday.

Services like Google Maps and Google Books let you embed their content using IFRAMEs.  For example, I've embedded "The Principles of Wave Telegraphy", and early book on radio, below:
 

To embed this Google Book, I first find it at http://books.google.com, then click on the link near the upper right:

google_books_link1

Then I can paste the Embed snippet into my Posterous post.  That's it!

This is very cool.  Next, I'll test Google Maps.

Comments (2)

Jun 24 / 12:16pm

A Yahoo! Pipes / Twitter API Example

I wanted to see where people are tweeting from, so I turned to Yahoo! Pipes to see if I could mashup the Twitter API and Pipes.

Twitter What Where

If you haven't used Yahoo! Pipes, its a web-based tool for manipulating data that's accessible on the web. You don't need to host anything yourself -- you just need a Yahoo! account. Anything beyond simple Pipes may challenge non-programmers, but the Pipes editor is visual -- you drag and drop modules and then wire them together. When you run most Pipes apps, you'll see a list of items output. This same information can also be output as RSS, JSON, or a number of other formats. But if your Pipes output has geographical info in it, like latitude and longitude, then the output is automatically rendered in an interactive Yahoo! Map:

This Pipe uses two Twitter APIs as data feeds. The first is Twitter Search. You simply craft a URL in the right format, and Twitter sends back data in XML. The Pipe user enters a search term and Pipes embeds that in the Twitter URL, calls Twitter and returns the XML for processing. But the Twitter Search returns little information about the person who tweeted. I can get their screen name from the item.author.name field with a little regular expression magic.

Ypipes_twitter_whatwhere1
 
Next I wrote a submodule, Twitter User Location, to get Twitter user information for each tweet item in the Twitter Search results. This information comes back as XML that includes an item.location element. This contains whatever the Twitter user assigned as their location. If its actually a place, like "Washington, DC", or "Paris, France", then the Pipes Location Extractor module can geocode that information and return a latitude and longitude.
 
The Location Extractor module only looks at certain XML elements, so I have to do a little creative renaming to get it to work. In this case, I copied the Twitter element "location" into the RSS element "description".

Comments (0)

Jun 22 / 1:03am

Embedding a Static Google Map

This is a simple test to see if I can embed a static map in a posterous
blog.
 

 
If it works, a map of Richmond, VA is above this sentence.

Comments (0)

Jun 16 / 11:53am

A First Post (and some code)

Embedding example code in a blog post would be useful.  Let's see how it works:

#include <stdio.h>
int main (void) {
    printf ("Hello, world.\n");
}

That's not too bad.  Unfortunately, in my browser (Firefox), the code shows up double-spaced.  It doesn't look like it does in the editor.  This could be annoying, especially for longer code snippets.  [It does look somewhat better under Internet Explorer.]

Late addition:  Actually, it looks okay from my newer version of Firefox.

Comments (0)