Uploading a Drupal module using TortoiseCVS

I just added the yolink enhanced search module to drupal.org.

The last hurdle in getting this module out in the world was adding it to CVS, which I prefer to do using the Tortoise CVS GUI. Following these directions, everything was going great. And it would have gone great right through to the end, but in the Create New Module step, I missed the part where I changed the path to the module. My module wasn't showing up in the contributions directory, and I kept getting errors that adding a branch wasn't allowed in /module_name.

Lots of Googling and some fail-dread-based procrastination later, I found this video, paused after every sentence, and managed not to skim over the critical module path setting. CVS let me add a branch with no errors in sight (cue angels singing the name of kyl191).

If you're looking to add a module to Drupal.org with TortoiseCVS, both of the linked sets of directions should work -- if you follow them!

Uploading to Freebase, part II: authenticating with OAuth

I'd hoped to have written the bulk of human knowledge to Freebase by now, but I came to a screeching halt when I found that I'd need cookies and sessions and such.

That is, you have to authenticate to write data in bulk to Freebase. Here's one way to do so using OAuth and PHP.

1. Sign in to Freebase and register an app. Take note of your Consumer Key and Consumer Secret.

2. Get oauth-php and add it to a directory where your code can see it.

3. On the page from which you'd like users to authenticate, include the following code (adapted pretty directly from the Twitter example):

require "oauth-php/library/OAuthStore.php";
require "oauth-php/library/OAuthRequester.php";

/**
* oauth-php: Example OAuth client
*
* Performs simple 2-legged authentication
*
* The MIT License
*
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/

// register at http://www.freebase.com/apps/create and fill these two
define("FREEBASE_CONSUMER_KEY", "FILL IN");
define("FREEBASE_CONSUMER_SECRET", "FILL IN");

define("FREEBASE_OAUTH_HOST","https://api.freebase.com");
define("FREEBASE_REQUEST_TOKEN_URL", FREEBASE_OAUTH_HOST . "/api/oauth/request_token");
define("FREEBASE_AUTHORIZE_URL", "https://www.freebase.com/signin/authorize_token");
define("FREEBASE_ACCESS_TOKEN_URL", FREEBASE_OAUTH_HOST . "/api/oauth/access_token");

define('OAUTH_TMP_DIR', function_exists('sys_get_temp_dir') ? sys_get_temp_dir() : realpath($_ENV["TMP"]));

// test
$options = array('consumer_key' => FREEBASE_CONSUMER_KEY, 'consumer_secret' => FREEBASE_CONSUMER_SECRET);
OAuthStore::instance("2Leg", $options);

try
{
// Obtain a request object for the request we want to make
$request = new OAuthRequester(FREEBASE_REQUEST_TOKEN_URL, "GET");
$result = $request->doRequest(0);
parse_str($result['body'], $params);

echo $result['body'];

}
catch(OAuthException2 $e)
{
echo "Exception" . $e->getMessage();
}

?>


When you load that code, you should see a token, good for at least one POST to Freebase. (I hope -- I'm writing this up as I go.)

Please stay tuned for the next exciting installment of Uploading to Freebase!

How to upload to Freebase part I: domains and types and properties, oh my!

I've been admiring the Wikipedia of databases, Freebase, from afar for a long time. I've kept my distance because if you think learning wiki syntax is a challenge, try figuring out where to even put factoids about your favorite books or musicians on Freebase. If you want your data to be useful to more people than just yourself, though, it is critical that you get the organization right. In honor of Open Data Day last weekend, I've decided to finally figure out how to upload data to Freebase (instead of adding three things by hand and giving up when the knowledge that I could add 1000 with a little PHP gets too unbearable).

The first step when you'd like to add data to Freebase is pretty easy: determine if your data would be useful to other people, and if you have a right to upload it. If the answer to both is yes, proceed!

But then we're on to step 2 -- where does your data belong? You need to determine your data's structure. My first through tenth passes at adding to Freebase probably went through the Basic Concepts wiki, which falls prey to the downfall of many a wiki page -- really bad organization. (It looks like someone played a hand of Yahtzee with all the abstract concepts you need to understand to put your data in the right place on Freebase, then wrote it up in wiki form. I'd re-organize, but every attempt to cut down redundant information and leave things in reasonable order on Wikipedia has left me reversed, scolded, and frustrated.)

So let me try to clarify here (feel free, Freebase wiki editors, to grab any content that's useful, but please don't just sprinkle it hither and thither within the page).

My first hope that I might make sense of Freebase yet came from the Freebase Schema Explorer app.

Just the name gave me hope. The schema is the structure of the data, so a schema viewer is just what I was in the market for.

Right away, I saw that Freebase data is organized in domains, like Books (accessed at http://www.freebase.com/view/book).



Domains (like Books) have types, like Poem, Short Story, and Book (accessed at http://www.freebase.com/view/book/book). This could be confusing, because other users, probably baffled by the wiki like me, have added things like ISBN and Book Character as types of books. I know we're in data hippie land and everyone is a special flower, but that's frankly wrong. "Book Character" is not a type of book.



Types (like Poem, Short Story, or Book) can have both instances and properties, accessed at http://www.freebase.com/view/book/book. For the domain Books of type Book, an instance would be something like The Catcher in the Rye. (It seems that Freebase calls instances "topics", but the Schema Viewer nails it better with "instance," I think.) Examples of book properties include Characters and Genre.



For the data I'll probably add to Freebase first -- podcasts -- the domain is Broadcast (/broadcast), with the type Podcast Feed (http://www.freebase.com/view/broadcast/podcast_feed).

As of this writing, there are 2,584 Podcast Feed instances/topics. Their properties include Name (example: Wired's Alt Text), Image, and Average Media Length. Already I see a flaw, which I'll need to correct if I'm going to use this data for my Podcast Finder -- there's no podcast creator (or Podcaster) property listed. And the Freebase wiki noted that one can't edit a schema created by another Freebase user -- I'd have to duplicate all 2,584 instances/topics and set up my own schema.

Will I figure out how to add the Podcaster property? Will I duplicate the Podcast Feed type? Will I lose my Internet connection because I stayed up too late blogging, slept through a WebEx, and lost my job?

Find out in the next installment of this exciting series, wherein I shall explore yet another query language, MQL, and hopefully start adding all information ever to an easily-queried free online database.

(Or I'll take another two years off blogging and come back in 2012 blogging about how excited I am about our new lady President. Or arsenic-based space aliens. Stay tuned!)

Comparing Amarok and Songbird

I love the tremendously uncool Windows Media Player (WMP), primarily because it has portable smart playlists. A WMP smart playlist can go with you to a new computer and still find music by the artist(s) of your choice, whereas an iTunes Smart Playlist(TM) will be looking for specific file locations. That makes a huge difference in what you can do with your music.

I've hoped for a while that open source music players for Linux like Amarok and Songbird would implement smart, XML based playlists instead of playlists that save file locations. Unfortunately, neither does yet. But while I was checking out their playlists, I decided to compare Songbird and Amarok on a few other criteria.

In addition to XML playlists, both players lacked a couple of other important features. Neither Amarok nor Songbird showed any interest in syncing music to my SD card, and neither deleted a file from my computer when I deleted it from my library.

After an initially more difficult install, Songbird edged out Amarok for usability. Amarok had too many "little" bugs (like not playing mp3s or scrobbling to last.fm). Sure, you can probably fix the problems, but they worked out of the box with Songbird. Amarok wins handily in terms of podcast support, but it wasn't enough to put it over the edge as the media player I'll keep.

Songbird
Pros: Plays mp3s right after install. Scrobbles to last.fm right after install.
Cons: Not easy to install. Podcast support not as strong as Amarok's. Doesn't play wav files.

The installation process was not as easy as Amarok's (in fairness, I didn't have to tweak Songbird afterward to get mp3s and scrobbling to work).

I had a brief flash of hope that I'd get my SD card synced by installing the FolderSync add-on. But after installation, I couldn't find any evidence of FolderSync's existence, and I couldn't understand the directions.

Songbird kind of has podcast integration through subscriptions, but nothing showed up with updated Onion or Buzz Out Loud subscriptions. Apparently not all feed formats work.

There are other cool things I haven't explored about Songbird: web pages as playlists, love and ban for last.fm right in the player, Flickr pics related to music you're listening to, Twitter integration.

Songbird Summary
Plays mp3s after install? Yes
Last.fm scrobble? Yes, by default
Truly smart playlists? No, just iTunes-style dropdown playlists
Sync SD card? Nothing obvious there by default
Integrated podcasts? Kind of
Rip CDs? No
Delete file when removing from library? No
Plays wav files? No
Apt-get install? No

Amarok
Pros: Easy install with package manager. Podcast integration better than Songbird's. Plays wav files.
Cons: Doesn't play mp3s right after install; you have to install an extra package. Last.fm scrobbling requires work.

To get mp3s to play (!), I had to install libxine1-ffmpeg. That's a number 1 after libxine, not a letter l.

Though it is integrated, I never got last.fm scrobbling to work. Apparently on Ubuntu 9.04, you have to upgrade Amarok, but after an hour hacking at it, I gave up.

Amarok added The Onion podcast using its integrated OPML database and Buzz Out Loud via pasting in the RSS feed. Both had updated and showed files within an hour.

Amarok Summary
Plays mp3s after install? No!
Last.fm scrobble? In theory
Truly smart playlists? No, just iTunes-style dropdown playlists
Sync SD card? Nothing obvious there by default
Integrated podcasts? Yes
Rip CDs? No (but maybe with a hack)
Delete file when removing from library? I think no
Plays wav files? Yes
Apt-get install? Yes

Tools of the Twitter famous

I enjoy Twitter, but I'm looking to improve a few things. The worst flaw I've found is that when people write me directly using my username or a direct message, I don't see it for days. I don't click the individual links to show me those messages. So through sheer bad design, I miss the parts of the Twitter conversation that are most important to me.

Another flaw is keeping track of followers. If someone is hitting me with a lot of sandwich tweets or a wall of twenty automated tweets in a row, I want to weed them out. If they're following me back, though, I'll think twice about unfollowing, as it might insult them and I'd probably lose a follower out of it. Unfortunately, there's no integrated way to see if people follow you back.

And then there's design. I like the simplicity of the Twitter interface, but I'd like to at least try something besides a single, static, tiled image.

Perhaps the many, many Twitter tools available could solve my problems. I decided to look first at tools used by Twitterers I like with more than a thousand follows.

This is unscientific, of course. Lance Armstrong, Rainn Wilson, and Martha Stewart could tweet using stone tablets and they'd get thousands of followers. But tools used by busy people probably at least provide efficiency.

In the end, none of the tools I tried solved the three main problems I set out to correct. But one tool, TweetDeck, at least shows replies and direct messages on a single page by default.

Here is a list of tools used by twitterers I follow who have more than 1000 followers.

TwitterBerry
The most popular Tweeter I follow is Lance Armstrong, with 860,957 followers. As of this writing, he had recently used both TwitterBerry and Tweetie. TwitterBerry, as suggested by the name, is a tool for BlackBerry mobile devices. If you have a BlackBerry, TwitterBerry is good enough for Lance Armstrong, so you should probably check it out. I don't have a BlackBerry and don't want one, so I won't review TwitterBerry. Other tweeters who use TwitterBerry: GarinKilpatrick (18,097 followers), SistersTalk (5,276 followers), and PGHolyfield (1,048 followers).

Tweetie
Tweetie is like TwitterBerry, but for iPhones. Popular users of Tweetie include Lance Armstrong (860,957 followers), Rainn Wilson (665,607 followers), Trent Reznor (515,302 followers), Stephen Fry (500,730 followers), ElijahManor (25,048 followers), GeekyGirlDawn (2,274 followers), Verso (2065 followers), TheSuicideKing(1,480 followers), and EvilSpinmeister (1,400 followers).

As with the BlackBerry, I don't have an iPhone and don't want one, so I didn't check this out, but if you have an iPhone, this app seems to very popular with quality tweeters.

Mobile web
Martha Stewart (620,374 followers) is the third most popular twitterer I follow. She posts frequently using "mobile web", which just increases my fondness for her that much more. She's, what, late sixties? and instead of thinking she needs a separate tool to "do this Twitter whatsit" like most intentionally-obtuse older celebrities, she just fires up IE or Opera on her phone and does it. Bravo! Wildly popular feline twitterer Sockington (506,681 followers) also uses the mobile web interface.

TweetDeck
In addition to Tweetie, Trent Reznor uses TweetDeck. TweetDeck is a tool for any major platform (that is, Windows, Mac, or Linux). Other popular users of TweetDeck are The_GMan (68,246 followers), StyleTime (20,683 followers), DataDirt (10,277 followers), Mur Lafferty (3,290 followers), SawHtoo (3,086 followers), SWhitley (2,292 followers),

The three-column design in TweetDeck solved my number one problem right away. Within five minutes of finishing my install, I got an @reply from an IRL friend I wouldn't have wanted to miss.

You can also create groups, so you see your most important followers (i.e., your real friends) first.

I need to change my profile pic, though. I used a full-size giant creepy photo of my face, to embarrassing effect in TweetDeck. It's amazing I've ever gotten a followback from a TweetDeck user.

TweetDeck Summary
Clicks required to see replies and direct messages: Zero
Easily see if someone you're following follows you back? No
Works on PC? Yes
Works on Linux? Yes
Send to Twitter, Facebook, or both? Yes
Easily add people to groups? Yes
Easily add RSS feeds? No
Have to install Adobe Air? Yes
Easily manage multiple accounts (not necessarily a good thing)?

Web
In addition to using the mobile web, Sockington is my most popular Twitter follow who frequently uses the plain old web to update. Good to know I haven't been hobbling myself by loading up twitter.com in Firefox to tweet. Other folks who hit the web to tweet: GeraldWeber, BreakingNewz (28,689 followers), Garin Kilpatrick (18,097 followers), Paste Magazine (18,039 followers), Timothy Adam (17,489 followers), Dana Willhoit (17,051 followers), and Steph Agrestra (11,294 followers), DataDirt (10,277 followers), Cloaker (10,205 followers), Alexandre Gervais (8348 followers), SistersTalk (5,276 followers), Jesse Thorn (5,270 followers), PlayTwivia (4,647 followers), Bitch Magazine (4,575 followers), GraphicRiver (3,518 followers), John Roderick (3,446 followers), ApeLad (3,221 followers), SawHtoo (3086 followers), TweetDr (2,610 followers), Go_GetIt (2,470 followers>, TechMemeFH (2,087 followers), BikePortland (1,597 followers), WomanistMusings (1,469 followers), Dave Hill (1,243 followers), WebChick (1,102 followers).

Web Summary
Clicks required to see replies and direct messages: Two
Easily see if someone you're following follows you back? No
Works on PC? Yes
Works on Linux? Yes
Send to Twitter, Facebook, or both? Yes, with work
Easily add people to groups? No
Easily add RSS feeds? No
Have to install Adobe Air? No
Easily manage multiple accounts (not necessarily a good thing)? Yes

Seesmic Desktop
The next most popular Twitterer I follow is Ustream (453,327 followers). Ustream uses Seesmic Desktop, which works on Mac and PC, but apparently not Linux, and looks similar to TweetDeck.

Seesmic's most obvious advantage looks like its ability to manage multiple accounts. That seems spammy and yucky to me, but one person's spammy and yucky is another person's livelihood, I guess.

Though it is not the default setting to show replies and DMs, there are two links to show those, and you can probably set it up to show those without clicking. I never figured out how to add a friend to a friend group.

Seesmic Summary
Clicks required to see replies and direct messages: Two
Easily see if someone you're following follows you back? No
Works on PC? Yes
Works on Linux? No?
Send to Twitter, Facebook, or both? Yes
Easily add people to groups? No
Easily add RSS feeds? No
Have to install Adobe Air? Yes
Easily manage multiple accounts (not necessarily a good thing)? Yes

Twitli
In addition to the web interface, GeraldWeber also uses Twitli, a Twitter tool for Android phones. Now, unlike BlackBerry and iPhone, I do want an Android phone, but as I'm stuck with Windows Mobile until I strike oil or find gold, I can't yet review Twitli.

Ping.fm
My next most popular Twitter follow, ChrisPirillo (61,885 followers), updates using Ping.fm. Ping.fm is an awesome tool that links one social network to another, and you should check it out whether you want to use it for Twitter or not. Someone posting to Twitter using Ping.fm may be using his Facebook status or FriendFeed or blog to post to Twitter. Ping.fm is a little dangerous, as you can end up spamming people you like with multiple links to the same thing, so if you use it, try to keep track of what's going where. And of course, if you want to keep parts of your online life separated, be very careful with Ping.fm.

Ping.fm Summary
Clicks required to see replies and direct messages? N/A
Has easy way to see if someone you're following follows you back? N/A
Works on PC? Yes
Works on Linux? Yes
Has way to send to Twitter, Facebook, or both? Yes
Easy to add people to groups? N/A
Easily add RSS feeds? Yes
Have to install Adobe Air? No
Easily manage multiple accounts (not necessarily a good thing)? N/A

Twitter Tools
After ChrisPirillo, the next most popular twitterer in this "study" is SitePoint (32,918 follows). SitePoint uses Twitter Tools, a WordPress plugin to tweet. This may be very useful for folks with WordPress blogs, but it's somewhat specific to those users. BikePortland (1,597 followers) also uses Twitter Tools.

bit.ly
In addition to TweetDeck, Styletime posts using bit.ly's bookmarklet. You put the bookmarklet in your browser toolbar, then click when you find an interesting link you'd like to share on Twitter.

Nambu
In addition to the web, PasteMagazine (18,039 followers) uses Nambu, a desktop Twitter application for Macs. Meh. Rasmus (2345 followers) also uses Nambu.

Twitterfeed
PasteMagazine also uses twitterfeed, which, like Ping.fm, will pipe the RSS feed of your choice to Twitter. Twitterfeed looks more versatile than Ping.fm, in that it seems to take any RSS feed, not just pre-approved ones as on Ping.fm. Such services can be used well with care, but it can also lead to the second most annoying Twitter behavior (after spam follows) -- posting an entire page of tweets at the same time. Like PasteMagazine, Alexandre Gervais (8,322 followers), SistersTalk (5,276 followers), and BitchMagazine (4,575 followers) use twitterfeed so I never noticed until I wrote this blog post. SciSolutions (2,051 followers) uses twitterfeed badly, posting ten updates at a time.

Txt
The most popular twitterer I follow who recently used a text message to update was Cloaker (10,205 followers).

ShareThis.com
In addition to the web and twitterfeed, AlexGervais (8,322 followers) used a ShareThis button at the bottom of a blog post to post to Twitter.

Digg
AlexGervais (8,322 followers) also used the Digg toolbar to post something he saw on Digg to Twitter.

CoTweet
In addition to the web interface, GraphicRiver (3,518 followers) uses CoTweet, a business-oriented tool currently in private beta testing.

Twitterific
In addition to Tweetie, Verso (2,065 followers) uses the iPhone app Twitterific. BikePortland (1,597 followers) also uses Twitterific.

HootSuite
IHeartBrunch (1,586 followers) uses HootSuite.

HootSuite's unique feature seems to be that multiple people with different passwords can post to a single Twitter account.

HootSuite Summary
Clicks required to see replies and direct messages? Two
Has easy way to see if someone you're following follows you back? No
Works on PC? Yes
Works on Linux? Yes
Has way to send to Twitter, Facebook, or both? Yes
Easily add people to groups? No
Easily add RSS feeds? Yes
Have to install Adobe Air? No
Easily manage multiple accounts (not necessarily a good thing)? Yes

Twhirl
Sarah Conner (1,282 followers) uses Twhirl, similar to TweetDeck and Seesmic, but very compact. PGHolyfield (1,048 followers) also uses Twhirl.

Twhirl Summary
Clicks required to see replies and direct messages? Two
Has easy way to see if someone you're following follows you back? No
Works on PC? Yes
Works on Linux?
Has way to send to Twitter, Facebook, or both? No (need Ping.fm)
Easily add people to groups? No
Easily add RSS feeds? No (need Ping.fm)
Have to install Adobe Air? Yes
Easily manage multiple accounts (not necessarily a good thing)? Yes

Twitterfon
TheBuzzByBrian (1,154 followers) uses Twitterfon, an iPhone app.


I'd love to hear about your favorite Twitter tools in the comments.

Blog status update

I'm deciding how best to separate code posts from music posts from angry political rants. Appreciator will return to its originally-scheduled cultural programming soon. Hope to see any readers who've encountered this tumbleweed of a blog then!

My first Drupal module(s)!

I'd nearly given up learning the Drupal content management system, after it took me two hours to change a font on a static web page. But I needed to procrastinate something even more difficult than changing fonts in Drupal, so I decided to try, try again.

The first interesting thing I wanted to try in Drupal was feed aggregation. I love the idea of feed aggregators like Tumblr, FriendFeed, and Swurl, but I'm unsatisfied with the limitations on what or how many feeds you can use. The output (from Tumblr in particular) can also be rough-looking. Why oh why can't they handle quotation marks in titles right. I'm no regex champion, but I suspect even I, PHP fledgling, could write the fix to that. Surely they've noticed that posts titled "Barack Obama - "Yes We Can"" aren't very appealing? (I had to go out of my way for Blogger to let me make it that ugly.) Finally, I'd like to enhance the output of at least one feed with additional content from my own database, which is completely impossible with Tumblr, Swurl, or FriendFeed.

So I've been checking out Drupal's feed aggregators. I tried Aggregator and FeedAPI my first day using Drupal, so I doubt I assessed their usability fairly. I'll try them again when (if) I understand CCK and Views. But when I tried the ActivityStream module, my enthusiasm for Drupal was restored! Not only did it do significant portions of what I want done (Flickr, last.fm, and delicious feeds), but it looked dead easy to add new feeds by making new sub-modules.

It was indeed dead easy. To add a Goodreads sub-module, I copied and pasted the delicious sub-module. I opened the two delicious files and replaced the word "delicious" with "goodreads". I opened my module list and saw my new module. I turned it on, entered my Goodreads info, and holy shit it worked! My recent Goodreads activity was interspersed with the Twitter, delicious, last.fm, and Flickr info I'd already added. I decided sleep could wait until I made five modules like my Goodreads modules, for Blogger, StumbleUpon, YouTube videos, YouTube favorites, and Hulu.

My new modules are far from perfect. I have a list a page long of improvements I want to make (like handling quotation marks in titles...). But the beauty of Drupal, and open source in general, is that I have access to the code to make those improvements. All I can do with Tumblr is gripe and hope they fix it.

Thus far, it seems to me that in Drupal, it's much more fun doing hard things like feed aggregation than doing easy things like changing fonts. I think this explains why people hate Drupal, then love it. To beginners doing something "easy" like trying to change fonts and put files where they want them, Drupal seems like a ridiculous thing to be excited about. To people who've just done something "hard" that they've wanted to do for years in the space of ten minutes (that's what she said), Drupal seems pretty awesome.