Showing posts with label php. Show all posts
Showing posts with label php. Show all posts

Saturday, December 11, 2010

Uploading to Freebase, part II: authenticating with OAuth

I'd hoped to have written the bulk of human knowledge to Freebase by now, but I came to a screeching halt when I found that I'd need cookies and sessions and such.

That is, you have to authenticate to write data in bulk to Freebase. Here's one way to do so using OAuth and PHP.

1. Sign in to Freebase and register an app. Take note of your Consumer Key and Consumer Secret.

2. Get oauth-php and add it to a directory where your code can see it.

3. On the page from which you'd like users to authenticate, include the following code (adapted pretty directly from the Twitter example):

require "oauth-php/library/OAuthStore.php";
require "oauth-php/library/OAuthRequester.php";

/**
* oauth-php: Example OAuth client
*
* Performs simple 2-legged authentication
*
* The MIT License
*
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*/

// register at http://www.freebase.com/apps/create and fill these two
define("FREEBASE_CONSUMER_KEY", "FILL IN");
define("FREEBASE_CONSUMER_SECRET", "FILL IN");

define("FREEBASE_OAUTH_HOST","https://api.freebase.com");
define("FREEBASE_REQUEST_TOKEN_URL", FREEBASE_OAUTH_HOST . "/api/oauth/request_token");
define("FREEBASE_AUTHORIZE_URL", "https://www.freebase.com/signin/authorize_token");
define("FREEBASE_ACCESS_TOKEN_URL", FREEBASE_OAUTH_HOST . "/api/oauth/access_token");

define('OAUTH_TMP_DIR', function_exists('sys_get_temp_dir') ? sys_get_temp_dir() : realpath($_ENV["TMP"]));

// test
$options = array('consumer_key' => FREEBASE_CONSUMER_KEY, 'consumer_secret' => FREEBASE_CONSUMER_SECRET);
OAuthStore::instance("2Leg", $options);

try
{
// Obtain a request object for the request we want to make
$request = new OAuthRequester(FREEBASE_REQUEST_TOKEN_URL, "GET");
$result = $request->doRequest(0);
parse_str($result['body'], $params);

echo $result['body'];

}
catch(OAuthException2 $e)
{
echo "Exception" . $e->getMessage();
}

?>


When you load that code, you should see a token, good for at least one POST to Freebase. (I hope -- I'm writing this up as I go.)

Please stay tuned for the next exciting installment of Uploading to Freebase!

Saturday, December 27, 2008

My first Drupal module(s)!

I'd nearly given up learning the Drupal content management system, after it took me two hours to change a font on a static web page. But I needed to procrastinate something even more difficult than changing fonts in Drupal, so I decided to try, try again.

The first interesting thing I wanted to try in Drupal was feed aggregation. I love the idea of feed aggregators like Tumblr, FriendFeed, and Swurl, but I'm unsatisfied with the limitations on what or how many feeds you can use. The output (from Tumblr in particular) can also be rough-looking. Why oh why can't they handle quotation marks in titles right. I'm no regex champion, but I suspect even I, PHP fledgling, could write the fix to that. Surely they've noticed that posts titled "Barack Obama - "Yes We Can"" aren't very appealing? (I had to go out of my way for Blogger to let me make it that ugly.) Finally, I'd like to enhance the output of at least one feed with additional content from my own database, which is completely impossible with Tumblr, Swurl, or FriendFeed.

So I've been checking out Drupal's feed aggregators. I tried Aggregator and FeedAPI my first day using Drupal, so I doubt I assessed their usability fairly. I'll try them again when (if) I understand CCK and Views. But when I tried the ActivityStream module, my enthusiasm for Drupal was restored! Not only did it do significant portions of what I want done (Flickr, last.fm, and delicious feeds), but it looked dead easy to add new feeds by making new sub-modules.

It was indeed dead easy. To add a Goodreads sub-module, I copied and pasted the delicious sub-module. I opened the two delicious files and replaced the word "delicious" with "goodreads". I opened my module list and saw my new module. I turned it on, entered my Goodreads info, and holy shit it worked! My recent Goodreads activity was interspersed with the Twitter, delicious, last.fm, and Flickr info I'd already added. I decided sleep could wait until I made five modules like my Goodreads modules, for Blogger, StumbleUpon, YouTube videos, YouTube favorites, and Hulu.

My new modules are far from perfect. I have a list a page long of improvements I want to make (like handling quotation marks in titles...). But the beauty of Drupal, and open source in general, is that I have access to the code to make those improvements. All I can do with Tumblr is gripe and hope they fix it.

Thus far, it seems to me that in Drupal, it's much more fun doing hard things like feed aggregation than doing easy things like changing fonts. I think this explains why people hate Drupal, then love it. To beginners doing something "easy" like trying to change fonts and put files where they want them, Drupal seems like a ridiculous thing to be excited about. To people who've just done something "hard" that they've wanted to do for years in the space of ten minutes (that's what she said), Drupal seems pretty awesome.

Wednesday, December 17, 2008

How to write a playlist maker script

I got an email asking for tips on writing a playlist maker script.

Using PHP, my process was this:
1. Choose a playlist format. My favorite playlists are Windows Media Player WPL files, which use the SMIL subtype of XML. There are tons of other types of playlists. Songbird and Winamp both use M3U, for example. Unfortunately, most playlist file types, including M3U, are not as "smart" as WPL files, in that they want the exact path to a single song on a user's computer, rather than just an artist name or song title. WPL files will take an artist or song name and give you a playlist of everything in your library that matches. I don't understand why Songbird and Winamp don't handle WPL or SMIL files (yet?). After I figured out which playlist format I wanted, I right-clicked a WPL file I'd made, opened it in Notepad, and used it to cut and paste the top, bottom, and repeating bits of the playlist code where needed in the following steps.

2. Find a source of artists or song names you want in your playlist. The best way was to use an RSS feed, but for my iLike and last.fm tag playlist makers, I scraped the source code of some web pages, which is messy, but it works. You can also offer a way to paste in or upload a list of artists.

3. Write an HTML form (called, for example, input.html) that takes user input (like a last.fm username) to get the source you want. The "Submit" button on the form will take you to your playlist making code (playlistmaker.php, for example).

4. Write code to convert your input (such as a last.fm username) into a URL to an XML or HTML file and get that file ready to be used. For example:
$handle = fopen("http://ws.audioscrobbler.com/1.0/user/".$username."/topartists.txt?type=overall", "r");

5. Write code to make an array of only artists or song names using your source. The exact code will vary depending on the source of artists or song names. This is the trickiest part, and I'd give examples of how I've done it, but it's different nearly every time, and I'm sure my ways are not especially elegant. One thing I'd like to do better is parsing the actual XML instead of finding where the artists turn up in the array and using numeric indices to grab the artists or song names.

6. Create a variable that will contain the entire text you want in the playlist file ($wpl, for example). Paste in the top bit of the playlist text. For a WPL file:
$wpl = "<?wpl version=\"1.0\"?>
<smil>
<head>
<meta name="\" content="\">
<title>".$username."</title>
</head>
<body>
<seq>
<smartplaylist version="\">";

7. Write a loop that adds the section of the playlist that repeats for each artist or song name to your playlist variable ($wpl., for example). Concatenate in the artist or song name variable. Make sure you convert HTML characters, as ampersands will render your playlist utterly worthless. For example, where the artist names are in $data[2]:
while ($data = fgetcsv($handle, 1000, ",")) {
$wpl.="<querySet>
<sourceFilter id=\"{4202947A-A563-4B05-A754-A1B4B5989849}\" name=\"Music in my library\">
<fragment name=\"Album Artist\">
<argument name=\"condition\"<Contains>/argument>
<argument name=\"value\">" . htmlspecialchars($data[2]) . "</argument>
</fragment>
</sourceFilter>
</querySet>";
}

8. Finish off your playlist text variable by pasting in the end text of the playlist variable (again, $wpl.). For a WPL file:
$wpl.="
</smartPlaylist>
</seq>
</body>
</smil>";

9. Create a new file containing your playlist variable text as follows:
$handle = fopen("filename.wpl","w");

10. Offer a link to download the new file.

11. Try it out!

Optional tweaks:
12. WPL playlist files offer you the option of finding an artist name that "Contains" the word "Bell" (for example), or finding an artist that "Is" the word "Bell." After the band "Bell" made Belle and Sebastian turn up in a festival playlist I made (oh, the brief crushing excitement), I added a conditional so that if the artist name is 5 characters or shorter, I use "Is", and if it's longer, I use "Contains". It's still inexact, and it's an area I'd like to improve. Handling "The" (Pixies, Beatles, Raveonettes) is another thing I'd like to work out at some point. Here's an example of one of my conditionals where the artist names are in $data[2]:
while ($data = fgetcsv($handle, 1000, ",")) {
$limit=6;
$num=count($data[2]);
for ($i=0;$i<$num;$i++) {
$len[$i]=strlen($data[2]);
if ($len[$i]<$limit) {
$wpl.="<querySet>
<sourceFilter id=\"{4202947A-A563-4B05-A754-A1B4B5989849}\" name=\"Music in my library\">
<fragment name=\"Album Artist\">
<argument name=\"condition\">Is</argument>
<argument name=\"value\">" . htmlspecialchars($data[2]) . "</argument>
</fragment>
</sourceFilter>
</querySet>";
}
else {
$wpl.="<querySet>
<sourceFilter id=\"{4202947A-A563-4B05-A754-A1B4B5989849}\" name=\"Music in my library\">
<fragment name=\"Album Artist\">
<argument name=\"condition\"<Contains>/argument>
<argument name=\"value\">" . htmlspecialchars($data[2]) . "</argument>
</fragment>
</sourceFilter>
</querySet>";
}
}
}

13. Greasemonkey! Using a tiny bit of Javascript, you can put a link to your playlist making code on relevant source pages. Clicking this link will give you the option to look at one of my Greasemonkey scripts. You'll especially want to change:
  • @include to reference the page you'd like to put a link on.
  • The link to the script that makes your playlist.
  • @name
  • @namespace
  • Colors (I'm using last.fm grey. Meh.)

You can grab the username for the last.fm URL they came from by putting this at the top of your playlist maker script (playlistmaker.php, for example):
$url = getenv("HTTP_REFERER");
$elements=explode("/", $url);
unset($url);
$username=$elements[4];
unset($elements);

Here's a zip folder with code for the input form and playlist maker, as well as a Greasemonkey playlist maker for a last.fm listener's most-played artists. This was one of the first things I did when I was learning PHP, so it's far, far, far from perfect. But feel free to use any bits of it you like. I'm not litigious.


I am crawling up a learning curve (Drupal) myself right now, but if you have questions or suggestions, comments are open.

For anyone more interested in making playlists than playlist-maker scripts, my playlist makers are here.

Monday, September 15, 2008

I shoulda learned Java

Alas, my bank account has little more than tumbleweeds in it, so my year or so of un-/self-employment is coming to an end. I must job search in earnest and stop turning up my liberal elite nose at corporations looking for automated pollutant emitters and chicken torturers.

The question now is what programming skills such corporations want me to use to design their new improved Humvee fan sites, and since I've sent out my quota of three resumes today, I get to blog those skills.

As I've opened various job postings, hoping the single handful of programming languages I'm truly comfortable with are just what they're looking for (FORTRAN anyone? Anyone?), I've been keeping track of what they are, in fact, looking for. This is completely non-scientific, of course. One glaring flaw, for example: every third company wants a straight-up Java programmer. I want to know Java (that's how the awesome, awesome, awesome Legend of Zelda for your phone! was written), but I don't honestly know Java, so I don't open those ones.

Here are the programming languages/skills companies are looking for in order of popularity in my unscientific survey:

Javascript - 8 companies
Green Ventures, Inc.
Mystery Company
Auction company
Smarsh
Mystery Company
Dealerpeak
Inspiration
ConceroTechnology

HTML - 6 companies
Sharepoint
Smarsh
Intersoft
Mystery Company
Auction company
ConceroTechnology

Java - 6 companies
Amazon
Sharepoint
Axiom
Mystery Company
Dealerpeak
DB Professionals

Linux - 4 companies
Amazon
Mystery Company
Intersoft
VxWorks

PHP - 3 companies
Green Ventures, Inc.
Mystery Company
Intersoft

MySQL - 3 companies
Green Ventures, Inc.
Mystery Company
Auction company

Visual Basic - 3 companies
Smarsh
Axiom
Selectron

Visual Basic.NET - 3 companies
Lifeport
Sharepoint
ConceroTechnology

XML - 2 companies
Intersoft
Dealerpeak

Perl - 2 companies
Amazon
Intersoft

C++ - 2 companies
Amazon
Axiom

C# - 2 companies
Selectron
ConceroTechnology

C - 2 companies
VxWorks
Mystery Company

Coldfusion - 2 companies
Mystery Company
Dealerpeak

.NET - 2 companies
Sharepoint
Intersoft

CGI - 1 company
Intersoft

CSS - 2 companies
Smarsh
Mystery Company

ASP - 2 companies
Sharepoint
Smarsh

Flash - 2 companies
Auction company
Inspiration

SQL - 1 company
Dealerpeak

AJAX - 1 company
Green Ventures, Inc.

Joomla - 1 company
Green Ventures, Inc.

Drupal - 1 company
Inspiration

SQL Server - 1 company
Smarsh

ActiveX - 1 company
Smarsh

Saturday, April 19, 2008

My obsession with WMP playlists

For programming practice, I've been writing PHP scripts that will turn various lists of musicians into playlists for Windows Media Player.* Basically, you find a list of musicians that you'd like to listen to as a set, put that list in the appropriate script (see below), and out comes a playlist you can open with Windows Media Player to listen to your list of artists. For long lists of artists, this saves a lot of time over the usual method of putting them in one at a time.

The musician lists you can use for this purpose so far with my little tools are:
  • Last.fm Tagged Artists
    *Playlist maker
    *Example data source
    *Example playlist (rock.wpl)
    *Greasemonkey script (for users of Firefox with Greasemonkey installed)
  • Last.fm Calendar
    *Playlist maker
    *Example data source
    *Example playlist (portland408to808.wpl)
    *Greasemonkey script (for users of Firefox with Greasemonkey installed)
  • Last.fm User's Most-Listened Artists
    *Playlist maker
    *Example data source
    *Example playlist (jamidwyer.wpl)
    *Greasemonkey script (for users of Firefox with Greasemonkey installed)
  • iLike User Most-Played Artists
    *Playlist maker
    *Example data source
    *Example playlist (jami.wpl)
    *Greasemonkey script (for users of Firefox with Greasemonkey installed)
  • Eventful Calendar
    *Playlist maker
    *Example data source
  • Comma-Separated Values File
    *Playlist maker
    *Data source is a list of artists only saved as a CSV in Excel or OpenOffice Calc.
    *Example playlist (sasquatch.wpl)

    *I know Windows Media Player is not hip. But unbeknownst to even the tech-savviest, music-lovingest people I talk to, Windows Media Player's playlists will dynamically add and sort music by criteria you set. Instant mix tape! This was far more powerful than the playlists iTunes had back in 2004, when I took my first faltering steps to quit stealing the mp3 files I wanted, I discovered that iTunes had indeed sold me intentionally-crippled aac files that indeed wouldn't play on my phone's mp3 player, and I quit using iTunes in fury.

    So no, I will not be writing anything, ever, for intentionally over-priced, intentionally under-compatible Mac software. I would, however, like to write playlist makers for Songbird or Amarok, when I feel a little more bored of learning more PHP and MySQL and want to learn whatever it is I need to learn to write real programs.


    (Updated to fix links. Check for new playlist makers and updates on my code page.)
  • Tuesday, April 15, 2008

    Programming server problems

    The web server I'm using to do programming practice is giving me server errors when I load my PHP pages. This is the second time my site has been obliterated for at least a day by the server with no warning, and with lots of lecturing instead of trying to get my page running again. I don't write the tightest code yet, and I'm grateful for advice, but when three months' work is gone without warning, with no hint at when it will return, it might not be the best time for a lesson.

    Basically, nothing I've written for the past three months works right now. It will -- I'm moving everything to a new server (and trying to install my own Linux server at home). But if you're having trouble using any of my playlist makers or Podcast Finder, it's because I'm moving everything to a place where it'll work. Sorry for the inconvenience.

    Sunday, March 9, 2008

    Podcast Finder

    I've been making decent progress learning PHP and MySQL since I finally got started last month. My proudest achievement thus far is a podcast finder, called "Podcast Finder", that recommends podcasts based on a couple of podcasts you know you like.

    I'm a total podcast junkie (try doing PCR in a dead-silent lab for two years, then winning an iPod shuffle, and see if you don't turn addict). But finding new podcasts when you've exhausted an archive or ten is not as simple as giving your favorites five stars on Amazon or Netflix and seeing what they recommend. My old "system" to find new podcasts basically added up to Googling combinations of my favorites to see what else came up (zip), combing through all the most-dugg podcasts on Digg, and scouring the last.fm artists of other big NPR listeners.

    The last.fm method gives pretty good recommendations, but it's slow. Podcast Finder speeds it up by using a database (fairly small right now) of last.fm favorites. As with all my little projects, there's room for improvement, but if you're a podcast fan, I'd love for you to give it a try. If I'm missing your favorite (or you have any other feedback), feel free to leave me a comment here.

    Tuesday, February 12, 2008

    CSV to WPL

    Here's my first PHP and SQL program, called CSV to WPL. It creates Windows Media Player playlists from comma-separated lists of bands.

    It's a niche program that has football stadiums worth of room for improvement (it's not a beta -- it's an alpha), but I know I'll use it a lot just as it is.

    Update: I did use it a lot, but I've written various improved playlist maker scripts, including a version that allows you to cut and paste a list of artists instead of making a spreadsheet of them.