Harshdeep 2.0

January 27, 2009

The Making of Latest in Music – Using Youtube API

Filed under: Latest in Music, Programming, Video — harshdeep @ 2:12 pm

This is the second part of the Making of Latest in Music trilogy (here is part 1). Latest in Music is a Youtube mashup that scrapes the listings of top songs from various websites and shows their music videos by searching for them on Youtube. This is the first time I built a web application and I’m amazed by how quickly one can build something interesting with the available tools.

An important part of LiM is interaction with Youtube. Youtube generously provides an API to access the service. Initially I didn’t feel the need to use it. All I had to do was search for a song. This can be done simply by inserting the query string at the right place in the URL – http://www.youtube.com/results?search_query=<my_query_string> and fetching that page. However, this approach turned out to be insufficient for two reasons.

Firstly, some videos on Youtube cannot be embedded on external pages (like this one). Extending the basic approach to determine whether a video is embeddable would require another http page fetch. With Youtube API however, it is only a matter of setting an additional parameter in the search request (format=5).

Secondly, the initial users complained that there is no way to play all songs one after the other. I figured this could be done by creating a Youtube playlist with those songs. This definitely required the use of the API.

Using the API was pretty straightforward. But I learnt a few things the hard way. This might be a useful read if you are going to use the Youtube API for the first time

  1. Login

    The generic Google login does not work for the Youtube API calls that require authentication. It probably works fine for other Google APIs like the ones for Google Docs. But for Youtube API, you’ll get “Service Forbidden” errors with it. You need to create a login specifically on Youtube.

  2. HTTP version

    Youtube API requires the HTTP version 1.1. If you are using Ruby (version >1.6), the default http version is 1.2 and that causes errors. You need to call Net::HTTP.version_1_1 before sending any requests to make sure the Google servers are happy with you.

  3. API call frequency

    If you make a lot of Youtube API calls in a short time, you would start getting Forbidden errors. I couldn’t think of a better way to handle it than reducing the call frequency artificially by putting a sleep between them.

Advertisements

January 21, 2009

Useless Comments

Filed under: Uncategorized — harshdeep @ 9:20 pm

I am sure that when Randall Munroe published this XKCD cartoon…

… he had not seen Amitabh Bachchan’s blog. Here is a sampling of comments on the latest post of India’s most popular person –

Ravi g says: January 19, 2009 at 2:29 am
Dear Sir,
waiting for ur post for so long……..great to see you back….thanks to you..

Nikki (London) says: January 19, 2009 at 2:50 am
Parnam mere dost
Good morning, I don’t know where you are at this moment, but I’ve just got back from the gym after a 45 min workout, 30 min swim, 15 min jacuzzi, 15 min sauna, feel totally pooped but refreshed.
About to start the eve family meal of khadhi chawal, so will enter your world laters, my love.
Nikki

Shachi says: January 19, 2009 at 2:50 am
I dont belive is becoming a fan of a person without meeting that person….
dont kknow why but i am a huge fan of urs….
ur in perfect in everything in the world i can think of…
i am not from ur field… but for sure wana be like u at 67 all overloaded with energy… full of passion ….
and most imp. i am glad ur not in plotics…

Joshua David says: January 19, 2009 at 3:08 am
Dearest Amit ji,
Happy to read that U had a safe journey and are back home.. It snowed in scotland today, i guess the city realised that the warmt of Amit ji was not with them any more !!!
Sir ji do reply to my comment at least once… Please..
Do take care
Luv u hamesha
Luv n respects
Joshua D
Glasgow

shakir says: January 19, 2009 at 3:10 am
Dear Sir,
please sir show some more pictures like this.

… you get the drift. There may be some well thought out comments worth reading in there but they are lost in this sea of abject fan adulation. That makes it more like a newspaper column than a blog – no two way communication.

Update: There is a Firefox extension that removes ‘undesirable’ comments from the Youtube comment stream – Youtube Comment Snob. It uses various rules to identify the comments to be removed including the formatting (all caps/no caps) and use of profanity. I’m sure the same rules could be used to weed out undesirable comments from other sites (like Big B’s blog) as well in a generic way.

January 14, 2009

The Making of Latest in Music – Ruby on Rails

Filed under: Latest in Music, Programming — harshdeep @ 11:28 am

Last week I unleashed www.latestinmusic.com to the unsuspecting world. Keeping in touch with music is never going to be the same again. You don’t go finding the new songs, they come to you (in your RSS reader).

Coming back to Planet Earth, it’s a modest little site that I thought would be useful for me. Hopefully it would be useful to others as well. It took me less than a week to build it. This being my first web application, I was learning as I went – a seasoned web developer would probably take less than a day.

I used Ruby on Rails for development. The decision was primarily based on all the hype that the platform has been getting for simplicity and elegance. In my case, the hype turned out to be completely justified. Ruby, as a language, is sheer pleasure to write code in. Rails takes care of the mundane low-level things like maintaining connections with the databases, providing a set of powerful abstractions to work on top of. It does take some time getting used to and there is definitely a lot of scope of improvement in documentation, but once you cross the initial hurdles, it lets you be very productive.

One of the hang-ups that I have from my desktop/mobile development experience is the availability of an all-encompassing IDE like Visual Studio, XCode and Eclipse. Nobody should have to do serious development in Notepad anymore. Thankfully I discovered Aptana RadRails pretty early. It’s not without its share of annoying bugs, but I think it does the job pretty well. You can edit Ruby code, JavaScript code, layouts in .html.erb files and css stylesheets in the coziness of the same IDE and it lets you visually debug the code while running the application locally. TextMate is a popular choice of Ruby developers but it is available for Mac OSX only. Aptana is cross-platform.

The next question was where to get the application hosted. I first tried GoDaddy because I’d used it before to register domains. They do support Ruby on Rails but for some reason I could not get my app running with them. I tried contacting their customer care and they duly told me that it’s not them, it’s me.

Being a newbie, I thought it would be easier for me to host my app with one of the new hosting providers that focus exclusively on Ruby on Rails apps. Surely enough, I could set it all up with HostingRails in a couple of hours. Their FAQs section turned out to be particularly useful.

Overall, it was fun working with Ruby on Rails. I’ll hopefully use it for more projects.

January 12, 2009

Latest in Music

Filed under: Latest in Music, Music — harshdeep @ 6:18 am

This time when I came to visit Delhi, I realized that I’m completely out of touch with Bollywood music. When I stayed here, it was literally impossible to not know the latest chart busters by heart. You listen to them on FM in car (the distances and traffic of Delhi means you spend quite a lot of time in the car) and on TV at home – even news channels play Bollywood music sometimes. They are inescapable.

It’s a totally different story in the US. Sure there are ‘top 10’ websites where you can look up the names of the popular songs. To listen to them, I would generally search for individual songs on Youtube. Very inconvenient.

Latest in Music

All I want to do is stay in touch with the latest in music. So I wrote a script to scan those song listings on the web, search for the songs on Youtube and show all the videos at one place – www.latestinmusic.com. The script also generates an RSS feed so that the videos of new songs appear in my aggregator (I use and highly recommend Google Reader) automatically. All those songs are also assembled in a playlist using Youtube API so I can hear them one after the other without having to manually switch between them.

The song listings are scrapped from Dishaant, Planet Bollywood and Raaga. Once I was done with this, it was not too hard to do the same thing for English songs as well, scrapping the song listing from iTunes’ US and UK top lists here.

Youtube search with a song title (along with the movie name for Bollywood songs and artist for English ones) is definitely not guaranteed to return the actual song as the first result but it’s uncanny how often it works right. Check it out for yourself and let me know if you have any comments or suggestions.

Blog at WordPress.com.