Skip to content

Rotate This RSS Revisited

3 July 2010, 11:23 pm

About five years ago, I wrote a script to scrape the web­site of Rotate This and repu­blish their con­cert tic­ket lis­ting as an RSS feed. Every six hours for five years I’ve hit their site and upda­ted my data­base. The RSS feed has quite a few users, using Feed­Bur­ner, Goo­gle Rea­der, Blo­gli­nes and a few other mis­ce­lla­neous RSS vie­wers. It’s one of the most fre­quently hit pages on my site.

I wan­ted to try out the new Goo­gle AppEn­gine ser­vice, and was loo­king for a sim­ple pro­ject, so I figu­red I could port my little scra­per. It see­med like per­fect fit.

Goo­gle AppEn­gine is a uni­que appli­ca­tion hos­ting ser­vice. It’s not really clas­si­cal web hos­ting, and it’s not really a vir­tual ser­ver. You gene­rate a Java web appli­ca­tion, bundle it as a .war file, and deploy it to AppEn­gine. Google’s magic voo­doo makes sure that your app is hos­ted on at least two ser­vers, that it will scale auto­ma­ti­cally for load, that it will start up quickly, and that it will always have access to its data. There are some res­tric­tions, howe­ver: no request to your app can take lon­ger than a few seconds to pro­cess, and there are strict size limits for data sto­rage access and request/response size. It can’t start threads, it can’t access the disk, it can’t use too much memory.

There is no real “bac­kend” access to your app either; all access must hap­pen though a URL. The Cron ser­vice simply calls a spe­ci­fied URL within your appli­ca­tion on regu­lar intervals.

A big advan­tage to AppEn­gine is that their basic ser­vice is free. The quo­tas for requests, data trans­fer and CPU time under the free ser­vice level are very high. Anything that excee­ded them I would call a very suc­cess­ful app — it is unli­kely I will ever have to buy more capa­city. The free ser­vice inc­lu­des over a million requests and a gigabyte of data trans­fer per day.

I spent a cou­ple of days wor­king on it — AppEn­gine was easy to learn but had a few quirks. My older code was in Perl (the scra­per) and PHP (the RSS gene­ra­tor) using MySQL as a bac­kend. The AppEn­gine code is all Java, using Google’s datas­tore through Java Data Objects. I had never used JDO before, so there was a bit of a lear­ning curve there. Everything else was straight for­ward, using the URL Fetch ser­vice to get the Rotate This tic­ket page, and using the Cron to sche­dule the scrape every six hours.

On the out­put side I was able to take advan­tage of the rich Java ecosys­tem, using ROME to gene­rate the RSS feed, and using ICal4J to gene­rate the new iCal feed that I was adding on. Both wor­ked fine on AppEngine’s res­tric­ted Java runtime.

So, here are the links, give it a try!

Rotate This Tic­kets RSS Feed: http://rotatethisrss.appspot.com
Rotate This Tic­kets iCal Feed: http://rotatethisrss.appspot.com/ical

Let me know how it goes!

p.s. I redi­rec­ted the old link to AppEn­gine — if you were using this before, you shouldn’t have to change anything.

Filed under general.
2 Comments

Roomba 560 Repair!

21 June 2010, 1:09 pm

My Roomba 560 was making clun­king noi­ses as it drove around… I think it was the safety clutch somewhere in the brushes dri­ve­train. I took out the brushes and clea­ned as much cat hair as pos­si­ble, but there was some jam­med in one of the square dri­ves in the side of the brush hou­sing. I couldn’t get that out, so I deci­ded to take apart the Roomba and its brush module and see what was going on.

Disas­sembly was easy. There are quite a few different-sized screws, so keep them orga­ni­zed… but take it easy, don’t force anything, and you should have no trou­ble get­ting into the gears. I lubri­ca­ted things with some Tri-Flow synthe­tic grease (which is safe for plas­tic) and put it back together.

Here’s a little video I made of the pro­cess:
YouTube Preview Image

Watch it in HD for best results! Note the screw that got away when I pulled off the spin­ning brush at the front… I found it again, but take some pre­cau­tions so this doesn’t hap­pen to you as well.

Filed under general.
Comment

23 April 2010, 2:47 pm

Filed under general.
Comment

Social Network Downsizing

22 April 2010, 4:32 pm

A few days ago, I got a sud­den fee­ling that I was on too many social net­works and my infor­ma­tion was spread too far out of my con­trol on the inter­net. I was also rea­ding a lot about Face­book’s pri­vacy pro­blems and their ero­sion of user con­trols in order to bet­ter mone­tize their infor­ma­tion 1, 2, 3. During the pro­cess of quit­ting Face­book, the site sho­wed me pic­tu­res of my friends, and told me that they would miss me. It said “your 153 friends won’t be able to com­mu­ni­cate with you any­more!” Scary.

I was also regu­larly using Fours­quare, which is a social net­wor­king appli­ca­tion that lets you “check in” at various phy­si­cal loca­tions and see who else is there. If you check in to a place more than anyone else, you become “Mayor,” and if you check in at the right sequence or num­ber of pla­ces, you can win “bad­ges.” This was fun for a while, I was Mayor of seve­ral pla­ces, and I had lots of bad­ges, but I didn’t feel I was get­ting anything out of it, and Fours­quare was get­ting a lot of valua­ble infor­ma­tion. I for­got to “check in” at work for a few days, and when I rea­li­zed this, it was time to finish with Foursquare.

I also had seve­ral near-dormant accounts with Ins­tant Mes­sage ser­vi­ces. I had an MSN account, a Yahoo account, an AIM account and a Goo­gle Talk account. I’m on Lin­ke­dIn, and Twit­ter, Last.Fm, You­Tube and Vimeo, and who knows what other ser­vi­ces I don’t remem­ber at this point. Too many.

I have been using Pass­pack to orga­nize my online pass­words, which I’ve been happy with. They have a very strong secu­rity policy, a sane pri­vacy policy, and I pay for ser­vice, which feels a bit more safe as well. As I’ve been chan­ging pass­words and ente­ring them into Pass­pack, it has given me a view of just how many accounts I have! I have over 40 ente­red so far, and there are a lot more to go.

So, I’ve been clea­ning this up a bit. I’ve deci­ded that I want to focus on my blog/website, my Twit­ter, and my Goo­gle Talk. I’m going to share videos with You­Tube, at least until Vimeo impro­ves their iPhone sup­port. Almost everything else, I’m going to get rid of. If you hap­pe­ned to talk to me on one of the ser­vi­ces I’ve dele­ted, you can always find me here. Check the “Con­tact Info” box on my blog, or just call me on the phone.

Remo­ved so far:

  • Face­book
  • Fours­quare
  • Mic­ro­soft Live Messenger
  • Yahoo Mes­sen­ger
  • AOL Ins­tant Messenger
  • Last.fm
  • Lin­ke­dIn
  • MyS­pace

… and more to come.

So far, this feels pretty good! I’ve been focu­sing on ser­vi­ces that feel less evil, don’t try to collect as much per­so­nally iden­tif­ying infor­ma­tion, and who are at least more subtle about their attempts to mone­tize me.

Filed under general.
3 Comments

Controlling MasterConsole IP 116 with Groovy

13 April 2010, 3:46 pm

At work we have a KVM switch for our test rack, the very nice Mas­ter­Con­sole IP 116, from Rari­tan. The switch has a com­pe­tent web user inter­face, but no published auto­ma­tion API. I guess it’s rea­so­na­ble to assume that nobody would want to auto­mate a KVM switch, but once in a while, you might find a need.

We have a large screen hoo­ked into the switch’s video out­put, and we want to switch which input we’re watching on a fairly regu­lar basis. One click access would be ideal, but the web UI for­ces you to log in, select the desi­red input from a drop­down menu, and click a “switch input” button.

I wrote a small Groovy script using the Htm­lU­nit tes­ting fra­me­work to auto­ma­ti­cally navi­gate the web UI, which was totally pain­less and didn’t require any reverse engineering.

Check out the script… you might find the con­cept use­ful for auto­ma­ting your own interactive-only device.

package com.onestopmedia.kvmcontrol

import com.gargoylesoftware.htmlunit.WebClient
import com.gargoylesoftware.htmlunit.html.*

class KvmControl {
  public static main(args) {
    def cli = new CliBuilder (usage: "KvmControl")
    cli.u(longOpt: "username", "set username", args: 1)
    cli.p(longOpt: "password", "set password", args: 1)
    cli.i(longOpt: "input", "set input (0-15)", args: 1)
    cli.q(longOpt: "query", "query current input only")
    cli.h(longOpt: "help", "usage information")

    def opt = cli.parse(args)
    if (!opt) return
    if (opt.h) cli.usage()

    def webClient = new WebClient();
    def page = webClient.getPage("http://192.168.7.3/auth.asp") as HtmlPage
    def loginForm = page.getForms().get(0) as HtmlForm

    loginForm.getInputByName("login").setValueAttribute(opt.u)
    loginForm.getInputByName("password").setValueAttribute(opt.p)
    def button = loginForm.getInputByName("action_login") as HtmlImageInput
    def page2 = button.click() as HtmlPage

    def dataFrame = page2.getFrameByName("data")
         .getEnclosedPage() as HtmlPage
    def selectForm = dataFrame.getForms().get(0) as HtmlForm
    def inputSelect = selectForm.getSelectByName("ECG_kvm_active_port")
        as HtmlSelect
    if (opt.q) {
       def currentlySelected = inputSelect.getSelectedOptions()[0] as HtmlOption
       println ("currently selected: "+currentlySelected.getText()
          +" ["+currentlySelected.getValueAttribute()+"]")
    } else {
       inputSelect.setSelectedAttribute(opt.i,true) as HtmlPage
       def switchButton = selectForm.getInputByName("action_switch")
       def page3 = switchButton.click() as HtmlPage;
    }
  }
}
Filed under geek.
Comment

Ever wonder why your Wi-Fi is slow?

10 April 2010, 9:23 pm

I can see 62 dif­fe­rent access points from my apart­ment. Only two or three are carrier-grade hard­ware, the rest are just people’s home rou­ters. I can only see three that are on 5GHz though, and one of those is mine. If you’re in a really WiFi-dense area (ie, anywhere down­town) you will get quite a bit of bene­fit from a 5GHz (802.11n) upgrade, at least for a while…

Filed under general.
Comment

OS X Chrome Bookmarks Export

4 February 2010, 12:31 am

google_chrome_logo
Chrome for OS X is lac­king a book­mark export fea­ture. I’m sure it will have one in the future, but it doesn’t have it yet. It will hap­pily import your mozi­lla book­marks and let you get star­ted using the brow­ser, but even­tually you will notice that your book­marks are loc­ked in. It only recently got a very rudi­men­tary book­mark mana­ger in the latest nightly builds, but there is still no export.

The Chrome book­mark file in OS X is a sim­ple JSON data­base, so I whip­ped up a quick tool in Groovy to do the export. This was really a single-use pro­ject but I figu­red I would post the source and bina­ries in case someone else needs it.

Usage: java -jar ChromeExport.jar > mybookmarks.html

By default it will find the current user’s “Default” Chrome pro­file. Since Chrome doesn’t seem to have any UI for mana­ging pro­fi­les, I don’t see why you’d have a dif­fe­rent one.

Requi­re­ments to build/run:
* Groovy 1.6+
* Commons-Lang (for the Strin­gEs­ca­peU­tils)
* json-simple (for the JSON parser)

Here is a link to the source: ChromeExport.groovy
Here is a link to the fully bund­led binary (which pro­bably breaks somebody’s license agree­ment somewhere, but com­pliance is not worth the anno­yance): ChromeExport.jar

Enjoy! Let me know what you think, or if it works for you!

Filed under general.
8 Comments

Hacklab Introduction to PIC Microcontrollers

17 September 2009, 10:46 am
Basic PIC Circuit with PICKit3

Basic PIC Cir­cuit with PICKit3

Yes­ter­day I did a small lecture/demo intro­du­cing seve­ral mem­bers of the Hac­klab to the Mic­rochip PIC plat­form, gave a walkthough of a basic, free toolchain for deve­lop­ment, and demons­tra­ted using the PICKit3 programmer.

In the sli­des you will find links to impor­tant resour­ces, a sche­ma­tic for the sam­ple cir­cuit, and source code for a mini­mal pro­gram to test your circuit.

Thanks to Andrew Kil­pa­trick for hel­ping out with the resour­ces and ans­we­ring ques­tions at the presentation!

Here are the sli­des from that presentation!

PIC Pre­sen­ta­tion Sli­des 20090916

Filed under geek.
Comment

Simplification

17 July 2009, 9:54 pm

I’ve been tid­ying up some things around the condo, mostly in the IT realm. I’m trying to reduce the num­ber of devi­ces, cables, con­fi­gu­ra­tions, and gene­ral anno­ying­ness in my daily exis­tence. So far, this is what is new:

  • AppleTV + 1.5TB USB Drive

And this is what I’ve eliminated:

  • MediaPC Run­ning Myth­Buntu (a cons­tant source of aggravation)
  • Net­work Switch
  • Soe­kris Net4501 Router
  • Gamer PC Run­ning Windows
  • Long ugly phone wire run­ning from bedroom to living room
  • About six cat5 patch cables
  • Four wall warts

So right now, my inter­net comes out of the wall and runs to the clo­set where the DSL modem and the Air­Port Express sit. The Air­Port is now my rou­ter, and the net seems fas­ter. My Mac­Book and the AppleTV both con­nect wire­lessly (802.11n). The AppleTV sup­ports Air­Tu­nes and also gets a syn­ced copy of my iTu­nes library, so my music is Just There. Same with my pho­tos. Much much easier. We’ll see how long it lasts.

The only pro­blems with the AppleTV so far: No remote volume con­trol when Quick­Time plays a video. And it doesn’t seem to be able to put my screen to sleep over HDMI… I have to use the power button.

Filed under general.
Comment

Roomba Update

20 April 2009, 11:03 am

roomba_discoveryRoomba and I have been get­ting along pretty well over the last few months. It has been clea­ning my floor regu­larly, and doing a good job at it. It gets dust bun­nies from pla­ces that I never did with the vacuum, and it usually makes it home to its char­ging base.

I recently rebuilt its bat­tery pack… the bat­te­ries have a finite lifes­pan, and living in a Roomba is a hard life. They get deple­ted and fully char­ged regu­larly, and the little robot uses a lot of current.

I bought a rebuild kit off of eBay, which inc­lu­ded a pre­pa­red set of cells (already con­nec­ted together, with leads ready for the roomba) and the secu­rity screw­dri­ver to open the pack. I’m not sure why they used secu­rity screws — the pack is also glued clo­sed and requi­red SIGNIFICANT effort to open. We’re tal­king vio­lence here. But it did even­tually yield to my screwdriver-and-mallet technique.

The new bat­tery is wor­king… run times are nice and long, the bat­tery char­ges pro­perly, and it hasn’t burst into fla­mes yet.

Filed under general.
Comment
« Older Entries