Dread Pirate PJ’s House of Hacks and Tricks

I’ve been at my new job only four weeks, and my first project is nearly finished. My task these past four weeks has been the re-implementation of the company’s website in Rails. I was to do this from scratch, while keeping the site’s design and existing link structure. Among my requirements is adding support for multiple languages, as the company wants to show it knows how to make websites for a diverse, multi-ethnic audience. Another requirement was, that I was to do this re-implementation by myself. The company only has one other experienced Rails developer, and he is busy doing maintenance work on older non-Rails projects for company customers.

It’s been a pleasure working on this project, and I am very pleased in the progress I’ve made in such a short time. I look forward to going live in another two to three weeks, once the site has been through QA and any changes required.

—

This coming Wednesday, May 16, I am off to RailsConf 2007, in the lovely city of Portland, Oregon. Portland is great, I have been there twice and I’ve had a great time each trip. I look forward to the visit very much.

Two of my coworkers from the Williams F1 project, Kyle Drake and Nick Wright, are going to be attending RailsConf as well. It’s gonna be great to meet them face-to-face. Daniel Browning, one of my fellow coworkers at my new job, lives just north of Portland. My college buddy Ken Williams lives in the Portland area too. I plan to meet with them at some time during this, my third visit to Portland.

There are so many sessions I want to attend at RailsConf. But interestingly enough, I feel I won’t get any value out of the tutorial sessions, so I didn’t sign up for any of them. That means I’m out of the tutorial-attending n00b league, yeah! We’ll see whether I can hang on to that thought in the other, more advanced sessions.

A few of the sessions sound downright dull. I guess some of the n00bs will find them interesting, since they’re looking for ‘insight’ and whatnot. Whatever.

What am I looking for from RailsConf? I’m looking for some nitty-gritty I can sink my teeth into, some new tech I can experiment with and learn more from. I’m looking forward to have my brain blown by something cool but complicated that I can learn over the next few months. I want to learn about some new-to-me techniques and plugins or gems that I can master and take my Ruby and Rails to a higher level.

I also look forward to meeting some people face to face. Tim Bray is going to be delivering the keynote on Saturday morning. If you’ve been reading my blog for the last year, you already know what I think of him: he’s the d00d!

The JRuby guys will be presenting one session, and I think their work is of importance for the growth of Rails outside the leading edge. I think JRuby will be the thing to help Rails cross the chasm and be adopted by mainstream.

At OSCON 2005, DHH signed my Agile Web Development With Rails 1st Edition, so I will try to get Robert Martin, Dave Thomas, and Andy Hunt to sign some of my other books that they wrote.

Anyway, I’ll talk to you all later!

Technorati Tags:
rails, ruby, hacks, open.source, web.architecture, web.technologies, railsconf

Last Wednesday, I sent emails and résumés for various job posts, and by that afternoon, I started getting nibbles. It was weird! Whereas I barely got any attention for over a dozen job applications I sent out between September and early February, I suddenly had five companies interested in me in a matter of hours! Maybe companies were waiting for post-holiday economy reports to start hiring? I wonder.

So to cut to the chase, I got a great hourly-pay remote development gig working on a Ruby on Rails project, on a really cool sounding website, for a very-high-profile vehicle racing company in the UK. It’s right down Ken’s alley, but if I’m not mistaken, this sport usually has a definitely more European racing audience. I think American racing fans like Ken, might resent these are not American cars, and just don’t watch. LOL

I cannot disclose any more information because of client confidentiality, but I think I may already have said too much. :-p

I’ll post a link here when the site goes up, at any rate. Only then can we drop all this silly NDA crap. It’s a bummer, because I can’t discuss application architecture specifics that would be of interest to my readers.

But fret not. There’s no rule about my discussing the project’s architecture in general. I just can’t say right out what architecture I’m dealing with on this gig.

I’ll be writing more about Rails in the weeks ahead, but just cuz I write about it, don’t mean it’s got a thing to do with work, you hear? :: rolls eyes ::

Technorati Tags:
rails, ruby, hacks, life, software.development, open.source, web.architecture, system.architecture, web.technologies, web.services

Mateusz, Lenny, thanks for the comments.

Update: Reader Mateusz points out that there is an effort under way to port the XNA frameworks to other platforms using Mono. It’s great to see what the subversive, creative types come up with all on their own. It would be hilarious if PS3 owners could run XNA games by simply booting Linux. Take that, Redmond! Welcome to our social. LMAO!

My own take is that Sony is taking a laissez-faire approach, a la Franklin D. Roosevelt and the economy: let’s provide the tools, and whatever happens, happens. Let the market take it where it may. (I hope I am not embarrassing Mrs. Freite, my history teacher in high school, by mixing my economic strategies and presidents.)

While that is great, I think they could be a bit more proactive. They could encourage garage game development with the PS3 Linux project, by marketing it as such. A program like the old PSX Net Yaroze, but with PS3s and Linux, could attract some great talent. (See? Sony has been thinking about this garage developer thing for quite a long time.)

IBM and Toshiba see this as another marketing venue for the Power processor line, especially the work on the Cell Engine that they did. Toshiba only promotes Cell within its embedded processor market, not putting an oar in to help with the garage developer thing. IBM is a bit more active, but only within their established promotional circles: developerWorks and alphaWorks. And it is hardly a big push.

Microsoft is, of course, going all out, promoting XNA as a garage developer’s haven, especially generating excitement with the possibility of garage developers selling games on XBox Live Marketplace for both PC and XBox360. In my opinion, MS is doing a better job at pushing their ideas.

It’s like IBM and Sony don’t quite see what it is they have to do, having just provided the tools and sitting back to see what develops. While it’s great they’re not stirring the pot too much, the stew is gonna ruin if there’s no cook minding the stove.

OK, that’s enough analogies for today. Thanks for keeping the conversation going, guys. Comment away, and let’s keep talking. Anyone else have any thoughts to share?

Technorati Tags:
hacks, software.development, game.development, open.source, free.culture, linux, windows, sony.ps3, xbox.360, ps3.linux, xna

Programming high-performance applications on the Cell BE processor, Part 1: An introduction to Linux on the PlayStation 3

The Sony PlayStation 3 (PS3) is the easiest and cheapest way for programmers to get their hands on the new Cell Broadband Engine (Cell BE) processor and take it for a drive. Discover what the fuss is all about, how to install Linux on the PS3, and how to get started developing for the Cell BE processor on the PS3.

Let’s ignore the $600 USD “cheapest” flame bait and concentrate on the business significance of Sony and IBM providing and promoting Linux for the PS3, shall we?

The PS3 is not the first game console officially sanctioned by the manufacturer to run Linux out of the box. For a few years, Sony sold a $99 Linux kit for the PS2, available only through their online store. It was discontinued early last year, and is now only available on eBay.

But why would Sony provide Linux for any of their video game consoles? What do they have to gain? In business, nothing is done “just” for the benefit of the consumer, much less for pure “free as in freedom” idealism. In the end, it all comes down to driving revenue and profits up, which ultimately pleases the shareholders. Cynical, perhaps, but do you really believe a public company as large as Sony does anything if not for the sake of revenue and profits? Any activity that does not somehow help improve those two factors, will not be seen very positively with the shareholders.

But still, what does it mean? How does PS3 Linux translate into revenue and profits for Sony? Let’s examine this some more, in the context of an obscure game console manufacturer with a similar strategy.

In Korea, the GP32 game console may not have run Linux, but the manufacturer provided on its website, a full set of open source programming tools that ran on Windows. Linux and Mac OS X support was added by the GP32 developer community shortly later. This made the GP32 the first off-the-shelf game console where the users were officially encouraged to make games for it. GamePark, the makers of the GP32, followed up nearly four years later with the Linux-powered GP2X, which also had a free, full set of development tools, including most of the source for the device’s implementation of Linux.

The response to the consoles was great, both in its native Korea and around the world. There is a healthy global open source game development community. Code-savvy game geeks from all around the world have taken to the GP32 and GP2X. Instead of dismal failure competing against Nintendo and Sony, GamePark has survived and grown against all odds. The gadget-happy Koreans responded despite fierce competition, and a healthy Korean commercial game developer community has grown to serve the Korean GP32 market. Many of these commercial developers were founded by the proverbial “two guys in a garage,” propelled to creativity by the open community GamePark fostered.

So back to the PS3. Sure, it’s freakishly expensive for just a single threaded, single tasking, web browser, digital photo gallery, photo slideshow player, propietary-format movie player, game console. But with Linux and the Blender Game Kit, OpenGL, SDL, etc., this is a garage game coder’s dream workstation on the cheap, with high-definition accelerated 3D graphics, surround sound, and all kinds of fun wireless tech at the developer’s fingertips. At $600 for a 60 GB model with 512 MB of RAM, it would make a nicer Linux computer than most sub-$1000 Windows PCs out there.

Perhaps a cottage gaming industry will grow out of the PS3 Linux strategy, like the GP32’s in Korea. Which makes this Linux play by Sony the direct competitor to Microsoft’s XNA move. You see, both Sony and Microsoft see independent game developers as the future savior of the game industry.

The big game publishers and developers screwed themselves, just as the MPAA and RIAA did, by making production ever more expensive, formulaic, and insipidly uncreative, and by depending on the quarterly mega-blockbuster to survive. They did this at a time when more people than ever have access to the means of production and distribution of digital creative content.

The goal of Sony’s Linux strategy and of Microsoft’s XNA, is to attract the independent game developers to their respective platforms. Microsoft believes that giving the tools away for “free as in free beer,” while keeping the secret sauce bottled up, is the successful strategy. Sony believes making the tools as open as possible without screwing the pooch is the the best strategy.

Now, as an open source fan, I will argue that a truly open community of coders, without central control and oversight, is going to prevail over the “you must please the gatekeeper” culture Microsoft wants to impose with XNA on XBox360.

But let’s not be too idealistic and see this independent game developer thing for what it really is. Just like IBM’s open source moves and Tim O’Reilly’s publishing strategy, both Microsoft’s and Sony’s strategies are all about the alpha geek mind share, and self-interest in the increase in revenue it generates.

But cynical as that may sound, open source is still the better thing. It gives more control to the developer and imposes less order, which leads to greater opportunity for creativity to go where it may. Anything less than an open technology and open community will stave off innovation before it even has a chance to set in.

Technorati Tags:
hacks, software.development, game.development, open.source, free.culture, linux, windows, sony.ps3, xbox.360, ps3.linux, xna

I receive some mailings from Sun every month. And for the most part, they are entertaining. As in laugh out loud hysterical. But this one takes the cake.

Easier, faster, and more secure the Solaris 10 OS is the only operating system robust enough to address enterprise security, compliance, and business continuity concerns, and at a lower cost and running across more hardware platforms than any other OS on the market.

Excuse me, but “more hardware platforms than any other OS on the market?” I don’t know what Sun’s definition of platform is, but last I checked, Solaris didn’t run on HP PA-RISC and IBM S/390s. Linux does, though. Linux runs on all three platforms that Solaris runs on **snort, yeah they’re beating their chest over three platforms** and then some. And I bet that when they say “OS on the market,” they mean commercially supported OS. Which conveniently disqualifies NetBSD, which is available for over forty platforms.

Hear from Sun’s Vice President of Software Marketing, Peder Ulander, and Vice President of Services Marketing, Brian Winter about how Sun’s Solaris 10 Operating System — the only multiplatform, free, and open source enterprise-class OS on the market — allows you to leverage existing infrastructure investments and anticipate rapid growth by bringing applications and services online more quickly, and at lower support costs, than any other commercial OS offering, including Red Hat, HP-UX, AIX, and Windows.

Whoa, Nelly! “The only multiplatform, free, and open source enterprise-class OS on the market”. That’s likely to get you thrown in court in most free countries, never mind hanged in those run by Sharia law! I’m not saying that Solaris isn’t free, or that it isn’t open source, or that it isn’t multi-platform. I guess 3 platforms does count as multi-platform.

But my beef is with “only”. Say what?!? Are you so desperate, Sun Microsystems, that you have to resort to outright lies? No outgoing links on this post for you, Sun!

For the record, I do like Solaris. It’s false claims like these from Sun which I despise.

Technorati Tags:
open.source, system.architecture, cross.platform, linux

This sounds like fun!

Whistle while you work: Use the open source sndpeek program and a simple Perl script to read specific sequences of tonal events — literally whistling or humming at your computer — and run commands based on those tones.

I can just imagine the “Star Trek XXXIV: Superfluous” movie scene now, with the brand spanking new Enterprise, the NCC-1701-ZZXYZ.

Lieutenant Atad: Snognilk on the bridge!

Captain Dracip: Commander Rekir, set the auto-destruct sequence on the main computer, quick!

** Commander Rekir begins whistling the tune from “Bridge Over The River Kwai” a.k.a. “Colonel Bogey March” **

Unix server administrators everywhere are probably too familiar with connection timeouts when administering remote servers. There you are, dutifully editing that configuration file, when suddenly your text editor no longer responds to your cursor keys. A dozen seconds later or so, SSH reports the connection has been lost. If you didn’t save your file recently, you probably mutter a curse, for you probably have to redo your work all over again.

There is nothing worse than having to administer a server through a bad connection. When you are disconnected, having to reconnect, login, and navigate to where you last were, can knock you out of the flow of what you were doing, severely affecting your productivity.

screen to the rescue

Back in the time of the dinosaurs and dial-up, some GNU hackers, tired of the inconvenience just described, invented the screen application. screen creates a virtual console running on the server, which keeps the user logged in to the server, even when the network connection is lost. With screen, you will never lose your place while logged in remotely again, baring a reboot, shutdown, or power-loss to the server itself (alas, screen is not magic.)

screen can do much more than this simple description so far hints at. You can learn more from the online manual (type man screen at the command-line.) But for this hack, the following is all you need to remember:

Type in screen -DR when you first connect to your server, and your login session will be preserved through network disconnections. If you get disconnected, login again, and type screen -DR to return to your session, just as you left it.

The screen trick is all well and good. But we like to do things thoroughly at pjtrix, and try to find a full solution to the problem. So on to autossh we go.

autossh: aim for the kill

Keeping with the dinosaurs and dial-up theme, autossh is like an auto-redialer. It faithfully tries to reconnect you to your server again as soon as it detects the connection is gone. It does this by creating an SSH tunnel loop between your workstation and the remote server, and monitors it repeatedly for lost messages.

Using autossh is almost like using ssh: except for the connection monitoring specifics, autossh simply passes all command-line parameters to ssh, and after entering the command-line, all user interaction is done through the running ssh instance. To use autossh effectively with screen, type the following command to first connect to your remote server (use this command instead of ssh):

autossh -M monitoring-port -t username@server.domain.tld “screen -DR”

For example:

autossh -M 20000 -t webmaster@pjtrix.com “screen -DR”

With the above command, autossh creates an SSH tunnel loop with ports 20,000 and 20,001 for monitoring the connection; the -t parameter makes ssh create a pseudo-terminal when it connects to the server; ssh connects to the server pjtrix.com as user webmaster; and finally, ssh runs the command screen -DR remotely on the server immediately upon a successful connection and user authentication.

If for any reason the connection to the server drops, autossh detects this, and starts another ssh session, with the same commands. This effectively reconnects you to the server, logs you back in, and returns you to your screen session, right where you left off.

Used together, autossh and screen are real peace of mind savers for Unix system administrators. Couple this hack with an SSH authentication agent and SSH keys, and you don’t even have to remember your server password between reconnects.

Truly, there can never be greater Unix administration bliss than this.

Technorati Tags:
hacks, ssh, open.source

In the months ahead, I will be making use of this weblog to discuss software development technologies and processes I’ve learned to use in my ten-year career. I will most likely be writing about web technologies and web services, and open source technologies rather than proprietary ones.

I am more likely to cover Ruby on Rails and Ruby as a language and cross-platform development technology versus other open source technologies, as this is what I now prefer. But you may also see posts on C and C++, Java Enterprise Edition technologies, Python, Mozilla technologies such as XUL and XULRunner, and software development ideas in general. I may cover C# and .Net, but only because you can develop cross-platform applications with them using Mono.

My article proposals for “the secret online geekly articles site” are most likely to be accepted if they are about Java technologies. As I research my articles, I am bound to write about Java subjects here. But I’ll try to make the weblog posts more general than the articles. If the article proposals get turned down after a few rewritings and retries, I will publish the subject here or try to get them published somewhere else, like InfoQ and The Server Side.com. (BTW, that should be a hint that those two sites aren’t “the secret online geekly articles site.”)

I will also cover some Unix administration topics. For example, I feel I ought to cover the details of pjtrix’s Subversion configuration, as I found the online manual a bit wanting in specifics. Other like-minded geeks, the very people I’d like to bring to my weblog, might welcome a more direct approach. There are also more ssh tricks I haven’t begun to cover.

All of this writing will hopefully be bringing new readers to my humble weblog and other parts of the website. Armed with this hope, I continue to get pjtrix.com ready for 2007.

Tuesday night, I signed up to Google Analytics and Tools for Webmasters, and added to the WordPress and Trac templates the bits of JavaScript that report to Google where you all come from. The reports Google provides are nice and pretty, but I find Google’s solution wanting, specially in their support of blogging and citizen media. The tools are more geared towards “website” traffic analysis. That’s just lame.

Not everyone that will come here will do so with a JavaScript-powered browser. Feed readers and podcast clients, for instance, only speak HTTP and RSS or Atom. They will leave no trace in Google Analytics’ logs.

To complement Google Analytics, I’ve installed the Webalizer Apache log analysis tool, which will help me study my web server logs in more detail without my being swamped in hundreds of megabytes of raw text.

I also installed the Popularity Contest Wordpress plugin. This should help direct search-passers-by to what’s hot on this site, or just make it more clear sooner that they got to the wrong place.
Hopefully, the combination of Google Analytics, Webalizer, and Popularity Contest, will together help me learn who my audience is, learn what is “my voice,” and learn how to make this weblog more valuable to my readers.

Technorati Tags:
rails, ruby, hacks, ssh, software.development, open.source, web.architecture, wordpress, weblogs, web.technologies, web.services, cross.platform

Happy New Year, everybody! It’s five and a half hours to the big hour here at Chez PJ, so I am taking this time to update you all about what’s going on.

Last weekend, I finally got Trac and Subversion installed like I wanted: read-only anonymous checkouts through http://svn.pjtrix.com/projectname, and read/write access for certain users only through svn+ssh://svn.pjtrix.com/var/svn/projectname.

As is almost always the case with things Unixy, the problems all had to do with setting the right permissions on the repository folder, and making sure that all related processes create new files with the right permissions set. The fix was a litany of addgroups, modusers, chowns, and chmods. Then I had to make substitute shell scripts for all the subversion commands, which simply set umask 002 before running the actual command.

After the Trac and Subversion headaches came Christmas, so I took a three-day break, visiting family, starting to read one of a few dozen fiction books from my “to read” pile, and generally chilling out. Then from Thursday through Saturday, I only worked on proposals for articles for “the secret online geekly articles site.” I hope to get at least one article proposal accepted in the next weeks.

I also found a potential employer, a place in Amhurst, MA called Assembla.com. They are a remote development consulting company, looking for Ruby on Rails developers that can work remotely. Their main product is something like a mashup between a wiki, a social network site, and a project management repository. They also do custom remote development for customers.

The interesting thing about Assembla.com, is that the hiring process consists of a type of paid audition: instead of sitting through interviews, getting your references and work history checked out, and usual job-getting stuff, they assign you a task out of their real workload, and pay you for a week or two while you get it done. If you accomplish the task to their satisfaction, you’re offered a permanent contract. If not, well, you can keep trying. I’m going to be applying this coming week.

I need to get my time organized, that’s certainly step one of the New Year. I want to work on iCal to GCal Sync and do more pjtrix goodness. I also will need to work on whatever tech article gets the a-OK. And I also need to concentrate on passing a paid trial for Assembla.com. Easy peasy. (Hint: that was sarcasm, folks. But I’ll try nonetheless.)

What I will probably end up doing, is divide my days into five-hour periods, not including sleeping, eating, my mandatory 20-minutes-away-from-the-computers-every-two-hours, and hygiene time. I’ll work on only one thing each period, then rotate among tasks every five hours. It will be like working three part-time at-home jobs! I’ll be like a homemaker mom.
Alright, that about does it with the updates. If I think of anything else, it will be my first post for 2007! See you then!

Technorati Tags:
hacks, rails, ruby, life, software.development, open.source

Through the magic of social networking site LinkedIn, a friend of a friend introduced me to a friend. This friend is looking for a technical writer for a website with a geek audience interested in open source, Linux, web technologies, and software development. It’s not as big as Slashdot in readership, but it is sizable and has good credibility.

It’s not a permanent job, it’s more like “we’ll pay you decent money if we publish your article.” Print magazines pay a pittance for a two-thousand word article. This website pays pittance x 3, making it a lot more worth the trouble. If I can get on the stick and write an article per month, it could make a decent source of part-time income.

I know I’m being vague about the site. Once I get my first article published, believe me, I’ll post all the details.
—-

I have finished setting up Trac for hosting my open source projects. As if installing Subversion last week wasn’t enough, Trac configuration took a lot more trial and error. My web hosting provider uses Plesk for setting up web domains. Even though I know my way fairly well around Apache, I had used the Plesk configuration app to set up this web site and my friends’ websites on this server. I practically painted myself into a corner, as Plesk doesn’t like geeks messing around the Apache config files, which you have to do to set up Trac and Subversion. ** sigh **

I am still fighting with the configuration of the Subversion code repository. I want to have check-ins through ssh+svn, and anonymous checkouts through HTTP via Apache mod_dav_svn. I hope to have that all set up tonight so that I can check in the first files of my open source project’s code.

—-

I like and use Google Calendar, but as a bona fide code geek, I am more interested in getting my calendar data out and integrating Google Calendar into desktop and web applications.

My first open source project for pjtrix is a Mac OS X application to integrate together iCal and Google Calendar. In the last few weeks, I’ve been researching the Google Calendar API, writing some test applications to insert calendar events into Google Calendar, pull calendar events out, change calendar event data and put it back in, etc. It’s all done in Ruby, which has made it all really nice.

Right now the integration between iCal and Google Calendar is only one-way. You can import your iCal calendars into Google Calendar. Or you can import your Google Calendar calendars into iCal. But you can’t have changes flow from one application to the other without creating a mess of duplicates every time you change the date or time of an event.

You can read more information about my project at the project’s Trac home page.

Technorati Tags:
web architecture, software development, open source, ruby, life