All Things Patrick - Working Hard to be Lazy!

While working at RCS (see Jobs, Jobs, & More Jobs - Part 1) one of the projects I ended up working on was trying to figure out a way to upload streaming video to Amazon's S3. Googling around I found several articles, but only about 3 different PHP implementations for S3 - all 3 used at least 1 PEAR library. After wasting half a day trying to find something that didn't use PEAR I started a conversation with the lead programmer. What he said revealed a lot.

Before we get to the actual conversation on instant messenger I want to cover a few things for those that aren't familiar with PHP and PEAR. If you're familiar with Perl and CPAN, there is a similarity in association, but there are differences - I'm not familiar enough with Perl and CPAN to make those distinctions though. PEAR is a library of common reusable PHP components. Rather than having to re-write certain things from scratch, PEAR allows PHP developers world wide to have a common base to work from. There are sometimes reasons for not using a PEAR library, but if you're needing something quick and a PEAR component already does it, it doesn't make much since to waste time to completely re-write the same code. There are a very few PEAR components that have been moved into the PHP base code.

PEAR is a completely separate project from PHP. However, PEAR is such an integral part of the PHP world that if you download the PHP source and compile it it defaults to including PEAR these days. I have yet to come across a single PHP web host that does not have PEAR available (the specific PEAR components that are available do vary between web hosts). Most of the core PEAR components are used within thousands of web sites and in some instances they're buried under several layers and the end PHP developer is not even aware that they're using the PEAR components.

For those that are not familiar with the PHP MSI (some sort of Microsoft specific installer) installer I'll walk through the steps. After running the installer you have to accept the PHP license, select where you want to install PHP (which hard drive & directory), and then comes the first hard part. You have to select which server (and sometimes how you want to set it up on that server) you want - Apache 1.3.x Module, Apache 2.0.x Module, Apache 2.2.x Module, Apache CGI, IIS ISAPI module, IIS FastCGI, IIS CGI, etc (even Do not setup a web server!) - RCS is using the IIS ISAPI module. The next step is to select which Extensions and Extras you want installed - the version I have (from November '07) starts off with absolutely nothing selected. At this point you have to start selecting extensions you want to use (including the MySQL, MySQLi, and a very few other extensions we actually had available to us on the server). Considering how many are listed and if you're unsure, it may just be better to have it install all of them and then comment them out of the php.ini file later (and if you're really paranoid, move the extension dll file to a different directory). Then just click Next, click Install, wait for it to finish installing, and then click Finish... T'dah, PHP is installed in IIS - you may however need to set up IIS to allow CGIs to execute.

If you forgot to install something and need to install it later, just go through the exact same steps, making sure you're using the same PHP directory you used before, and it will update your php.ini with the updated information.

PEAR doesn't even need to be installed via the MSI installer (although it's a bit easier). Just copy the go-pear.bat script (and the PHP files it uses) to your PHP server and run it. You can use the defaults or set your own specific directories for where to store things. Now PEAR is installed... man, this stuff is just so hard.

Now here is the instant message conversation I had with the lead programmer.

12:40 PM <patrick> any chance we can get pear installed on the server for s3 ? 12:45 PM <yinrunning> nope. <yinrunning> no workie <yinrunning> herky jerky <yinrunning> so <yinrunning> we'll just have to write some headers. <yinrunning> I was looking into it yesterday. <yinrunning> gonna do some more reading when I get a chance. <yinrunning> and jacking around w/ it. <yinrunning> All the Pear class does is abstract header() as far as I can tell tho. <patrick> why can't we get pear installed on the server? <yinrunning> just won't do it. <yinrunning> breaks the PHP installation. <yinrunning> I spent part of a week jacking w/ it at some point. <yinrunning> couldn't ever get any satisfaction. <patrick> how does it break the php installation? <yinrunning> if I try to install pear, php goes bye bye <yinrunning> and since we've never been dependent on Pear in the past, I didn't have much time to devote to it. <yinrunning> wasn't critical. <patrick> i'm not sure why pear wasn't installed as part of php... it comes with as default <yinrunning> mm, not really. <yinrunning> php doesn't depend on pear at all. <yinrunning> it's just an addon. <yinrunning> alot of people think that it's a default. <yinrunning> but in all reality it's a completely different project. 12:50 PM <yinrunning> all it is is code abstractions anyway. <yinrunning> there's nothing you can do with Pear that you can't do with just php. <yinrunning> including this. <yinrunning> so I was just going to say F it, write our own class for it, and then maybe put that out there for other people. <yinrunning> on the Interwebs somewhere. <patrick> i realize pear is a different project... it used to be pear was a completely different install process <yinrunning> <shrug> <yinrunning> doesn't matter to me. <yinrunning> Pear stuff has always pissed me off anyway. <yinrunning> At its worst, I once watched a business's $25K/mo revenue stream almost completely tank because of Pear dependency. <yinrunning> So honestly I don't really trust anything that uses it in general. 12:55 PM <patrick> If you have a recent installation of PHP, you can relax: The PEAR base installation is already there, unless you have compiled your PHP with the ./configure flag --without-pear. <yinrunning> no configure. <yinrunning> doesn't compile. <yinrunning> it's the binary MSI. 1:00 PM <yinrunning> and btw, that doesn't hold true for all PHP installs. <yinrunning> even if they are compiled. <yinrunning> If that was from the PHP.net site FAQ or something, it was probably user-written and they assumed that their downstream version was just like everyone else's. <yinrunning> that's from the pear user docs on installation... that's also not the only way to install it <yinrunning> they're the same way. <yinrunning> honestly. <yinrunning> they assume quite a bit about the environment. <yinrunning> the worst is trying to find documentation in PHP CLI. especially for Windows. <yinrunning> But the other docs out there don't fare much better. <yinrunning> Took me and Jer the better part of a week to get PHP to function correctly on the new new servers. <yinrunning> was a fun time. <yinrunning> docs were totally useless. <yinrunning> as much as I love PHP in general terms, there's times when it really really frustrates me. <yinrunning> And those times are when I'm trying something new and trying to figure out how it works.

I'm still not quite sure what he means by the fact that PEAR breaks the PHP installation. I don't ever recall mentioning that PHP depends on PEAR (PEAR depends on PHP!). I realize that PEAR is just an abstraction and we can hand write our own, but considering the project deadline... why waste time when it's already been written and does what we need?

I think his statement on trying to find documentation on the PHP CLI quite funny. While there's not a whole lot within the PHP docs (currently enough that someone who's worked with the PHP CLI before can get a refresher), googling for info turns up a lot of information. Not only that, but once you get past the initial $argc and $argv it's pretty much normal straight forward PHP. I will however agree with him that the CLI on Windows is lacking, but that's not the fault of PHP, but rather a fault with Windows. Oh, and that PEAR he hates? The go-pear.bat file gives you great examples as to how to use PHP CLI in Windows!

After wasting the rest of the day I finally found something hidden away in the deep recesses of Google Code. It looked like somebody had started a project to deal with all of the various Amazon services. The class for S3 had had a lot of time put into it, but it didn't seem complete for what we needed. It did, however, use the curl binary rather than the PHP Curl lib to upload files. This was great because most things that make use of the PHP Curl lib directly don't take into account large files. In other words they'll try to load an entire 20 or 30 meg file into RAM before transferring it. The better way is to get a file segment and then push it to the server before getting the next file segment - this way you don't have to have more RAM than the size of the file you want to transfer.

The problem was that it expected that you were using a Unix (Linux, BSD, whatever) server that has the Curl binary in a logical location or that it can at least use yet another unix command to find where it was located at. Because the web servers are running windows we had to find a Curl binary that would run on Windows - luckily the Curl project has binaries for just about every OS and Linux distro you can think of. I set up a config file to point to where Curl was located and modified the S3 class so it would accept a path for Curl.

Rather than thinking ahead to possible future issues, the lead programmer installed the non-SSL version of Curl.

Sometimes I think if we started programmers off with a line editor it would either decrease the number of people pretending to be programmers or else it'd make them more appreciative of the frills.