All Things Patrick - Working Hard to be Lazy!

Back in September of last year I wrote a post about my job hunt in No More Vegas and an ending note about the jobs in Kansas City. Where else to start except at the beginning (of the end?). To protect the names of the innocent (and the not so innocent) I won't mention any names.

Downtown Kansas City, Take 1, RCS, beginning of October to end of November -

I ended up with a job in Kansas City at a marketing agency I'll call RCS. It had started years ago as a print media company and as the internet blossomed it started adding web sites to it's list of offerings. The business started at about 66%/33% print/internet and now they're about 66%/33% internet/print - their business has completely shifted. Their initial web offerings was limited to static web pages, later they outsourced dynamic pages to a ColdFusion developer, & finally they ended up with their own in house development staff.

When I went through the interview process I asked certain questions. Maybe I didn't ask the right people the correct questions because after I got started I found a completely different picture than what I had thought had been painted during the interview process. I had asked about LAMP (Linux, Apache, MySQL, PHP), source code versioning (subversion, cvs, etc), development cycle, OOP, a framework or at the very least a library of code, and several other things. It sounded like it was a good match - they did inform me though that they did have two Windows servers which were required for the legacy ColdFusion and ASP code. I figured no problem, I won't have to deal with those unless I have to mess with some of the ColdFusion or ASP code. I especially thought this was a good match not only because the lead developer programmer/server admin liked Gentoo (even his desktop was running Gentoo) and disliked Windows, but the long-time owner of the marketing company was a Mac person and very anti-Microsoft (she also owned the Mac store down the hall which is also certified to do Mac repairs!)

And then it all came crumbling down...

They had just started a server move. They were in the process of moving all of their sites from 3 different web hosting companies to their own dedicated servers. They had 5 dell servers that were the same and then a 6th dell server that had a raid5 array. The front 2 dell servers were running Linux and played the roll of DNS and load-balancers. The 2 web servers were running Windows and using Microsoft's IIS web server! The 5th same dell server was running Windows & some email server I've never heard of. The 6th server with raid5 was running Linux and was the MySQL server.

I applaud them on their move, but I can't understand their choice of OS for the web and mail servers. Especially considering their new development was all PHP. Even the novice & hobbyist PHP programmer should know that you don't want Windows as your webserver (even if they don't know or understand all the reasons why). It would have been better to have used Linux as the server OS and then virtualized the windows servers. Especially considering some of the ColdFusion sites were known for having crashed the server they were on in the shared hosting environment. Had the servers been virtualized then the parent OS (Linux) could monitor the virtual server (Windows) & restart it if necessary.

I don't want to put Apache out there as a cure all (some people prefer Lighttpd), but I think in this situation and because it is so flexible that apache would make quick work of handling both it's own websites and/or determining what server it needed to forward the request to based on the url. You could even virtualize the Linux/(Apache|Lighttpd) server for new PHP development. At that point your hardware server running Linux/Apache would intercept calls to the webserver, determine the url being requested, and then pass it off to which ever virtualized server (Windows+IIS or Linux+Apache) was responsible for that url. Windows just doesn't seem to make the most use of system resources (it has to run the GUI whether it's being used or not) and IIS just isn't as easily configurable (setting specific PHP configurations) on the domain level as Apache is.

Conclusion - this isn't a LAMP environment... it's a WIMP (Windows, IIS, MySQL, PHP) environment.

The we're looking at using version control, bologna that I was fed during my interview process turned out to be completely false. The 2 guys who were responsible for me deciding to accept the position (based on the questions they asked and our geeky banter during the interview process) were starting to use it. Later I was forwarded an email originally written by the lead developer programmer, that had been sent out prior to me being hired, that was yet another instance of I have no idea what I'm talking about.

Conclusion - version control will never be implemented because it's to hard to figure out.

The library or framework that they had in place wasn't much more than their own version of hotscripts.com. Not only that, but none of it was stored in any kind of central location so that all of the projects could share the same files... Each project had their own copy! Not only that, but in order to add X to your project, you had to know which previous project had used X. As far as I could tell there wasn't any kind of a place that listed what was available.

But what about OOP? Surely this allowed for some sort of re-usability? The OOP that they referred to was that each project had it's own single admin class and at the end of the class definition it was immediately instantiated - $admin = new admin();. This class handled everything from database connectivity to sending emails. It was literally the class that knew to much. A better description of this would be to call it a namespace for functions - it wasn't like any attempt at making a class that I'd ever seen before... At least not since Windows (forms) as objects, with all the functions needed for the project, in classic VB. Not only was this project object that bad, but it didn't even use any of the new PHP 5 features for defining variables and functions - private, public, protected, static, const... Yes, they were using PHP 5, but only in that they were using the PHP 5 cgi to process the PHP files. Looking at the code, you would be hard pressed to see any of the new features of PHP 5 being utilized.

Ok, well maybe they were using a 3rd party framework or library... like PEAR? You would think that in a high project turn over environment you would want to utilize something that somebody else had already put together. This way you're not wasting time re-writing lots of code. Especially something that thousands (if not millions) of projects and websites are already utilizing in a production environment. I don't think I've ever seen anybody be so hostile towards 3rd party libraries. Asking to use PEAR was like somebody who cusses so well they make a sailor blush do so in front of the Pope. Apparently this lead developer programmer sat there and watched a company lose somewhere in the neighborhood of $25k a month due to a PEAR dependency. What kind of a developer sits there and watches? Not only that, but what caused this problem? My guess is that the programmers involved weren't using version control or even any kind of backup, updated to a broken alpha or beta PEAR package, and then had no idea how to revert the PEAR package back to a previous version... and then they just sat their and waited for a new package to be uploaded to PEAR.

Conclusion - No framework, no library of code, no OOP. Very hostile towards using anything that isn't hand written by staff - especially if it utilizes an API. This includes using 3rd party javascript libraries such as YUI (from Yahoo for those that don't know).

So what about the development cycle? Surely they have some sort of test server - especially considering how many websites they have? I was told to make changes directly to the live code. I was also told I needed to post my fixes often so that if somebody else needed to fix something they could pull a copy with my latest changes. This was also part of the reason he didn't want to use version control - it wouldn't allow him to make quick fixes to the website.

Conclusion - Make your fixes directly to the live site - don't worry about completely destroying the entire live site while you make an update to a part of the code base that requires a change on every page (which is possible considering their programming style).

I don't know... I'm guessing it's going to take a complete melt down of their current processes for them to understand the benefits of what's out there. The point of version control and test servers is so that you don't destroy the live site. Not only that, but with version control if you completely jack something up you can always go back to a previous working version.

I did forget to mention something about 2 of the guys, Ravi and Ryan, that were in the group that interviewed me. It was because of the questions they asked that I ended up accepting the job. My first project was to finish up a project that the lead programmer had started... When I dug into the code I discovered how bad this thing was...

Ravi and Ryan were the 2 young upstarts at their first real programming job with a company and didn't know how things were in the real world... They had gotten jobs here hoping that they would learn something new. After 6 months of working here before I showed up they had learned something, but not what they had wanted to learn. Rather than learning good practices and better ways of doing things they had discovered they were going to have to forge their own paths with no guidance other than their past experiences. I know Ravi at least had had a serious talking to from the lead programmer about the woes of OOP.

It didn't help their position any that the first major project Ravi was given ended up becoming a complete nightmare. I don't know all the details, but from what I understand Ravi was doing all the programming on it and it went through various designers (I think starting with Ryan). Because of various designers going on vacation or being moved to other projects at different points, because none had Ryan's skills, and because there was no process or standard for how to do anything... each new designer at best tacked onto what was there and at worst had to completely start over - 1 required Dreamweaver templates. It devolves into a morass of he said, she said, and ends with Ryan having to quickly finish the mess.

Because of the Dreamweaver templates problem there's no header/footer included file nor are any of the block sections pulled out into any kind of site wide template. Any time something needs to be changed site wide time has to be wasted to update every single page. Ravi did hand code a form builder object that did a decent job of building and validating the forms (especially on a first attempt!) - had he been allowed to use PEAR he probably could have saved a lot of time by using the HTML_Form or HTML_QuickForm.

Unfortunately because of the all the changing of hands, nobody knowing how anybody else works, and having no process as to how to do things... The entire project became the OOP/web standards nightmare. Rather than it being an eye opener for the lack of procedures and project standards it became the nightmare inflicted upon them by Ravi and Ryan.

I thought I might be able to stem the tide or help change the process for the better, before I knew about the above project, but I needed help. Because they were eager and hungry for better ways of doing things I thought for sure I could find the help I needed from them. Unfortunately after my first 2 weeks Ravi quit to go work elsewhere and 2 weeks later Ryan left as well.

At that point I realized the boat had sank and I needed to look elsewhere before I got fired for inflicting good programming practice, procedures, and web standards upon them. After a total of 2 months of working there I finally found work elsewhere.

Please note that I did enjoy working with the people and having the cats around the office. Everybody there was very nice and helpful. Every morning when I came in, Yoda, the cat, would come scampering over and waited for me to sit down so he could jump in my lap.

It was the programming environment and the unwillingness to effect any change upon it that caused me to leave. I'm definitely going to miss the cats...

Yoda

Melia