The forgotten cron job

Posted on September 22, 2010 by mschwar99 — No Comments ↓

Diamonds might be forever, but cron jobs are no slouches.

Upon adding a new cron task to a development machine I noticed that I had a cron job that had been running on the hour, every hour, since sometime in February.  I had completely forgotten that it was there.

This cron job has been calling a ruby script that has been loyally scrapping the Google “Hot Search” page since February and faithfully tallying the results in a MySQL table.

I’m adding doing something useful with this data to the ever growing “list of things to do when there is time”.  Maybe something JavaScripty like making a huge tag cloud out of them.  Must… stop… starting… new projects….

Here is the mysqldump if anyone is curious – I know there are sites online that let you search through old Trends – but I don’t recall seeing dumps of them.

The schema is indicative of the quick hack that the script was.  The text of each trend is in “trends” and if it appeared more than once, each occurrence gets its own tally in “timestamps” along with the timestamp of the occurrence.  “Count” is incremented in “trends” to show the total number of occurrences.

mysql> describe timestamps;
+------------+----------+------+-----+---------+----------------+
| Field      | Type     | Null | Key | Default | Extra          |
+------------+----------+------+-----+---------+----------------+
| id         | int(11)  | NO   | PRI | NULL    | auto_increment |
| trend_id   | int(11)  | NO   |     | NULL    |                |
| created_at | datetime | YES  |     | NULL    |                |
+------------+----------+------+-----+---------+----------------+
3 rows in set (0.00 sec)

mysql> describe trends;
+-------+------------------+------+-----+---------+----------------+
| Field | Type             | Null | Key | Default | Extra          |
+-------+------------------+------+-----+---------+----------------+
| id    | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| name  | varchar(256)     | YES  |     | NULL    |                |
| count | int(11)          | NO   |     | 0       |                |
+-------+------------------+------+-----+---------+----------------+
3 rows in set (0.00 sec)

Its somewhat depressing looking through them all – most are whatever celebrity gossip was all the rage that day:

mysql> select * from trends where name LIKE '%tiger%';
+-------+-------------------------------------------+-------+
| id    | name                                      | count |
+-------+-------------------------------------------+-------+
|   776 | joslyn james tiger woods                  |     1 |
|   983 | how old is tiger woods                    |     1 |
|   991 | tiger grant awards                        |     4 |
|  1002 | tiger woods press conference              |    10 |
|  1202 | what time is tiger woods press conference |     2 |

                 .........

|  5223 | tiger woods divorce                       |    23 |
|  8986 | tiger tiger burning bright poem           |     8 |
| 12882 | tiger woods divorce settlement            |    10 |
| 14269 | goliath tigerfish                         |     3 |
| 18877 | rudy ruettiger                            |     8 |
+-------+-------------------------------------------+-------+
38 rows in set (0.00 sec)

As a PS – I don’t think this is against Google’s ToS or anything – it goes out in a public feed as well as being on that page.  If I’m wrong I’ll take it down so please don’t smite me Mr. Google.

Posted in Geekery, Hackery, omgCron

New Venture: WP Plugin FU

Posted on April 27, 2010 by mschwar99 — No Comments ↓

WordPress: insecure spaghetti code or platform extraordinary and plenipotentiary?  Meh.

Its the biggest kid in the room.  18 million active installations. Several million of which aren’t even zombie spam blogs. (maybe)

I threw up a new site to host some plugin development I have been working on:

WP Plugin Fu

Couple live wires already:

Posted in WordPress

Lessons Learned

Posted on April 24, 2010 by mschwar99 — No Comments ↓

Slideshare.net is one of my very favorite sites.  If you haven’t been blown there by the tides of the random web before you are missing out. SlideShare is the YouTube of PowerPoints.

If you have worked in the 9 to 5 corporate world at all, you might have a distaste for PowerPoints.  As much as any single workplace item, they are the stamp of the mid-level technocrat.

Often they are a personification of marketing over substance, gleaning over grokking, bullet points over in depth knowledge.

What’s the next ten words? These bullet points are the tip of the sword: just enough knowledge to get you into trouble. (thank you Jed Bartlet).

Sorry, evidently I needed to vent a bit.  If you can get past all that, SlideShare is a great collection of knowledge.  A couple of SlideShares I found tonight struck a cord with me.  One is a summary of “what we have done right” from a small company I really admire: Dropbox.

Dropbox is a great model for aspiring  entrepreneurs: really smart people doing something really hard and making it seem easy.

[slideshare id=3836587&doc=dropbox-startuplessonslearned-100423230315-phpapp02]

Next to this SlideShare was one from Pownce.  Two-ish years ago Pownce was the apple of the start-up world’s eye.  Founded by notable folks and staffed by with-it techies it seemed bound for IPO. Somewhere it fell off the tracks. Never would have guessed at the time of this similar looking presentation.

[slideshare id=128491&doc=pownce-lessons-learned4283]

My future is uncertain right now and I am coming up on the end of the contract I am currently working on.  I have been thinking a lot lately about where my train tracks lead from here. While my company might re-win the right to do the work I am doing now, I’m not sure what I pursue if that falls through.

I spent around nine years of my life running bars.  After year four or five of those years the novelty wore off.  My best friends were there, it was great cash for a guy my age, but I wasn’t quite sure where it was going.  What I did know was that I was getting a chance to be responsible for a volume of business that no one my age could be reasonably expected to take on.

It was a challenge.  It was a rush.  I have found out since I started in the corporate world that it was also the best experience that I have ever had: I learned more about responsibility and customer service in those years than any internship could have taught.

I kept grinding the last years out on the thought that it was my path to equity. With equity a person can pick their future, without it one works 9 to 5.  Somewhere I drifted off those tacks.

I think its time to get it back on them.

Posted in Employment, Geekery

Hacking the Job Hunt

Posted on February 15, 2010 by mschwar99 — No Comments ↓

If you are job hunting you have seen your fair share of nausiating stock photography like this.

Over the last couple of months lots of people have chatted me up about finding employment in the IT sector.  The economic ass-plosion of 2008 is still reverberating in 2010, and from the few datapoints I have seen in my everyday life, it can still be a long slog to find a new gig.  While I don’t have an endless network of contacts or many secret ins to hand out to people, my time working for an IT mega-corp has provided me with one insight that I think is valuable to people in their job search: It is almost an absolute certainty that you are wasting your time submitting application after application to a large company’s web site.

Application Shot-gun Style

The typical job search for a newly out of work or graduated IT worker seems to be:

  • Google, Google, browse, bookmark
  • Copy and paste 10 resumes into the job application sites of large companies
  • Brace themselves for the eventual onslaught of HR workers overjoyed at the reception of a perfect resume

What has actually happened is that they have spent a few hours flushing their resume down e-black holes.  After going through the hiring process you get a peek behind the mega-corp HR curtain and understand what a waste of time that was.

The point is this: large IT companies do almost all, if not every last drop, of their hiring of junior to mid level positions through staffing companies.  The general workflow for one of these companies when they want to fill a junior or mid position is:

  • Requisition someone from a staffing agency they have a relationship with
  • Hire that person through the agency on a 4-6 month contract
  • Evaluate the person and transition them to a permanent employee after the 4-6 months if things work out

Big companies find this process to be advantageous because its both safer for the company (they can axe someone if they don’t think it will work out without anywhere near the risk of terminating a full time employee) and because outsourcing the HR and headhunting for junior level workers is more efficient.

I have found that only a few types of people generally get hired directly from a big company’s website:

  • Senior level positions: 8-10 years of experience or an otherwise very thick resume.
  • People with very specific or hard to find skill sets.  If you have years of experience with a relatively obscure piece of software or hardware there is hope for you.
  • Rockstars with very impressive skill sets and resumes flush with big time gigs or Open Source coding projects.

The irony of course is that the above people are the very ones that will be the least likely to need to resort to shot-gunning their resume across the internet to find a job.

In fact, 90% of the non-senior level postings you see on company’s websites are not “real” postings.  When a company wants to move a contract agent to a permanent position, they have to take the step of publicly posting their job position on their web site.  However, they already know who they want to fill the job and won’t be responding to random applicants.

Hacking the Job Hunt and Actually Getting Hired

The above is not necessarily bad news though.  It just means that you need to understand how the process works and figure out where it is most advantageous to insert yourself.  Hint: its not by flushing your resume down the mega-corp website, its by making contact with the headhunter.

The upside of this arrangement is that there are thousands of staffing agency headhunters that are running around desperately trying to fill jobs for big companies.  Like a real estate agent, these people only get paid when they successfully place a candidate.  These folks are your friends. They want to find you a job.  Its the only way they make any money.

Sure they might not understand what it is you do for a living.  Conversations with them are often painful.  You will invariably have to explain to three of these people a day that its not possible for you to have eight years experience with a three year old technology.

But who cares?  Odds are these days that if you are shot-gunning your resume you just want a damn job and would be happy to put up with the ridiculousness of all that comes with it.  You can worry about leading a life of of quiet desperation as a corporate sheep after you stop worrying about your car getting repo’d.

The key then becomes getting your self found by these headhunters.  The single best way to get yourself noticed is to optimize your resume to be found by agencies.  Again, understand the process:  these people find contacts by searching through the large employment websites for suitable candidates with keyword searches.  They type in “PHP” and troll through the 8,000 results looking to match up Company X’s list of keywords with Job Seeker Y’s keywords.

Make a new resume just for this purpose and post it on careerbuilder.com, monster.com, and other similar web sites.  Load it up with buzzwords.  Have an intro paragraph chock full of buzzwords.  Have a bulleted list of 9 to 12 skills you can legitimately claim to have that might match a simple keyword search.  It doesn’t need to be pretty, it just needs to be keyword rich.

Once you list yourself that way you will start getting random pings from headhunters.  Whether the gig is appropriate for you or not respond to the person and chat them up.  Describe what skills you have and tell them what you are looking for.  These people get paid when they place people so they are happy to have every lead they can get their hands on.  You are not a special flower with interesting career goals to these people, you are a dollar bill with ears.

Sure some of them might think its odd that you contacted them if you weren’t a good fit for the position they were advertising.  But hey, screw those people, they probably aren’t very good at their job anyway.

Senior level IT workers get jobs by networking and reaching out to contacts.  If you are a junior level worker you don’t have that network yet so use the one that is available to you: headhunters.

Posted in Employment, Geekery

Open Source in Obama’s Gov 2.0?

Posted on January 22, 2009 by mschwar99 — 1 Comment ↓

The inauguration has come and gone (wonkette.com) and as the Mall puts itself back together and licks the wounds left by 2 million interlopers and their discarded hand warmer wrappers, the Obama administration is taking office.  Since I am currently working in (well, next to) government IT, I continue to be very curious how a new, and very possibly technically hipper, administration will effect every day goings-on in government data-centers and on Beltway desktops.  As a campaign that embraced all sorts of web technologies that came of age on the consumer web it will be interesting to see how this group shapes policy as it reconciles with entrenched opinions and federal record keeping statues.  Word has it that their first glimpse of the technology that is standard on government desktops failed to thrill (washingtonpost.com).

Open Source for Uncle Sam?

A blurb over on CNet (CNet.com) asks the question: “Obama wants to know: Why open source?”.  The debate over the benefit of open source adoption in government IT is neither new nor original.  Open source advocates will roll out their list of benefits of open source use including:

  • improved security,
  • higher-quality software,
  • lower costs,
  • higher reliability

The argument above and beyond the free-as-in-beer adoption cost is that open software developed and maintained in the bazaar (wikipedia.com) has the benefit of such a massive pier review that defects and shortcomings are flushed out before becoming problems. Additionally, the open nature of the software enables the users themselves to to correct any problems without relying on manufacturers to observe the problem, care about the problem, and distribute a patch in a timely manner.

The are certainly other opinions, many centering around the idea that the TCO (Total Cost of Operation) of open source software is actually much higher than its “free” price tag (ZDNet.com).  The usual argument is that ripping desktop users or system administrators away from their accustomed interface and sets of expectations and replacing them with entirely new alternatives will create a huge and expensive demand for support and retraining.

As with any heated argument I think that the truth lays somewhere in the middle and that the advocates at either end of the spectrum miss solid business cases for either type of product.  Personally I certainly lean toward the use of open source solutions where possible.  As far as the desktop goes, my anecdotal experience says there certainly is a case to be made that the switch from Office 2003 to Office 2007 that has stymied so many of my customers can not possibly be all that much more dramatic than a switch to Open Office.  Additionally, government agencies all currently contract out massive desktop support solutions.  Whats really required is not money spent on the retraining of government support staff, but rather changing the text in government RFQs that mandate the support of whatever systems they deem fit.  These poor support people are going to be answering interface questions about Office 2007 and eventually Windows 7, why not OO and Ubuntu?

This is not a new idea.  The German government began adopting open source software almost a decade ago and claims that where it is used, open desktops provide lower costs (osor.eu) than their competitors.  Our own Defense Department inserted language (CNet.com) into the National Defense Authorization Act for Fiscal Year 2009 singling out the use of open source software as a cost saving, security enhancing, and quality ensuring measure.

Building IT Infrastructure

However, there is a larger meta-point to be raised when listing the benefits of open source solutions in government.  President Obama’s coming economic stimulus plan is said to be large on building infrastructure (NYTimes.com) rather than just cutting every citizen a check like Bush style tax cuts and stimulus checks.  The rational is that you get multiplier effects both from creating new jobs to build the infrastructure and from harnessing the benefits that the new infrastructure provides to your community.

I would argue that the exact same rationals are reasons to use open source software in the government.  By introducing thousands of new use cases to the bazaar you are adding quality and security to the product that the government is spending tax dollars on.  Rather than the government’s IT dollar being spent on licensing fees that wind up being transitioned rather directly into dividends, the dollar is better spent on support solutions that will contribute back to a wider community.  Code optimization and customization that would be developed in government data-centers and support centers could wind up in the hands of of the projects that they are using.

In my head the analogy is fairly direct.  In the same way that an economy leverages tangible economic benefit off of government dollars spent on roads and bridges, the wider IT community leverages real economic benefit off of government’s use of open source software.  Licensing fees are stimulus checks.  Infrastructure investment is Apache and Tomcat in the data-center.  Does use necessitate contribution?  I’m not really sure, but again – the Germans did it by releasing their desktop packages to the public.

All Good Things in Moderation

As a previous paragraph said, the truth is somewhere in the middle.  While I do think that there are great business cases for integrating open source, forcing it in will be detrimental to the government and to open source’s reputation as a whole.  An interesting Ask Slashdot (slahsdot.com) shows that given the right set of circumstances, even in the den of the most strident FLOSS advocates, advocating proprietary technology can be modded Insightful.

Posted in .gov 2.0, Geekery