Leaving NASA

This has been a hard post to write.

I was delighted that on January 3rd, 2005 I started my first day working for the National Aeronautics and Space Administration (NASA). While I wasn't working on science efforts, I was at least contributing to the cause. In 2005 I was introduced by co-worker Chris Shenton to Python, which became my favorite programming language ever. I also learned tools like Zope, Plone, and Django. Over the past five years, I've met a lot of fascinating people in and around the agency, a list that seems endless in size and scope. That includes astronauts, scientists, engineers, developers, managers, and so much more.

This meant so much to me, and maybe because my first memories of television as a child were the moon landings of the early 1970s. I dreamed as a child of being an astronomer or astronaut, and sometimes I plot how I would redo my life to fit these dreams if I got a second childhood.

In the past year I've had some incredible opportunities present themselves to me. I've been presenting frequently on Django and Pinax. I've had the singular honor of writing course material for Holdenweb, LLC on behalf of the O'Reilly School of Technology. Representing NASA as a contractor to the Python and related communities has been both enjoyable and a great honor.

Yet all things, even good ones, must come to an end.

I've decided to become an independent consultant. My first project will be working with Revolution Systems (Jacob Kaplan-Moss and Frank Wiles) on a neat stealth project that looks very promising and once launched will help people. The project will be Python/Django/Linux based, and the client insists on accessibility, testing, and quality work. We'll be exploring the boundaries of what has been done with those tools and besides what must remain proprietary, a lot of our work will end up going back to the community. Sounds like my kind of thing!

My last day is April 1, 2010. I'm both excited to explore this new project, and saddened that my professional world for the past five years is coming to an end. Yet the overlap in technology and the participation of the NASA SMD python group in the open source world means that my work with NASA isn't coming to an end, its just transforming into something different.

Nevertheless, this is the end of an era for me.

Which is partly why I'm happy that I'll still be in touch with my fellow NASA SMD Python contractors such as Katie Cunningham, Chris Shenton, Chris Adams, James Saint-Rossy, and others. I also plan to be real friendly with the awesome Ames Research Center Python/Django/FOSS groups such as the intrepid Mark Friedenbach and the entire incredibly awesome Nebula team.

I'll miss having the pleasure of working with Leslie Cahoon, Jessy Cowan-Sharp, John Kasmark, Bob Ryan, Candace Solomon, Bill Keeter, Gamble Gilbertson, Meredith Mengel, Malik Ahmad, Jenny Mottar, Mike Brody, Virginia Butcher, Dawayne Pretlor, Michele Montgomery, Jim Consalvi,Siew Chin Hon, Hans Goetzelt, Shannon Lantzy, and many more.

Lastly, I'll miss the honor of serving civil servants such as Gretchen Davidian, Sharron Sample, and Ruth Netting and others.

Learn something new every year

As developers, we all make that pledge, don't we? Some of us learn new languages, new frameworks, new APIs, new methodologies. Rarely do we learn a new operating system.

30 years ago I was using whatever they called the operating system on the Apple ][ series of computers. 20 years ago I learned to use MS-DOS. 17 years ago I added Windows 3.1 to that mix. 15 years ago I ran into Windows 95 and the subsequent set of 98, 2000, Me, and XP. 10 years ago I started to play with Linux and Unix, mostly learning the simple shell commands needed to shuffle files around. 3 years ago I commenced working on Mac OS X.

During my Windows 98 days I was forced in doing help desk and system administration work. I hated it. The pop-up menus were not intuitive. Documenting what you did was hence agony. I begged my job to make me a pure developer but they forced me to continue down the path of Windows administration. Eventually I left that job and became a pure developer.

Over the years I got better at Unix and Linux. I even set up two servers for production use, one running Apache 1.3.x around 2001 and and the other a whole bunch of JBOSS stuff in 2003. Finding documentation was a pain, but the sense of logic compared to the chaos of windows pop-ups was a relief. After that when I would touch Unix or Linux it was basic file management stuff and nothing sophisticated, and my beginner skills never got a chance to grow further.

Flash forward to 2007 and beyond where I did serious Python development for Plone and then Django applications. Development was on Mac OS X and production was Linux. My neophyte level skills no longer where quite enough. Our system administrator and other co-workers helped out or did the work but it limited and frustrated me. Nevertheless, the time to really bone up on Linux never happened.

Well, this past week has been a crash course in ramping up my Linux skills. The next version of SMD Spacebook will be put into production as an RPM. Its been assigned my task to do it. This would involve a lot more in the way of Linux that what I was used to, since I needed to create an environment to run the RPM build commands, and that couldn't be our testing or production environments. That meant a chunk of setup and configuration of a lot of packages, and various proprietary bits and pieces in order to meet government and corporate requirements.

I tried not to bug my co-workers too much except for when their documentation didn't quite explain everything, and then I added to their documentation. I'm still a beginner but I feel like I've finally gotten a chance to spread my wings in this arena a little bit and learn a lot.

This will make it easier for me when I finally get around to getting this blog off blogger and into what will probably be Django-Mingus.

Code I'll reuse

When I'm evaluating a package to use in my work or play I tend to look at five things. I think many of my on-line colleagues look at a similar list. If its missing too many of these things then odds are I'll go somewhere else for my needs or roll my own.

Documentation
Did the author bother with a README file? How about some sphinx documentation? How complete it it? Does it get me started and give a few basic examples?

I'm okay with typos and mistakes. These happen. But I want to see

Licensing
Everyone has their own idea of what they like for licenses. I like the MIT/BSD thing. I can understand the attraction to LGPL and GPL although they aren't for me. What I can't stand and won't use are monstrosities like GPL/Commercial used by such libraries as ExtJs.

Want to make money off your software? Easy... let anyone use it and charge for support. Worked damn well for communities and companies like Python, Django, Plone, various Linux distributions (Redhat anyone?), etc...

Eggification
Is your software constructed so that it can be installed via easy_install or pip? And yes, this is a bit of mild embarrassment for me, so I'm happy enough to eggify other people's work.

Tests
Do you have tests? Even a nearly empty tests file or folder? How about a test application? If you have no tests then your package is suspect. How do I know it will work independently of your personal computer?

Code Quality
Does the code smell bad? Can it be easily extended? If its innovative but the code needs work is it on a DVCS so more people can easily contribute?

22 more posts to go!

I don't like Integrated Development Environments

I really don't like Integrated Development Environments (IDEs). I don't like code completion, class browsers, object inspectors, class heirarchy diagrams, source control management in whatever I am editing coding with. I find such tools arcane and frustrating.

Why?

Its because I want to be able to feel the design of a module. When I manually introspect things I feel like I am sifting through the sand of the module to see what it gives me. I feel it in my gut that this is how I learn a language and use it best of all. Python makes that very easy for you with its powerful introspection capabilities which you use on the shell. I find switching to the shell lets me separate the capabilities of the module from the code I are working on. Which for some reason I find a lot more comfortable and intuitive. Your own mileage may vary.

If I do need a class hierarchy diagram, I just write a fun little python script which generates some dot notation and run graphviz's dot or neato utility.

If I do need source control management, thats what the command line is for!

Also, there are times I have to go and do things on systems besides my own. I can't expect to have Eclipse or NetBeans there. Or if they are there by some weird chance, they won't be configured the way I want.

Keep in mind I do like code highlighting. So I guess that makes me a text editor fan. And now on to my favorites:

Textmates (Mac only) http://macromates.com/
Kendall Clark introduced me to this tool back in April/May of 2006. I was very quickly hooked. It was much better than the TextWrangler that was giving me grief. And also much less arcane than Emacs. It is the one piece of software I'll actually pay to buy to use on the Mac!

Emacs http://en.wikipedia.org/wiki/Emacs
Back in the 1990s I did Perl for a short time and was introduced to Emacs and Vi. Emacs clicked for me, because even its arcane commands worked better for me than Vi. These days my Emacs skills are not superb, but I can get by on any machine that has it installed. So that means any Mac or Linux machine I stumble across.

Textpad http://textpad.com/
So my dark secret is that until December 2006 I did much of my work in Windows. Yes, my first python work was all developed on Windows! Anyway, I had stumbled across Textpad during a Java job and kept using it across other languages and efforts. It was light, did code highlighting, and kept out of the way. Perfect! Well, maybe not since it crashed about once a day. Still, it was better for me than more sophisticated alternatives.

Can't seem to use zope debug mode anymore.

I drop out very quickly from my Mac OS X laptop running 10.5. Going to go home and try it on Ubuntu. Grrr... this is a royal pain.

Update: This was because I had a broken version of readline on my Mac. I did sudo easy_install readline and now it works.

Don't bet on closed source third party software!

US Government Agency story
Sometime in the late 1990s a US government agency (NASA) made a decision to go with a certain set of form software. Basically, this software allowed a person to use a desktop application to easily generate complex government form with data fields. The ability to easily generate paper forms that could be effortlessly converted to PDF was allowed via Adobe Acrobat Professional. All was good.

Over the next few years hundreds, perhaps thousands of forms were generated this way. A few select staff of the agency became experts in creating these forms, or converting existing paper forms into this system. Changes required for whatever reason were easily and quickly implemented, and the forms reached a pinnacle of quality. All was good.

Then, in late 2004, the company that made agency's chosen form software was purchased out by a competitor. And the first thing the purchasing company did was to cancel the products of their competitor. You couldn't buy the forms product, nor could you get support. The product was dead.

At the agency's HQ, the person using the forms software left the agency for other things. The purchased copy/copies of the forms software was somehow lost. This meant that making any changes to agency HQ forms would require rebuilding forms from scratch. Since government forms are complex, you are talking about hundreds of hours of work, especially when stuck with limiting tools such as Microsoft Word, a tool not designed for this kind of work.

At this point I stopped paying attention because it was too depressing. My guess is that the agency fixed the problem with Microsoft Word, since it was already paid for, even though using it for this role is not very efficient. Or they might have a different solution, and I pray its something good, something in the open source arena, because then even if the project collapses, writing a migration system to pull your data out is a bit easier since you have access to the source code.

This sort of thing happens from time to time. People bet on close source third party software and it nails them hard. And sometimes they don't even realize its happening...

Democracy story
Let us imagine a powerful nation which decides to rid itself of paper or mechanical ballots and go modern and electronic. They pick a security firm to deliver computerized voting machines. The delivered voting machines were pretty much proven insecure and the tallying code was beyond review since it was compiled, patented, and closed. Electoral workers and security analysts both complained that since the code was closed, there was no way to ensure that election counts were accurately tallied.

Yes, I'm talking about Diebold in the United States.

Getting past the rather intimidating security issues, the issue of closed source tallying code is terrifying. How do we know that the systems can't be used for doctoring elections? Because Diebold says so? Even if everyone implicitly trusted Diebold, do we really want to bet our future on just one small group of hidden people when the whole country could be vetting their code?

Flashpaper Story
Until I moved to Mac OS X in 2007 I hated Adobe's Portable Document Format (PDF). It didn't seem very portable, the readers were slow, and just displaying it even in modern computers slow my machine down horrifically.

And yet, back in 2002 Blue Pacific released Flashprinter which Macromedia bought and released as Flashpaper. The beauty of it was that it was part of the Flash system, which meant that any browser that supported Flash (all of them I think) supported Flashpaper documents. It printed out nicely, displayed beautifully, was lighter than PDF, and tools were easy to write for it.

In December of 2005, Adobe bought out Macromedia. Amazingly, they didn't cancel Flashpaper right away. Nope, they waited until September of 2008 to cancel this competitor to PDF. Of course, most of us shied away from Flashpaper, since the writing was on the wall.

Not everyone shied away. Young internet startups specializing in documentation seem to have really focused on the technology. What the heck were they thinking? Didn't their Venture Capitalists (VC) examine the technology base at all? I'm hoping they have alternatives ready soon, but my guess is that at least one new venture is going to fail as a VC pulls out his money and rethinks how they do analysis of proposed efforts going forward.

Summary
This isn't to say that closed source is bad. Well, maybe it is bad.

As for me, when I have a choice I use Linux. I can get away with doing my code in open source text editors (although I do prefer the relatively inexpensive and superlative closed source Textmates on Mac OS X).


Note: On September 3rd, 2011 I updated this to state that the agency listed in the first section of this article was NASA.

I've not quite done LAMP, or have I?

It hit me just now that while I've played with all the individual bits of LAMP (Linux, Apache, MySQL, Python), I've never actually done LAMP per se. I tend to play with Plone/Zope professionally, and in my goof off time I prefer to delve into wxPython, with my Django work being all too rare. I'm wondering if it might be worth it to do some formal LAMP work, and there might be available work at NASA for it real soon.

On the other hand, according to the Wikipedia entry on LAMP the 'M' in LAMP can mean 'middle-ware'. In which case I am professionally doing LAMP every work day.

Funkload to be revisited!

Benoit Delbosc (I assume its him because the Blogger sign was 'Ben') last week responded to these two posts I made in February:

• http://pydanny.blogspot.com/2008/02/funkload-charting-woes.html
• http://pydanny.blogspot.com/2008/02/ditching-funkload.html

Benoit is the author of Funkload, a promising python based functional and web load tester I explored earlier in the year. Its biggest shortcoming was the problem with getting graphical charting to work because the graphing library it was using was impossible (for me) to find. Sure, numbers are important, but a picture says a thousand words.

Especially to managers.

Well, Benoit told me that the latest Funkload calls for python-gdchart2, for which linux packages (at least Debian and Ubuntu) can be had. So installation across the many machines I touch will be trivial. I don't always use linux, but now I'll make sure to use it when I need Funkload to do what I need. It might also work with Mac OS X, something I plan to investigate tomorrow.

This makes me very happy. I really enjoyed playing with Funkload, and plan to do more of it in the near future.

And, just to make Benoit happy, even though it wasn't official, for the past 6 months or so I've used it in many unofficial load/functional tests for http://nasascience.nasa.gov as well as other internal efforts. :)

Sick but Ubuntu

I was out of work for a couple days. Stuck at home, the first evening I grabbed an old machine and installed Ubuntu plus a surplus wireless card. This was my second Ubuntu install, the first suffering from a Microsoft wireless card. I was amazed by how fast and easy the install was to do. I kept the old XP on a seperate partition and got on the house network in 30 minutes.

Of course the basic installation doesn't have all the python tools I like to use. That took me another 30 minutes.

The only real downside I discovered was a lack of Mac's Textmate. I can do emacs, but Textmate is my preference. So I toyed a bit with gEdit and was pleased. Its also free!

One thing that Ubuntu beats out Mac and Windoze for is speed. Granted, this is a new install but I have to say I love the speed of the machine. And this machine is about 3.5 years old.

Good times indeed.