brad's blog

SiteSprint III - The Reclaimation

2010-11-02T17:22:00.002-05:00

SiteSprint III - The Reclaimation

My personal website is a disgrace. As a web developer, I find it embarrassing that my own site is a cobbled-together mess of PHP and static html. Only slightly less embarrassing is my blog, which is obviously hosted on Blogger. Even though Blogger has recently added a few new themes, I'm still unhappy with the look-and-feel of my blog. So, since SiteSprint III has officially started, I'm taking this opportunity to Reclaim™ my personal website.

Tech Specs - the easy part

I've thoroughly enjoyed using Django over the last couple of years, and as any self-respecting djangonaught would, I'm moving my site over to that platform. Here's the planned tech:

nginx + gunicorn + postgresql on rackspace cloud
mezzanine
a revamp of my own django-notes

Design - the not so easy part

I'm not a graphic designer by any means, but I know I can do better than what I've got. I like Minimal designs, and I'd really like to focus on typography in my site. I've still got a lot of homework to do here. I also want make use of as much HTML5 as is feasible. In the next few days I'll be reading up on:

modernizr
font foundries (typekit, Typotheque, or kernest)
django-typogrify

Wish me luck, and feel free to point out any tools I might find interesting.

Python: stray commas cause tuples?

2010-10-15T11:26:00.003-05:00

As I try to debug a strange problem in a Django view, I notice a stray comma after a dictionary definition. So I jump over to a python shell, and guess what? Ending a literal dict with a comma creates a tuple.

>>> d = {1:'foo'},
>>> type(d)
<type 'tuple'>
>>> d
({1: 'foo'},)
>>>

A case for values_list

2010-07-19T16:25:00.002-05:00

Here's the Scenario: I have a model (lets call it Contact) with two Foreign Keys, one of which is related to User in Django's contrib.auth app. I need to build a form that lets me select an existing object, and a new user.

class ContactType(Model):
    name = CharField(max_length=128)

class Contact(Model):
    user = ForeignKey(User)
    contact_type = ForeignKey(ContactType)
    # possibly more fields...

I need to select from existing models, so my first thought might be to build a form that uses two ModelChoiceField's. I also want to modify the way that my form displays each choice, so I could extend ModelChoiceField by overriding the label_from_instance method:

class UserModelChoiceField(ModelChoiceField):
    def label_from_instance(self, obj):
        return "%s (%s)"%(obj.get_full_name(), obj.username)

class ContactModelChoiceField(ModelChoiceField):
    def label_from_instance(self, obj):
        return '%s (%s)' % (obj.type, obj.user.get_full_name())

class CopyContactForm(forms.Form):
    contact = ContactModelChoiceField(Contact.objects.all())
    new_user = UserModelChoiceField(User.objects.all().order_by('first_name', 'last_name', 'username'))

This actually provides a solution to my original problem, but it's not very efficient. Notice that both the UserModelChoiceField and the ContactModelChoiceField call methods on each object with the latter accessing a foreign key. In an app with 600 Users and 600 Contacts, this form would generate around 1200 queries!

There's actually a very efficient way to generate the same sort of form using values_list, especially, when you realize that the form really just needs to contain something like the following:

<select>
<option value="1">John Doe</option>
<option value="2">Jane Doe</option>

</select>

So a more efficient solution to my problem looks something like the code below, which yields two queries.

class CopyContactForm(forms.Form):
    contact = forms.ChoiceField(choices=[(c[0], '%s (%s %s)'%(c[1],c[2],c[3])) \
        for c in Contact.objects.values_list('id', 'type__name', 'user__first_name', 'user__last_name')])
    new_user = forms.ChoiceField(choices=[(u[0], '%s %s (%s)'%(u[1],u[2],u[3])) \
        for u in User.objects.values_list('id', 'first_name', 'last_name', 'username')])

PyGraphviz on OS X (SL) with virtualenv

2010-07-14T22:14:00.002-05:00

There's this cool project called django-extensions that (among other things) adds a lot of commands to django's manage.py offerings. One of which is ./manage.py graph_models [appname] which will generate a nice graph displaying the relationships among all of your Models. This comand needs pyGraphViz, though, and I was a little disappointed when i discovered I couldn't install pyGraphViz with pip install pygraphviz. (ok, a lot disappointed).

I eventually got this working, and here's how:

First, install the development snapshot of GraphViz: http://graphviz.org/Download_macos.php (I used graphviz-2.27.20100713.0445.pkg)

Next, download the pygraphviz source code from http://networkx.lanl.gov/pygraphviz/download.html. (Again, I got pygraphviz-0.99.1/ )

Here's the fun part. You need to edit pygraphviz's setup.py file (located at pygraphviz-0.99.1/setup.py for me). Find the following two lines of code:
```
library_path=None
include_path=None
```
And change them to
```
library_path='/usr/local/lib/graphviz/'
include_path='/usr/local/include/graphviz/'
```
You also might want to verify that the above directories exist.

Now, active your virtualenv (workon <whatever_your_virtualenv_is_named>)

Finally, just run python setup.py install

That's it! This installed pygraphviz in my activated virtualenv, and I can now use ./manage.py graph_models to my heart's desire.

I should mention that I'm using distribute (and it may be safe to assume that if you're using virtualenv, you're probably also using distribute).

Vim Syntax highlighting for Apache Config Files

2010-06-09T09:12:00.004-05:00

I use Django with Apache and mod_wsgi. Each project that I work on has different apache config files, so I like to keep those in the same mercurial repo that contains my django project's code.

For some time now, it's been bugging me that vim doesn't do syntax highlighting for those apache configs (nor the wsgi files). I finally decided to do something about it, and I'm glad I did, because it's a fairly simple fix:

My project directory layout often look like this:


~/django_projects/projectname/
    __init__.py
    apache_configs/
        projectname
        projectname.wsgi
    manage.py
    settings.py
    myapp/

Vim does syntax highlighting for apache files based on their path. To get vim to recognize the files in the apache_configs directory, you need to edit (or create) ~/.vim/filetype.vim and add the following:


au BufNewFile,BufRead ~/django_projects/de_concierge/apache_configs/*wsgi setf python
au BufNewFile,BufRead ~/django_projects/de_concierge/apache_configs/* setf apache

You'd have to add variants of this for each separate project, but this works well for me.

There's several other ways you can modify this, and the solution that I used comes from this thread on the vim_mac mailing list. (There's also a lot of good info in :help new-filetype ).

Convert Tables to Unordered Lists

2010-05-28T15:39:00.004-05:00

If you've ever had the pleasure of working with old HTML content, you've surely seen some <table>'s where they don't belong. Lately, that's the sort of thing I've been dealing with on a regular basis, and for some reason, I often see a list of information in a table.

Wouldn't it be nice if there were an easy way to turn these tables into unordered lists? Thanks to BeautifulSoup, this is really not that difficult.

Here's the code:

from BeautifulSoup import BeautifulSoup, Tag 

def table2ul(content, flatten_rows=False):
    """ 
    Convert a <table> into a <ul>.
    Each cell, <td>, gets converted into a list item <li> unless
    the flatten_rows paramter is given. In this case, all content from 
    a table row, <tr>, gets converted into a list item.
    """
    soup = BeautifulSoup(content, convertEntities=BeautifulSoup.HTML_ENTITIES, smartQuotesTo="html")

    for table in soup.findAll('table'):
        ul = Tag(soup, 'ul')

        if flatten_rows:
            for row in table.findAll('tr'):
                li = Tag(soup, 'li')
                for cell in row.findAll('td'):
                    li.contents.extend(cell.contents)
                ul.append(li)
        else:
            for cell in table.findAll('td'):
                li = Tag(soup, 'li')
                li.contents = cell.contents
                ul.append(li)
        table.replaceWith(ul)

    return soup.prettify()

Now, suppose we had the following HTML snippet:

<h1>Some heading</h1>
<p>Some paragraph with stuff in it</p>
<table>
<tr><td> row 1, <strong>col1</strong></td><td>row 1, col2</td></tr>
<tr><td> row 2, col1</td><td><em><a href="http://google.com">row 2</a></em>, col2</td></tr>
</table>

<h2>A second heading</h2>
<p>more peee</p>

Passing this in to table2ul would convert each cell into a list item, <li>.

>>> table2ul(content)

<h1>
 Some heading
</h1>
<p>
 Some paragraph with stuff in it
</p>
<ul>
 <li>
  row 1,
  <strong>
   col1
  </strong>
 </li>
 <li>
  row 1, col2
 </li>
 <li>
  row 2, col1
 </li>
 <li>
  <em>
   <a href="http://google.com">
    row 2
   </a>
  </em>
  , col2
 </li>
</ul>
<h2>
 A second heading
</h2>
<p>
 more peee
</p>

But what if we don't want each <td> converted into an <li%gt>? What if we want all the content from entire row in an <li>? In that case, just set the optional flatten_rows parameter:

>>> table2ul(content, flatten_rows=True)

<h1>
 Some heading
</h1>
<p>
 Some paragraph with stuff in it
</p>
<ul>
 <li>
  row 1,
  <strong>
   col1
  </strong>  row 1, col2
 </li>
 <li>
  row 2, col1
  <em>
   <a href="http://google.com">
    row 2
   </a>
  </em>
  , col2
 </li>
</ul>
<h2>
 A second heading
</h2>
<p>
 more peee
</p>

This has been somewhat useful for me. Hope it's useful for you!

Pretty options for Django's auth.User

2010-04-22T09:07:00.004-05:00

Several of my Django Apps have Foreign Key relationships to django.contrib.auth.model.User. In Django's admin app, these show up a select elements displaying the username attribute. For some people, that may be OK, but for most of the people with which I work, it's not. We want to see prettier options, i.e. each User's full name as the options in that select element.

So, here's how it works. We override the ModelChoiceField (for ForeignKeys) and the ModelMultipleChoiceField (for ManyToMany Fields):

from django.forms import ModelChoiceField, ModelMultipleChoiceField

class UserModelChoiceField(ModelChoiceField):
    ''' 
    A ModelChoiceField to represent User 
    select boxes in the Auto Admin 
    '''
    def label_from_instance(self, obj):
        return "%s (%s)"%(obj.get_full_name(), obj.username)

class UserModelMultipleChoiceField(ModelMultipleChoiceField):
    ''' 
    Similar to UserModelChoiceField, provide a nicer-looking 
    list of user names for ManyToMany Relations...
    '''
    def label_from_instance(self, obj):
        return "%s (%s)"%(obj.get_full_name(), obj.username)

Then, to customize the admin, you need to create a custome ModelForm for your Model. So, if I had a Model that looked like this:

class MyModel(models.Model):
    user = models.ForeignKey(User)

You'd need to create the following ModelForm:

class MyModelAdminForm(forms.ModelForm):
    user = UserModelChoiceField(User.objects.all().order_by('first_name', 'last_name', 'username'))

    class Meta:
        model = MyModel

Now, when you create a ModelAdmin class for the MyModel, you specify the above form:

from models import MyModel
from forms import MyModelAdminForm

class MyModelAdmin(admin.ModelAdmin):
    form = MyModelAdminForm
admin.site.register(MyModel, MyModelAdmin)

At this point, the choices for User objects in the admin should contain the user's full name and their username in parenthesis.

Dealing with Unicode and ASCII using Python

2010-03-25T15:21:00.002-05:00

Dealing with Character Encodings is (sometimes) hard. It's especially confusing for those who've never done it before. Converting text from unicode to ascii can be tricky.

A lot of times, I'll import some data from a text file, and I just want to convert everything to ASCII and ignore anything that's not ascii (like MS Word's smart quotes). Luckily, this is fairly easy:

mystring = mystring.decode('ascii', 'ignore')

There's tons of great Python resources (and code!) for all your character encoding needs. In no particular order, here are a few I've found:

A Crash Course in Character Encoding
Dive Into Python's Chapter on Unicode
Beautiful Soup gives you Unicode, Dammit and there's the companion: ASCII, Dammit
There's also unaccent.py, which seems to convert various unicode characters to their ascii equivalent.

There's probably more, but most of these have helped me get the job done.

On select_related()

2010-03-23T15:02:00.003-05:00

If you use Django, and your models have relationships that span across multiple tables, you need to read this: http://docs.djangoproject.com/en/dev/ref/models/querysets/#id4

select_related() is awesome.

That is all.

Building PIL on OS X: Snow Leopard

2010-02-25T15:40:00.002-06:00

There are several places online that discuss problems installing PIL on Mac OS X Snow Leopard

This is how I got it to work.

Install lib jpeg using homebrew (which is super-aweseome!)
```
brew intall jpeg
```
.This installs the library into
```
/usr/local/Cellar/jpeg/7
```

Install libfreetype the old-fashioned way (./configure, make, sudo make install). I used freetype-2.1.10.pre-20050511.

Download, unpack PIL (I used Imaging-1.1.6). I had to make the following changes to setup.py

FREETYPE_ROOT = "/usr/local"
JPEG_ROOT = ("/usr/local/Cellar/jpeg/7/lib", "/usr/local/Cellar/jpeg/7/include")

Then, build PIL:
```
python setup.py build_ext -i
```

If the build works without any errors, you can run the tests:
```
python selftest.py
```
which should yield the following: 57 tests passed.

(optional) I like to use virtualenv, so I activate that:
```
workon myproject
```

Install PIL:
```
python setup.py install
```

At this point, code like the following works for me:

import Image
im = Image.open("/path/to/pretty/picture.jpg")
im.show()

Specialization

2010-02-24T13:50:00.002-06:00

Thanks to @mxcl for tweeting a link to this: http://gist.github.com/313640

Things I've done (or similar)

change a diaper
plan an invasion (warcraft counts, right?)
butcher a ~~hog~~ squirrel
design a building (with legos!)
write a ~~sonnet~~ song
balance accounts
build a ~~wall~~ fence
comfort the dying
take orders
give orders
cooperate
act alone
solve equations
analyze a new problem
pitch manure
program a computer
cook a tasty meal (yum!)
fight efficiently (yay for taekwondo!)

...not quite there yet:

conn a ship
set a bone
die gallantly (here's hoping!)

Hooray for Humanity!

Adventures in Bread-Baking: Part 1

2010-01-23T09:57:00.003-06:00

Last weekend I attempted to make Mark Bittman's Speedy No-Knead Bread... sort of. Being the adventurous experimenter that I am, I altered the ingredients:

2 cups whole-grain flour + 1 cup all purpose flour
2 cups of really warm water (instead of the 1.5 cups)

I followed the rest of the recipe perfectly. However, the results were really unsatisfactory. The center was super dense and gooey, and the bread had a really really hard crust (ok, I also cooked it an extra 30 minutes because of the gooey center).

I didn't take any pictures, because the results were so embarrassing.

Here's what I learned: Bread Flour != All Purpose Flour (for the non-programmer, that means they are NOT the same thing!)

Fortunately, there are several articles online that expound on the differences between bread flour and all-purpose flour.

The secret is wheat gluten (or gluten additive). Essentially, you must add 1tsp to each cup of all-purpose flour to create bread flour. Unfortunately, this is not something I keep. So part 2 will have to wait until I go shopping...

Add JSON syntax highlighting in Vim on OS X

2010-01-15T15:03:00.004-06:00

This is how I got Vim to do syntax highlighting for JSON files (Max OS X - Snow Leopard)

Download the syntax file from http://www.vim.org/scripts/script.php?script_id=1945. (I got json.vim version 0.4)
If it doesn't already exist, create a .vim/ftplugin in your home directory.
Put json.vim into .vim/ftplugin
Add the following lines in your .vimrc file (mine is located at ~/.vimrc)
```
au BufRead,BufNewFile *.json set filetype=json
au! Syntax json source /Users/brad/.vim/ftplugin/json.vim
```
Note that you'll have to change the path to your json.vim file so that it works on your computer.
Now, got edit a .json file!

Resize the iTunes Window

2010-01-14T09:47:00.003-06:00

My Macbook Pro is my main machine. At the office, I connect it to a 20" Cinema display. At home, I connect it to a 24" Samsung Monitory (Model 2494SW Glossy Black, which I really like, btw.)

There's a down-side to all this, though. I mostly just notice it with iTunes. When I'm working on the laptop without an external monitor, my iTunes window is larger than my screen :( Unfortunately, clicking the Maximize button doesn't do what I want (which is to make the window fit in the screen. So here's a little Applescript to to fix it:

tell application "iTunes"
 set the bounds of the first window to {50, 50, 1024, 640}
end tell

The arguments are the distance from the top and left of the screen (both 50), the width of the iTunes window (1024) followed by its height (640).

I just saved this in a file called resize_itunes.scpt and put it in my Documents/Applescripts/ folder. I'm a big fan of Quicksilver, so when I need to resize my iTunes window, I just activate Quicksilver (Ctrl+Spacebar), type in "resize" (which finds the applescript file), and then hit "enter" to run the script. Easy.

Ubuntu Audio, YES! Windows XP, NO.

2009-12-13T14:57:00.003-06:00

I run a dual-boot Ubuntu64[1] and Windows XP system. The windows partition really exists for a single purpose, and I occasionally just reboot the machine, choose the XP partition from the Grub menu, and all is well.

However, a few weeks ago, I upgraded Ubuntu, but when I rebooted the machine and chose the XP partition, I noticed the sound[2] stopped working. What!? The audio worked fine in ubuntu!

The Secret: I had to completely power down the machine before booting into Windows. Then, audio works just fine.

Unfortunately, I didn't discover this until I'd re-installed the windows drivers (which didn't work), reformatted/reinstalled windows and all drivers (which only worked because I powered down in the process), fixed my grub boot loader, and incorrectly assumed the problem was fixed until I booted back into Ubuntu and back into Windows only to find that the problem remained.

Sheesh. So, if this has happened to you, I'm sure you've found the myriad article on how to get sound working in Linux... which for me, has always worked just fine.

[1] Output of uname -a:

Linux homeserver 2.6.31-16-generic #53-Ubuntu SMP Tue Dec 8 04:02:15 UTC 2009 x86_64 GNU/Linux

[2] A Soundblaster Audigy LS. Output of lspci:

02:02.0 Multimedia audio controller: Creative Labs CA0106 Soundblaster

Gahhh!! Django, virtualenv, and cx_Oracle

2009-11-24T13:47:00.005-06:00

UPDATE: Thanks in advance to the comments from Graham Dumpleton whose comments below pointed me in the right direction!

This wasis a plea for help.

I've got django installed and configured with apache and virtualenv. I also have one particular app (named myapp) that queries an Oracle database directly (django is configured to use MySQL). All of the apps work, except for anything thatbrequires the myapp app... which includes the admin!

Requesting any view that uses cx_Oracle results in a cryptic error simiar to:

ViewDoesNotExist at /some/url/
Could not import myproject.myapp.views. Error was: libclntsh.so.10.1: cannot open shared object file: No such file or directory

The full traceback follows:

Environment:

Request Method: GET
Request URL: http://mydomain.com/myapp/foo/
Django Version: 1.1.1
Python Version: 2.6.2
Installed Applications:
['django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.admin',
 'django.contrib.admindocs',
 'django.contrib.flatpages',
 'django_extensions',
 'myproject.userprofile',
 'myproject.myapp',

 ... snip...

 'myproject.utils']
Installed Middleware:
('django.middleware.gzip.GZipMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.middleware.doc.XViewMiddleware')


Traceback:
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  83.                     request.path_info)
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/urlresolvers.py" in resolve
  218.                     sub_match = pattern.resolve(new_path)
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/urlresolvers.py" in resolve
  218.                     sub_match = pattern.resolve(new_path)
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/urlresolvers.py" in resolve
  125.             return self.callback, args, kwargs
File "/home/django/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/urlresolvers.py" in _get_callback
  134.             raise ViewDoesNotExist, "Could not import %s. Error was: %s" % (mod_name, str(e))

Exception Type: ViewDoesNotExist at /myapp/foo/
Exception Value: Could not import myproject.myapp.views. Error was: libclntsh.so.10.1: cannot open shared object file: No such file or directory

The strange thing is that tests using cx_Oracle run correctly (as does code typed into the interpreter via ./manage.py shell), so cx_Oracle is installed correctly in the virtualenv. Orignally, I thought that apache was somehow misconfigured. The problem, however, is that Apache just doesn't know where to find the shared libraries for cx_Oracle's C extension modules. On my system (Ubuntu), shared libraries are located in /usr/lib and /usr/local/lib .

When building cx_Oracle, you must include an environment variable for ORACLE_HOME and update the LD_LIBRARY_PATH so that it also includes ORACLE_HOME. While logged in as the django user, I updated .bashrc so that it includes the following:

export ORACLE_HOME=/home/django/oracle/instantclient_10_2
export LD_LIBRARY_PATH=$ORACLE_HOME:$LD_LIBRARY_PATH

For me, all of the cx_Oracle shared libraries are located in /home/django/oracle/instantclient_10_2. So, (with my virtualenv activated) I built, installed, and tested cx_Oracle, and everything worked fine, because my user environment was set up so that these shared libraries are available.

To make these available to Apache, I created hard links to all of the shared libraries from /usr/local/lib, and then ran ldconfig. Note that ldconfig ignores symbolic links!

cd /usr/local/lib
sudo ln /home/django/oracle/instantclient_10_2/libclntsh.so.10.1
# also creating links to all other libs
sudo ldconfig

After restarting Apache, my django apps worked as expected. For the curious, my wsgi script and apache config follows.

My .wsgi script

import sys
import site
import os

vepath = '/home/django/.virtualenvs/myproject/lib/python2.6/site-packages'
prev_sys_path = list(sys.path)

# add the site-packages of our virtualenv as a site dir
site.addsitedir(vepath)

# add the app's directory to the PYTHONPATH
sys.path.append('/home/django/django_projects/myproject_root/myproject/')
sys.path.append('/home/django/django_projects/myproject_root/')
os.environ['PYTHON_EGG_CACHE'] = '/home/django/.python-eggs'

new_sys_path = [p for p in sys.path if p not in prev_sys_path]
for item in new_sys_path: 
    sys.path.remove(item)
sys.path[:0] = new_sys_path

from django.core.handlers.wsgi import WSGIHandler
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings'
application = WSGIHandler()

The Pertinent part of the Apache Config, located within a VirtualHost Directive:

Alias /media/ /var/www/media/
Alias /site_media/ /var/www/static/

# Run WSGI in Daemon Mode:
WSGIDaemonProcess myproject user=www-data group=www-data threads=25
WSGIProcessGroup myproject
WSGIScriptAlias / /home/django/django_projects/myproject_root/apache_mod_wsgi_conf/myproject.wsgi

NOTE: Graham Dumpleton suggests four options to get around the original problem (and I paraphrase):

Rebuild cx_Oracle setting the LD_RUN_PATH - I tried this, but was unnsuccessful.

Put the shared libraries in /usr/lib - I hard linked to the libraries from /usr/local/lib. This is the solution that worked.

Modify /etc/ld.conf so that it includes the path to the shared libraries - I considered adding a file to /etc/ld.so.conf.d/ which would have included the path to my ORACLE_HOME directory. However, the above options worked, so I went with that instead.

Edit /etc/apache/envvars so that it includes the appropriate environment variables - I also tried this approach, including the same environment variables that I added to my .bashrc, but this also did not work

Hope this helps someone else!

Sticky Groups

2009-11-24T12:07:00.003-06:00

I often deploy web projects in a directory that's not owned by the user under which my webserver runs. Therefor, I often have to change permissions so the webserver can read from or write to certain files. So, for this example, let's assume I'm logged in to my linux box as brad, and I'm using apache which runs under the user www-data.
To give apache access to my public_html directory, I'd change ownership for the directory and all of its contents:

chown -R brad:www-data public_html

I may also need to allow apache to write to a certain directory:

chmod g+w public_html/somedir

Now, if I modify any files in public_html, I don't want to have to change the group permissions (that is, apache's permissions), so lets make the group ownership sticky:

chmod g+s public_html

Now, any new files that I add beneath the public_html directory will be part of the www-data group.

Mercurial installation woes on Mac OS X

2009-09-30T12:15:00.006-05:00

I started using mercurial around version 1.2, and I'm pretty sure I used the Mac OS X installer (from http://mercurial.berkwood.com/) to install 1.2.1. This placed hg in /Library/Frameworks/Python.framework/Versions/Current/bin/.

Now, I've decided to upgrade to 1.3.1, and I again grab the Mac OS X installer (again from http://mercurial.berkwood.com/), which installs hg in /usr/local/bin/.

Ok, but my path is set up so that the older version is used by default... Is there a preferred way to remove the older version of mercurial?

I just renamed the file: /Library/Frameworks/Python.framework/Versions/Current/bin/hg to hg-1.2.1, and source'd my .profile so that the new version (/usr/local/bin/hg) gets used instead.

This does not seem like a good way to upgrade. :(

this is all relevant to Mac OS X 10.5.8

A for AJAX - OR - Dynamically generating options for a select element.

2009-07-14T11:42:00.008-05:00

I don't do a lot of AJAXy web development, but when I do, I usually make use of Prototype. I've recently created a form containing a <select> element whose <option>s are dynamically generated via an AJAX request. The problem however, is that a selected option was already in the form. So before the AJAX request, my HTML looked something like this:

<select name="s" id="s">
<option value="val1">Value 1</option>
<option value="val2" selected="selected">Value 2</option>
<option value="val3">Value 3</option>
</select>

We can then use Ajax.Request method to dynamically add items into this list. (Note here that http://example.com/dynamic/options/ would be a server-side script giving us our dynamically generated <option> tags.)

new Ajax.Request('http://example.com/dynamic/options/', {
    method: 'post',
    onSuccess: function(transport) {
        if (transport.responseText)
            Element.insert($('s'), transport.responseText);
    }
});

Our resulting HTML might look something like the following:

<select name="s" id="s">
<option value="val1">Value 1</option>
<option value="val2" selected="selected">Value 2</option>
<option value="val3">Value 3</option>
<option value="val4">Value 4</option>
<option value="val5">Value 5</option>
<option value="val6">Value 6</option>
</select>

One problem that arises, is that once the above <option>s are inserted into the original <select> element, the originally selected option is no longer selected. (In my experiments using Firefox 3.5 on OS X, the last item — in this case, Value 6— becomes selected).

That's not so bad, though, because a little more code will just set the correct <option> as selected. The following code looks at all the options until it finds one with the attribute: selected="selected", and then ... selects it!

var selected_index = 0;
while (!$('s').down(selected_index).defaultSelected) {
    selected_index += 1;
}               
$('s').selectedIndex = selected_index;

However... one must be careful where one puts this code! You must take into account that the Ajax Request is asynchronous (that's the A in AJAX)!

Here's what NOT to do:

function append_options() {
    // Request the dynamically generated options
    new Ajax.Request('http://example.com/dynamic/options/', {
        method: 'post',
        onSuccess: function(transport) {
            if (transport.responseText)
                Element.insert($('s'), transport.responseText);
        }   
    }); 
    // Make sure the correct on is selected.    var selected_index = 0;
    while (!$('s').down(selected_index).defaultSelected) {
        selected_index += 1;
    }
    $('s').selectedIndex = selected_index;
}
document.observe('dom:loaded', function() {
    append_options();
});

The Ajax request is sent, but the portion of the code that selects the option containing selected="selected" may get run before there is a response to that Request. Therefore, the selected option gets set, then the dynamically generated options get inserted. This is not what we want!

The proper way to do this is to use the onComplete callback in Ajax.Request. This will ensure that the code to select the appropriate option is run after the Request is completed.

function append_options() {
    // Request the dynamically generated options
    new Ajax.Request('http://example.com/dynamic/options/', {
        method: 'post',
        onSuccess: function(transport) {
            if (transport.responseText)
                Element.insert($('s'), transport.responseText);
        },  
        onComplete: function(transport) {
            // Make sure the correct on is selected.
            var selected_index = 0;
            while (!$('s').down(selected_index).defaultSelected) {
                selected_index += 1;
            }
            $('s').selectedIndex = selected_index;
        }
    }); 
}
document.observe('dom:loaded', function() {
    append_options();
});

This is fairly basic AJAX behavior, but for those of us who don't live, breathe, eat, and sleep AJAX, this is an easy mistake to make.

So the moral of this story? Know your tools! It's always good to Know what's going on!™

My PYTHONPATH bit me.

2009-06-30T14:23:00.003-05:00

I'd just finished the first version of a new django app (myapp), and so I pushed it out to my development server. All the new code was in place, so I ran python manage.py syncdb. The result?


Traceback (most recent call last):
  File "manage.py", line 11, in <module>
    execute_manager(settings)
  File "/usr/local/lib/python2.6/site-packages/django/core/management/__init__.py", line 340, in execute_manager
    
  File "/usr/local/lib/python2.6/site-packages/django/core/management/__init__.py", line 295, in execute
    
  File "/usr/local/lib/python2.6/site-packages/django/core/management/base.py", line 192, in run_from_argv
  File "/usr/local/lib/python2.6/site-packages/django/core/management/base.py", line 210, in execute
  File "/usr/local/lib/python2.6/site-packages/django/utils/translation/__init__.py", line 73, in activate
    
  File "/usr/local/lib/python2.6/site-packages/django/utils/translation/__init__.py", line 43, in delayed_loader
    
  File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 209, in activate
  File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 198, in translation
  File "/usr/local/lib/python2.6/site-packages/django/utils/translation/trans_real.py", line 181, in _fetch
AttributeError: 'module' object has no attribute 'myapp'

What!? I had all the correct files in place, and myapp was listed in my settings' INSTALLED_APPS. I was confused. As it turns out, my PYTHONPATH pointed to my production code (which is actually on the same system, configured for a separate virtual host). So no, it couldn't find my new app because my path said to look in my production code.... :(

So, I just temporarily reset my PYTHONPATH, ran my python manage.py syncdb, and I'm rolling for now...

"Adding" Q objects in Django

2009-06-26T09:47:00.003-05:00

I've got a Django app with the following Model:


class Story(models.Model):
    title = models.CharField(max_length=255)
    content = models.TextField()

The Problem:
I wanted to build a simple search feature that OR'ed all the search terms. Essentially, I wanted SQL resembling the following:

SELECT * from myapp_stories where 
    title LIKE '%term1%' OR content LIKE '%term1%' OR 
    title LIKE '%term2%' OR content LIKE '%term2%';

The Solution:
You can add django's Q objects together! This is a feature not currently discussed in the docs, but I dug through the source code and I discovered that a Q object is really just a node in a Tree! More specifically, Q is a subclass of django.utils.tree.Node (check it out, it's cool!) A Node has a attribute called a connector. Q objects have two possible connectors: AND and OR. But how do we connect Q objects? Well, a Node has a handy add(node, conn_type) method whose parameters include another Node and a connection type.

As previously mentioned, the possible connection types for Q objects are AND and OR, so Q objects can be added together by doing something like this:

# ANDing Q objects
q_object = Q()
q_object.add(Q(), Q.AND)

# ORing Q objects
q_object = Q()
q_object.add(Q(), Q.OR)

So, the solution to my Search view is as follows:


from django.db.models import Q
from models import Story

def search(request):
    ''' 
    Generic Search: GET should contain the following: 
    terms - the search keywords separated by spaces
    '''
    terms = request.GET.get('terms', None)
    term_list = terms.split(' ')

    stories = Story.objects.all()

    q = Q(content__icontains=term_list[0]) | Q(title__icontains=term_list[0])
    for term in term_list[1:]:
        q.add((Q(content__icontains=term) | Q(title__icontains=term)), q.connector)

    stories = stories.filter(q)

    return render_to_response('myapp/search.html', locals(), \
            context_instance=RequestContext(request))

Needless to say, Q objects are quite powerful!

Data Truncated Errors

2009-05-06T15:03:00.004-05:00

I recently ran into some of the Data truncated for column ... errors in my django apps. After a little digging, I've discovered that my particular problem lie in the structure of the underlying MySQL tables. Particularly with varchar columns.

I have a model that contains a FileField:

class MyModel(models.Model):
    file = models.FileField(upload_to="files/%Y/%m/%d")

Note that the MySQL table generated by this model will look something like the following:

+-------------+--------------+------+-----+---------------------+----------------+
| Field       | Type         | Null | Key | Default             | Extra          |
+-------------+--------------+------+-----+---------------------+----------------+
| id          | int(11)      |      | PRI | NULL                | auto_increment |
| file        | varchar(100) |      |     |                     |                |
+-------------+--------------+------+-----+---------------------+----------------+

See the varchar(100)! If you're users are uploading files with long names (mine are!), this may not be enough! Note that my model also has an upload_to that includes a year, month, and day. That's already 17 characters...

So when someone tries to upload a file named:

Who_Said_One_Hundred_Characters_Would_Ever_Be_Enough_For_A_REALLY_LONG_Filename_draft_one_.docx

It will get pre-pended with '/files/2009/05/06/', and what gets stored in your table?

files/2009/05/06/Who_Said_One_Hundred_Characters_Would_Ever_Be_Enough_For_A_REALLY_LONG_Filename_draft_one_.docx

That's 112 characters... and it won't work!

The solution?
You could alter your table so that the file column is a varchar(255)... which may help.

alter table myapp_mymodel modify file varchar(255);

OR, use shorter filenames!

Good Luck!

Have Apache Force File Downloads

2009-05-01T14:27:00.003-05:00

I have a Django app that lets users upload files. Any kind of file. It's nice that Apache will let me force file downloads based on the files extension.

<LocationMatch "\.(gz|tar|pdf|docx|doc|xls|xlsx|bz2|zip)$">
    SetHandler None
    Header set Content-Disposition attachment
</LocationMatch>

So, in my HTML/templates: all I have to do is this:

<a href="SomFile.docx">Some File</a>

Push Your SSH Public keys using Fabric

2009-04-30T15:45:00.005-05:00

This came across my twitter radar today from @bitprophet (aka: Jeff Forcier), who just happens to be the new maintainer for Fabric:

def push_key():
    keyfile = '/tmp/%s.pub' % env.user
    run('mkdir -p ~/.ssh && chmod 700 ~/.ssh')
    put('~/.ssh/id_rsa.pub', keyfile)
    run('cat %s >> ~/.ssh/authorized_keys' % keyfile)
    run('rm %s' % keyfile)

Everything you need to push your public key to an external server using Fabric.

Restricting Access by Group in Django

2009-04-24T18:36:00.006-05:00

Django's authentication system provides built-in support for Groups. When developing an app, you may want to prevent users in a particular group from accessing part of your app.

For example, if you were building a tool to be used by Faculty and Students, it's quite possible that there would be parts of the app you wouldn't want Students to access (like the part that allows a User to change grades!)

Luckily, there's a decorator called user_passes_test that allows you to easily perform this sort of thing. Let's see an example:

from django.contrib.auth.decorators import login_required, user_passes_test

@login_required
@user_passes_test(lambda u: u.groups.filter(name='Student').count() == 0, login_url='/myapp/denied/')
def some_view(request):

    # Do whatever this view should do

The view above (which lacks any content) actually uses two decorators. The first login_required simply requires that a user be logged in. The second, user_passes_test, requires a function as the first argument. This function must accept a User object and return True or False. If True, the User can view the page. If False, the user cannot view the page.

We define this function using a python Lambda Expression. It simply uses the ORM to check if a User is in the Student group. In this example, u would be an instance of django.contrib.auth.models.User.

lambda u: u.groups.filter(name='Student').count() == 0

Additionally, you can specify the keyword argument login_url to the user_passes_test decorator. Normally, if the user failed the test this would redirect them to a login page. However, in our case, they're already logged in (because of the login_required decorator), so this just acts as a redirect page. In this example, it would redirect to a url that we've specified which might contain an Access Denied message (with giant red blinking letters). Just kidding.

UPDATE: There is one slight caveat to this solution so far... it breaks if your user is not already logged in. In fact, this will result in the following AttributeError:

'NoneType' object has no attribute '_meta'

So, to account for that, we need to put the logic that tests for Student-group membership into its own function. The result would look something like the following:

def not_in_student_group(user):
    if user:
        return user.groups.filter(name='Student').count() == 0
    return False

This makes sure the user object exists before trying the test, and if not, we assume the test fails (by returning False).

Now, our decorator will look something like this:

@login_required
@user_passes_test(not_in_student_group, login_url='/advising/denied/')
def some_view(request):
    # ...

Dynamically Displaying Fields in a ModelForm

2009-04-21T13:31:00.005-05:00

The Problem: I want to dynamically include some fields in a ModelForm based on some external criteria. Sometimes I want the fields displayed, sometimes I don't.

I'm going to try to explain this scenario through a (albeit contrived) example. I have a Model that looks like the following:

class Suff(models.Model):
    foo = models.CharField(max_length=255)
    bar = models.BooleanField(default=False, blank=True)

    def is_foo_bar(self):
        ''' is this model's foo attribute set to 'bar' '''
        return self.foo == 'bar'

Normally, if I needed a Form for this Model, I would subclass a ModelForm like the following:

class StuffForm(models.ModelForm):
    class Meta:
        model = Stuff
        fields = ('foo', 'bar')

However, if I do NOT want the 'bar' field to be displayed by default I would need to remove it from the ModelForms list of fields (or use something like exclude = ('bar', ) ). But, if this form is created with an instance of Stuff whose foo attribute contains the string bar, I would like for the Form's 'bar' field to be displayed.

I originally tried to accomplish this task by overridding StuffForm's __init__ method, and adding a new BooleanField when the desired circumstances arose... However, I stumpled across Ross Poulton's Dynamic ModelForms in Django, and then I realized it would be much easier to prevent a ModelForm's Field from being displayed than it would be to dynamically create one.

In order to accomplish this, the StuffForms's __init__ method would look something like the following:

def __init__(self, *args, **kwargs):
    super(StuffForm, self).__init__(*args, **kwargs)
    
    # If this form is created without an instance, OR
    # If the instance's foo field is != 'bar'
    if not kwargs.has_key('instance') or (kwargs.has_key('instance') and \
        not kwargs['instance'].is_foo_bar()):
        # Remove this field from the form.
        del self.fields['bar']

Done. I get all the benefits of a ModelForm, and the bar field is not displayed unless it should be.

How to Set up a Foreign Key Constraint in MySQL

2009-04-01T14:49:00.004-05:00

The default storage engine in MySQL (MyISAM) does not support Foreign Key constraints. If you want to use Foreign Keys in Mysql, you need to use InnoDB.

The following is a simple example that illustrates Foreign Key constraints, we'll create tables to store information about Authors and their Books. The Foreign key will link a book to an Author. Note, that in MySQL we need to use the InnoDB storage engine to support Foreign Key Constraints.

First, we need to create a simple table for Authors. There are only two columns: a primary key and the author's name

CREATE TABLE author (id integer primary key auto_increment, name text) ENGINE=InnoDB;

Next, we create a simple table for Books. Again, we need a primary key (id), the title of the book, and the column that will be used as the Foreign Kye (author_id). The author_id column will be a Foreign Key that references the author table's id column (i.e. it's primary key).

CREATE TABLE books (id integer primary key auto_increment, title text, author_id integer NOT NULL) ENGINE=InnoDB;

Finally, we alter the books table to add the Foreign Key constraint. Below, the author_id_refs is just a name for the constraint, and this could be anything that we want (as long as it's sensible!)

ALTER TABLE `books` ADD CONSTRAINT author_id_refs FOREIGN KEY (`author_id`) REFERENCES `author` (`id`);

Another example is available in the MySQL documentation that covers Foreign Key Constraints.

An Example

Insert a couple of Authors:

insert into author (name) values ('Brad Montgomery');
insert into author (name) values ('John Doe');

Let's see what's in the author table:


mysql> select * from author;
+----+-----------------+
| id | name            |
+----+-----------------+
|  1 | Brad Montgomery | 
|  2 | John Doe        | 
+----+-----------------+
2 rows in set (0.00 sec)

Lets put some stuff in the Books table. Note that author_id column corresponds to the id column in the author table above.

insert into books (title, author_id) values ('Brads book', 1);
insert into books (title, author_id) values ('John Does book', 2);

Lets see what the books table looks like and what's in it:


mysql> describe books;
+-----------+---------+------+-----+---------+----------------+
| Field     | Type    | Null | Key | Default | Extra          |
+-----------+---------+------+-----+---------+----------------+
| id        | int(11) | NO   | PRI | NULL    | auto_increment | 
| title     | text    | YES  |     | NULL    |                | 
| author_id | int(11) | NO   | MUL | NULL    |                | 
+-----------+---------+------+-----+---------+----------------+
3 rows in set (0.00 sec)

mysql> select * from books;
+----+----------------+-----------+
| id | title          | author_id |
+----+----------------+-----------+
|  1 | Brads book     |         1 | 
|  2 | John Does book |         2 | 
+----+----------------+-----------+
2 rows in set (0.00 sec)

Try to Delete Something

When you try to delete an author, an Error will occur

delete from author where id=2;
ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails 
(`brad/books`, CONSTRAINT `author_id_refs` FOREIGN KEY (`author_id`) REFERENCES `author` (`id`))

This happens because the data in the books table depends on the data in the author table. The Default constraint prevents you from deleting these books, without first deleting the author


mysql> delete from books where author_id=2;
Query OK, 1 row affected (0.00 sec)

mysql> delete from author where id=2;
Query OK, 1 row affected (0.01 sec)

When you delete the author's books first, the author no longer has any dependencies. You can therefore delete the author.

Public Keys don't like write access!

2009-03-25T15:20:00.003-05:00

The first part of my Automatic Backups with cron, tar, and SSH details how to set up remote login (sans passwords) using RSA public key.

Apparently this doesn't work if your home directory allows groups or others write access. So if used ssh-keygen -t rsa to generate a public key which you added to a remote host (under .ssh/authorized_keys), but ssh is still prompting you for a password, try setting your home directory's permissions to something like 0755 (or remove write permissions from other and groups)!

mod_python checks your blood pressure

2009-03-24T09:28:00.003-05:00

I'm deploying a django project using mod_python... now, usually I will just use my package management tools to install a binary version, but this time I need to build it from source.

As I get ready to go through the whole configure/make/make install process, I peruse the output of configure just to make sure everything is ok...

checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for ar... ar
checking for a BSD-compatible install... /usr/bin/install -c
checking whether make sets $(MAKE)... yes
checking for main in -lm... yes
checking for an ANSI C-conforming const... yes
checking your blood pressure... a bit high, but we can proceed
configure: checking whether apxs is available...
checking for --with-apxs... /usr/sbin/apxs executable, good
checking Apache version... 2.0.52
checking for Apache libexec directory... /usr/lib/httpd/modules
checking for Apache include directory... -I/usr/include/httpd
checking for --with-python... /usr/local/bin/python2.6
checking Python version... 2.6
checking Python install prefix... /usr/local
checking checking where python libraries are installed... /usr/local/lib/python2.6
checking for Py_NewInterpreter in -lpython2.6... yes
checking what libraries Python was linked with... -lpython2.6 -lpthread -ldl -lutil -lm
checking linker flags used to link Python... 
checking where Python include files are... -I/usr/local/include/python2.6
checking for --with-python-src... /home/bkmontgomery/Python-2.6.1/src
checking for --with-mutex-dir... no
Using MUTEX_DIR /tmp
checking for --with-max-locks... 32
Using 32 MAX_LOCKS.
checking for --with-flex... /usr/bin/flex
found /usr/bin/flex, we'll use this. Use --with-flex to specify another.
checking flex version... configure: WARNING: Flex version 2.5.4 found.
    Version 2.5.31 or greater is required.  You can generally ignore this
    warning unless you need to regenerate psp_parser.c from psp_parse.l.
    If you do need regenerate psp_parser.c, use --with-flex to specify the
    location of the correct flex version. See the README for more information.
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating Doc/Makefile
config.status: creating src/include/mod_python.h
config.status: creating test/Makefile
config.status: creating test/testconf.py
config.status: creating dist/setup.py
config.status: creating dist/Makefile

Thank you for caring for my health, mod_python.

Scheduled Tasks (or cron jobs) with Django

2009-02-20T10:15:00.005-06:00

This is my take on setting up cron jobs for the apps in a Django project. It is based on my own convention, and it solves my initial problems where I want to perform some action on all of my Django apps at a periodic interval (currently this is a once-a-day task).

In order for this to work, I create a cron.py module for all of my INSTALLED_APPS. This module must contain a run method. Other than that, it can work just like any other python module (using django's internals as necessary).

For example, if you had an app called utils (possibly located at mysite/utils/), and if you just wanted to delete all sessions with old expire_dates, your cron.py (which you would put in mysite/utils/cron.py) might look something like this:

from datetime import datetime
from django.contrib.sessions.models import Session

def delete_old_sessions():
    Session.objects.filter(expire_date__lt=datetime.now()).delete()

def run():
    delete_old_sessions()

Now, the meat of this solution checks for the cron.py module in all of the apps in mysite.settings.INSTALLED_APPS, and invokes its run() method. I've also named this module cron.py, but this gets stored in Django's project directory (e.g. mysite)... the same directory where your settings.py is located.

#!/usr/bin/env python
"""
Project-wide Cron Job... A Command-line Django Script.

This script gets scheduled and run by cron (or whatever).
It then calls the `run` method of each app's cron module, 
if it exists (should be `appname/cron.py`)

This script should be invoked after setting the 
DJANGO_SETTINGS_MODULE environment variable.

You chould do this in a BASH script as follows:
    export DJANGO_SETTINGS_MODULE=mysite.settings
    python /path/to/mysite/cron.py
"""
from django.conf import settings

def my_import(name):
    '''
    __import__ helper function to import modules inside packages
    e.g.:  where name is something like 'package.module.mod_i_want',
           would return mod_i_want
    
    See: http://www.python.org/doc/2.5.2/lib/built-in-funcs.html
    '''
    mod = __import__(name)
    components = name.split('.')
    for comp in components[1:]:
        mod = getattr(mod, comp)
    return mod

def run():
    for app in settings.INSTALLED_APPS:
        if not app.startswith('django'):
            output_info = '%s.cron'%app
            ## Dynamically import a module called 'cron'
            ## from each INSTALLED_APP (if it exists)
            try:
                cron_mod = my_import(app+'.cron')
                output_info += '... FOUND'
                print output_info
                ## 3. Execute the cron's run method (if it exists)
                if hasattr(cron_mod, 'run'):
                    #print '---> calling run()'
                    cron_mod.run()
            except ImportError:
                # ignore packages that don't have a cron module
                output_info += '... SKIPPED'
                print output_info

if __name__ == "__main__":
    run()

The final piece of this puzzle lies in the BASH script used to invoke the above python module. It makes sure the appropriate environment variables are set and then it invokes the above module. I also store this in my django project directory (as cron.sh), and I use cron to schedule it to run.

#!/bin/bash
# This is a Django, Project-specific Cron script.
# Separate Projects would need a copy of this script 
# with appropriate Settings export statments.

PYTHONPATH="${PYTHONPATH}:/path/to/django/project/directory"
export PYTHONPATH
export DJANGO_SETTINGS_MODULE=mysite.settings

python/path/to/django/project/directory/mysite/cron.py

Using cron, you'd schudule this to run every morning at 6am by editing your crontab and adding the following:

#m h  dom mon dow   command
  0   6    *        *      *     /path/to/django/project/directory/cron.sh

That's it. This has been working for me, but there is at least one major pitfall: All of your app's cron tasks get run at the same time. This works well if your apps need to do something once a day (which has been my requirement), but this probably won't work well if you have some apps that need to run tasks at differing times.

There are also other solutions to this (none of which I've tried). There's a snippet at http://www.djangosnippets.org/snippets/1126/ which looks interesting. Then there's the django-cron app which seems to be fairly flexible in how it works, and it doesn't require cron (so this is a plus if you cant set a crontab or if you're on Windows!)

Thanks in advance for any feedback... suggestions are always welcome!

A Custom form for Django's Automatic Admin.

2009-01-14T16:08:00.003-06:00

A huge selling-point for Django (at least for developers) is its Automatic Admin. However, the ease at which the Admin can be set up, might make one second-guess an attempt to customize what is provided by default. Of course, the default admin site may not be without its drawbacks...

Many of the django Apps that I have built, tap into Django's User Authentication System. Simply put, when I build a model, it has a Foreign Key to django's User Class.

Here's an example Model:

from django.contrib.auth.models import User
from django.db import models

class Book(models.Model):
    author = models.ForeignKey(User)
    title = models.CharField()

The problem here is that when I create or edit a Book object using the Automatic Admin, the author field is represented by a select element, whose options contain ALL User objects... listed by username! Wouldn't it be nice if we could have that listed as "firstname lastname" or even as "lastname, firstname"? You can! And here's how:

First of all, Django's admin makes extensive use of ModelForms, and fields with a Foreign Key relationship are represented by a ModelChoiceField. So, all we need to do is extend the ModelChoiceField so that we have something that can be used on any Form that represents a Model with a Foreign Key to a User object. The label_from_instance method accepts an object (in this case, a User object), and returns a string that will be used between <option> elements. In the example below, I've chosen to format that as "firstname lastname (username)".

from django.forms import ModelChoiceField

class UserModelChoiceField(ModelChoiceField):
    def label_from_instance(self, obj):
        # Return a string of the format: "firstname lastname (username)"
        return "%s (%s)"%(obj.get_full_name(), obj.username)

Now, create a ModelForm for your Model, which specifies the new Field to be used for the author. Note that we need to pass it a queryset of Users. Below, I've named this ModelForm, BookAdminForm since I'm only going to use this form for the admin pages.

from django.forms import ModelForm
from django.contrib.auth.models import User

class BookAdminForm(ModelForm):
    author = UserModelChoiceField(User.objects.all().order_by('first_name'))
    class Meta:
        model = Book

Now we set up the ModelAmin for the Book Model. In it, we can specify the form that is used by Django's automatic admin (Note that this MUST be a ModelForm!). Your admin would look something similar to the following:

from django.contrib import admin
from forms import BookAdminForm
from models import Book

class BookAdmin(admin.ModelAdmin):
    form = BookAdminForm

admin.site.register(Book, BookAdmin)

Now, when you use the Automatic admin to add or edit existing Book entries, the drop-down list of Author names will be a bit more user-friendly.

Add a Context Processor for your Django app using Sites

2009-01-05T19:29:00.004-06:00

I've recently refactored a significant number of my Django Apps so that they include the "sites" framework. Essentially, this allows me to use the same code (and database) for multiple sites.

For Example, if I was was building a CMS (and I am!), I might have a model that defines a "page":

from django.db import models
from django.contrib.auth.models import User
from django.contrib.sites.models import Site

class Page(models.Model):
    title = models.CharField('title', max_length=255)
    content = models.TextField()
    author = models.ForeignKey(User)
    sites = models.ManyToManyField(Site, 
                  help_text="This page will be displayed on the selected Sites")

Note that the Page class has a ManyToManyField relation ship to a Django Site, which allows a page to be associated with one or more Sites. The neat thing about this, is that when I publish content on a Page, it can be published to one or more sites!

While working with this, I discovered I often wanted to write template code that knew which which Site it was associated. For example, I might have a base template containing a block inside html head elements so templates that inherit it can include external javascript or CSS files. For example, base.html might include the following:

<html>
<head>
<title>
    {% block title %}{% endblock %}
</title>
    {% block head %} {% endblock %}
</head>

Then, in any template that inherits from base.html, I could do something like the following:

{% extends "base.html" %}
{% block head %}
    {% ifequal current_site.domain "www.whatever.com" %}
        <link rel="stylesheet" type="text/css" href="/media/whatever.css" />
    {% endifequal %}
{% endblock %}

There trick here, though, is "How is my template going to know what site is being requested?"

I could put something like the following in every view I write...

current_site = Site.objects.get_current()

BUT, that's a lot of extra stuff to type. Especially if you have a lot of views.

The clever thing to do, would be to write code so that a Site object containing the currently requested site is automatically added to the current Context. We can do that by writing our own Context Processor!

And that's just what I did! The following code is fairly simple. It just retrieves the current Site from given request object (using Site.objects.get_current()), then returns a dictionary mapping the current site to the variable name current_site.

from django.conf import settings
from django.contrib.sites.models import Site

def current_site(request):
'''
A context processor to add the "current site" to the current Context
'''
    try:
        current_site = Site.objects.get_current()
        return {
            'current_site': current_site,
        }
    except Site.DoesNotExist:
        # always return a dict, no matter what!
        return {'current_site':''} # an empty string

For this to work, we've got to add the function above to the list of TEMPLATE_CONTEXT_PROCESSORS in our project settings file. My project directory is called "mysite", so I created a folder called "context_processors", and in it, I created a file called "current_site.py". That file contains the function current_site defined above.

To get my new context processor working, I've got to edit mysite/settings.py, which now looks something like the following:

TEMPLATE_CONTEXT_PROCESSORS = (
    "mysite.context_processors.current_site.current_site",
    "django.core.context_processors.auth", 
    "django.core.context_processors.debug",
    "django.core.context_processors.i18n",
    "django.core.context_processors.media", 
)

Voila! Now, while writing template code, I can always access the current_site variable!

Morphology is Fun!

2009-01-04T14:27:00.004-06:00

Morphological operations are very common in image processing. The two most basic of these are Erosion and Dilations, and from these, additional morphological operations can be performed. While there is an abundance of literature on the topic of morphology, I often like to view concrete examples of the operations.

So, I've published a few images that illustrate the output of 1-to-5 iterations of various morphological operations--erosion, dilation, opening, closing, gradient, top hat, and black hat--with various structuring elements--rectangle, cross, and ellipse.

See it at: bradmontgomery.net/morphology/

The code to reproduce these images was written using OpenCV, and is freely available for academic purposes (that means its not "great" code, nor is it production-ready!).

Enjoy!

How to convert HTML to PDF using Python.

2008-12-17T19:37:00.007-06:00

I'm building web-based, data-driven apps using Django. Eventually (or unfortunately), I will need to generate some reports that are printer-friendly. Logically, PDF is the format for such files... so how am I going to convert my xHTML and CSS to a nice-looking PDF document?

The Django Book has a whole chapter dedicated to Generating Non-HTML Content. They seem to to be fond of ReportLab ToolKit. The caveat here, though, is that you need to know a bit about the internals of a PDF document. If you're familiar with this, the ReportLab toolkit seems to be the way to go! It has many features, and it seems to be a powerful PDF-generating tool.

Unfortunately, I know nothing about PDF internals, but I do know quite a bit about HTML and CSS. That's why xhtml2pdf.com caught my attention. If it delivers on it's promises, it parses HTML and CSS and generates PDFs (imagine that)! There's also a handy Activestate recipe using it: Recipe 572160: HTML/CSS to PDF converter.

I'm definately going to check this (HTML2PDF.org) out... so expect an update on this!

Any other suggestions?

UPDATE: xhtml2pdf works well. There's also a great
post by Greg Newman outlining how it's used in django.

radiohead - 15 steps video

2008-12-04T19:11:00.004-06:00

Updated... BIZRADv2... is a bit better.

That Looks about Right

2008-12-04T16:27:00.001-06:00


 history | awk '{ print $2 }' | sort | uniq -c | sort -rn | head
 152 cd
 145 ll
  73 vi
  20 rm
  18 svn
  13 exit
   9 ls
   8 grep
   7 php
   5 wget

via b-list.org

A simple Django example with Ajax

2008-11-24T15:11:00.008-06:00

I often employ Ajax in HTML forms in order to update the list of options in select elements. For example, suppose a form consists of two select elements, and the options in the second depends on the values selected in the first. A simple example of this might be an Automobile Rental website that lets you choose the type of vehicle as well as the color. Not all vehicles come in the same color, though, so you might have a form that looks similar to the following:

<select name="auto" id="auto" onchange="get_vehicle_colors();">
<option value="">-- select a vehicle type --</option>
<option value="car">Car</option>
<option value="truck">Truck</option>
<option value="motorcycle">Motorcycle</option>
</select>

<select name="color" id="color">
<option value="">-- choose a vehicle first--</option>
</select>

In this example, you would choose the type of automobile you wanted, then employ Ajax to set the appropriate color values for the color element.

A Django app that provides this sort of functionality, might have a Model resembling the following (omitting various methods and Meta classes):

class Color(models.Model):
    color = models.CharField(max_length=256)

class Auto(models.Model):
    type = models.CharField('auto type', max_length=256)
    colors = models.ManyToManyField(Color)

Likewise, a form (similar to the one above) could be built with:

from django import forms
from models import Color, Auto

class AutoForm(forms.Form):
    TYPE_CHOICES = [('', '-- choose a type --'), ] + [(t.type, t.type) for t in Auto.objects.all()]
    COLOR_CHOICES = [(c.color, c.color) for c in Color.objects.all()]
    COLOR_CHOICES.insert(0, ('', '-- choose a vehicle type first --'))
    
    type = forms.ChoiceField(choices=TYPE_CHOICES, widget=forms.Select(attrs={'onchange':'get_vehicle_color();'}))
    color = forms.ChoiceField(choices=COLOR_CHOICES)

Notice the use of the widget paramter on the Form's type field. Django Forms only render the most basic HTML, so in order to set an onchange attribute for a select element, we have to specify that attribute in attrs, which is a dictionary of element attribute name/value pairs. More information about customizing Form widgets can be found in Django's widget reference.

Another thing to note is the COLOR_CHOICES attribute. A ChoiceField will validate that any user-submitted content is conteint in its provided choices. So, the COLOR_CHOICES must contain all valid colors for an Auto. However, we make sure the first choice is a default value that will later get updated by our AJAX request.

For the most part, this form would be used as in any other Django app, but since we're adding in a little Ajax, I include my javascript libraries (in this example, Prototype) in the same template as the form. So, my template code looks something like this:

{% extends "base.html" %}
{% block head %}
<script type="text/javascript" src="/site_media/prototype.js"></script>
<script type="text/javascript" src="/site_media/my_ajax_function.js"></script>
{% endblock %}

{% block content %}
    {% if form_was_valid %}
        {# ... show whatever... #}
    {% else %}
        <form action="/auto/reserve/" method="POST">
        <ul>
        {{ form.as_ul}}
        <li><label for="submit">&nbsp;</label>
        <input type="submit" id="submit" name="submit" value="Submit"/></li>
        </ul>
        </form>
    {% endif %}
{% endblock %}

There are two items of note, here. The first is that this template builds on top of a base.html template which contains my sites layout and definitions for blocks. The second, is that one of those blocks--head--is inside my page's head element so that I can reference arbitrary javascript files (or CSS if I needed to) in only the templates that need them. It may be a minor note, but including your javascript libraries ONLY when you need them might save you some load-time and bandwidth.

Now, what does the stuff in the my_ajax_function.js look like? When a Django form gets rendered, every form element automatically gets an id attribute whose value is the name of the field, prefixed by "id_". So, our type and color Select widgets would have attributes id="id_type" and id="id_color", respectively.

So what is it that our Javascript needs to do?

When an auto type is selected (determined by the onchange event, grab the value of that type ($('id_type').getValue())
Construct the XMLHttpRequest with the appropriate POST data (the type of Auto chosen--accomplished using the $H shortcut to create a Hash)
Send that back to the webserver (at the appropriate URL, which we've set as /auto/ajax_purpose_staff/)
Listen for a response from the server,
And if that response contains any text (hopefully a list of available colors), update the select element with that text

function get_vehicle_color(){
    new Ajax.Request('/auto/ajax_purpose_staff/', { 
    method: 'post',
    parameters: $H({'type':$('id_type').getValue()}),
    onSuccess: function(transport) {
        var e = $('id_color')
        if(transport.responseText)
            e.update(transport.responseText)
    }
    }); // end new Ajax.Request
}

So now we've got a model and a form (outfitted with some nifty Ajax code, no less), how would we set up a view and a URLconf? Well, the URLconf works the same as in any other app, so we just have to set an entry that maps to the view that handles the Ajax request. If the name of this app is auto, and it lives in a project called mysite, our URLconf might look like the following:

urlpatterns = patterns('mysite.auto.views',
    (r'^ajax_color_request/$', 'ajax_color_request'),
    # ... everything else...
)

And it would map our URL (www.example.com/auto/ajax_color_request/) to a view named ajax_color_request.

Now for the view. Since our Ajax request is sending its data via post, we can pull it from request.POST (which is a dictionary-like object), and then retrieve all the colors associated with a particular type of Auto.

def ajax_color_request(request):
    # Expect an auto 'type' to be passed in via Ajax and POST
    if request.is_ajax() and request.method == 'POST
        auto_type = Auto.objects.filter(type=request.POST.get('type', ''))
        colors = auto_type.colors.all() # get all the colors for this type of auto.
    return render_to_response('auto/ajax_color_request.html', locals())

Now, all we have to do is send that data back to the client as a snippet of HTML which will get written to the appropriate select element. I've chosen to do this in a very simple template:

{% for c in colors %}
    <option value="{{ c.color }}">{{ c.color|title }}</option>
{% endfor %}

And there you have it. A simple bit of Ajax in a Django app.

Extending Django's MultiWidget: SplitSelectDateTimeWidget

2008-11-19T12:49:00.005-06:00

This entry is an update to SelectTimeWidget: A custom Django Widget.

The Problem: I want to use a Single widget object for a DateTimeField, but I want it to consist of select elements with appropriate options for month, day, year, hour, minute, and second. Additionally, I want to be able to specify a 12-hour format, so I would then need options for "a.m." and "p.m."

Fortunately, Django's SelectDateWidget (from django.forms.extras.widgets) takes care of the Date portion of this, and I've previously written a similar SelectTimeWidget. Now, I just need to find some way to appropriately combine the two widgets.

After a little digging in Django's source code, I found something called a MultiWidget (in django.forms.widgets). From it's docstring:

A widget that is composed of multiple widgets.

Wow! This sounds like JUST what I need! Luckily, just beneath it is the definition of a SplitDateTimeWidget(MultiWidget), which combines two TextInput widgets for DateTimeFields. So taking that as an example, I've written the SplitSelectDateTimeWidget.

To Use the SplitSelectDateTimeWidget you might do something similar to this:

# Default usage of SplitSelectDateTimeWidget
class TimeForm(Form):
    dt = DateTimeField(widget=SplitSelectDateTimeWidget())

A slightly more complex example hooks into the flexibility of the underlying widgets (SelectDateWidget and SelecTimeWidget):

class TimeForm(Form)
    dt = DateTimeField(widget=SplitSelectDateTimeWidget(hour_step=2, \
    minute_step=15, second_step=30, twelve_hr=True, years=[2008,2009,2010]))

The above example displays hours in increments of 2, minutes in increments of 15, and seconds in increments of 30. Likewise, only the years 2008, 2009,and 2010 are displayed in the years' options.

The output of a form using the SplitSelectDateTimeWidget looks something similar to this:

SelectTimeWidget: A custom Django Widget

2008-11-18T19:40:00.008-06:00

I've been meaning to write this blog post for quite some time...

Django Models provide a way to create a definitive source of data for web applications. Written as a python class, a Django Model consists of Fields that (among other things) define a type for your data.

Django's Forms provide a mechanism for generating HTML form elements and validating user input. A sublcass of Django's Form class is the ModelForm which essentially creates a Form based on the Fields in a Model. Form Fields and Model Fields are not the same--though there is a mapping from Model fields to Form fields, however. The Fields in a Form define the validation logic for the data entered into the html form elements generated by the Field.

When Django renders a Form, it chooses an appropriate html form element (such as input, textarea, or select elements) based on the type of Fields in the Form. For example a form to save a URL with an associated title might look something like the following:

from django import forms

class LinkForm(forms.Form):
    title = forms.CharField() 
    url = forms.URLField()

The HTML generated by calling this forms as_table() method might look something like this:

>>> f = LinkForm()
>>> f.as_table()
<tr><th>Title:</th><td><input type="text" name="title" /></td></tr>
<tr><th>Url:</th><td><input type="text" name="url"/></td></tr>

We can change the behaviour of this output HTML by specifying a specific widget for the Form. If for some strange reason we wanted the Title to render in a textarea, we would do the following:

title = forms.CharField(widget=forms.Textarea)

All of this is well-documented in Django's Widget documentation, which has been working well for me 80% of the time.

My big complaint here is that Date and Time information defaults to text input elements. In the past, I've always liked to use select elements for situations when users would need to enter things like dates (perhaps a different select element for Months, Days, and Years) and time (different select elements for Hours, Minutes...

Luckily, (as of version 1.0, I think), Django offers a SelectDateWidget (located in django.forms.extras.widgets) which accomplishes this very thing. It produces three select elements, and it can be attached to a Form's DateField. This is accomplished like so:

some_date = forms.DateField(widget=SelectDateWidget, initial=datetime.date.today())

That's Awesome! I can even tell the widget to default to "today's" date by passing it an initial parameter containing the value generated by datetime.date.today(). Ok... what about times? Would we ever want to enter Time information as three separate select elements. I do, so I basically altered the SelectDateWidget, creating a SelectTimeWidget, which is available on django snippets. It isn't perfect, but for now, it has accomplished what I've wanted... and its fairly flexible.

# Specify a basic 24-hr time Widget (the default)
t = forms.TimeField(widget=SelectTimeWidget())

# Force minutes and seconds to be displayed in increments of 10
t = forms.TimeField(widget=SelectTimeWidget(minute_step=10, second_step=10))

# Use a 12-hr time format, which will display a 4th select 
# element containing a.m. and p.m. options)
t = forms.TimeField(widget=SelectTimeWidget(twelve_hr=True))

However, I often use DateTimeFields in both Models and Forms (which consists of date and time info). All the code I need to represent select elements for such a field exists in both SelectDateWidget and SelectTimeWidgets. The problem? I just have figured out a good (read: proper) way to combine these two Widget classes.

Do I use multiple inheritance here... mixins... ? My experience doing this in python is a bit rusty... (erm, non-existent!) Hopefully the helpful django community will help me out. :)

UPDATE: This discussion is continued in Extending Django's MultiWidget: SplitSelectDateTimeWidget. I found a solution to this problem simply by digging through Django's source code. Huzzah for Open Source!

So if you had a project called mysite and an app called myapp, putting it all together would look something like this:

# in urls.py
urlpatterns += patterns('',
    (r'^test/$', 'mysite.myapp.views.test'),
)

# in forms.py
class TestForm(forms.Form):
    t = forms.TimeField(widget=SelectTimeWidget(minute_step=15, second_step=30, twelve_hr=True))


# in views.py
def test(request):
    from forms import TestForm
    
    if request.method == 'POST':
        form = TestForm(request.POST)
        if form.is_valid():
            info = form.cleaned_data
    else:
        form = TestForm()
        
    return render_to_response('myapp/test.html',
                               locals(),
                               context_instance=RequestContext(request))

And the test.html Template:

{% extends "base.html" %}

{% block content %}
    {% if info %}
        <pre>{{ info }}</pre>
    {% else %}

        <form action="" method="post">
        {{ form.as_p }}
        <p><input type="submit" value="go"/></p>
        </form>
    {% endif %}
{% endblock %}

Soup's On! And it IS Beautiful!

2008-11-17T12:41:00.004-06:00

Here's the problem: There's a BAJILLIION static html pages sitting out on a server, and I need to migrate all that content to a new Database-driven CMS. Additionally, I need to get rid of a lot of non-essential hard-coded presentational markup (like align="center" or font="whatever") and any inline styles that may exist... (you know, because external CSS is the way to go).

I could spend hours and hours just copy-/pasting stuff... but meh. Enter BeautifulSoup, urllib, and sqlite... all glued together with python.

Now, for the most part, the content I want to scrape from these static html pages aIso contains links to the other pages I want to keep. So, here's what we can do, use urlllib to grab the content from the web server, use BeautifulSoup to extract all the values from href attributes in our "a" elements, and keep a list of these (checking to make sure they point to sites we want to keep).

Then, I use BeautifulSoup to parse the page, keeping the bits that I want to keep. Lucky for me, the pages I'm particularly interested in have handy "InstanceBeginEditable" and "InstanceEndEditable" in comments (thanks Dreamweaver), so I just strip out everything before and after that, and parse what's left over. Of course, this sort of thing is going to be different for everybody, but luckily, parsing XHTML isn't that difficulty thanks to BeautifulSoup's mostly-comprehensive documentation.

So, the part you've been waiting for... Download the Code, try it out, and leave me some feedback!
http://bradmontgomery.net/files/migrate_web.zip

migrate_web.py by Brad Montgomery is licensed under a Creative Commons Attribution 3.0 United States License.
Based on a work at bradmontgomery.net.

Wordle - Cool Word Clouds

2008-11-12T10:17:00.005-06:00

I've just run across Wordle, a nifty app that generates artistic word clouds based on text you give it. It also consumes rss or atom feeds, so you can do something like the image below:

It's quite interesting what sort of things stand out in this... I just happened to notice that everyone want great web!

BarCampMemphis, the Day After

2008-11-09T18:53:00.007-06:00

I think everyone is in agreement: BarCampMemphis was a tremendous success!

I want to personally thank everyone who came to my Zero-to-Wiki in 30 minutes with Django session, and I also want to remind everyone that the Presentation and the code is freely available!

There were many great sessions available throughout the day, but I want to list the ones I got to attend... all of which were just excellent:

Dave Barger's LinkedIn tips

Will Murphy's insights on what's changing on the web

Scott Brown's Ruby on Rails demo

The great thoughts on Game Design offered by Mike Rowe, Scott Brown, and Kyle Dempsey

New insights on Etiquette in Social media moderated by Spencer Dillard

And thanks to Tollie Williams for a great list of iPhone Apps...

Relive the action as it happened, thanks to Zannel: http://www.zannel.com/bkmontgomery

A topic for BarCampMemphis

2008-10-22T16:41:00.005-05:00

I've recently watched the What is BarCamp video, and I've been thinking about a Django-related topic. I'm sure there are plenty of web developers out there looking for a better way...

Now, I've only been using Django for about 5 months, but I'm pretty sure I could give any interested kindred souls a good jump-start. And so that's what I'd like to do... show by example.

The Example? Why not a wiki... It's simple. (blog examples have been overdone)

So there it is... Anyone interested in 0-to-wiki in 30 minutes?

on beards

2008-10-05T13:22:00.004-05:00

I just ran across an interesting beard video on vimeo. Lots of cool links including beards.org, the World Beard and Moustache Championships, and the BLF.

Fun stuff.

hack your iphone earbuds...

2008-09-23T19:31:00.003-05:00

I've been complaining (inwardly) about my iPhone's earbuds since I first tried them on. Who knew it was so easy to just hack them!

iPhone / iPod Touch Headphone / Earbud Replacement

iTunes Genius Lies!

2008-09-16T13:48:00.004-05:00

I begrudgingly turned on the Genius Sidebar in iTunes 8 and now it's showing me the "Top Songs You're Missing"... except, I'm not missing them! Surely Apple doesn't think I should purchase music that's already in my library!

Maybe this is just a bug... or maybe iTunes is confuzed because my album is "Weezer (Blue Album)"... who knows.

Pondering a Robotic Lawn Mower

2008-09-13T10:23:00.004-05:00

Last weekend my gasoline powered lawn mower died. This prompted a search for a new mower, during which I stumbled upon some of Friendly Robotics' Robomow; a line of battery-powered robotic lawn mowers. Now, I have to admit that for the past 15 years or so, every time I've mowed a lawn, I've thought about the possibilities of a robotic lawn mower, and I'm not at all surprised that there's a company producing these. At first the Robomow seems like a really nice product, but it does a have a couple of major drawbacks that would prevent me from purchasing one:

Cost: it's at least $2500. I just can't afford that
It requires a low-voltage perimeter wire.
It takes a huge, heavy-looking battery.

The first problem is a big one for me (i'm broke), though it's a completely uninteresting problem.

The second problem is the one that really irks me though. The Robomow requires that you install a low-voltage perimeter wire around your yard. Now... if you have a rectangular yard with nothing it it, this may not be that big a deal, but if you have trees or any other obstacles (flowers, gazebos, tool sheds, playground equipment, etc), laying that wire is going to be a lot of work. It just seems like there's enough technology out there that would allow a robotic lawn mower to navigate a yard in a much smarter fashion. If I were to build a robotic mower it just seems that there are a number of options that would allow it to navigate a lawn without the perimeter wire. Now I'm just thinking out loud here, and putting this all into one machine may be cost-prohibitive, but if I were to build my own robotic lawn mower, these are the things I would consider:

GPS: your lawn mower is probably outside. Couldn't a robot "know" its position and stay inside some predefined boundary?
Touch Sensors: why not just let the robot (gently) bump into stuff. Using touch sensors it might be possible to just back up and mow around stuff. This may be especially useful for fenced-in yards.
Light or IR sensors: if for some reason boundary problems could not be solved using GPS, couldn't lasers or IR LEDs be deployed (I'm thinking stakes with battery-powered LEDS on two sides at each corners of the yard) yard)

There's got to be a compination of these that would allow a robot to navigate obstacles without using a perimeter wire.

I don't have a solution for the third problem. I like the idea of a quiet electric motor, but batteries are heavy, and (in my opinion) just aren't ideal for a piece of lawn equipment. For now, a small, efficient gasoline engine (small like those in a hand-held weed-eater... I'm not sure how efficient those are, though) might be the best way to go. Solar power may also be a possbility.

I'm also thinking a reel mower might be a better way to actually cut the grass. Could a small motor attached to each wheel provide enough control/power/propulsion to make this work?

(I just bought a Craftsman Reel mower, so I'm curious to see how well it works...)

I guess there's also a number of other significant questions that may need to be addressed when considering robotic lawn care. How do we take care of sticks and leaves? I have lots of trees, so these a big problem for me.

I'm beginning to think we're a number of years away from completely robotic lawn care...

This is a completely unorganized, incompletely thought-out post, but I just wanted to get some ideas out in the open...

Recursively deleting files (based on regex)

2008-09-09T06:20:00.002-05:00

While we're on the topic... (the topic being recursively doing stuff to files), I often want to delete all the files in a hierachy of directories that match a regular expression (or regex).

The typical scenario for this is, "I want to delete all of my compiled python files."

Here's how I do it:

find ./ -type f -name "*.pyc" -exec rm {} \;

find ./ -type f -name "*.pyc" should find all files ending in .pyc under the current directory. Once they've been found, the -exec rm {} \; will remove them.

Recursively Renaming files

2008-09-05T19:10:00.002-05:00

My hosting provider offers PHP4 and PHP5. Unfortunately, all files ending in .php get interpreted by PHP4, while all files ending in .php5 get interpreted by PHP5. So, how do I quickly change all of my files that end in .php to .php5?

Do a google search for "recursively rename files", and you might run across this:

http://seal-7.blogspot.com/2006/12/recursively-rename-files-with-regex-one.html

So, to accomplish my task, I use the following:

find . -type f -print0 | xargs -0 rename 's/.php$/.php5/g'

Awesomeness.

Looking Forward to BarCampMemphis

2008-09-05T09:05:00.003-05:00

BarCamp is coming to Memphis!

A project of LauchMemphis, this is yet another event focused on growing the local tech industry. It follows the successful Memphis Startup Weekend (which resulted in Spynnr).

The theme of BarCampMemphis is FLUX: Your Paradigm, and from the About BarCampMemphis page:

In a constantly shifting environment, what do you do? Entreprenuers and businesses are constantly trying to walk the line between chaos and control, between the cutting edge and solid ground. Things are in FLUX. Now enter the world of technology; tried and true constants shift quickly and the market gives businesses and entreprenuers a choice: change or fail. BarCamp Memphis is designed to explore the awesome questions that affect you. Maybe you will find solid ground amidst shifting paradigms!

If you're unfamiliar with the concept of a BarCamp, check out BarCamp.org and Wikipedia's BarCamp page, which starts with:

BarCamp is an international network of user generated conferences — open, participatory workshop-events, whose content is provided by participants — often focusing on early-stage web applications, and related open source technologies, social protocols, and open data formats.

BarCampMemphis looks to be exciting and informative!

Safari is Cooler than you Think!

2008-09-04T15:12:00.007-05:00

When you really start digging into Mac OS X, it's fairly mind-boggling how much extra stuff it has that your average user never sees. I recently encountered a problem on my MacBook Pro, where the Optical Audio was overriding my internal speakers, which prevented me from hearing any audio (without using headphones). Unfortunately, there's no easy-to-access preference pane to enable or disable various audio devices. This led me on a search for command-line utilities to manage system preferences.

(hang on, I'll get to Safari in a second...)

Eventually I ran into Amit Singh's list of OS X Hacking tools, where I stumbled upon the defaults command. defaults is the magic "change any setting for Mac OS X on the command line" tool that you (by you, I mean me) never knew about!

So, what does this have to do with Safari? Well, after looking for more information on how to use defaults, I stumbled upon Mac OS X Tips' Top 15 Terminal Commands for Hidden Mac OS X Settings, one of which really caught my attention:

defaults write com.apple.safari IncludeDebugMenu 1

This enables a "Debug" Menu ("Develop" menu on Safari 3). The Developer menu allows you to:

Change Safari's user-agent, so you can masquerade as IE, Firefox, Opera, earlier versions of Safari, or even the iPhone version of Safari
View a Web Inspector - allows you to view scripts, css files, and other information associated with a Web page...
View a network timeline - showing the time it took to download each component linked in a web-page...
Disable stuff - like javascript, css, java, caches...

It's not quite the Web Developer plugin for Firefox, but it does give you additional control over Safari.

And the Network timeline looks pretty darn cool!

ps: i never did find a way to disable the optical audio...guess i got side-tracked.

[Update: 10/05/2008] I just realized there's an easier way to enable this in the Advanced Preferences Tab... just enable the "Show Develop menu in menu bar" option! (as in the screenshot below)

I guess it pays to pay attention to your app's options!

Net Neutrality

2008-08-13T08:30:00.002-05:00

Save the Internet | Rock the Vote

How to update an input value with the value from a selected option using Prototype

2008-08-12T10:32:00.004-05:00

Today, I needed to set the value of an HTML input element based on the value of a option in a select element. This is fairly easy to do with Prototype's writeAttribute. Here's an example:

A simple javascript function to do the work:

function populate_input(){
    var field = $('tf_select').getValue(); 
    $('tf').writeAttribute('value', field);
}

A simple HTML snippet to see it in action:

<div>
<p><select id="tf_select" name="tf_select" onchange="populate_input();">
<option value="">- choose one -</option>
<option value="v1">value 1</option>
<option value="v2">value 2</option>
</select><br/><input type="test" id="tf" name="tf" value=""/></p>
</div>

And yes... I know the title of this post is almost longer than the post itself!

Extracting and Encoding Still images to/from Video files

2008-08-11T09:05:00.007-05:00

ffmpeg is a cool tool.

While it's got many many features and uses, I mostly use it to extract frames (or still images) from a video and create a video out of frames. While instructions on how to do this are posted on their list of FAQs, it's probably worth posting again.

To extract all frames from a video file named "video.mpg":

ffmpeg -i video.mpg frame%d.jpg

This will produce a series of JPEG image files named file1.jpg, file2.jpg... file10.jpg, file11.jpg... etc.

I often like to have a zero prepended to the number associated with a frame. For exaple, if I knew there were 500 frames in the entire video, I would like to have the first frame named "file001.jpg" instead of "file1.jpg". This can be achieved by specifying the number of digits to use in the output file name (such as 3 in the example below):

ffmpeg -i video.mpg frame%3d.jpg

Now, to create a video named "movie.mpg" from a series of images named something like "frameX.jpg" (where X is the frame number), you could do something like this:

ffmpeg -f image2 -i frame%d.jpg movie.mpg

The caveat here is that the input frames MUST be numbered as if they were extracted from the video. So, if you had zero-padded filenames (as in the case where we used "frame%3d.jpg" to extract images), you'd have to account for that when encoding those images back into a movie.

For example:

ffmpeg -f image2 -i frame%3d.jpg movie.mpg

A Note on image formats: In the example above where we created a video file from a set of still images, the "-f image2" option to ffmpeg specifies a video format. Available formats can be displayed using the command

ffmpeg -formats

Note that this is not the same as a container format (such as .mpg, .avi, .mov... etc). If this sounds a little confusing, you may want to read up on digital video encoding.

The Apple Dock is a Mirror!

2008-08-08T21:39:00.003-05:00

I guess I'm still a relatively new Mac User, but I consider myself fairly technical and probably more savvy that your typical Mac user (though there are a lot of mac users who are far more savvy than I!)

Imagine my surprise when, while working tonight, I discovered that the Dock in Leopard is a Mirror! Yes, I'd noticed the reflection of the Icons, but tonight, I actually saw my cursor (in Terminal) moving as I typed... that caught my attention! At first I though, "well that's just amazing!" I had to wonder just how much work went into making that happen. Then, I discovered that some people don't like this!

Oh well. To each his own...

PHP is chopping off my Access Memo Fields

2008-07-30T15:29:00.003-05:00

I've got a few simple web forms that use PHP to read and write to an Access database (running on IIS), and I just spent the last few hours frantically trying to figure out why some of my Memo fields were being truncated around 4000 characters.

I know Memo fields "should" be able to contain up to 65536 characters, and I could verify this by inserting data directly into the database. However, when querying the database through PHP (using odbc), I could only retrieve 4000 characters.

The culprit? A tiny little setting called odbc.defaultlrl! (lrl = long read length). This can be changed in php.ini, or you can use ini_set to modify this setting directly in your php script:

<?php
ini_set('odbc.defaultlrl', 65536);
?>

I found this info on the discussion for the odbc_longreadlen function.

Django: Generating an Image with PIL

2008-07-16T10:37:00.005-05:00

I've been reading through the Django Book, and in chapter 11 they talk about generating non-HTML content (such as PDF files, Images, RSS/Atom Feeds). They mention using PIL to generate images, but they don't give an example. So, I thought I'd post a simple example View that generates an image.

def pil_image(request):
    ''' A View that Returns a PNG Image generated using PIL'''

    import Image, ImageDraw 

    size = (100,50)             # size of the image to create
    im = Image.new('RGB', size) # create the image
    draw = ImageDraw.Draw(im)   # create a drawing object that is
                                # used to draw on the new image
    red = (255,0,0)    # color of our text
    text_pos = (10,10) # top-left position of our text
    text = "Hello World!" # text to draw
    # Now, we'll do the drawing: 
    draw.text(text_pos, text, fill=red)
    
    del draw # I'm done drawing so I don't need this anymore
    
    # We need an HttpResponse object with the correct mimetype
    response = HttpResponse(mimetype="image/png")
    # now, we tell the image to save as a PNG to the 
    # provided file-like object
    im.save(response, 'PNG')

    return response # and we're done!

This example just draws simple text, but the drawing code could be replaced by something more elaborate such as code that opens and scales existing images.

A note of concern: The Image object (im) MUST be saved as a PNG for this to work. Luckily, the Image.save method expects a file-like object as its first parameter, so we can use django's HttpResponse object here. Also, the example above doesn't make use of any particular Font for the drawing, so if you do want to draw text, you'll want to take a look at PIL's ImageDraw Documentation.

(on a side note: the html for the code above was generated using dpaste, which rocks!)

The Structure of a Django App

2008-07-15T10:57:00.006-05:00

Previously, I'd lamented the difficultly present in choosing an web development framework. I'd worked through several symfony tutorials, and though I could see the benefits down the road, it just didn't feel right to me (yes... "feel" is a technical drawback).

So, I checked out a copy of Django, and I haven't looked back. If you're the least bit proficient with python, and you need to build a database-driven web site, USE DJANGO! They have superb documentation, a free book (the The Django Book), and there's a built-in development server included so you don't have to get Apache and mod-python (or some other webserver) running somewhere before you can start writing code.

The fist thing that really made me like Django was the structure of it. First of all, a site is organized as a project. Multiple projects may be set up for different websites. Inside a project, you build an App. (Note that a project may have one or more Apps).

The App is where you do most of your work, it really consists of four parts:

Models
Views
The URLconf
Template(s)

Models (models.py) define your data. Instead of building tables in your database, you build python classes in your model. Django will give you the SQL to create your database tables. Now, what's really cool, is Django's Automatically generated Admin interface. All you really need is the model, and the Admin interface provides a password-protected, web-based, interface to your data.

Views (views.py) are essentially functions that extract the correct data based on some (or no) input. Some may think this is an over-simplification, but in Django, the View is the "view of your data". The functions in your View will pass data back to the Templates where they are displayed.

The URLconf (urls.py) is magic. Well, it sort of seems that way, but it just uses regular expressions to match urls and pass data to the View. So, "out of the box", Django supports pretty URLs so there's no need to do any url rewriting to get them.

Last (but not least) are the Templates. Django sports it's own template language, and apparently it also supports a host of pre-existing template engines. I personally have never used existing template engines, so this is all a bit new to me. However, the Django template language appears both concise and powerful.

Now, there's a bit more to it that what I've described here, but if you're wondering about Django (like I was), go ahead and check out their overview, skim the installaltion guide, and then start working the tutorial. I did, and I haven't looked back.

Why Bluetooth headsets didn't catch on in the '90s

2008-07-12T10:12:00.004-05:00

Games for Kids: dodger

2008-07-11T11:09:00.003-05:00

I've always thought it would be fun to build video games, and when my daughters were born, I knew I would eventually want to build some simple games that they could enjoy while they're young. My original attempt at this lead me to pygame, and the result was a simple alphabet/number game that plopped letters and numbers on the screen (very similar to AlphaBaby, but not nearly as good!). Unfortunately, I never continued to develop that one.

Since my oldest daughter (who is now 5) has recently discovered GCompris, I thought I'd give the game-writing another chance. This has resulted in dodger, which is also my first attempt at using pyglet. I know it can't compete with GCompris, but hopefully my daughter will like it.

I've released it under a CC license, so feel free to grab a copy, play it, use it, learn from it or whatever... Enjoy.

Mac OS X, Python, and Fink: Playing Nicely Together

2008-06-25T08:29:00.005-05:00

Since upgrading to Leopard, I've been using Mac's default install of Python (which is 2.5.1). For the most part it's worked well for me, namely because I've installed additionaly packages either using Mac installers or through easy_install. I recently needed to install python-ldap which didn't work using easy_install. So, I turned to Fink.

Unfortunately, Fink wanted to install it's own version of python (2.5.2) as well. (I say unfortunately only because I now how two separate python installations, which I think is a bit redundant.) So, the problem is this: How do I make the Mac's System python play nicely with any python modules I install through Fink?

My solution to this problem is to add the Fink python's site-packages directory to my PYTHONPATH. I do this by setting an environment variable in my .profile file located in my home directory. I added the following:


PYTHONPATH="${PYTHONPATH}:/sw/lib/python2.5/site-packages"
export PYTHONPATH

Now, when I run the python interpreter, I can import packages that were installed the Mac way, or through Fink. Hopefully, this won't give me any problems.

Lions, Tigers, and Web Development Frameworks, oh my!

2008-06-17T20:05:00.004-05:00

Apparently I've stumbled upon a problem that has recently faced many web developers. That is, I would like to adopt an open-source web development framework for mid-sized project. Well... searching that phrase only yields 200,000+ results. So how does one choose?

I guess Ruby on Rails sparked the whole "Web Development Framework" movement (among other things). I've typically used PHP for my web-based projects in the past, but over the last 2 years, I've also become a huge fan of Python. I'm not opposed to learning a whole new language (in fact, I'm quite for it!), but for this particular project, I'd like to stick with something I already know quite well. So, that limits my languages to PHP and Python.

Additionally, I would like that the framework support multiple databases on the back-end (especially SQLite!), and that there not be huge additional requirements to get things up and running. I would also like it to integrate nicely with Prototype, and, if it works on Mac OS X, that a huge PLUS!

So, for the PHP side of things, my top three candidates were:

I've already started working through symfony's excellent tutorials, and there are two main reasons that I chose symfony to start with:

An online book (and excellent Documentation)
An easy-to-use sandbox. This lets me get started quickly with it.

I've still got a soft spot in my heart for Python, and eventually, I'd like to let it into my Web Development world. I've read good things about both TurboGears and Django. Interestingly enough, Django comes bundled with its own (lightweight) web server (for testing purposes), so it's a great candidate for anyone wanting a framework with little overhead... It will probably be my overall number 2 choice.

Folding@Home

2008-05-24T07:59:00.004-05:00

I've recently started running Folding@Home. This project (out of Stanford) has been around for a number of years, and apparently they're still going strong! They simulate the way that proteins fold (or assemble themselves t) in order to investigate diseases that occur when the proteins don't fold correctly. Check out their link for more info.

Anyways, I've got these computers, so why not do something good with them while I'm not using them! Here are my Stats!

pyCropper

2008-05-20T10:01:00.003-05:00

Occasionally, I need to crop some images. I suspect this happens to a lot of people, and they simply use their favorite photo editor to perform the task. For a few images, this works just fine. Plenty of people have Photoshop, the Gimp, or even iPhoto.

This past weekend, however, I found myself needing to crop about 1000 images, and I need to do it quickly. This was more than my machine could load into the Gimp at once. Even loading 100 at a time was a bit slow for me. So I set out to find some simple tool that would let me JUST crop images. I found a few nifty web-based tools (picresize.com and resize.it) that would allow me to easily crop ONE image... but that's not really what I needed.

So, when I couldn't find what I needed, I wrote my own, and it's now available under the GPLv3.

If you find yourself needing to crop a large number of images, head on over to the pyCropper page. Enjoy!

A Go at California Rolls

2008-05-02T19:02:00.006-05:00

Well, I've tried to make some California rolls. I think they turned out OK. Not the prettiest rolls, but they tasted good... maybe I shouldn't have eaten them all, though!

These were made with the following ingredients:

1 cup of rice. (most recipes call for short grain rice, but I used long grain rice)
1Tbsp of Rice vinegar
dash of salt
Nori
Carrots
Cucumber
Imitation crab meat (or real crab meat if you have it)
1 Avacado

Boil 1 and 3/4 cups of water and salt. Once the water has come to a boil, add the rice, cover, and reduce to a simmer for 15 minutes. While simmering, julienne (or slice very thinly) carrots and cucumber. Chop crabmeat.

Once rice has finished, mix in the rice vinegar, and let cool for 10 minutes. Spread a thin layer of rice on the Nori, and top the rice with a small amount of carrots, cucumber, and crab, then roll tightly. Slice into thin pieces!

Enjoy with wasabi, soy sauce and pickled ginger!

Review: Beginning Game Development with Python and Pygame

2008-04-13T19:40:00.004-05:00

It's been out for a while now, and I'd always meant to write a review for this book... I pre-ordered it, and I've had it for a
while, but I haven't worked through the whole thing (though I've hit most of the latter chapters). Actually, this is not really a review, per se... just my succinct list of pros and cons.

So, without further ado:

Pros:

Very easy to read.
Great examples that actually work
Chapters 1 & 2 give a great intro to python, so this would actually be a good book if you've never touched python before (but did have some programming experience)
Lots of info using pygame+opengl
A lot of examples use his gameobjects library, so a lot of the grunt-work coding is available to use already.

Cons:

My biggest complaint is the lack of discussion on Sprites. Pygame is really a 2D library, and I think he left out a lot of very important information by not discussing how the Sprite class can be used.
Only cursory discussion Sound. If you're writing a game and just want the basic sound effects and/or background music, this is ok. However, if you want to do something really interesting with sound, you'll need to dig way beyond what this book offers.

Overall, I think the book is worth getting unless you've already done
a few significant projects using pygame. It's definately an Intro
book, and it does a really good job giving the user an idea how to put
a project together.

Disclaimer: I'm a relative noob to pygame (~10 months), and I've been
using pygame for research projects in Augmented Reality (not
necessarily games) for both visual and non-visual (sound-only)
applications.

Google blocking automated searches?

2008-03-04T10:14:00.005-06:00

This morning I was writing a little python, and I used Firefox's Google toolbar to search for isalnum. The strange thing about this, though, is that Google didn't do the search! Instead they gave me a message saying:

We're sorry...but your query looks similar to automated requests from a computer virus or spyware application. To protect our users, we can't process your request right now.

Luckily I could type in a captcha and continue the search. This really caught me offguard, though, so I decided to experiment a little. I had been logged in to my gmail account t the time, so I tried clearing the cookies and cache from my browser. I searched for "python isapha", and I got the same "We're sorry..." message. But this time, there was no captcha! I also tried searching for other terms (including "dogs"), all with same result.

So what happens if I don't have a Google account? I suddenly can't use their search engine?

So this got me thinking... Is the state of malware so bad that we'll eventually have to validate our identity before using a search engine?

Shrinking Images with Python

2008-02-15T18:26:00.005-06:00

My wife had a very large image, and she needed some smaller versions of it. Well, I thought I'd just fire up The GIMP and create a few shrunken versions, but by the time The GIMP loaded on my G4 Mac mini, I'd almost finished writing the python coded needed to do the task for me!

Writing code is good!

import Image
from sys import argv

def shrink(filename):
    im = Image.open(filename)
    w,h = im.size
    
    # Create images that are 10%, 20%....90% of the original
    sizes = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]
    
    for s in sizes:
        nw = int(w*s)
        nh = int(h*s)
        n = im.resize((nw, nh))
        f = "%s_x_%s_%s"%(nw,nh,filename)
        print "Saving %s"%(f)
        n.save(f)
        
if __name__ == "__main__":
    
    if len(argv) == 2:
        shrink(argv[1])
    else:
        print "USAGE: python shrink.py "

Download this code: http://bradmontgomery.net/files/shrink.zip

Tracking a Laser pointer with Python and OpenCV

2008-01-24T13:13:00.000-06:00

A recent thread on the OpenCV mailing list (entitled: "Tracking laser dots") discussed techniques that could be used to track the dot from a laser pointer. This sounded like something fun, so I finally got around to trying it out. Essentially this could be done acheived by the following algorithm:

Grab the video frame.

Convert the video frame to the HSV color space.

Split the frame into individual components:
(separate images for H, S, and V).

Apply a threshold to each compenent (hopefully keeping just the dot from the laser). It was originally suggested that just the Hue component be used to search for the laser's dot, but I actually got several false positives doing this. Therefore, using Value in addition to Hue gave me a more reliable result. I can see where finding good threshold values for all 3 components would be a good approach in some situations.

Finally, perform an AND operation on the 3 images (which "should" cut down on false positives)

I should note that my testing was performed using a red laser pointer and a large white sheet of paper in an well-lit office. Since I was only tracking the dot on the paper, this turned out to be a fairly easy task to accomplish. Finding good threshold values in other situations would be much more difficult.

Download this Python code: (laser_tracker.zip)

Super Spicy Creamy Veggie Chili

2008-01-14T13:51:00.000-06:00

I like food. I like to eat it... I like to make it.

A few days ago I got a craving for some chili, and I almost had all of the ingredients that I needed. What I didn't have, was meat. So, I decided to improvise, and what resulted was the recipe below. It's sure to give all the gastric goodness of any other spicy bowl of chili!

Ingredients:
1 can black beans.
1 can dark red kidney beans.
2 cans RO*TEL extra hot (good stuff!)
1 medium eggplant cut in small cubes.
about 1 cup of cubed cheddar
about 1 cup of sour cream
a hearty dash of chili powder.

Rinse the beans, and put them in a pan with the RO*TEL and the eggplant. Place on Medium heat until the eggplant starts to soften, then add the cheese and the sour cream. Once everything is all melted, it's ready to eat!

It's good with corn chips or crackers, and another dollop of sour cream doesn't hurt either. Enjoy!

Computing Correlation Coefficients in Python

2007-12-05T08:59:00.001-06:00

A useful technique for matching objects in images is to compute the images' Correlation Coefficients. Essentially, you take any image and compute the correlation between it and another, smaller image containing ONLY the object that you want to identify. The resulting correlation image should contain bright spots where there is a high correlation (or match) between the two images.

Here's a simple python script to compute the correlation between two images:
http://bradmontgomery.net/files/correlation.zip

It requires PIL and numpy.

Announcing pgSlideShow

2007-10-31T10:13:00.000-05:00

I've just released a simple little application called pgSlideShow. It's a free image slideshow application written in Python using pygame. It recursively searches a given directory for image files, and then displays them on a computer screen in Fullscreen mode.

Get it here:
http://bradmontgomery.net/pgSlideShow/

On the Move...

2007-10-16T21:04:00.000-05:00

Well, I've just created my account here at blogger.com, and I'll have to try it out for a while before I decide whether or not I'm happy. I've been using my own system for a while, and normally, I'm happy with a simple, stream-lined app, but I've begun wanting new features that I just don't have time to build for myself...

So, I'm here. We'll see how it works.

PyOpenGL on OS X

2007-10-16T20:17:00.001-05:00

Update: July 7, 2010: For the past year or so I've been using virtualenv, virtualenvwrapper, and pip to manage and install python packages, and I highly encourage everyone to use these tools! It may take a bit to learn how to use these tools, but once you do, the rewards are great! (especially when using 3rd party python libraries on Mac OS X).

To install PyOpenGL using the aforementioned tools, you'd simply do the following.

Create a virtual environment for your OpenGL installation. I'm naming it "gl":
```
mkvirtualenv gl
```

Then use pip to install PyOpenGL:
```
pip install pyopengl
```

That's all there is to it! I grabbed the demo apps from http://pypi.python.org/pypi/PyOpenGL-Demo, and verified that the gears.py script works, so this should get you started.

I'm leaving the old post below, but again, check out virtualenv and pip. You'll be happy that you did!

I actually got this working quite a while ago, but for some reason I've gotten busy and never documented it. In fact, installing PyOpenGL was almost so easy it's not even worth mentioning... Except... it was so easy, so it's worth mentioning!

Here are the steps to install PyOpenGL on Mac OS X (10.4.10):

You'll need to get setuptools, and the easiest way to do that is with EasyInstall (see http://peak.telecommunity.com/DevCenter/EasyInstall). You'll download the ez_setup.py file, and just run: python ez_setup.py.

Download PyOpenGL from their download page. (Side Note: I also grabbed the OpenGLContext). Now, once you untar/unzip it, you should have a PyOpenGL/ directory. Just open a Terminal, cd into that directory, and run python setup.py build. Once that is finished, run python setup.py install

Now, do the same for OpenGLContext...

Pygame on OS X with python 2.5

2007-09-23T20:38:00.000-05:00

I've used Pygame on Mac OS X in the past, but my installation recently stopped working for some reason, so I decided to grab the binaries and re-install. After checking out the Pythonmac list, I was a little disappointed to see that there were only Pygame binaries for Python 2.4. Below is a list of software that I installed (in the necessary order) to get pygame working with Python 2.5 on OS X. All of these (except Python 2.5.1) I got from pythonmac.org/packages.

Python 2.5.1 - from the Python download page.

PIL 1.16 - Python Image Library.

NumPy 1.0.3.1.

Numeric 24.2.

numarray 1.5.2.

PyObjC 1.4.

wxPython 2.9.3.0 (just in case)

Finally, I grabbed the Pygame 1.7.1 source from pygame.org/download.shtml.

Installation for most of the prerequisite packages went smoothly, since they all include OS X Installers. While pygame requires Numeric, I went ahead and installed NumPy and numarray, just in case.

The only real problem came when installing PyObjC. After installing, I opened a python interpreter to test it out by typing "import Foundation" (as it says to do in the PyObjC Tutorial. This let to the following error:


dyld: Symbol not found: __cg_jpeg_resync_to_restart
Referenced from: /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Versions/A/ImageIO
Expected in: /sw/lib/libJPEG.dylib

Apparently, if you've installed anything using Fink, the PyObjC installer gets confused when trying to choose to which libraries to link. For me, libjpeg was the culprit... hence the error above. To make sure PyObjC works with Apple's ImageIO, I edited my ~/.profile and commented out any lines that set a DYLD_LIBRARY_PATH environment variable (just place a # in front of the line). For me, this looked like this:

#DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/usr/local/lib:/sw/lib:/usr/lib
#export DYLD_LIBRARY_PATH

This prevents any conflicts when linking against libraries installed by fink (which are stored in /sw) and possibly duplicate system libraries. After doing this, I re-ran the PyObjC installer, and everything worked perfectly. Now, I built the Pygame source, and my simple Pygame applications will run.

So what's next? Notice there's no mention of PyOpenGL above, so any Pygame applications that use PyOpenGL won't work Until you Install PyOpenGL

Download this Pygame Package: http://bradmontgomery.net/files/pygame-1.7.1release-py2.5.1-macosx10.4.dmg

$100 iPhone Early adopter Credit

2007-09-15T20:42:00.000-05:00

There's been a lot of hoop-la about the steep price-drop of the iPhone, and a lot of people have widely ranging opinions about the $100 credit.

Well, I (actually, my wife) willingly paid the initial price, and I've been absolutely happy with my iPhone! I also grabbed my $100 credit. All I had to do was hit Apple's website and type in my phone number, and my iPhone's serial number.

$100 is $100! Woot!

The Never-ending "To-Learn" List

2007-09-10T20:43:00.000-05:00

I'm officially a student! August marked the beginning of a new semester for me, and this time I'm a full-time student. After being on the faculty side of things for a while, I'm actually having a great time being a student again. One of the major benefits of being a student is that I'm actually getting to dive into that never-ending list of technology tools that I've been wanting to learn how to use!

Just to name a few:

Python - I'm writing as much code as I possibly can in Python. I've used python minimally in the past, but it's actually nice to start working on larger projects using it. I don't really like to get into religious debates about languages, but Python does feel nice!

Pygame - I'm not building a game (yet), but it does make a nice foundation for putting pictures on a screen!

PIL - the Python Image Library offers an Excellent way to to Image manipulation!

Twisted - if you EVER need to build an event-driven network application from the ground up... use this!

OpenGL - GAhh! Why haven't I learned this before... there's a lot to learn!

Luckily, there's PyOpenGL!

That should get me started!

Building Apps for the iPhone

2007-07-13T20:44:00.001-05:00

So there was this little get-together called iPhoneDevCamp last weekend (July 6-8, 2007), and since I'm just over 2000 miles from San Francisco, I missed out. However, in just under a week, there seems to have been a flood of great iPhone applications. To find out what I mean, just check out a google search, OR, head right on over to the iPhone Application List.

So, in such a short time since the iPhone's release, how have so many applications been built? I blame Joe Hewitt. His User Interface library, iUI, makes building basic apps for the iPhone a cinch. So what is iUI? Well, to quote from his blog:

Its goal is simply to turn ordinary standards-based HTML into a polished, usable interface that meets the high standards set by Apple's own native iPhone apps.

So, there you have it. Try it out! I certainly will!

iPhone Disappointments

2007-07-08T20:46:00.000-05:00

Updated on 2007-07-10
(I forgot a couple of things, so I'm adding them below as numbers 1 & 2) Part 3 of my iPhone saga (Part 1 and Part 2)
After using the iPhone for a little less than a week, I do have a few disappointments. Don't get me wrong, I still think it's an incredibly cool device. Like my wife said, "It's the greatest thing since sliced bread."

I share some of my disappointments with Terry White, who gives a fairly objective review of the iPhone in general.

So to cut to the chase, here they are:

Limited Tilt... I've found that it's easiest to type on the iPhone when I tilt it sideways (in landscape) However, the app that I type most in (Mail) doesn't let you do this!! In fact, only Safari, Photos, and iPod take advantage of the iPhones accelerometer. Mail, SMS, Google Maps, and Notes (all apps that may require extensive "typing") don't! Hopefully a software update will fix this!

Where are the games!? I have to admit that I've already had a few neat ideas for games on the iPhone. Surprisingly, it doesn't come with ANY bundled games. The folks who put on the iPhoneDevCamp may have had some interesting browser-based games, but there's still a lot more you could do with a local app. Only time will tell what will happen for those interested in 3rd party development for the iPhone.

Fingerprints!: I'll admit, I'm OC when it comes to fingerprints on my monitor/TV/display, so I'm constantly wiping the iPhone on my shirt.

EDGE: OK, nothing wrong with the iPhone itself... the EDGE network (AT&T's data network) is just too slow.

Ringtones: All that music, and you can't use your tunes as a ringtone!

No Video! - While most cell phones give you poor quality video, at least they give you the option to capture video! Not so with the iPhone (yet, I hope)

Safari/Google Maps Crash!: This one was a big surprise to me! Safari seems to crash a LOT for me. In fact, I tried to post a blog yesterday, and halfway through writing it on the iPhone, Safari just disappeared. When I started Safari again, I was greeted with the site I'd previously viewed (http://slashdot.org/palm).
Apparently several others have experienced this, and the solution (which I found on Apple's Discussion Boards, is to simply reboot the phone. I'm confident that this issue will be fixed with an update in the future.

Bluetooth headset support: I've got a Jabra BT 350 headset, which the iPhone regonised and paired with perfectly. However, sometimes when I talk on the iPhone, it doesn't give me the option to switch the Audio to the headset.

It Locked up!: Ok, so yesterday, my iPhone completely locked up. Well, the touch screen wouldn't work, with is fairly devastating for the iPhone. I hadn't reset it since I'd activated it, and even when I tried to power it off... I couldn't move the slider to verify that I wanted it powered off! After a Reset, though, it seems to be working perfectly.

Overall, I'm still happy. Personally, I think the large screen is what makes the iPhone so cool. I really like my music, but I've been more than happy with my iPod nano for that... it's the iPhones integration with iPhoto that takes the cake for me. I've probably found myself looking a pictures of my daughters and my wife more than anything, and having the 2 megapixel camera rocks! The email/web/phone just happens to seal the deal for me.So, baby... Again, THANK YOU!

My wife bought 2 iPhones!!!

2007-07-04T20:51:00.000-05:00

I think I'm still in shock. Yesterday, I got a call from my wife. She simply said, "We both have iPhones! I got you the 8Gb model, and I got the 4Gb model..."

Yeah... I'm still in shock.

So, over the next few days, I've decided to blog about my iPhone fun. First, I'll start with some pictures!

So, here we have the iPhone with it's box, and all of the stuff that was include with it. There's the dock, a USB connector, an AC adapter to recharge the iPhone, the new stereo headphones with a mic, the manuals, and the complementary apple stickers.

One thing that I didn't know about was the new headphones. In addition to just being the basic earbud (like those bundled with an iPod), these headphones also include a mic, so they can be used to talk on the phone. The mic includes a "button" that allows you to answer phone calls. (it's not really an extruding button per se, but a "press-able" area surrounding the mic.

Now, it's really difficult to get an idea just how sleek the iPhone is, without holding one yourself. I've photographed my new iPhone next to a US Quarter and next to my old Nokia 6102. I have to admit, for a phone (or any portable device) the screen feels large, but the phone feels very slim. I don't really like to carry much stuff in my pocket, so I'm not sure how it will feel when I'm not using it.

As soon as I unpackaged my iPhone, I turned it on to start playing with it... Unfortunately, there's not much you can do until you activate it. You can, however, unlock the display, and see the keypad, so I assume it could be used to dial 911 without activating it (like most/all cell phones). Before I activated mine, I did punch some numbers on the keypad, just to see how it felt. I didn't really have any problem touching the correct numbers, so I'm happy with how the keypad feels. One other thing I noticed, is that the numbers on the screen decrease in size as you type more of them... this way, you always see the whole number you're dialing (unlike some other cell phones where you may only see 4-5 digits at a time, or where the digits will stack on top of each other)

So, I finally got around to plugging up the dock and connecting my iPhone. At first, I plugged the dock directly into my powered USB hub, but iTunes didn't detect the iPhone! So, I plugged the dock directly into the back of my Mac mini, and iTunes immediately detected it. Time to activate this baby!

As soon as iTunes detects the iPhone, it greets you with a welcome screen, and walks you through the activation process. My wife and I were already Cingular/AT&T customers, and luckily the activation process included options for this possibility. Unfortunately, activating the iPhone through iTunes only allowed us to add a data plan to our existing plan. The lowest-cost data plan is $20/per phone, so our monthly bill has been increased by $40. I read the att wireless website, and it seems that we should've been able to switch to a new plan. I suspect that you'd either have to call AT&T or stop by a store to do this. The other thing that I'm a little disappointed about is the FamilyTalk pricing for the iPhone. For regular phones, FamilyTalk is $9.99/per line. For the iPhone, it's $29.99/per line. So, as of now, our phone bill has increased an extra $20 on top of the $40 for the data access... I'm not exactly happy about that.

All in all, the iTunes registration process was really easy, and for both phones, it only took about 5-10 minutes.

Once my iPhone was registered, it seamlessly sync'd my iCal calendars, my photos from iPhoto, and my music from iTunes. Of course, since I'm a mac user, I expected this seamless integration. My wife, who's using Windows XP was able to pull music from iTunes and photos from her My Pictures folder. The iPhone also has the capability to sync mail and calendar info from Outlook or Outlook Express, but we haven't had a chance to try that out yet.

So, my initial reaction to being an iPhone owner: "COOL!"
And it is... very cool, though a little pricey.

As I explore my iPhone's new features and learn all of its quirks, I'll post my thoughts on this cool new gadget.

Using (and Abusing) the iPhone

2007-07-04T20:47:00.000-05:00

OK... so for part 2 of my iPhone saga:
We went to visit some family today, and of course I took my iPhone. For the most part, I've really tested out the camera today. I'm no guru, but I'm very happy with how quickly the iPhone takes pictures. Since my girls are now 4 and 2, this is a very important feature!! I've also got an HP Photosmart M527, and my non-scientific opinion is that the iPhone takes photos about as twice as fast as my HP. (Of course, it's a 2-megapixel camera vs. a 6-megapixel camera)

Just to give you an idea, here's a picture of my youngest daughter:

I convinced my wife (OK, so she offered :)) to drive home from out festivities today, so I could play with the iPhone in the car. While in town, I had no problems browsing the web or checking my email using the EDGE connection. However, once we got out of town, there were several locations where I had no network connection (this is rural NE Arkansas...)

Another thing that I've noticed is that there are a LOT of unsecured wireless networks... but that's for another blog!

As for typing on the iPhone, I'm fairly impressed with it's ability to suggest and correct misspelled words. Now, I don't really think that I have large fingers, but I very frequently misspelled words using the iPhone. However, the iPhone "guessed" what I wanted about 80% of the time! Things like "sxgppl" were correctly identified as "school" (As as side note, I'd guesstimate that I touch type somewhere around 60wpm with 90% accuracy, if anyone cares)So, when typing on the iPhone, I found myself just ignoring what I typed in hopes that the phone could figure it out. Overall, I'm impressed.

Now that I'm home, the geek in me is dieing to portscan this thing! So here goes! First a basic portmap with nmap:


nmap 192.168.0.11

Starting Nmap 4.20 ( http://insecure.org ) at 2007-07-04 19:41 CDT
All 1697 scanned ports on 192.168.0.11 are closed

Interesting enough... let's see what OS nmap thinks is running on this thing:


sudo nmap -O 192.168.0.11

Starting Nmap 4.20 ( http://insecure.org ) at 2007-07-04 19:43 CDT
Warning:  OS detection for 192.168.0.11 will be MUCH less reliable because we did not find at least 1 open and 1 closed TCP port
All 1697 scanned ports on 192.168.0.11 are closed
MAC Address: 00:1B:63:E1:ED:D8 (Unknown)
Device type: general purpose
Running: Apple Mac OS X 10.3.X|10.4.X|10.5.X, FreeBSD 4.x
OS details: Applie Mac OS X 10.3.9 - 10.4.7, Apple Mac OS X 10.4.8 (Tiger), OS X Server 10.5 (Leopard) pre-release build 9A284, FreeBSD 4.10-RELEASE (x86)
Network Distance: 1 hop

OS detection performed. Please report any incorrect results at http://insecure.org/nmap/submit/ .
Nmap finished: 1 IP address (1 host up) scanned in 16.991 seconds

So, while I'm not really a security professional, and I know these are very basic scans, it seems like there's no blatantly obvious security gaps in the iPhone (through its wifi connection anyway). If anybody has any more advanced nmap scans they'd like to see performed, just let me know.

Compiling Warzone 2100 on Mac OS X

2007-06-16T20:53:00.000-05:00

I recently ran across The Warzone 2100 Resurrection Project, and I thought to myself,"cool, I wonder if there's an OS X version..." So, I hit their downloads page only to find the message: "(The MacOSX diskimage will follow soon)"

So I thought, "oh well, I'll just grab the source code and compile it!" After about two hours of digging, I finally got it compiled, and here's how! (NOTE: I've got quite a bit of Unix stuff installed via fink, so your mileage may vary)

Here's the list of stuff I had to get/install/compile before I could even start on Warzone 2100: cmake (to compile physfs), physfs, a newer bison, and SDL_net

First I grab cmake 2.4.6. Luckily, they have a universal .dmg for Tiger.

Next, I grab the source code for physfs-1.1.1 from their download page. At first I tried compiling the "Unix Way", but that didn't work, so I used cmake to generate an Xcode project (pretty cool!). From a terminal window, I cd into the physfs directory and type:
```
cmake -G Xcode .
```
Then, after compiling physfs with Xcode, I copied the libraries to my /usr/local directories. Again from a Terminal:
```
sudo cp libphysfs.* /usr/local/lib
sudo cp physfs.h /usr/local/include
```

Next, I open up fink commander and grab the newest version of bison from fink

Now I need SDL_net. While the project offers a Mac OS X Framework and SDL_net is available through fink... I grabbed the source anyway. It compiled just fine the Unix way:
```
./configure
make
sudo make install
```
This put all the libraries in /usr/local/lib and /usr/local/include

Finally, time to compile Warzone2100, so I grabbed the source code, which contains a handy-dandy file called COMPILE. I suggest you read it. To start of, I opened a terminal and cd'd into the warzone2100-2.0.6 directory and typed the following command:
```
./autogen.sh
```
Then, I typed the following to configure it using my fink libraries and headers:
```
./configure --prefix=/Applications/Warzone2100-2.0.6 LDFLAGS=-L/sw/lib CPPFLAGS=-I/sw/include
```
This will also put the final binary in my /Applications Directory. So, the last to steps are to compile, and install:
```
make
sudo make install
```
This did the trick for me.

Now that I got Warzone2100 compiled, I can open a terminal and type the following:
```
/Applications/Warzone2100/bin/warzone2100 --fullscreen
```
And this "should" lauch the application!

Enjoy!

Thunder Demons live here!

2007-05-26T20:55:00.001-05:00

While walking across campus, I noticed this particular image attached to a large green box that I knew to contain electrical "stuff". Now, I know the dangers of mixing humans with direct electrical current, but I'm not so sure this image evokes the correct message. I'm not sure that a monstrous blob of electricity has ever actually jumped out of a box in order to heinously inflict harm to humans as they unknowingly walk by... I could be wrong, though. Perhaps I should avoid that particular sidewalk next time I'm on campus.

Easy Video Conferencing?

2007-05-22T20:56:00.000-05:00

I just skimmed through the list of PC World's 100 Best Products of 2007, and one of the things that caught my eye was
SightSpeed. Apparently it's supposed to be the world's best video conferencing system. What interests me, though is the price (FREE!) and that it works on Mac OS X (and Windows). I'm going to give it a shot and see how it works.

Hello Torque, again.

2007-05-16T20:57:00.000-05:00

Well, I've just begun to dig into the Torque Game Engine again, but this time, I'm building it on a Linux box. Once I downloaded the Linux source, it took me a while to figure out how to get it compiled. TGE's configuration is a little different than the typical "./configure and make" process.

First of all, I tried to make sure I had all of the development tools necessary to build TGE, including nasm, libogg, libvorbis, and libtheora along with any -dev versions of many packages. Then, once you unzip the source you need to issue the following command to set up your configuration:

make -f mk/configure.mk

This will display the current build configuration and it will tell you if it's valid or not. To get a valid configuration on my system, I used the following configuration:

make -f m/configure.mk OS=LINUX COMPILER=GCC4 BUILD=DEBUG \
DIR.OBJ=/home/brad/TorqueGameEngineSDK-Linux-1-5-0/build

Notice that I had to use an absolute path for the (optional) object Directory. At first, I just used "DIR.OBJ=build", which resulted in some compiler errors. Anyway, this placed all the object files in a directory called "build.GCC4.DEBUG"

The next change I had to make was in an actual source file. Apparently, GCC didn't like some things... I'm not going to post any details, but I did eventually get it working, so contact me if you're interested.

Good Luck!

MacMathematics

2007-04-23T20:58:00.002-05:00

One of my Favorite Mac Apps is Grapher. It lets you graph everything from simple 2D equations to complex 3D equations. For example, want to see all of the Axes of symmetry for a cube? Just enter the equations x=0, y=0, z=0, x=y, x=-y, x=z, x=-z, y=z, and y=-z, and voila! Pictures make math fun! You can also save these in .gcx files. Download my cube: CubeAxesOfSymmetry.gcx

MacMathematics

2007-04-23T20:58:00.001-05:00

One of my Favorite Mac Apps is Grapher. It lets you graph everything from simple 2D equations to complex 3D equations. For example, want to see all of the Axes of symmetry for a cube? Just enter the equations x=0, y=0, z=0, x=y, x=-y, x=z, x=-z, y=z, and y=-z, and voila!Pictures make math fun! You can also save these in .gcx files. Download my cube: CubeAxesOfSymmetry.gcx

Casual Games

2007-03-09T21:01:00.000-06:00

It seems like I keep running into this term, "casual games." Apparently this is a burgeoning industry with the advent of more advanced handheld devices (like cellphones). There were even two keynotes (Trip Hawkins of
Digital Chocolate, and Robert Tercek of PeopleJam and GDC Mobile) regarding causal games at the recent Game Developers Conference.

Basically, any simple (hopefully addictive) game that's quick and easy to learn falls in the category of a casual game. Two of my favorite examples are Atlantis Sky Patrol (from Big Fish Games) and Insaniquarium (from PopCam). [Side note: It seems like I remember playing insaniquarium before it belonged to a company... did I just dream this?]

I don't where I read it, but I vaguley remember reading a quote somewhere that had something to do with casual games, and it said something similar to this, "If you don't think you're a gamer, it's just because you haven't found the right game!"[Another Sidenote: Basicially, these are the the types of games I would like write during my free time (if I had any!)

Mac mini disassembly

2007-03-05T21:04:00.000-06:00

I've been a mac user since my wife bought me a Min in 2005. During that time I've enjoyed being a Mac user, but until now I've not performed a lobotomy on my little mac. I finally decided to upgrad to 1Gb of Ram, and thanks to NewEgg, this was fairly inexpensive. Also, thanks to Russel Beattie and smashsworld.com, I found it fairly easy to get into the mini.

Though I only upgraded the RAM, I was interested in discovering how difficult it would be to upgrade the Hard drive as well. To do this, you have to first remove the combo drive (which is attached to a daughter board along with the hard drive).
Once this has been done, you need to remove the air vents and the processor's cooling fan in order to get to the screws that hold the drive in place. While doing this, I noticed that the power switch's wire was taped to the vent, so I just cut this (The TAPE, not the WIRE!). Once all of the screws that held the air vents to the motherboard were removed, I found it fairly easy to just lift the hard drive and it's daughterboard out of the motherboard.
In doing this, I was fairly amazed at how small the Mac mini really is... I guess you have to see it to believe it!

Javascript: What is the standard?

2007-02-27T21:06:00.001-06:00

I've recently been writing a little javascript, and I needed to chage the value of some text inside an html/xhtml element. It seems there are several ways to do this, but evey browser may or may not support the same method for doing it (big surprise, here) I'm not sure what is considered the "standard" way. Here's a little script that I use to help me decide which browsers support which methods for altering text within an html/xhtml element:


<html>
<head><title>blah</title>
<script language="JavaScript">
<!-- Hide code from older browsers
function loader() {
  pp = document.getElementsByTagName("p")
  pp[0].innerText = ".innerText DOES work!"
  pp[1].style.content = ".style.content DOES work!"
  pp[2].innerhtml    = ".innerhtml DOES work!"
  pp[3].childNodes[0].nodeValue = ".childNodes[0].nodeValue DOES work!"
}
// End hiding-->
</script>
</head>

<body onload="loader()">
<h1>Changing text with javascript</h1>
<p> .innerText DOES NOT work.</p>

<p> .style.content DOES NOT work.</p>
<p> .innerhtml DOES NOT work.</p>
<p> .childNodes[0].nodeValue DOES NOT work.</p>
</body>

</html>

(Thanks to the users of codingforums.com, especially Roy Gardiner, whose code from which this script was adapted.)

magic SysRq

2007-02-09T21:07:00.000-06:00

Occasionally I goof up. Yes, as much as I hate to admit it, it's true. However, thanks to this handy tutorial on liquidweather.net, I've learned about some nifty ways to kill things in linux.

In addition to the traditional ways to kill a process, this tutorial lists some magic SysRq key combinations that--if enabled in your kernel--can provide a nice option to just pulling the plug...

Alt+SysRq+K - Kills all processes (SIGKILL / kill -9)

Alt+SysRq+E - Terminates all processes (SIGTERM / kill -15)

Alt+SysRq+I - Interrupts all processes (SIGINT / kill -2)

Alt+SysRq+U - Force unmount and remount of all filesystems readonly

Alt+SysRq+S - Syncs all disks

Alt+SysRq+B - Reboots

(Note to self, and any other developers: When using a language without built-in garbage collection, don't forget to free up memory when you're done!)

wifit - an iwconfig tool

2007-02-08T21:09:00.000-06:00

I've recently installed Mepis Linux on a couple of laptops (a Compaq Presario 2195US and a Dell Latitude D610, both of which use NdisWrapper for wifi drivers).I'm fairly mobile, so I needed a way to quickly change my wireless settings. I wrote this simple bash script to let me do that, and I thought I'd just share it.


#!/bin/bash

## wifit - the wifi tool
## This is a script that accepts a wifi-enabled interface, 
## essid, and an optional ascii key for a wifi network connection

if [ $# -lt 2 ] || [ $# -gt 3 ] ; then 
 echo "wifit - the wifi tool"
 echo "-------------------------------------------------"
 echo "Usage: wifit   [Optional: ]"
 echo " Example: wifit wlan0 somenet "
 echo " Example: wifit wlan0 somenet mypassword "
 echo "-------------------------------------------------"
else
 echo "Setting up wifi connection for "
 echo "Interface $1, ESSID: $2..."

 iwconfig $1 essid $2
 iwlist $1 scan
 iwconfig $1 mode Managed
 if [ -n $3 ] ; then
  iwconfig $1 key restricted s:$3
 else
  iwconfig $1 key open
  iwconfig ap any
 fi
 iwconfig commit
 dhclient
fi

To use this, just copy and paste the code above into a file named wifit (or wifit.sh, or whatever you want). You'll then need to make that file executable using chmod:

chmod a+x wifit.sh

Then, simply call the script (possibly using sudo) passing it any arguments you need:

sudo ./wifit wlan0 somenetwork

Alternatively, you could place this file in your /usr/local/bin or /usr/bin directories so that it would be included in your PATH. If you find this useful, or if you decide to add to is, feel free to Contact Me!

Enjoy.

Automatic Backups with cron, tar, and SSH

2007-01-10T21:10:00.000-06:00

Everyone knows that backups are important, but how many of us regularly back up our own websites, blogs, or whatever? Well, I've put together a relatively simple way for my Mac to log into my Linux-based webserver, archive some content, and download it for me. All of this is done Automatically, too!

The first thing I had to do, was set up my home machine (a Mac, but any Unix-based system should work, too) so that it could log into a remote host without requiring me to type a password. This is accomplished by generating a public key, which I can store on my webserver. To do this, I opened a Terminal, and typed the following:

ssh-keygen -t rsa

This generated the following output, and I accepted all default values... even the empty passphrase. This is important, because I don't want to type anything to log into my server!


Generating public/private rsa key pair.
Enter file in which to save the key (/home/brad/.ssh/id_rsa):
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/brad/.ssh/id_rsa.
Your public key has been saved in /home/brad/.ssh/id_rsa.pub.

Now, I need to store the public key (id_rsa.pub) on my webserver. Again, from my Terminal (in my home directory), I issue the following command to log into my webserver (Note that you'd need to change the following command to fit your needs).

cat .ssh/id_rsa.pub | ssh myusername@mywebserver.com 'cat >> .ssh/authorized_keys'

I'll need to type my password here, but after executing this command, I should be able to ssh into my remote server without ever typing my password again!

Now that I've accomplished that, I need to write some simple bash scripts that will archive my web content. On my Webserver: For simplicity sake, let's save this in our home directory in a file named www.sh. Inside that file we put the following shell script, which uses tar to archive my web directory and then calls bzip2 to compress the tar file. Upon execution, I should have a file with a name similar to 2007-01-10.tar.bz2. Notice that you could also modify this script to invoke mysql_dump or pg_dump if you needed to backup a database as well!


#!/bin/bash
DATE=`date +%F`  # Grab the date
TARFILE=$DATE-www.tar # Use it to create a filename
tar -cf $TARFILE public_html
bzip2 $TARFILE # compress the tarfile

Now that I have a script that will archive my content, I need to schedule a cron job to run it on a regular basis. I can use the following command to view my cron jobs

crontab -l

And, to edit my cron jobs, I just use:

crontab -e

This will open your system's default editor (vi for me), where we need to add the information that tells cron when to run our www.sh script. I added the following:

30 1 * * * ~/www.sh

This tells cron to run my backup script every morning at 1:30am (system time). See wikipedia for more info on cron.

Almost done! I've now got my webserver generating automatic nightly backups, but how can I transfer them to my local machine? Well, we'll make use of our newly generated public key to do this! First I need to write another simple bash script that will use scp to copy the backup that I generated on the server.On my local machine, I create a file called transfer.sh, which contains the following:

#!/bin/bash
DATE=`date +%F` # Grab the date
# I'm downloading file with this name
FILE=$DATE-www.tar.bz2 

# put the file on my local Desktop
scp myusername@mywebserver.com:~/$FILE ~/Desktop

# delete the file from the server
ssh myusername@mywebserver.com rm -f ~/$FILE

Now all I have to do is schedule a cron job on my local machine, and I've got an automatic website backup! Since I'm using Mac OS X, this works just like it does on a Linux box. In my terminal I type the following command:

crontab -e

This opens vi for me, so I type the following, which will exectute my transfer.sh script every morning at 7:30am.


30 7 * * * $HOME/transfer.sh

That's It! One side note, however... the bash scripts that I've written must be executable by the user under which cron runs. Usually, the following command will make a file executable for its owner:

chmod u+x filename

Resources:
debian-administration.org
Wikipedia
linuxproblem.org

Migrating PHP scripts to MySQL from PostgreSQL

2007-01-05T21:13:00.000-06:00

I've recently had to work on a project where I needed to convert some very basic PHP code that accessed a postgresql database so that it would work with mysql. For the most part, this has been fairly simple thanks to rpl. Many of PHP's database functions have very similar names, so I simply use rpl to convert the existing code.
Here’s a simple bash script that I put together to convert some of my postgresql functions to mysql:


#!/bin/bash
if [ ! -n "$1" ]
then
  echo "Usage: `basename $0` "
  exit 1;
fi

X=$1
echo "---- Relacing pg_ with mysql_ in $X"

rpl -R -x'.php' 'pg_pconnect' 'mysql_pconnect' $X
rpl -R -x'.php' 'pg_connect' 'mysql_connect' $X
rpl -R -x'.php' 'pg_query' 'mysql_query' $X
rpl -R -x'.php' 'pg_fetch_row' 'mysq_fetch_row' $X
rpl -R -x'.php' 'pg_fetch_assoc' 'mysq_fetch_assoc' $X
rpl -R -x'.php' 'pg_close' 'mysq_close' $X
rpl -R -x'.php' 'pg_num_rows' 'mysq_num_rows' $X
rpl -R -x'.php' 'pg_numrows' 'mysq_num_rows' $X
rpl -R -x'.php' 'pg_last_error' 'mysql_error' $X

Just in case your curious about converting from postgresql or mysql to sqlite, I’ve compiled the following table of analogous functions from php.net:

Postgres	MySQL	SQLite
pg_connect	mysql_connect	sqlite_open // open or create database
pg_query	mysql_query	sqlite_query
pg_fetch_row	mysql_fetch_row	sqlite_fetch_array (ordinal and associative)
pg_fetch_assoc	mysql_fetch_assoc	sqlite_fetch_array (ordinal and associative)
pg_close	mysql_close	sqlite_close
pg_num_rows or pg_numrows	mysql_num_rows	sqlite_num_rows

Blocking outgoing UDP traffic using iptables

2007-01-03T21:15:00.000-06:00

Since rebuilding my server (after having it used to propogate a UDP flood DoS attack), I’ve been advised that I should set up iptables to block any unnecessary outbound UDP traffic. So, how do I do this?
Well, the first thing I’ll do is update my apt repository, and install iptables using the following two commands:

apt-get update
apt-get install iptables

Now, the quick and dirty solution is to just add append a rule that blocks all outgoing UDP packets from my server. You can do this based on the systems IP addess. Assuming my server’s IP addess is 192.168.0.1, I would use the following rule:

iptables -A OUTPUT -s 192.168.0.1 -p udp -j DROP

Essentially, this rule says, match any outbound UDP packets whose source address (-s) is 192.168.0.1, and jump (-j) to the DROP chain. That will drop the packet. Now, just to be safe, I’ll add the same rool using my loopback address, as follows:

iptables -A OUTPUT -s 127.0.0.1 -p udp -j DROP

Now, let’s just hope that keeps me covered until I can find a little more advanced solution that will also write to a log when a packet gets dropped...

Resources:

netfilter’s documentation

linuxquestions.org forums

The manpage! (man iptables)

Building a Debian-Based Webserver

2006-12-31T21:17:00.000-06:00

Well, my server was recently compromised and used to perform a UPD flood Denial of Service attack (or so my provider claimed), so I’ve had to rebuild my server from scratch. Now, this seems like a daunting task, but it has actually been quite easy. I started out with a minimal Debian Linux system, and after a few apt-get install commands, and a few hours of uploading backups, I’m up and running again.

How’d I do it? Well, i just set up my apt-sources:

### debian
# stable
deb http://http.us.debian.org/debian stable main contrib non-free
deb http://non-us.debian.org/debian-non-US stable/non-US main contrib non-free
deb http://security.debian.org stable/updates main contrib non-free
# testing
deb http://http.us.debian.org/debian testing main contrib non-free
# unstable
deb http://http.us.debian.org/debian unstable main contrib non-free
### dotdeb
deb http://packages.dotdeb.org stable all
deb-src http://packages.dotdeb.org stable all

Then I ran the following apt-get commands:

apt-get update
apt-get upgrade
apt-get install libperl5.8
apt-get install postgresql
apt-get install libapache2-mod-php4
apt-get install php4-pgsql
apt-get install gd
apt-get install php4-cli

And I’m all set! (yeah... I’ve still got a few PHP4 sites, and I grabbed the PHP GD library and the PHP command-line Interface). Notice that I updated my apt sources before I started installing packages... this is kind of important! I also knew that I needed some perl libraries before installing Postgresql (my preferred database) All in all, things went fairly well. The worst part of (re)building this server was uploading my backups! (AND I CANNOT EXPRESS THE IMPORTANCE OF BACKUPS!)

My Wife's New Laptop

2006-12-23T21:18:00.000-06:00

Well, the semester has ended, so I've been working from home (trying to catch up on all of the "extra work", and, while out to the gym, My wife gives me a call and starts asking about laptop specs!? It turns out she's got her eye on a nice little bundle at Radio Shack!

It seems they had an HP Pavilion dv2000 notebook (Turion 64 - 2Ghz/512Mb Ram/100Gb HD/nvidia Geforce go 6150) with a bundled 6.0 megapixel digital camera, and a printer/scanner/copier for less than $1000! So, she makes a purchase, and the notebook is a surprisingly nice little machine. It's a little lacking on Memory, but luckily, it seems NewEgg has just what we need to fix that. I'm particularly impressed with the graphics capabilities of this machine (which are usually lacking in notebooks).

So, while looking around for something to test its capabilities, I ran across the War Rock demo. I thought I'd just try it out, and it plays quite nicely. Hopefully, a memory upgrade will provide even greater performance, though. Perhaps one of the best features of this notebook is its display - a widescreen 14 inch, which runs at 1280x800 by default. Despite the relatively small size, it offers a very nice view for a laptop computer. Overall, I'm quite proud of my wife's purchase, and I've probably used it more than she has!

Spammers never die!

2006-12-17T21:20:00.000-06:00

I've recently received a lot of spam through my contact form, and it's starting to get on my nerves. Apparently the spammers have noticed that many developers are trying to thwart their attempts to sneak extra header information into PHP's mail function, so they've resorted to spamming websites directly! (or am I the only one?)

The messages I've been getting actually include a long list of URLs of the form: [URL=http:www.whatever.com]whatever[/URL], so I'm guessing some web app that uses this type of markup has a security hole... Time to patch that up developers!

That's Amazing

2006-12-02T21:21:00.001-06:00

'Tiz the season... so today I went to Wal-Mart to get a Pre-Lit Christmas Tree. I'm still a little amazed at how easy this thing was to assemble when compared to a real tree of the same size. Sure it doesn't smell like a real tree, but I don't have to worry about watering it so that it doesn't become a fire hazard. Besides, the girls really liked it, and that's all that counts! In fact, Dakota (3yrs) walked into the den, looked at the tree, and just said "That's Amazing". Where does she learn these things!

Voice Observation

2006-11-09T21:23:00.000-06:00

Apparently, I haven't quite beat the flu... or whatever it is that's making me cough. So head over to my local Medical Practitioner and get a nice pharmaceutical conglomeration injected into my hip. Apparently, I could also use some antibiotics, too... so I head over to walgreens and drop off the magical note that lets them know it's OK to give me drugs. I wait several hours, and then I discover that my insurance has rejected a claim... Uhm, OK. So my first thought is "Did someone pull an Office Space maneuver on me and forget to tell me I don't have a job!"
I hope not.So, I call up my Insurance company to see if there's something wrong, and of course, I get an automated system. However, instead of giving me a menu and letting me push a button, I'm greated by a voice that asks me to speak my responses to the menu... (like I'm a dog or something). Some people call this Voice (or Speech) Recognition. I didn't notice much recognizing though. I guess computers aren't designed to interact with people who cough. Basically the transcript of my conversation with the menu went like this:

Menu: Do you need information about Health or Dental insurance?
Me: Health (cough)(cough)
Menu: I'm sorry, please say Health or Dental
Me: Health (I then turn my head away from the phone and cough)
Menu: Please say Health or Dental
Me: HEALTH
Menu: Did you say "Start Over?"
Me: Ahhh!

Menu: Welcome to... *click*

Yeah, I hung up the phone. Well... actually I just hit 0, which normally bypasses the automated system and sends you right to an operator. Unfortunately, their offices were closed. I guess I'll just call tomorrow and try to see why my insurance was cancelled.