Quoth the Johns:
This song was written with childhood friend Mr. Matthew Hill. ... The lyrics are as factual as we could make them with the reference books handy. James Knox Polk, the 11th President of the U.S., was a dark horse candidate who unexpectedly won the Democratic nomination and the election based on his popularity in the South with his stated goal of annexing Texas, the Southwest, and the Oregon Territories. Once in office he fanned the flames of dispute between the U.S. and Mexico to achieve part of this aim. (The Mexican War is still commemorated in the expression "Remember the Alamo!") Personally, we find his expansionist policies ruthless and unscrupulous, but the existence of the Western U.S. is largely due to him. The spooky sound halfway into this recording is a "singing saw," an actual metal saw stroked with a bow by Mr. Julian Koster.
John Linnell and John Flansburgh
[source: "A Guided Tour of Factory Showroom," TMBG Info Club Newsletter, Holiday 1996 edition]