I used sqlldr to import the file and ran into a few issues along the way so I thought I’d share my experience for a couple reasons: 1) to help me remember what I did, and 2) to help other poor souls who run into problems when faced with similar circumstances.

The first obstacle I faced was figuring out how to tell Oracle that the fields are tab-delimited.  (I always use the pipe as a delimiter because you are not likely to find pipes in your fields and there is no problem determining null columns.)  A quick search led me to an asktom.oracle.com thread that provided the answer: X’9′. Easy enough.  I imported the file.  It did not import any rows because the ordering of the columns was thrown out of whack whenever there was a null value in a field.  When the values shifted left, 20 character varchar2 values did not fit in 2 character columns.  Next I adjusted the sizes and types of the columns to see which data was being imported where.  All the rows imported, but the ordering of the columns was off whenever there was a null in a field.  This was a big problem because there were three columns with more null values than populated values.  I tried a number of different options in the ctl file that did not work, mostly centering around the NULLIF operator.  Oracle was never able to discern when a column was null as long as I had the line OPTIONALLY ENCLOSED BY ‘”‘ in the ctl file.  So I removed it and used the following:

LOAD DATA
INFILE “\SERVERNAMEFILEPATHFILENAME.txt”
REPLACE
INTO TABLE SCHEMA.TABLE
FIELDS terminated by X’9′
TRAILING NULLCOLS
(
ACSID,
EMAIL_ID,
FNAME,
MNAME,
LNAME,
FULL_NAME,
DIVISION_CD,
GROUP_NAME,
RG_SOURCE,
RG_DT,
EVENT_PARTICIPATION
)

All the data loaded in the correct fields.  Hooray!  One last problem: every field was enclosed with quotation marks so, in addition to being aesthetically displeasing, if I wanted to join this new table with any existing tables, I’d have a problem.  Removing the quotation marks was easy; I ran a simple update statement:

UPDATE TABLE
SET ACSID=REPLACE(ACSID,’”‘,”),
EMAIL_ID=REPLACE(EMAIL_ID,’”‘,”),
FNAME=REPLACE(FNAME,’”‘,”),
MNAME=REPLACE(MNAME,’”‘,”),
LNAME=REPLACE(LNAME,’”‘,”),
FULL_NAME=REPLACE(FULL_NAME,’”‘,”),
DIVISION_CD=REPLACE(DIVISION_CD,’”‘,”),
GROUP_NAME=REPLACE(GROUP_NAME,’”‘,”),
RG_SOURCE=REPLACE(RG_SOURCE,’”‘,”),
RG_DT=REPLACE(RG_DT,’”‘,”),
EVENT_PARTICIPATION=REPLACE(EVENT_PARTICIPATION,’”‘,”);

Mission accomplished.

• VPN to connect to the office network
• meebo for instant messaging
• gmail
• google calendar
• google reader

I have tried using Internet Explorer 7, Safari, Firefox, Chrome and Flock. It is kind of annoying but I don’t mind needing to have two browsers open: IE7 for VPN and then another browser to use for everything else.  Most of the time I just use my laptop as a terminal to get to servers at my office anyway.  What irritates me and leaves me unsettled is the fact that I cannot get a rather ordinary set of sites/services to cooperate in one browser.

Negatives

• The VPN client I use for work (Sonicwall Aventail) does not work in Flock or Chrome.
• The auto-scroll stops working in meebo chat windows in Firefox and Safari (probably Flock too since it is built on Firefox but I have not confirmed).
• In Chrome, the icon that indicates a refreshing tab is in constant motion on the meebo tab.
• Chrome crashes too frequently with moderate to heavey use, at least every other day.
• IE is slower and clunkier than the other browsers in terms of opening tabs and navigating through sites and it is more vulnerable to malware.  It is also uglier.
• Safari is a memory hog that slows down other applications.
• The longer Firefox stays open, the slower it gets (memory leak?).
• Flock’s RSS reader is feature-poor.

Positives

• In Firefox, meebo messages are previewed in the tab header–a very nice feature.
• Firefox extensions like FireFTP and Twitterfox are very useful.
• My VPN client is configured for IE.
• Flock’s integration with social networking sites like facebook, twitter, and youtube make it easier to stay up-to-date with friends.

Dream Browser
My dream browser would be as fast as Chrome and as elegant-looking as Safari.  It would not require another open tab for instant messaging but rather would integrate IM like twitterfox integrates twitter into Firefox.  My work’s VPN client would work in it. Its RSS feed reader would have all the rich features of Google’s Reader, but again integrated with the browser rather than requiring an open tab.  It would never crash and not get slower over time.  This, I dream.

Blogged with the Flock Browser

This show what parts of the brain that were dominant during writing.

ISTP – The Mechanics

The independent and problem-solving type. They are especially attuned to the demands of the moment are masters of responding to challenges that arise spontaneously. They generelly prefer to think things out for themselves and often avoid inter-personal conflicts.

The Mechanics enjoy working together with other independent and highly skilled people and often like seek fun and action both in their work and personal life. They enjoy adventure and risk such as in driving race cars or working as policemen and firefighters.

Hat tip: Greg Mankiw

The two things I find most interesting are where visitors are geographically located and what search terms lead people to ahelms.com. Most of the searches are from recruiters looking for data architects/analysts, but people also love a good car crash as evidenced by the different variations of searches looking for bicycle accident photos: bad bycicle accident pictures, bicycle crash photos, photo of cycle accident, photos of mountain biking accidents, mountain bike accident photo

….and graphical analysis of rap music: analysing rap lyrics,rap lyrics analyse, rap lyrics analyzed, rap lyrics and analysis, rap lyrics in graphical form, rap lyrics to do a critical analysis on, cool soccer rap lyrics, graphical interpretation of rap lyrics, graphical rap, graphs rap lyrics, lyrical analysis rap, lyrics analysis rap, soccer rap lyrics

Seemingly irrelevant search terms often amuse me.  Exhibit a: black man growing a beard – don’t think ahelms.com will help that searcher.  Then there was a person using a computer in Mauritius, an island nation off the coast of the African continent in the southwest Indian Ocean, about 560 miles east of Madagascar, who was certainly disappointed after using the search phrase how do rainbows make life enjoyable and happy and ended up on ahelms.com.

Sometimes I feel real pity for searchers, like the people who searched using the phrase my rootcanal is hurt and root canal local ansthesia didn’t work.  Ouch!  Or for those people who are perhaps aspiring to grow an awesome beard like I did and quite possibly are trying to find someone to comiserate with: keep trying ye searchers who googled little beard growing, little beard under lip, and you’ve got beard.  Perhaps the saddest example is the person who searched for unattractive wife–what was that searcher trying to accomplish? Sorry, can’t relate there.

Then there are the searches that are a bit personal and hint at my growing legend: worst passport photo ever and andrew helms kentucky soccer are two such examples.

Nice to see you, glad you found me.

This morning I received an email from Dropbox that invited me to download and install its python-based desktop client for online storage. The client creates virtual folders that automatically sync to an Amazon S3-backed storage ‘facility’, what TechCrunch author Mark Hendrickson esoterically calls ‘the cloud’. It was his blog post last month that motivated me to register with Dropbox. In my first day using the app there are two things that I find really cool: (1) it is wicked fast and (2) the ability to upload a file to a public space and then right-click the file to get a public link is a simple way to share files with multiple machines or friends. In fact, the Dropbox image in the header of this post is hosted in a public Dropbox folder which can also be accessed here in a photo gallery format (Dropbox provided the three stock photos). It sure beats emailing files or uploading everything to my ahelms.com server.

I had the privilege of spending three days at The Data Warehousing Institute‘s (TDWI) 2008 World Conference in Las Vegas this week.  The event runs Sunday to Friday; I attended Monday to Wednesday.  TDWI puts on a first-class event.  The main reason I went and what I most enjoyed about the three days was the opportunity to learn from experts in the courses that were offered.Each day I went to a day-long data modeling course: the first two courses were more high-level, basic courses–one on modeling, one on dimensional modeling–taught by consultant and author Steve Hoberman; a more advanced dimensional modeling course was taught by Laura Reeves, principal at StarSoft Solutions and co-author of The Data Warehouse Lifecycle Toolkit.  Steve and Laura both have over 20 years experience as data modeling practitioners.

Monday’s class, TDWI Data Modeling: Data Analysis and Design for BI (Business Intelligence) and Data Warehousing Systems, was a great primer for the classes that followed.  It was mostly conceptual, and provided a framework from which we could dive into more practical topics later in the week.  Steve repeatedly stated that with enough time and money we have the ability to model anything, including world peace.  We went through the progression of modeling, from contextual to conceptual to logical to structural to physical.  The time we spent discussing the differences between dimensional modeling and relational modeling was helpful.  It has been my experience that it is difficult to satisfy a business unit’s requirements by implementing a purely dimensional model:  there is usually some need to replace a list that is generated by whatever current system we’re replacing.  Steve acknowledged that most marts are indeed hybrids and that there is usually cause to break the ‘rules’ of modeling to meet a business need.

Tuesday’s class, TDWI Dimensional Data Modeling Primer: From Requirements to Business Analytics, was very practical.  It presented a specific methodology to follow to build a dimensional data mart.  The main takeaway for me was the necessity to spend time probing the business and logically modeling a solution before moving into the physical.  My tendency is to let the business work out the requirements with a business analyst from IT who translates the requirements and then to prototype solutions and then iteratively improve it until the business is satisfied.  Handing off the dirty, not so fun requirement gathering work is very tempting but not a good habit.  I am currently taking an MBA course called Software Requirements Management and it is only reinforcing my distaste for the whole process.  Conversely, Tuesday’s course underpinned the value of asking why questions and of truly understanding what is trying to be accomplished rather than simply what the requirements are.

The final class I attended, Dimensional Modeling beyond the Basics: Intermediate and Advanced Techniques, was very practical.  The course instructor, Laura Reeves, repeatedly emphasized that modeling is an art and there is no one right answer.  If something works, it works.  There was a ton of experienced technologists in the room, many who wanted free consulting from a subject-matter expert.  The topics covered–Multiple Roles, Complex Hierarchies, Date and Time, Many-to-Many Relationships, Fact Table Design, and Aggregation–are all stumbling blocks modelers seem to encounter on even the most straight-forward projects.  Laura drew on a bevy of experience and was willing though sometimes embarrassed to share the dirty tricks she’s used to implement a solution.  I was at least nominally familiar with most of the concepts presented because she is a co-author of the reference book I use when faced with challenges.

The three days were extremely draining.  In the evenings when the vendors were putting on ‘exhibits’ with free booze, go-go dancers, Elvis and Mike Myers look-a-likes, all I wanted to do was sit in front of the television and zone out.  TDWI does a wonderful job ensuring that the courses are sales-pitch free and that the vendors are separated from the educational area.  If you want to speak with vendors, there is ample opportunity.  Equally comforting is the ability to avoid them altogether.  I spent some time watching new product demos and reading the latest literature, but steered clear of whoring my contact information in order to enter contests for gift cards and cool electronics.  My favorite contest was for the Sony Wii: I wonder what Nintendo thinks about that.

Now it is back to reality, back to the grind.  Armed with my new knowledge I can surely do some damage.

I’ve been captivated by the presidential nomination process and have been consuming all the information I can handle relating to the debates, speeches, endorsements, posturing, etc. The polarizing, counter-productive nature of American politics bothers me but I make it a point to try to stay informed. So as not to alienate or terribly offend anyone, I subconsciously adopted a common, practically failsafe method of engaging in political conversations with co-workers and acquaintances: never take a side, always play the cynic. It’s really a cowardly, easy out. In Good Will Hunting, Sean (played by Robin Williams) confronts Will (played by Matt Damon) and his similar approach to women: “I think that’s a super philosophy, Will, that way you can go through your entire life without ever having to really know anybody.” Don’t worry, I’m not going to address my cowardice here.

CNN put together a cool tool called Political Market that is right up my alley. According to CNN, the “goal of CNN Political Market is to combine the opinions of a diverse group of people to try and predict the probability of an event occurring or the value of something”. I don’t have to take sides; I just have to predict outcomes. I created an account today and bought shares. A couple of widgets that track performance on the answer to a question I am particularly interested in are pasted below. If you’re inclined give it a shot.

I really enjoy browsing Google’s annual Zeitgeist lists. What is a Zeitgeist list?  Google’s explanation:

To get a glimpse of what’s been on our collective consciousness, we mined billions of search queries to discover what sorts of things rose to the top. We encourage you to check out our findings to see if you, too, reflect the zeitgeist — the spirit of the times.

With a brief glance at the list, I’m not sure it is obvious what is on our collective consciousness. I was excited because I was not familiar with a few of the terms and thought I was in for a cool, learning experience. The excitement was tempered, however, when I discovered that the 3rd fastest rising term tmz is a website that provides an inside look at the latest celebrity scandals. Blech. So it seems Americans love using the Internets to do four things: the other three are social networking [webkinz, youtube, club penguin, myspace, facebook], learn celebrity gossip [tmz, anna nicole smith] and (window) shop [iphone]. A look at the global list reveals that global googlers have similar interests as there is a lot of crossover in actual terms and content [badoo and hi5 are social networking sites, dailymotion is a youtube competitor].

I’m not very interested in the newsmakers and showbiz segments because they are dominated by gossip although I will point out that each of the television shows in the top 10 are one-term shows [hereoes, lost, house, 24, bones, jericho, reba, scrubs, greek, caveman]. Wow, Reba? My favorite segments are the All the Rage and Top of the Mind segments. Google plots somewhat similar terms on the same group so you can easily compare the popularity of search terms.

My favorite

On the who is list ‘god’ is first, followed by ‘who’, ‘lookup’, ‘jesus’, ‘it’, ‘buckethead’, ‘calling’, ‘keppler’, ‘this’ and ‘satan’. That is kind of comforting.

Friending is a topic that is discussed ad nauseum on blogs and in newspapers and magazines. The online space certainly adds a new dimension to relationships.  I’m not so popular in the Robert Scoble, Mark Cuban sense that I have the problem of reaching limits.  I would say that I value quality over quantity but frankly speaking most of my friends are of questionable quality.