Posted in Open Source | no comments 
James
I had an unusual Christmas Eve. While my wife was entertaining the family, I was in my office programming until nearly midnight. (Have I mentioned how much I adore my wife?!)
What had me so tied down at such an odd hour? One of the open source libraries Highgroove maintains: FasterCSV.
The Ruby core team gave me the nod to replace the standard CSV library with FasterCSV just before the Ruby 1.9.0 release. While a little more warning would have been nice, I was happy to do it.
I think it’s important to discuss why Highgroove, not just allows, but actually prefers that I spend some of my time maintaining our open source libraries, like FasterCSV.
The reason we created the library is nice and simple: we needed it. We import or export spreadsheet data at some point for most of our applications. We work with large data sets and we needed that to be fast. We also frequently use headers in the CSV files to map directly to database fields so a broader feature set to support such things was important to us.
Now we could have built it and just used it internally, but we decided to share. Did we do it because we are just really nice guys? Not really. (We are though, of course!) We did it because we wanted it to be great.
There are many subtle variations and edge cases for CSV. We debugged the library the best we could, but we’re human and we missed things. That meant our library would have trouble with some inputs. We (hopefully) would have noticed those eventually, but we’re just one set of users and by releasing it on the world we multiplied the user base by thousands. Needless to say early users found problems for us much faster than we could have alone. Better than that, they sometimes even fixed them for us! Even when they don’t, they usually send in an example that leads us straight to the problem which is almost as helpful.
That’s not all. Remember all of those features I told you we love? I wish I could brag and tell you I invented all of them, but some of my favorite features in FasterCSV were sent in by the users. I got the ball rolling with what I could see us needing, but the users ran with the ideas and made my library better than I knew it could be.
Highgroove isn’t losing hours on me applying a few patches here and there. They are gaining the help of many extra employees. We love that.
To top it all off, Matz gives us the ultimate Christmas present. By blessing our work, he saves us the trouble of even installing it for new projects. Very soon we will be able to count on our code being available in all modern Ruby installs. That’s just one less thing we need to worry about.
With open source everybody wins.
2 comments
Derek
I’ve been researching ways to consolidate bank account information into a single, automatically updating view. There are a couple of new web-based options – most notably Wesabe and Mint – but those have a consumer focus. I’m more interested in viewing trends in our business – a real-time financial dashboard.
Open Financial Exchange (OFX) Standard
In January of 2007, the Open Financial Exchange standard was announced by Microsoft and Intuit. Many financial institutions provide support for a portion of the standard, namely the portion used by Quicken.
The best list of OFX connection settings is here. A bank information generator is also available that retrieves connection data from the Microsoft Money website.
Retrieving OFX data
Once you obtain the OFX connection details for a financial institution, you can then connect to it to retrieve financial transactions. Getting to this point often isn’t straight forward. For many banks, the online login information isn’t the same as the data needed for the OFX connection. Some banks, like Wells Fargo, charge a monthly fee for OFX access. Finally, chances are your bank’s customer support line won’t know anything about OFX. Your best bet is to ask how to enable Quicken access (which uses the same OFX standard).
There are a couple of libraries available for retrieving OFX data:
- LibOfx is the library used by GnuCash to connect with banks online. Full documentation is available, but I didn’t find any tutorials for getting started.
- OFX for Ruby is similar to LibOfx, but obviously in Ruby. There are a couple of examples included in the library download.
In general, retrieving this data isn’t a straight forward process. OFX connection details are scattered, some banks haven’t made it easy to enable OFX, and there isn’t much information available on building an OFX client. It’s definitely primitive.
Additionally, OFX transactions can be cryptic – a large part of Wesabe’s value is in its ability to provide better transaction formatting.
How are others doing it?
Wesabe and Mint are 2 fairly new web startups that automate the process of collecting account transactions. How they collect the data is very different.
With Wesabe, you record a Firefox browser session, walking through the download process. You can then re-run the session and Wesabe then updates your account data using the script.
Mint provides the most seamless experience. You simply provide your bank details (even PayPal works) and it grabs your transactions. Mint uses “Yodlee” to actually connect with financial institutions. Yodlee has been doing this for a while – they have a service called MoneyCenter that is remarkably similar to Mint – it actually has more features but lacks Mint’s visual appeal. My guess is Yodlee charges a pretty hefty licensing fee.
The Future
I wouldn’t bet on banks making it any easier to automate data retrieval. Erick Schonfeld of Techcrunch recently wrote a post that covered the API landscape very well – which types of companies are most likely to offer APIs and why companies that push APIs are very careful with what they are actually opening up. Opening up a core component of your business can spur some pretty dramatic changes – it will take a younger, upstart bank to spur progress.
While it’s not here yet, I’d pay for a service that:
- Made connecting to financial institutions as easy as Mint (just a user name and password – no OFX connection details needed)
- Scrubbed transactions like Wesabe
- Published an API for retrieving this data that didn’t require a customer account (in other words, the ability to query for bank account transactions from an account not directly setup through the provider)
