Post#11 Reports

After spending a lot of time testing and fixing defects on the Transactions View and Tag page, I finally decided to move to Reports. Theoretically, its the last piece. You register, create accounts, upload statements, categorize transactions and then, finally you are supposed to see how you did over time.

This reports page, a basic one posted good challenges of moderate complexity. I prefer moderate complexity problems. They pick your neurons, and solving them gives you a sense of satisfaction.

Here is how I define the basic problem.

- The Report will always be 2 dimensional. The Columns represent time in months. As this is individuals we are dealing with, no quarterly or six-monthly. Each column represents a month.
- The Rows can be any of these
   - Categories
   - Accounts
   - Labels
   - Categories and Subcategories together

Personally I think my database design could be better. With reporting, I now need to convert rows to columns and columns to rows. For example, Months are columns (month1, month2 etc.). Another one, I have Labels in a row In query results I get transaction date in a single column. Now to accomplish this using sql query is cumbersome, to say the least. With passage of time and increasing number of records, such queries degrade fast.

To avoid this, I decided to do the entire transformation on the client side. Here is my philosophy. When I crank up my CPU, I pay for the cycles. I can decide to take up as much work as I want and then deliver only the final results to the customer. However, this costs me a lot. Why not leverage the browser power to do some work. Its distributed and its free. The laptop CPUs are fast and capable. js engines have really become quite efficient and its economical even from a bandwidth pov, to move minimum data and do much of the work at the client side.

So, I now send the query result over to the browser as json. From there on the JQgrid and javascript takes over. I render all the views with that data entirely on the client side, without making any round trip. The good thing is that I do not bring all the transaction data, I just get grouped data. The grouping is fine-grained. That allows me to create all 4 views on the client side.

Played a lot with JQgrid. Loved its features. It took me 4 days to reach where I am today. All these views are done. I have a big js function of 250 lines that does the work. Its neat. I like it. BTW javascript array.sort is one expensive feature. But then I need it...

iOS6

I downloaded the latest upgrade to the iPhone about an hour after it was released by Apple. I was wondering what's new and how my experience will change magically. A whole new world will open up and it'll once again be like Alice in Wonderland.
But no, the incremental improvements are like ms office upgrade. You hardly notice them.
The user interface has changed a bit in a nice way. The crescent moon on the top bar was keeping me mystified. Now I know it's DND. But overall it's been boring. It might also be because the existing experience is so good.
The biggest let down is the maps. Google did a far better job of it. Apple app can't understand any place outside of USA. When I asked it to locate Hard Rock Cafe in Mumbai, it was stumped. Same for any other place in India.
Poor choice by Apple. I understand that it at war with Google, but that does not mean you screw your customers. Thank god, Google chrome takes care of some of the angst. But this is sheer poor execution.
iTunes browsing experience / dial keypad color / music app palate have changed. Can't say it's better than before. Just different.
Facebook app is integrated. I had liked its revamped performance. Now you get a listing of all the apps that use Facebook data.
Passbook may be a cool thing in the US, but in India, its not much help.
In the mails, if I have to mark anyone VIP, he needs to be on my contact list. Its interesting. Maybe I'll find a use for it.
Several features are not available for iPhone 4 users such as Siri, Panoramic view, fly view of the city.

All in all, it does not change my world. Maybe its time to upgrade the hardware :)

Post#10 Txn Management and Error Handling

This was a tricky one. There are several ways of handling Transactions. I chose to use Annotation. It simplifies the coding a lot. With Spring Txn management is more about configuration. You could use a template and manage manually or you could manage through Txn: The challenge was largely how to setup transaction manager.
That led to revisiting the past and doing some cleanup. I decided on using Spring connection manager rather than jndi lookup. There was lot of duplication. I removed some of it as I cleaned the XML config files.

Then it was onto my favourite crib for about a decade. How to manage exceptions.
I had always wanted to manage the exceptions in a centralised fashion. Spring enabled me to do that beautifully. I wrote a Spring interceptor. It helps with instrumentation (basic) and also checks for error flag. If there is an error it would log useful info such as request params. The method arguments etc. all in one place. That way you do not have to write error handling code everywhere.
To centralise error handling I used AspectJ for Spring. Wrote an AOP @Throwafter annotation. It took a lot of time configuring it. But it was worth it.

I could suggest this to other program's too. I think it helps with debugging. I have wrapped up the initial implementation. It will of course evolve as we go along. To make it all work together I also leveraged ThreadLocal to store information.

Post#9 @Transactional

As mentioned in my last post, I've decided to take a break and do some clean up. There are 2 things I'm targeting for now.
- Transaction Management using Spring Annotation @Transactional.
- Exception Management. I'm thinking of doing exception management using Spring AOP.

I spent whole day yesterday trying to set up Spring Transaction Management. And what a hair puller it turned out to be. I thought I've grown wiser and better with previous 3 months of experience, but no. I made the same mistakes (spelling mistakes), just like my daughter does.

It was not that easy either. Got a lot of learning on setting up spring too. Like, you can't @Autowired a class if you use @Transactional because spring creates its own proxies. I must credit myself to be able to get to that point. That's not the most obvious piece.

There were other contentious issues too. I've been using jndi based datasource. I really could not set that up for use, so I fell back to org.springframework.jdbc.datasource.DriverManagerDataSource instead. I also realized, I had set up beans in multiple contexts. A lot of clean up is needed. Perhaps between now and end of September, I'll get these things sorted out. Further development such as reports and budgets will have to wait till Oct.

Post#8 Incorporating Feedback

Couple of weeks ago I hounded my wife to take the work in progress for a test drive. The idea was to see her reaction. She is not the most tech savvy person. But she knows enough to get by. She obliged and after spending good 5 min gave me some feedback. The task of uploading the statements was especially arduous. Once I have gone through the pain of uploading a statement for an account once the site should remember the column mappings and apply the same whenever I upload a statement for the same account. I have been working to incorporate the feedback which I finally finished today.

Other than that I corrected the issue with uploading the statement through signed in user URL. Uploadify was giving me problems integrating with Spring security. Every time I submitted the statement the Spring security would send me 302. It would redirect me to login.
Finally I worked around the problem by explicitly setting the upload URL at the time of upload button. This was fortuitous. I was trying to append the jsessionid to the URL but could not get to read the cookie using document.cookie. I still haven't resolved that issue but in trying that I resolved the upload problem so thats good.

Not sure what I should be doing next. I think I should do a clean up and some architecture related stuff before proceeding. The more I delay the bigger the cleanup.