Post#16 Done With Budget Set Up

I've completed the basic Budget Set Up and reviewing the performance against budgets. Currently, its completed with Reports, it still needs to be integrated with Dashboard Graphs. But I've decided to keep it aside for the time being.

I'm preparing myself for hosting. For that, I need to make some general pages, such as hmmm.... Home Page!

I'm really bad at visual design. I admit it. Hence, my attempts at creating a home page came a cropper. None the less I soldier on. There is no expert help available here. Its a one man show. So, it will embody all my shortcomings... That's just the way it is.

Once I'm done with these generic pages, I still have to work a little on consolidation, commenting the code and clean up before I go live.

The biggest nuisance will be making it compatible with IE8. Currently, IE8 breaks the application on most pages... This is going to be a headache.

I expect a bunch of defects to roll in as and when anyone other than myself tries to use it. Also, a flood of feedback. To track all that I think I need some system. I think I need to write a small usecase to track these.

Another disheartening thing struck me while I was writing the "How it works" section. It sounded like a lot of work to upload a statement. I always thought of it as a challenge but writing about it made me feel like it was a huge mountain to climb and most people will give up halfway. The value comes only if we get over the hump, and it sounded like crossing the Alps...  Sigh!

In the long term, I think I'll be the only one using it. But I'll use it anyway... I built it for myself, right?

Post#15 Managing Rules

When I tried uploading my actual statements, I realised I needed a way to edit the rules. You create rules based on your immediate needs and you try to make them generic. Unfortunately over time a different type of transaction comes along but gets caught in generic rule. One needs a way to analyse and correct this. Hence rules review is like a cleanup activity one needs to perform from time to time.

Interestingly, given the size and complexity involved it was completed surprisingly fast. Reuse and experience can improve productivity by a big factor.

Now for the big one. Creating custom reports. It's the same as creating your budget goals and tracking them over time. This is the one I've been moving towards all this while. It's now time.

Running On...

I've decided to give up my Gold Gym membership. Not just because its expensive, which it is. Its because, a friend of mine got annual membership for Rs. 8500 at Andheri. I'm Rs14K. The difference is too huge. These days, I've stopped going there for weight training. I only use the tread mill. My Community Gym is well equipped. I do not do heavy weight training and hence no need for extra sophistication . I told this to the service guy, when he called up for renewal. I asked him for Rs.10K. I thought it was a fair offer, he reduced to 13K, I decided to give it up.

I'll miss the running sessions. I liked the treadmills there. Not because they have TVs. I never watch TV while running(even at home I don't)... But they are very stable and sturdy. They are broad and have nice cadence.

My gym has treadmills too. But they are a little unstable. Also, there is no AC. It gets a little stuffy...

My membership will end this month. From now and end of the month, I'll use it to the hilt... Even today, I did   the entire half marathon. In fact 21.3Kms and did it in my personal best time on treadmill :) 115 min. If only I could keep it up, I sorely want to finish this time in less than 2 hours.

From Nov, I'm hitting the road. Lets see how that. My biggest concern is my knees. Even today they hurt a bit...

Post# 14 Wrapping Up Reports / Dashboard

I'm on a High... I've used Mint.com and I'm a subscriber to ICICI Money Manager. Mint.com has a gorgeous interface and some awe inspiring graphics. I give it to them, it fetches all the transactions and categorizes them for you.

But then they fall short. Lets take a couple of situations...
- I made some big ticket purchase in March. I want to consider it for my annual expenses, but do not want to to screw up my monthly averages.
- I went out with friends. We split the bill. I paid using cc but my friend reimbursed me his share.
- There are some annual purchases I make. I want to Tag them. Not just categorize them per category.
- I withdrew cash and made some cash payments. Especially in India 40% of the payments are in cash. I want them squared off against my cash withdrawals.
- I have some recurring cash payments every month. I don't want to enter it every month
- My FD matured, I want to split that into principal and interest. Principal should be ignored from calculations, while the interest should be counted towards income

All of this is now possible in my site. And it has Google Visualization Graphics. Though not as stunning, they are pretty nifty..

The best part was the client side script I wrote. I get grouped amounts to the client side and then render the data in various views at client side itself. The graphics take time to load from Google along with the data. Once loaded the views are created in a jiffy... I love it.

Now to take it live...
I'm gunning for Diwali. Need some major clean up activities before that. Hope to close it out before that.
Mind you, even with all I've achieved, its only 20% of what I want. This will keep my busy into 2013...

Post#13 Dashboard and Graphs

Having done the reports I've moved to the dashboard. Conceptually these are similar but the dashboard will eventually have a lot more features. Especially when I implement goal setting etc.

For graphs I'm using google visualisation API. It's a great tool to use and I'm only beginning to learn it. I will not use much of its muscle power. I'll use bar chart and pie charts mostly.
Hope to use dashboard tools too to reflect trending.
I've already completed the barchart and working on the pie chart. I plan to host it once this piece is done. Probably close to Diwali.

Post#12 Incremental Continued...

Incremental has an amazingly long tail... While in Bhopal, I did make some progress, but nothing new to show. Just a bit of stuff here and there. Here is what I did
- Discovered spring API to handle exceptions. Much better than the interceptor I wrote. The problem was, the exceptions did not bubble upto the interceptor if I just let them keep going up... So here comes a little help from Spring org.springframework.web.servlet.handler.SimpleMappingExceptionResolver. Nifty little helper. All I did was to extend it and it did exactly what I wanted from my exception handler, which is if its a runtime exception, don't catch it till it reaches the top. I tied this up with AfterThrowing advice, which collated all the important information about method arguments and logged it all together in one place. It worked. I then went around removing try-catch from all the DAOs. At the end of it, I now have a custom 404 and error page along with some useful logging.

- Improving on jqgrid. Though I did get most of the information to look okay, there were some pieces, that did not look okay. So spent some time cleaning up JQ grid. For example, put the navbar on top as well as bottom. Putting "View x of y" etc.

- Changed the Tag Transaction Page to show actions only when you check on any of the loaded transactions.This proved tricky. It's true. If you have a page that does a lot of stuff, you will go crazy untangling the permutations. It seems to have stabilized for now. But I'm sure, it'll change again.

- I also introduced modal for loading. I used jqueryui dialog for the purpose. For the loading icon, I found a very useful site http://ajaxload.info/. Truely useful. My need is simpler than what dialog offers. Let me see how I can strip it down.

Bhopal and its irresistible charm

I was away for about 4 days. I had gone to my native place. And it never ceases to surprise me. 4 days of decadence. Landed on Friday evening. I was handed a bottle of beer just as I entered the car. One for the road... or to kick start an extended weekend... I usually prefer the train but Friday was my nephew's birthday and so I decided to fly instead.
At the birthday party, I was welcomed with a round of neat Whisky(Chivas), just to set the tone. And thus the party continued. After wrapping up the party @midnight, I drove to another friend's.  Back to beer. But let me confess. This place had the best view of the city. Absolutely awesome. Its a residential house converted into a Service Apt. I was just blown away by the view. At night, you could see the entire shoreline lit by streetlights the interconnect road. It contours the shore for good 5 kms. Here are a couple of shots.

Day time and a night time scenery for posterity...

Post#11 Reports

After spending a lot of time testing and fixing defects on the Transactions View and Tag page, I finally decided to move to Reports. Theoretically, its the last piece. You register, create accounts, upload statements, categorize transactions and then, finally you are supposed to see how you did over time.

This reports page, a basic one posted good challenges of moderate complexity. I prefer moderate complexity problems. They pick your neurons, and solving them gives you a sense of satisfaction.

Here is how I define the basic problem.

- The Report will always be 2 dimensional. The Columns represent time in months. As this is individuals we are dealing with, no quarterly or six-monthly. Each column represents a month.
- The Rows can be any of these
   - Categories
   - Accounts
   - Labels
   - Categories and Subcategories together

Personally I think my database design could be better. With reporting, I now need to convert rows to columns and columns to rows. For example, Months are columns (month1, month2 etc.). Another one, I have Labels in a row In query results I get transaction date in a single column. Now to accomplish this using sql query is cumbersome, to say the least. With passage of time and increasing number of records, such queries degrade fast.

To avoid this, I decided to do the entire transformation on the client side. Here is my philosophy. When I crank up my CPU, I pay for the cycles. I can decide to take up as much work as I want and then deliver only the final results to the customer. However, this costs me a lot. Why not leverage the browser power to do some work. Its distributed and its free. The laptop CPUs are fast and capable. js engines have really become quite efficient and its economical even from a bandwidth pov, to move minimum data and do much of the work at the client side.

So, I now send the query result over to the browser as json. From there on the JQgrid and javascript takes over. I render all the views with that data entirely on the client side, without making any round trip. The good thing is that I do not bring all the transaction data, I just get grouped data. The grouping is fine-grained. That allows me to create all 4 views on the client side.

Played a lot with JQgrid. Loved its features. It took me 4 days to reach where I am today. All these views are done. I have a big js function of 250 lines that does the work. Its neat. I like it. BTW javascript array.sort is one expensive feature. But then I need it...

iOS6

I downloaded the latest upgrade to the iPhone about an hour after it was released by Apple. I was wondering what's new and how my experience will change magically. A whole new world will open up and it'll once again be like Alice in Wonderland.
But no, the incremental improvements are like ms office upgrade. You hardly notice them.
The user interface has changed a bit in a nice way. The crescent moon on the top bar was keeping me mystified. Now I know it's DND. But overall it's been boring. It might also be because the existing experience is so good.
The biggest let down is the maps. Google did a far better job of it. Apple app can't understand any place outside of USA. When I asked it to locate Hard Rock Cafe in Mumbai, it was stumped. Same for any other place in India.
Poor choice by Apple. I understand that it at war with Google, but that does not mean you screw your customers. Thank god, Google chrome takes care of some of the angst. But this is sheer poor execution.
iTunes browsing experience / dial keypad color / music app palate have changed. Can't say it's better than before. Just different.
Facebook app is integrated. I had liked its revamped performance. Now you get a listing of all the apps that use Facebook data.
Passbook may be a cool thing in the US, but in India, its not much help.
In the mails, if I have to mark anyone VIP, he needs to be on my contact list. Its interesting. Maybe I'll find a use for it.
Several features are not available for iPhone 4 users such as Siri, Panoramic view, fly view of the city.

All in all, it does not change my world. Maybe its time to upgrade the hardware :)

Post#10 Txn Management and Error Handling

This was a tricky one. There are several ways of handling Transactions. I chose to use Annotation. It simplifies the coding a lot. With Spring Txn management is more about configuration. You could use a template and manage manually or you could manage through Txn: The challenge was largely how to setup transaction manager.
That led to revisiting the past and doing some cleanup. I decided on using Spring connection manager rather than jndi lookup. There was lot of duplication. I removed some of it as I cleaned the XML config files.

Then it was onto my favourite crib for about a decade. How to manage exceptions.
I had always wanted to manage the exceptions in a centralised fashion. Spring enabled me to do that beautifully. I wrote a Spring interceptor. It helps with instrumentation (basic) and also checks for error flag. If there is an error it would log useful info such as request params. The method arguments etc. all in one place. That way you do not have to write error handling code everywhere.
To centralise error handling I used AspectJ for Spring. Wrote an AOP @Throwafter annotation. It took a lot of time configuring it. But it was worth it.

I could suggest this to other program's too. I think it helps with debugging. I have wrapped up the initial implementation. It will of course evolve as we go along. To make it all work together I also leveraged ThreadLocal to store information.

Post#9 @Transactional

As mentioned in my last post, I've decided to take a break and do some clean up. There are 2 things I'm targeting for now.
- Transaction Management using Spring Annotation @Transactional.
- Exception Management. I'm thinking of doing exception management using Spring AOP.

I spent whole day yesterday trying to set up Spring Transaction Management. And what a hair puller it turned out to be. I thought I've grown wiser and better with previous 3 months of experience, but no. I made the same mistakes (spelling mistakes), just like my daughter does.

It was not that easy either. Got a lot of learning on setting up spring too. Like, you can't @Autowired a class if you use @Transactional because spring creates its own proxies. I must credit myself to be able to get to that point. That's not the most obvious piece.

There were other contentious issues too. I've been using jndi based datasource. I really could not set that up for use, so I fell back to org.springframework.jdbc.datasource.DriverManagerDataSource instead. I also realized, I had set up beans in multiple contexts. A lot of clean up is needed. Perhaps between now and end of September, I'll get these things sorted out. Further development such as reports and budgets will have to wait till Oct.

Post#8 Incorporating Feedback

Couple of weeks ago I hounded my wife to take the work in progress for a test drive. The idea was to see her reaction. She is not the most tech savvy person. But she knows enough to get by. She obliged and after spending good 5 min gave me some feedback. The task of uploading the statements was especially arduous. Once I have gone through the pain of uploading a statement for an account once the site should remember the column mappings and apply the same whenever I upload a statement for the same account. I have been working to incorporate the feedback which I finally finished today.

Other than that I corrected the issue with uploading the statement through signed in user URL. Uploadify was giving me problems integrating with Spring security. Every time I submitted the statement the Spring security would send me 302. It would redirect me to login.
Finally I worked around the problem by explicitly setting the upload URL at the time of upload button. This was fortuitous. I was trying to append the jsessionid to the URL but could not get to read the cookie using document.cookie. I still haven't resolved that issue but in trying that I resolved the upload problem so thats good.

Not sure what I should be doing next. I think I should do a clean up and some architecture related stuff before proceeding. The more I delay the bigger the cleanup.

Post#7 Drop Down Menu

I've been thinking about what to take on next. Of the various options available I took on creating a Drop Down Menu. It took me a while. One of the things I'm really bad at is UI design. Not for the tags etc. but the colors. I truly appreciate the challenge of a great UI. Mint.com has a great. UI. It takes a lot of creativity to design a great Drop Down Menu. It might look deceptively simple, but it's not. And it's not the JavaScript. It's the interface.

I did not want a Hover drop down menu which a lot of sites sport. In fact many a sites have completely done away with it. They show simple Tabbed menu. You click and get to where you wish to go. But lately, in the last 3 years I've seen a lot of sites create monster menu. In fact I love it in some sites. Hence I decided to go with it.

It's tough to use that much real estate effectively. I soon realised. I tried to create a help on the features but it too small for effective help and too big for showing just a couple of links.

Also the color scheme is tough to nail. I spent 2 days just playing with the palate. Finally I came up with some hideous colors.

Another challenge is usability. The main hover menu may be on the right but the sub menu options are on the left. When I moved the cursor across the menu itself changed as soon as the mouse moved over the adjoining tab.

One of the options I tried is to leave some blank space at the top do that when you move the mouse you do not go over the neighbouring tab. It made the menu look so empty, it sucked.

I finally created a small timer function to circumvent the issue. Now the menu would change only if your mouse stays over the link for more than 500ms.

I liked the idea that implemented something that does not ruin the experience. No empty spaces needed.

I'm done with Drop Down stuff. So from tomorrow it's back to the transaction processing. I'll start with Label Creation and then move to user registration

Post#6 Completing Rules and Split

Finally I finished the first cut implementation of the Rules, Split Section. Its basic development. Lots of alternate scenarios and validations are missing. But its largely done. Now you could
- Create a new rule based on a given transaction. Currently its based on "contains".
- You could classify individual transactions(single or multiple). Ignore them from calculation, Assign Labels and or apply Categories.
- You could Split a Transaction into multiple ones.
- You Could create new Labels
All of this using JQgrid. It was fun and a lot of learning.

At times I was surprised at how I was able to diagnose a problem. This is veteran me. For example, realizing that the reason the function is not called is because I create these tags dynamically and these are not registered with jQuery. There were a few others, I'm forgetting now. But essentially, these were things that were not obvious. Good job me!

This probably brings me to completing about 10% of the overall functionality. I'm not sure what I should take up next. Probably, reporting. Thats an important piece. Let me look around for any charting tools I could use. Learn a new thing again :)

I tried accessing the site from one of my other home. It worked fine. I tried viewing it on my iPhone, Ah it sucks :(. Its slow. I think I'll need to remodel it so that it can be viewed properly on a smartphone.
That of-course will take till end of 2013.

One more thing I wish to do. I wish to do a Function Point Analysis of the app to size it. I have my company standards to estimate. Lets see what its worth.

There Goes My Back Again

Is it age or is it some kind of weakness in my back. This is the second time this year my back has given up on me. It happened in April and now again. It's the exact same location as before.

The first time it happened in 2007 My doctor told me to reduce weight and exercise. I did both and yet. Still planning to go to office. Hope it does not worsen. Last time I was laid up for 2 days.

IPhone revival

So after living with a paralysed iPhone for about a month I finally got it fixed. What a relief!
Essentially it's the sweat from my weekends running that brought about the disaster.
First the home button went. Then came the silent killer. The speakerphone died. No ringing and no speaker.

Then came the eerie voice control app. The phone automatically went into the voice control mode randomly arbitrarily anytime of the day or night. It would pick up ambient sounds and decide who to call. A couple of times, once at around 10:00pm it decided to call one of my women office colleague. It was embarrassing.

I read up the net and found out that putting the phone in a rice sack can help. It would dry up the moisture. I did that and it helped. The home button started working but only for sometime. In a few hours it went bad to bad old habits.

Finally I decided it was time to visit the doctors. I took the phone to Maple the authorised apple store. They said they don't open the phone. They simply replace a faulty one with a new one. Essentially for Rs13,500 they would swap my phone for a brand new iPhone 4. The offer was tempting but then I decided to visit a local repair shop. The guy immediately caught on and confirmed it was water damage. In fact he said it looks more like sweat than water. The charging plate had corroded. He changed the home button, speakerphone circuit and the charging plate all for Rs 1500. Ain't that something!
So finally after a wait of about 1 hr I for my phone back. All like new.
I'm writing this post on my all rejuvenated iPhone. Yooohooo!

Post#5 Managing Transactions through JQGrid etc

The JQGrid integration was tough, but I could get most of it. There are still some niggles but largely the data is coming up fine.
The big challenge is around usability. Once the user lands on the transactions page and views the transactions / some classified based on prior rules, some not, it becomes cumbersome.
There are so many things he can do with this data. For example,
- He could filter on Unclassified Transactions
- He could then multiple actions

• Ignore Txn
• Assign one or multiple labels to the transactions
• Classify Txns
• Create new rules and Labels and then apply them on transactions.
• Split any transaction

The transaction page is the gateway. How do I get all of these on a single page or weave them into navigation without confusing the user or pissing him off.
I'm working on the actions bar at the top, but its not coming out nicely.

I've learned about mouseover menus, buttons with images. All great stuff. Now, how to make it stick together....

I plan to finish this usecase by the end of this week. I'm taking couple of days off and that should certainly help.

After that its reports. That will be interesting too, I'm sure...

In terms of consistency of design and implementation, I think I need to make 2-3 passes before I can make this production ready. Hope I'm good with year end tgt.

Post#4 Completing Transactions Upload

Just about done. Great Experience. Learnt about jquery, uploadify and a few finer points in spring. I still need to establish spring observation i.e. if in the middle of a navigation, you switch urls from restricted access to open and back, you may lose some variables you put in session. Not sure, but I think that's happening. I'll check that out. Also, could not post to restricted access url from uploadify. Will have to check on that too.

The trouble is when I'm are up against a timetable, I do not have time to validate learnings. Thats bad. This is as much a product as a learning opportunity. I've to learn to balance the two.

So now I move into the terrain of jqgrid. From what I've seen, its a great product. I'll have to learn to use it effectively. Another month or so...

On a side note, my iPhone miseries continue. Its constantly going to voice control and dialing out to various people. Its embarrassing. I've put in a passcode. That's taking care of dialing out part for now. But its also coming in the way of working. Yesterday, I was on a skype call and it kept dropping due to voice activation. Bad iPhone...

Post#3 Making Incremental Progress

I could finally achieve closure on Accounts Set Up use case last week. It was grueling. For a long time jqueryui combobox did not work on ie 8. I tried various ways moving things around but it would not. Finally I gave up.

There was a lot of development that happened on that page. I split it up into common code vs use case specific. After moving them into their respective files, I tested again, to double check everything was okay, and lo! it worked in ie8. Don't ask me how but it did.

I've since moved on to the next use case. Upload Transactions. This was the most fun when I implemented it the first time. There is a free utility called uploadify. An awesome piece of work. They help you with uploading files. In the past my biggest challenge has been identifying the size of the file being uploaded beforehand. HTML does not allow for it. Hence you end up uploading a 1 mb document, and then the server reports that the size is too big.That's dumb and it also a wasteful use of server resources. Uploadify uses flash to read the size of the file and warns you if the file is bigger than allowed limit.


I did struggle a bit with configuring Uploadify but the documentation is good. The product is still evolving. I had 2.4 version from last December which I upgraded to the current 3.1 version and it solved my problem. 5 stars to Uploadify. They have made a real contribution to the user experience. I'd like to donate money to them. If I ever go live with this, I'll pay up.


Late nights have been hurting my exercise routine. For the last 2 days, I've been sleeping late and getting up all groggy in the morning. Yesterday night, Tomcat crashed. Its the second time. Fortunately, this time before reinstalling it, I took back up of web.xml, server.xml and context.xml. Saved me the effort of re-configuring the data source. 


Weekend looks busy. I have a late night birthday party to attend. Tomorrow, I have to take my daughter and her friends to their weekly class. On Sunday, I'll try running 18 kms. That will screw up the rest of the day for sure. So going slow. But making progress....

iPhone Disaster

When I run I sweat.... A lot!. I have an arm band to hold my iPhone and I wear it during my weekly long runs. Today however, was a bad day. Somehow the sweat leaked in and my iphone conked off.  Whatever I did it would not come up. I felt horrible. I had damaged the phone. I had bought it last year in may, and since have absolutely loved it. This was a disaster. I was not prepared. This was not happening to me.

I frantically searched for a solution. I read up on the net. There were some really useful tips. Based on a few of these, I wrapped up the phone in paper napkin and put it inside the rice jar. The article said it sucks out the moisture. And it did. After 4 hours when I pressed the power button, it sprang back to life. I was relieved ecstatic only to find out that the home button did not work. Ouch!

Some more frantic searching. After a while I found the solution. I went to settings->general->accessibility->assistive touch and turned on the feature. Its amazing how it saved the day for me. The home button is gone for good, but the phone still works. Its a little inconvenient to get to home, but its bearable. Hope I can keep it safe for another year before I upgrade. Hope iPhone 5 will be out by then. I have the perfect excuse to upgrade. I just need to keep this going till then.