Spaces, we don't want no stinking spaces

Dropdown List anomaly in asp.Net

For the first time in ages, I was stumped by what seemed like a trivial issue with formatting and a pesky ampersand.  The problem was that when users selected an item from a dropdown list that had a & character in it.  The result was a rather unfriendly validation error thrown by asp.Net.

Invalid postback or callback argument. Event validation is enabled using '<pages enableEventValidation=“true”/>'

Now there is a very easy solution to this, and it involves setting the <pages enableEventValidation=“false”/>', but this wasn't optimal as it would leave the page open to cross site scripting attacks.  I set about trying to HtmlEncode the postback value to escape the troublesome characters.  In hindsight this was never going to work, but it did lead me down another path.  After some googling to old blog posts and a few examples on stackoverflow (mainly revolving around Javascript) I thought I'd found the answer I was after.

protected override void Render(HtmlTextWriter writer)
{
    //register the items from the dropdownlist as possible postbackvalues for the listbox

    foreach (ListItem option in ddAssignedTo.Items)
    {
        Page.ClientScript.RegisterForEventValidation(ddAssignedTo.UniqueID, option.Value);
    }


    base.Render(writer);
}

This still didn't work, and I was about to admit defeat and just set the EventValidation to false.  I stepped through the code to check what values were being sent back to the page and I was quite shocked.  The Values being posted back didn't match the values in the dropdownlist.  It was a subtle difference but the Dropdownlist had removed additional spaces from the raw data.

At first I thought it was some funky javascript one of the front end guys had put into the code but it turns out this wasn't the case.  IE actually had an override on the Rendering of the Controls that removes this additional whitespace.  The trouble is, the Validation Values assigned to the controls happen after this has taken place.

Unfortunately I was unable to change the data in the database so I had to change my code to accommodate this feature of the web Rendering the page.  I had two options.  Strip the additional whitespace before assigning the value to the listitem in the dropdown box, or somehow force the dropdown to keep the additional spacing.  I opted for the latter and used the <Pre> tag  to ensure that all Text and Values in the Dropdown List were treated as strict values.

I am unsure if this affects other controls at the moment but I will run some simple tests when I get some downtime and append to this blog with my findings.

Engineering vs Product Release

For a long time it has bothered me that inferior products manage to top the charts of their prospective fields.  Maybe this was because the marketing department was better at the company with the inferior product,  or maybe it was that the marketing team sucked at the company that chose to invest in engineering.

Today I was reminded of what can go wrong when the engineering arm of a company invests too heavily in that field without looking at the overall picture of what is required for a good product roll-out.

A company was offering character recognition software for automating the entry of invoices into a system.  The premise is very simple, take some text on a scanned document and enter it into some fields that can then be exported to the customers database.  Everything about this product ticked all of the boxes except for one thing, the interface.  For some reason, someone within the IT department managed to convince everyone else that silverlight was the way to go.  There is nothing wrong with silverlight, don't get me wrong, provided silverlight is the best option to go with and no other tool can offer the functionality required.  Otherwise you are locking yourself into that particular vendor and restricting the platforms you can actually run the application on.  If the customer wants a web based system they can also view on their phone / tablet, silverlight really isn't going to do you any favours.

Functionality vs Complexity

So what drove this company to go with silverlight?  Maybe they thought it was right for their customer base at the time, or maybe the devs wanted something new to play with and thought they could bolster their CV with a bit of silverlight on it.  This is where the complexity comes in.  If you have a tool that you think aids development, ask yourself this question.  If you're going to be employing someone new into your company, does having experience in that product become a prerequisite to the job?  If the answer to that question is yes, then the tool isn't actually helpful.  Take resharper and SSMS Tools for instance.  Both great plugins for visual studio / SQL Server Management Studio.  However you wouldn't state that a developer had to have experience in using them before joining your team.  However if your company has decided to use an ORM layer like nHibernate or Entity framework, where do you go from there?  All you are doing is diluting the pool of applicants you might be able to source for the job.

Bad Product Release

On the other hand, there are times when a bad product release can undo all of the good work of the engineering team.  Not setting sensible values for defaults leaves people thinking your product is inferior.  80% of your user base is not going to take the time to wade through pages of options and configuration.  They are trusting that the Release team chose a sensible set of values that cater for the majority of their customers.

So what is the right level/trade off for all of this?  I honestly don't know, it depends on your user base and what they want.  The only way you're going to find this out is through market research.  So the emphasis is back on the marketing team again!  I hope that one day the user base will improve so they can make a choice based on engineering superiority rather than aesthetics.  If that happens, companies like Saab will still be in business.

When To Upgrade (and when not to)

I have often found people divided on this matter.  Some people always want to be on the latest version, it doesn’t matter what it is, but if there’s a newer version out there, they want it.  The other extreme is people clinging on to their old version and wanting it to run forever.  So much so that if they are forced to upgrade because of the operating system, they would rather run the older operating system as well.  There are arguments for both cases but camp A will get stung eventually when a new software release is detrimental and they can’t go back to the old version.  Camp B will get stung when they can no longer get support for their version and they are forced to upgrade.

So which is the better option?

Version Numbering

It comes down to understanding version number.

Version numbering comes in 3 Parts, Major Minor and Revision. A forth can also exist that refers to the incremental build number.

So we have version 1.4.8 of an application and they release version 1.4.9.  What can you expect to have changed in that release?  Not a lot in all honesty.  The most likely thing is bug fixes which were deemed urgent enough to make a release for.  It is these updates that you should avoid unless you have experienced the bugs in question.

The next upgrade might be version 1.5.2 this is a minor release; there may be improvements in security or a new feature requested by a number of users.  Again, I would be reviewing what the differences are between the two versions before upgrading.  If I’m never going to make use of those new features, it probably isn’t worth taking the risk of updating the software.

The Major release is the biggest upgrade you’ll encounter.  This could be a major overhaul of how the code works and the number of features that are available.  Not all major releases will have features added.  Some may be taken away because the development team have found they are troublesome to maintain and test.  Or based on user feedback have decided not to keep that feature.  Other reasons behind this could be incompatible operating systems or other software on the market or there could be legal reasons behind it.  The UI may have been overhauled giving it a different look and feel.  Users of Word 2003 to 2007 will have noticed a huge difference in how the application looks and operates (does anyone actually like the ribbon functionally compared to the humble toolbar?)

Switching Vendors

So there are reasons why you would never want to upgrade your software provided it is working as it should.  But what’s the other type of software upgrade?  This is where vendor switching comes into play.

Say you have Crystal Reports 8.5; you’ve kept with this version because it allows you to compile the reports and distribute them as Executables, thus eliminating the need to install an expensive licensed piece of software to all of the machines that need to run then.  Version 9 or later doesn’t allow you to do this so you’ve avoided upgrading it. After all, it works and you’re happy with it.

Sooner or later Crystal Reports is going to look very outdated compared to other companies software.  The terminology used within it doesn’t match any other products of a similar nature and guess what, anyone else joining your company isn’t going to have been trained in that version of Crystal Reports or maybe hasn’t even heard of Crystal Reports because it is that old.

So at some point you’re going to have to upgrade to something or face an extortionate bill finding the last person on the planet willing to get out of bed to support the legacy application.  I for one hope to make a mint out of writing conversion packages to take a report from one system and convert it to whatever other product is on the market e.g. SSRS.

Its not my fault, the mechanic did it!

Whose fault is it anyway?

I recently had the misfortune of having to deal with a difficult "customer" but I was glad it happened because I've been waiting to use this analogy for ages.

The usual story with chaotic IT departments is that they get the blame for everything.  The customers lose faith in the IT departments ability to deliver or maintain and as a result whenever something doesn't work as intended the propensity is to blame IT.


The problem that was put to me was that if the system wasn't working, the stores couldn't process orders and do their work, and this creates a problem.  The regional manager wasn't getting that what he'd described was not our problem in anyway.  It was beyond our control and the users were actually part of the problem due to their lack of instruction, propagated by the lack of leadership in this area.  When the supervisor doesn't know how to do something, they can't teach their direct report staff how to do it either.  Enter the analogy.

The analogy

I asked him how he used to do things before the computer system was there.  A little bit of a struggle but in the end, he accepted there was always manual ways things could be done and a good old piece of paper and pen was always to hand.  I related this to his journey to work.

Before you had a car you walked to work, the invention of the Car made it much more convenient for you to get to work and cut down the commute time immensely.  You were happily commuting to work in your car for many weeks, you couldn't even remember how you got to work without it.  One day, the car breaks down and you get a quote from the mechanic saying it will take at least 3 days to fix.


So in the meantime what do you do?  If you don't go into work there will be consequences but who's fault is that?  Not the mechanics fault, you can't hold them responsible for you not being able to get to work.  You can shout at the mechanic in the hope they'll work faster but if they have got to order in parts and wait for them to be delivered that won't really help.  It's not the mechanics responsibility for you to get to work, the car is just a tool to aid you in doing the job.  If your car is unreliable, consider getting a new one (see my next blog When to upgrade and when not to).  The analogy can go deeper here about the similarities between vendor lockin and leasing deals on cars but that's not the idea of this blog.

Implementing SQL Emails in SQL 2000

Stop with the Polling

In lots of legacy systems there is a problem of polling.  A job or routine comes along on a regular basis and checks to see if there is some work to do.  When it finds something, it actions it and moves on thinking everything is great.  In even more disjointed systems, they will then have another polling job that should run as part of the other job, but for whatever reason decides to do its own polling to see if work should be done.

This creates a problem in many cases because Job A may come and poll again before Job B has had a chance to do its bit.  What if Job A is cleaning up after itself or overwrites what it did the first time and Job B hasn't had a chance to get what it needed to complete?  This is a common scenario for polling and one of the reasons it is bad in modern computer and why event driven processing is preferred.  Quite often this is solved by implementing a listening pattern but my problem didn't need to go that far.

Legacy Emails

One of the features of SQL Server 2008 is the ability to send out emails using the inbuilt stored procedures for handling mail.  This can be a very useful feature as it saves you having to have a mail client installed on the server and saves another failure point to check if emails aren't being sent.  My problem was the Jobs in question weren't on SQL 2008, they were on SQL 2000 which doesn't natively support this email functionality.  My initial thought was to write a client that would interface to the smtp server and I could call into this with xp_cmdshell.  I stopped myself short at this point because of code smell and realised this problem must have been solved before me.  We're in 2012 FFS!

Sure enough after some searching I was able to find the article that MS have written on this very matter
http://support.microsoft.com/kb/312839

Golden Hammer

The trouble is, now my work colleagues think this is great and have started flooding email addresses when jobs complete, fail, take too long etc.  I need to remember them that there's a time and place for everything and not all solutions can be managed with a simple email.  Plus if you're getting a lot in one day, surely a report on a daily basis would give a better indication of how big your problem is.  You may even want to keep an audit trail of when these things happen so you can get some meaningful stats in the longrun.

Perversions in Source Control

Working for a small company has some plus points and some bad points; one of the plus points is that the number of licences required for software is small.  You could argue that granting people more access than is necessary is a good thing, and the lack of paperwork that is required to make this happen is limited or non-existent.  Things actually happen when they need to rather than waiting around for the sherif to authorize things.

Source Control tools are readily available in both free and paid license form.  Having convinced the bean counters that this was completely free and wouldn’t cost them anytime to set it up, I was free to implement the much needed versioning control.

About 1 week after this was all implemented I came to realise it was all for nothing.  The fact that everyone still had full access to the database and files on the system in a live environment, people were still releasing stuff straight to live and completely negating the point of version control.

Lack of communication, are you part of the problem?

The Problem

An incident happened Today where I was reminded about the lack of communication and the harm it can do.  But it was also a reminder that people can be too quick to assign the blame.

I arrived early in the morning at work to find that some services on machines were not running and some jobs on SQL were still running that should have finished hours ago.

I was just about to start the services and kill the jobs when I suddenly had a thought about events from the day before.  I decided to ring my boss and ask him if he knew of a reason why the services were stopped.  His response went along the lines of this:

"Err yes, err, I had to errr, stop them, errr, because I needed to run one of the month end jobs again because it went out of sequence with another one"

"OK, I came in and saw that the services were stopped and was about to start them, I'm glad I checked now.  I didn't see an email though, are the others aware?"

At this point another of my work colleagues came along and he was filled in about the problems and what was bing done.

Later that day I went in to see a colleague and he was furious about what had happened earlier.  He said he'd spent hours trying to diagnose why things weren't running and slow performance on the network etc etc.  His main gripe being there was no communication to any of us that such a major decision / change had been made.

He was, of course correct.  My Boss hadn't said anything to anyone.  I told one person in passing after I found out and that person hadn't relayed the information to them etc despite the proximity of all people being so close.  However I couldn't help but feel that this guy was also part of the problem.  Like an employee trying to move a mountain one spoon/shovel at a time, they didn't stop for one minute to think whether they should be doing anything about it, or whether or not they should call someone to check.  I reminded them that communication goes both ways.

Proactive Solution

I always find it prudent now to be proactive and ask, "everything running OK?" to someone who is already in the office.  Rather than waiting to hear bad news or just assuming someone will eventually shout if I'm needed.