Etikettarkiv: agile

Tools to support agile ways of working in large organizations

We are using JIRA to manage user stories and features in an agile setup. I was asked to make some tweaks to the JIRA setup to make the work more efficient.

As with many large organizations, the JIRA setup is managed by a central organization.  Their approach to their work is somewhat less than agile 🙂



Me JIRA admin
May I have a JIRA project to play around in and test some configurations?  I do not want a standard configuration as my changes will affect other projects in that case  
  What standard configuration do you want?
None, I want to be able to change configurations myself.  
  Do you want to be in test or production?
What’s the difference for me?  
  I think you should be in test
  Here’s a project
But it has a standard configuration  
  Oops, here’s another project
Well, this project did not have a standard configuration, but a default-configuration.  But I still can’t change anything.  
  What, you want to change things?
Yes, that was sort of the whole point of this….  
  But then you need full JIRA admin access
Why full access, I only need to change in my project…  
   Then you need full JIRA admin access
Can I have JIRA access?  
  You already have JIRA access if you can log in
I mean JIRA admin access…  
  No you can’t
Can I have my own standard configuration to play with  
  No you can’t
Who manages the standard configuration that is in use here?  
  JIRA admin
Is that your team?  
Who should I talk to  
  JIRA admin

I think I give up……

Fail-fast and garbage files

Today it happened again… Why don’t we have a better way of handling this?

My project (as I probably mentioned before) is a data warehouse solution to calculate market risk for a bank. As all data warehouse solutions much of what we do deals with data in files and data in tables. The tables themselves are represented as files also (we are using SAS solutions). Over time things change, names change, definitions change, machines change etc. But in many cases the old files that are not relevant anymore do not get deleted and they become garbage that lays around the various directories of our system. Everything works fine and nobody is worried.

Then one day we move to a new machine, or to a new environment, or to something else. All of a sudden, the garbage is not there anymore. And we get a crash. Turns out that everything was not fine, since the system did not read the files we thought it read but instead it read the garbage. With no garbage, nothing to read, and therefore a crash. This has happened to us at least four or five times over the past 3 weeks (we are in deployment mode right now).

The agile movement talks about failing fast. You should design your systems so that any bugs make the system crash and crash early. This way, the bugs are found early, and can be fixed quickly and cheaply.

Our data warehouse system, based on SAS has evidently not heard about failing fast yet. Also, I do not see that mindset in the community of data warehouse developers that I have met in this project. Could this be a big opportunity for improvement? Time for me to start some convincing…..

Why you need information modeling and/or simple ways to test and deploy

My current assignment deals with a data warehouse system where some calculations should be done based on the data in the data warehouse.  We also have some tables that control how the calculation should be performed (e.g. how data in separate data sets should be mapped to each other).

Over time, we have introduced more features to the calculation and most of these features need some control table structure so therefore the control tables have evolved into being more and more complex in their structure and in their interrelationships.

Now, with the great benefit of 20-20 hindsight, it is easy to see some areas of the structure of the control tables that are sub-optimal at best and downright ugly at worst.  There is one example where the same column of the same control table is used to map to two separate concepts in the model, depending on what exact data is used.

Why did this happen?  Why were we not able to control the structure of these important tables over time in the project.  I see a few different elements of an explanation:

  1. We did not have a good enough understanding of the total problem when we started.  (of course this is not what you typically have in an agile project, which leads us to the other explanations)
  2. The project, like all projects, was really pressed for time, so nobody wanted to redo work that was already invested in the early versions of the control tables
  3. The test and deployment processes of the project takes a long time, therefore rework was not really wanted.

So, what do I think we should have done instead

  1. Invested some time in some information modeling early on, so that we could have a better view of the full picture of these control tables early on
  2. Created reusable test cases and possibly some level of automated tests so that we could easily retest something that was reworked.
  3. Spent more time after the initial functionality was created to cleanup and fix these kinds of technical debt issues.