Observations In How Things Go

Have you ever noticed the way new ideas and innovations seem to decay with their mainstream acceptance and growth?

The usual scenario plays out something like this….

1) A new idea or innovation is developed. The core community which includes the originators of the concept and a small group of early adopters contribute to its betterment, fine tuning and working the concept. These early adopters also promote the particular idea, innovation or methodology.

2) The community is comprised of mostly like-minded people all contributing their particular talents for the betterment of the whole. This is the pinnacle of the communal symbiosis for any new idea or innovation. The community is focused only on the betterment of the concept.

3) Now things begin to get noticed by the larger population and the core community starts to grow. This seems a good thing but in reality, the growth of the group leads to the decay and ultimate corruption of the original value of the idea of innovation.

So why is there; this inevitable degeneration and how does it happen?

What seems to happen is that as more people become aware of the concept, the fewer true collaborative contributors you get entering the community. The new idea quickly becomes the latest trend and is USED in the truest sense of word. The new community becomes widely known and becomes seen as a must do/have item. It is during this transitioning that the foundation principles of the original community are under the greatest threat.

The core community becomes the latest bandwagon to jump onto by the rest of the broader community. It is this extra weight of the “Hangers On” jumping on the Bandwagon that breaks the axel and makes the wheels fall off. These “Hangers On”, USE the new idea or innovation as a means to virtue signal and big note themselves. The focus drastically shifts from collaboration and the concepts improvement, to a “What’s in it for me” narrative.

This often leads to the decay and ultimate failure of most new ideas.

These “Hangers On” join the new community not out of like mindedness or a collaborative philosophy with a willingness to contribute and embrace the foundational idea or philosophy of collaboration. Their motivation is to be seen as part of, what I call, “the leading herd”.

So how do you identify the “Hangers On”?

They rarely understand the innovation or the collaborative nature of learning, they just want a quick fix to their problems. Just like students wanting to know the answers rather than putting the effort into actually learning and understanding the lessons they are taught.

Some final thought to share.

“With great acceptance comes greater corruption.”

If you are a member of a core group be watchful of the decay which will degrade the foundations of your community. This doesn’t mean not to share your ideas or innovations but to understand that you can only really learn when you have reached the particular mind set or mental maturity to actually learn and understand.

Knowing something does not mean you understand it.

Knowledge is not Wisdom but only its starting place.

Anything learnt needs to become part of your daily experience.

Not everyone is ready to learn and embrace new and foreign ideas at any particular time.

The Factory Analogy for Software

This is a reflection piece working on similar themes to a post by Tobbe ‘The Software Factory — an idea wrong enough to be useful’

When we think about automation, it is easy to imagine factories. The standardisation of work that was brought in with ideas such as The Principles of Scientific Management by Frederick Winslow Taylor allowed us to automate jobs that were previously performed by craftspeople. So it is pretty easy to think of software development as like a factory because we are automating functions – but the ‘goods’ we are producing are transactions – not physical items.

I like Tobbe’s idea of the flexible factory line where we have teams and architecture that support delivering software in more flexible ways. Most of us do not think about factories like that. I had a part-time job packing biscuits in a health food factory working my way through university. So when I think about factories, I am thinking about highly repetitive (and boring) tasks that are getting automated.

Where we trip ourselves up is using factories as an analogy and mis-applying the metrics that are useful for high-volume repetitive tasks to tasks that are less repetitive. Even though I was manually assembling the boxes for the biscuits, I would pride myself on finding more efficient ways to do this – and shaving a few seconds off this part of the process (which I did at the start of each shift) was useful as well as rewarding. This is where tools such as control charts, six sigma and paying attention to changes in trends is very useful. The production version of a piece of software is similar to a factory in this way – I really want to know if my many transactions per minute start to take even a tiny bit longer to execute – this could indicate a problem that is easy to fix and avoid a big outage. I once looked after a system that lost sync between the 2 databases and it took 24 hours to re-sync (with degraded performance for the end users over that entire day). We made sure that when we found out what had caused the issue, we watched the early warning signs with a lot more rigour and I’m happy to say that it did not happen again.

Installing new code is like changing the setup of a factory. In my health food factory example, it might be automating the biscuit packing job that I did. I imagine that if we automated the whole process, then the factory might be offline for a few days – losing a few days of production and therefore sales. Taking an agile approach, perhaps we just automate parts of the process in one go, such as starting with the box assembly. I’m not sure how big a machine to assemble cardboard boxes would be, or how much it would cost – but we could get in the experts who have done it before and they could tell us. We might use project metrics such as on time, scope and budget to ensure that we are not causing too much disruption to the factory production and sales. We can also go back to the production metrics to see if we are making a difference to cycle or lead times, once we have installed the new parts. There might be a way to install some of the packing automation without disrupting production and we would need to design the factory in a different way to the one that I was working with if we needed to have continuous production while upgrading the machinery. This example is like the release management process when we deploy new software into the production environment – we should be using metrics that allow this process to become more efficient and project-based metrics would be appropriate – especially if we add the quality check about the impact the change has to the production environment.

Software development is like deciding what we need to automate and then designing the machines to perform that automation. If no-one had built a box assembler before, then we would need to observe the manual process, the steps surrounding it and then design the machine. We would need to take into account how it would be maintained – this is a bit like the architecture role in software design – in the factory, we would not be making large machines out of gold – it is too soft, very expensive and would be hard to maintain – but it would be nice and shiny. In software design – how do we make sure that it integrates well into the existing software and requires minimal operational intervention? We do not want to be having to re-index the database every day – just like we would not want to have to service machines every day.

What if we decide to produce a different type of item. In the health food factory we also had lines that produced pies and cakes – which were separate to the machinery that produced biscuits. What if the company decided to manufacture something different, it could be a small difference such as a new type of biscuit or a large difference might be to manufacture coffee mugs. Whether there are sales out there for a new biscuit or coffee mugs is very similar to identifying if there are needs for new software features and we can use the same human centred design or user experience toolsets for physical and virtual things. Once we’ve identified the need we then need to decide if we should start a separate production line or modify an existing one. It is hard to imagine being able to manufacture pottery mugs and biscuits on the same factory line (and health regulations would likely not allow it) – but I have seen people try and add new functions to a piece of software that was never designed to do that new function. An example is where we were asked to add a fraud-handling function to an application which would have incurred a huge cost (in time and money) when we already had a separate system that could do what was required with much less cost.

How we measure the effectiveness of new items in a factory or features in software is very different to how we measure the efficiency of the production environment for both. We look at how sales are progressing, the level of interest shown by Customers etc. At this point the analogy is breaking and becoming less useful – the key point of this post is that using a factory as an analogy for software development can be useful in the ways described. We need to remember that an analogy is only saying that elements of software development and operation are like a factory – and we need to be careful that we do not equate building software with assembling items on a factory production line – especially if we then try to apply factory production metrics to software development.