This is part 4 of a six part series titled “Frustrations of the Data Analyst Life…And How To Fix Them“
Picture the scene. Monday morning team catch-up with a senior executive who has just returned from a business trip to a conference he got a free junket to and had no business being at.
“So, I’ve been reading a magazine article on the plane all about the blockchain, big data and machine learning.
It’s revolutionising everything so we need to get involved.
It’s LITERALLY taking over everything.
We need to move all of our data to the Hadoop right now or we’ll get left behind, can you get onto it straight away?”
Is there anything more destructive to the productive time of an analytics team than a senior executive with a magazine or website link and the bare minimum of technical knowledge to help them apply it?
If you’ve been around long enough you’ll have seen this scenario come up over and over again, only with a different Buzzword Bingo sheet each time.
Right now, it’s Machine Learning and AI and every analyst became a Data Scientist overnight.
A couple of years ago it was “Big Data” (how big was actually “big” seemed to be left deliberately vague).
This led to a mad rush to decommission relational database systems and data warehouses in favour of data lakes, unstructured databases, ELT instead of ETL and other such innovations.
I don’t want to appear like an analytics Luddite.
I’m still (relatively) young enough to want to get on with developing and building the next generation of analytical tools and processes and my hankering for the glory days of MS Access 2003 are thankfully far behind me.
What I do object to is change for change’s sake.
There is, of course, a place in the forward-thinking organisation’s analytics arsenal for these technologies.
For the right use case, they are pushing analytical capability far beyond what we have seen up to this point.
And that’s obviously a good thing.
But it’s not for everyone, all of the time.
The analyst’s frustration then should be with having to continually chop and change their stack to suit the latest fashion or trend being pushed by the technology giants that want to sell them onto the publicity seeking C-level exec with a reputation to build.
Analysts will always want to work on cutting edge tech. It’s good for the CV, good for workplace morale and good for the soul.
What they don’t need is to have to change everything they’ve built every other year until the end of time.
How do we find the middle ground between progress and consistency?
Damn good question. Prepare for splinters from fence sitting.
Set the buzzword bingo cards aside, let our teams work on a consistent stack and concentrate on driving business value rather than chasing rainbows and the latest fashion.
If a new technology can potentially make a substantial improvement to the business then do a Proof of Concept on a small specific problem and see how that flies.
Don’t throw the baby out with the bathwater if the PoC is a success but don’t miss the chance to innovate where you can in small, incremental stages when the opportunity arrives.
And please ban those pseudo-tech business magazines on flights. Maybe then we’d all be a lot more content with what we have, not always chasing what we don’t.