Interesting and well written article Pedram. Though while I agree with nearly all the content, I disagree just a bit with the overall message (or perhaps have a slightly different spin on the message).
I whole wholeheartedly agree we're in an era where most firms do not need folks with deep math knowledge, nor need folks to innovate methodologically. They really just need folks to know how to leverage libraries (even if they don’t understand what’s going on under the hood) and by supplementing their work with business-domain knowledge, conduct analyses that provide business value as well as assist in or directly build elements of internal/external products.
However, here is where I have a bit of a different take on your message. The role I just described above (plug-and-chug through libraries with business domain knowledge, perhaps most focused on informing the building of products, but without fully understanding the math nor pushing for methods innovation) is functionally a bread-and-butter Analyst. Perhaps a Senior Analyst, but still very much an Analyst. Now I realize that in the last 5 years it has become popular for many firms to re-brand their Senior Analyst roles as Data Scientist roles, and that is certainly a firm’s prerogative. But the function of the role itself hasn’t changed much, just the job title. This is effectively a role that has existed for decades.
Now before people yell at me, I’m in no way saying these roles aren’t important or valued. On the contrary. These Analyst roles are hugely important, and any firm needs skilled and enthusiast folks in these roles. I have several Analysts working for me, and my firm would have a hard time getting anything done without them.
Getting back to your piece Pedram, I believe the overall message is that very few firms actually need functional Data Scientists. Certainly in terms of what a "Data Scientist" was most widely considered to be 5 or 10 years ago. What they want and need are Analysts. Depending on what firm you’re working for, you may be serving an Analyst function but have a Data Scientist job title. This is often the case even at huge companies like Facebook and Amazon (what Amazon calls a “Data Scientist” are predominately Analyst roles, and what they call “Research/Applied Scientists” I would consider Data Scientist roles). But regardless, whatever work you’re doing, if it makes you happy and fulfilled then great! Everyone should be so lucky to be happy with their work. But if you have a similar definition of “Data Scientist” as I do, agree that it’s not just a re-branded Analyst but rather an actual Scientist role, and want to be part of pushing the envelope methodologically and innovating, than we need to realize these jobs are few and far between. As you rightly pointed out, these roles only exist in huge firms with research labs and small firms particularly driven by methods innovation.
Anyway, that's my two cents. Again, well written article :)