Data Engineering Career Path: To Code or to No-Code?
Which direction do I have to grow in before ultimately topping out in this discipline?
As a Sr. Data Engineer, I find myself questioning what’s next. Many companies consider their Data Engineers an extension of analytics (I’m looking at you, Meta.) Many others, such as Apple and Netflix, see their Data Engineers as an extension of Software Engineering. My current skills are more focused towards low-code and no-code, such as writing SQL and scheduling scripts in tools such as DBT/Airflow. While I have consumed APIs, I certainly haven’t written any. I began to question; where do low-code/no-code Data Engineers go from here? Is there a ceiling in sight? Which direction do I have to grow in before ultimately topping out in this discipline?
I believe this divide has been caused by the excellent data platform products from the likes of DBT, Snowflake, DataDog, Palantir (if anyone actually knows what they do, hit me up) and DataBricks. These tools allow SQL-specialists to harness the transportation capabilities relegated to the Software Engineers or Data Engineers of times past. Clearly these tools are empowering companies to deliver value at a higher velocity, and the market is taking notice.
