Evolving Workflow Orchestration // Alex Milowski // #291 MLOps.community podcast

Evolving Workflow Orchestration // Alex Milowski // #291

10M ago 1:14:34

Content provided by Demetrios. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Demetrios or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

Alex Milowski is a researcher, developer, entrepreneur, mathematician, and computer scientist.

Evolving Workflow Orchestration // MLOps Podcast #291 with Alex Milowski, Entrepreneur and Computer Scientist.

// Abstract

There seems to be a shift from workflow languages to code, mostly annotation Python - happening and getting us. It is a symptom of how complex workflow orchestration has gotten. Is it a dominant trend or will we cycle back to “DAG specifications”? At Stitchfix, we had our own DSL that “compiled” into airflow DAGs, and at MicroByre, we used an external workflow language. Both had a batch task executor on K8S, but at MicroByre, we had human and robot in the loop workflows.

// Bio

Dr. Milowski is a serial entrepreneur and computer scientist with experience in a variety of data and machine learning technologies. He holds a PhD in Informatics (Computer Science) from the University of Edinburgh, where he researched large-scale computation over scientific data. Over the years, he's spent many years working on various aspects of workflow orchestration in industry, standardization, and research.

// Related Links

Website: https://www.milowski.com/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

// MLOps Swag/Merch

⁠https://shop.mlops.community/⁠

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Alex on LinkedIn: https://www.linkedin.com/in/alexmilowski/

Timestamps:

[00:00] Alex's preferred coffee

[00:20] Takeaways

[01:27] Workflow Systems Overview

[05:54] Everything as a DAG

[09:25] Workflows, Layering, and Ownership

[12:46] Workflow Awareness for Success

[17:31] Process Engineering Challenges

[21:20] ML Workflow Trends

[27:57] Data Engineering and ML Tools

[33:18] Infrastructure as Code Pros and Cons

[36:20] DSLs in Organizations

[43:04] Agents vs DAGs

[47:48] Agent Workflow Selection

[50:31] RPA Business Automation Tools

[55:00] LLM Integration in BPA

[1:01:14] Diversity in Decision Workflows

[1:03:09] Shadow AI Governance Risks

[1:08:34] Centralized ML Workflow Standardization

[1:12:50] Wrap up

489 episodes