architect
Slack Eliminates SSH in EMR Pipelines, Migrates 700+ Jobs to Rest-Based Architecture
Source:
feed.infoq.com 1 min read
Share
You are reading a summary. The full content is hosted on feed.infoq.com.
Slack modernized its data platform by replacing SSH based execution in Amazon EMR pipelines with a REST driven orchestration layer called Quarry. The migration covered 700 plus Airflow operators, improving security, reliability, and observability while eliminating direct SSH access across production clusters and enabling a server side job lifecycle model. By Leela Kumili
Read the full article on the original website
External link to feed.infoq.com
Related Articles
architect
WebMCP Standard Proposal for Agentic Web Actuation Now Available in Chrome (Origin Trials)
1 min read •
architect
The digital pivot: How HSS transformed hire with agentic AI
1 min read •
architect
Scaling Security Insights: how we achieved a 10x increase in global scanning capacity
1 min read •