Today we’re talking with Divya Parashar whose 16 years of driving enterprise-scale innovation have spanned conversational AI, native platforms and decision intelligence. He holds a US patent in data augmentation and has led transformative initiatives that blend real-time and batch processing at scale. As a recognized leader on architecture boards, patent committees and global tech forums, Divya is passionate about mentoring engineers and encouraging inclusive, impactful innovation. In this conversation, we’ll explore how he balanced latency and volume when redesigning a data and AI platform, aligned AI projects with business outcomes and built architectures that adapt as needs evolve.
Can you walk us through a time when you successfully scaled an AI or data platform for real-time and batch processing? What were the key challenges, and how did you overcome them?
In one of my previous roles, we were tasked with redesigning a data and AI platform to support both real-time decision-making and batch analytics across multiple business units. One of the key challenges was balancing the latency requirements of real-time use cases with the volume and cost of batch workloads. We approached this by decoupling the compute layers—using a streaming pipeline with Apache Kafka and Flink for real-time processing, and a Spark-based system for batch ETL. The tricky part was ensuring consistency and synchronization between the two. We invested time in standardizing data contracts and schema registries, which made the ecosystem more manageable. What made it work wasn’t just the tech—it was getting buy-in across data engineering, ML, and business stakeholders to align on shared priorities.
Many AI projects never move beyond the proof-of-concept stage. From your experience, what’s the secret to aligning AI initiatives with measurable business outcomes?
I’ve learned that success in AI starts with the problem, not the model. Too often, AI efforts begin as tech-first explorations rather than business-driven solutions. What’s worked best for us is embedding data and ML teams closer to the business, co-creating with product owners and domain experts early in the process. We focus on framing the problem with clear success metrics, like improving conversion, reducing churn, or cutting operational costs. I also try to avoid over-promising early on—it’s better to prove value through small wins and iterate. AI doesn’t need to be flashy; it just needs to move the needle.
When building enterprise AI systems, what architectural choices have made the biggest impact on long-term scalability and performance?
Standardization and modularity have been game-changers. Designing reusable components for feature engineering, model training, and monitoring allows teams to move faster without reinventing the wheel. We also made early decisions to separate data storage from compute and invested in event-driven architectures, which paid off as use cases scaled. Another key learning was the importance of model observability—tracking drift, performance, and usage metrics in production. It’s not the most glamorous work, but it’s foundational for maintaining trust in AI systems at scale.
You’ve worked at the intersection of enterprise architecture and machine learning. Where do these disciplines complement each other, and where do they tend to clash?
They complement each other when there’s shared intent—enterprise architecture brings the lens of scalability, maintainability, and governance, while ML introduces experimentation and agility. The clash usually happens around timelines and flexibility. Enterprise systems value stability and predictability, whereas ML teams often need to move quickly and iterate on data and models. Bridging the gap requires empathy on both sides. I’ve found success by introducing ML platforms that abstract complexity and adhere to architectural standards without stifling innovation. It’s about finding the right level of control without becoming a bottleneck.
How do you ensure that the AI systems you design are adaptable, especially as business needs and data sources evolve?
I believe adaptability starts with loose coupling. By designing systems with clear interfaces between data ingestion, model training, and serving, we can swap components as things change. We also rely heavily on configuration-driven pipelines rather than hard-coded logic. This makes it easier to adapt to new data sources or retrain models without significant rewrites. Just as importantly, we stay close to business teams and product managers to anticipate shifts in priorities. Technology is only half the equation—early communication and flexible processes keep us ahead of the curve.
What lessons have you learned from AI projects that didn’t go as planned? What would you do differently next time?
Some of the most valuable lessons came from projects that didn’t deliver the impact we hoped for. In one instance, we developed a technically sophisticated model that was never adopted by the business team. Looking back, we hadn’t invested enough in stakeholder alignment or user experience, so even though the model worked, it didn’t solve the problem in a usable way. Now, I focus much more on the full lifecycle—from understanding the user workflow to ensuring models can be easily interpreted and integrated. It’s a reminder that even great models won’t matter if no one uses them.
Looking ahead, what are your predictions for how enterprise AI architecture will evolve over the next three to five years? What shifts do you think leaders should prepare for now?
I think we’ll see a continued move toward AI platforms that democratize access, making it easier for non-specialists to leverage models safely and effectively. Foundation models and APIs will reduce the barrier to entry for many tasks, but they’ll also introduce new concerns around cost, security, and explainability. Enterprises will need to rethink their governance models and develop more robust ML Ops capabilities. I also expect to see tighter integration of real-time analytics and AI into business operations, blurring the lines between insights and actions. For leaders, the key is to invest not just in tech but in talent, culture, and ethical frameworks that support responsible AI at scale.
