DeveloperWeek Management 2024 + AI DevSummit 2024 (+ DW Europe/LATAM 2024)

Hareen Venigalla, Uber, Applied Science Manager
Vidhya Suresh, Atlassian, Senior Software Engineer

This presentation explores how data science and predictive modeling optimize the performance and scalability of information retrieval (IR) systems. We'll examine the impact of query analysis, document ranking, and result aggregation on user satisfaction. Our research demonstrates that techniques like keyword extraction, intent analysis, and custom deep ranking models can reduce irrelevant results by up to 26% while decreasing computing costs by more than 39%. We'll address the challenges of scaling IR systems to handle massive datasets and high query volumes, highlighting how predictive models streamline resource-intensive processes. Finally, we'll present optimization strategies leveraging distributed computing, multi-stage caching, and predictive ranking models to enhance throughput, reduce latency, and minimize computational overhead. This presentation offers valuable insights for those interested in the intersection of data science and information retrieval.