Skip links
Splunk to AWS Migration for Scalable

Overcoming On-premises Failures to Deliver Secure, Scalable Insights on AWS

Overcoming On-premises Failures to Deliver Secure, Scalable Insights on AWS

Splunk to AWS Migration for Scalable

Executive Summary

A top retailer was experiencing significant operational difficulty with their data analytics platform, as their IT troubleshooting was being done in an ad hoc way, inefficient, and improper way despite having deployed Splunk. The customer had a suboptimal experience with their on-premises Splunk Enterprise, resulting in the entire infrastructure going down for close to a week following an upgrade to version 6.3. Besides these technical failures, they also faced the daunting task of moving their teams to AWS while simultaneously handling the process of training and onboarding over 125 teams within the company.

To address these challenges, Crest Data provided professional services that improved the infrastructure management of the retailer and stabilized their environment. By implementing DevOps and self-monitoring, self-healing automation, Crest consultants successfully onboarded data from over 800 sources, such as web servers and payment gateways. This end-to-end approach led to a sevenfold improvement in dashboard performance and reduced Splunk administration costs by more than 50%, giving the retailer deep business insights and a competitive edge.

About the Customer

The customer is a leading global retailer and owns the largest supermarket chains, holding a dominant market share. The customer has also diversified into clothing, telecommunications, and financial services. By integrating these diverse offerings with a strong online presence, the customer maintains its position as a primary hub for essential consumer needs across Europe and Asia.

Customer Challenge

One of the primary challenges that the customer faced was that of inefficient and slow IT troubleshooting, even after deploying Splunk. The customer also faced a major technical setback when an upgrade to Splunk Enterprise 6.3 resulted in their IT infrastructure going down for nearly an entire week. Such a critical failure of their system highlighted the fragility of their infrastructure.

Furthermore, the retailer had a very bad experience when they had to migrate their teams to AWS on an urgent basis due to their on-premise Splunk deployment. Also, the customer faced the challenge of educating, training, and onboarding 125+ internal teams to ensure that they can be efficiently used across the enterprise.

Customer Solution

Crest Data offered specialized professional services and Day 2 support for monitoring and administrating the Splunk infrastructure to mitigate retailers’ problems like operational inefficiencies and infrastructure instabilities. Within 90 days, Crest Data placed 2 consultants who worked with more than 50 internal teams to stabilize the environment in order to transition to AWS.

Key essential aspects of the solution implemented include:

  • Infrastructure and Performance Optimization: Crest Data primarily focused on reducing the AWS footprint used by Splunk while simultaneously improving indexing performance. Moreover, dashboard performance was improved 7x by fine-tuning complex search queries.
  • Automation and Process Improvement: Consultants implemented change management processes and DevOps methodologies to enable long-term stability and reliability. They also created self-monitoring and self-healing automation for the Splunk deployment to avoid future outages.
  • Enterprise-Wide Data Ingestion: The team ingested data from more than 800 sources across Web servers, Payment gateways, AWS, database inputs, and other enterprise applications.
  • Team Enablement and Onboarding: Educating and onboarding more than 125 internal teams was a vital part of the solution, as this ensured that the platform was effectively utilized across the customer’s side.

Outcomes

The customer could experience a stabilized and highly optimized Splunk environment that provided significant operational and financial benefits to the customer:

  • Substantial Cost Savings: The customer could achieve a reduction in Splunk administration costs by more than 50% while gaining a competitive advantage through deeper business insights.
  • Significant Performance Gains: Crest enhanced dashboard performance seven times (7X) by fine-tuning search queries and successfully optimized Splunk’s indexing performance.
  • Increased Infrastructure Stability: By successfully implementing self-monitoring and self-healing automation for Splunk deployment, Crest Data ensured a stable infrastructure, preventing outages.
  • Scaled Data Ingestion: The project successfully scaled up data ingestion from more than 800 sources. These included data from web servers, payment gateways, AWS, and enterprise applications.
  • Broad Organizational Enablement: The customer addressed the challenge of onboarding and teaching more than 125 internal teams as Crest Data worked with over 50 teams across the enterprise in the first 90 days.  

About Crest Data

Crest Data is a data and AI-first product engineering and technology solutions provider with deep expertise in cloud and AI, cybersecurity, observability, data analytics, and workflow automation. In this case study, Crest Data relied on its CloudOps and DevOps expertise to assist the customer in moving out of a precarious on-premises environment into a cost-effective and performant cloud setup on AWS, with self-healing automation, indexing performance optimization, and proactive monitoring and administration.

With 1,200+ experts and a track record of 5,500+ successful projects across 150+ global customers, and backed by strong partnerships with Google, AWS, Microsoft, Datadog, Dynatrace, ServiceNow, and NetApp, Crest Data delivers outcome-focused solutions that strengthen security, improve platform reliability, and enable sustainable digital growth.