Skip links
Crest Data
Mastering Distributed Security with Real-Time Federated Monitoring and Automated Deployment for Global XDR

Mastering Distributed Security with Real-Time Federated Monitoring and Automated Deployment for Global XDR

Author
Published on:
Published in: CloudOps
Case Study

Mastering Distributed Security with Real-Time Federated Monitoring and Automated Deployment for Global XDR

Mastering Distributed Security with Real-Time Federated Monitoring and Automated Deployment for Global XDR

Executive Summary

The customer, a cutting-edge cybersecurity company, was looking for an architectural solution to deploy its core Extended Detection and Response (XDR) product to multiple and varied end-user environments. In particular, they needed to deploy their product within these environments without requiring access to and control of their infrastructure. This required the development of a zero-touch deployment model that could manage complex security needs – such as encrypted traffic and VPNs – automatically. Failure to design an efficient solution would result in costly operational complexity, delayed deployments, and vulnerabilities that would negatively impact the customer’s product adoption and customer confidence.

Crest Data built a cost-effective and secure cloud solution on AWS with Infrastructure as Code (IaC) and GitOps principles to automate tenant onboarding. Their design included a federated observability system, which leveraged Prometheus and the ELK stack to offer a single view of real-time monitoring across all the tenant environments, while ensuring security through mutual TLS (mTLS). This automated onboarding process meant deployment time was reduced from hours to less than 30 minutes, and infrastructure costs were reduced by 35%. As a result, the deployment enhanced operational efficiency and facilitated 70-80% shorter time-to-detect and time-to-respond for security incidents.

About the Customer

The customer is an innovative cybersecurity firm focused on developing cloud-native, scalable, and intelligent security solutions tailored for modern enterprises. Their flagship XDR platform emphasizes advanced threat detection, automated response, and deep observability across complex, distributed environments. By leveraging AWS, the customer ensures secure, resilient, and high-performance deployments that conform to industry standards and best practices.

Customer Challenge

The customer, an innovative cybersecurity firm, faced a unique and complex challenge: deploying their main XDR (Extended Detection and Response) product directly in end-users’ cloud environments. Unlike typical SaaS models, the customer did not have direct administrative access or control over these third-party infrastructures.

The key challenges include:

  • Requirement for Zero-Touch Automation: To scale effectively, the customer needed a fully automated deployment framework that could operate reliably across different customer infrastructures without manual intervention.
  • Complex Security and Connectivity: The architecture required automated management of VPN tunnels and secure data flows to ensure safe communication between the distributed tenant environments and the central platform.
  • Observability Gaps: There was a critical need for a unified observability framework. Without it, the customer lacked consistent, real-time visibility and operational insights across the various remote environments where their product was deployed.
  • Operational and Strategic Risks: If these challenges were unresolved, the customer could face deployment delays and high operational costs. These technical issues could introduce security vulnerabilities and threaten product adoption, eroding customer trust and hurting the overall end-user experience.

Ultimately, the customer needed to find a way to uphold zero-trust security principles and data integrity while operating within environments they did not own or manage.

Customer Solution

Crest Data created a scalable, secure, and automated cloud infrastructure on AWS to provide zero-touch provisioning of the client’s XDR product in various end-customers’ environments. The solution adhered to DevOps and GitOps principles to provide a consistent, secure, and reliable deployment and operation without direct administrative access to customers’ and third-party environments.

The solution consisted of:

  • Comprehensive Infrastructure Automation: Crest Data developed the infrastructure using Infrastructure as Code (IaC) with Terraform and Helm charts. AWS CloudFormation was used to create basic networking infrastructure (e.g. VPCs, subnets, NAT gateways), and GitHub Actions and ArgoCD were used to deploy these stacks on merge.
  • Federated Observability Architecture: To offer a single pane of glass to monitor all remote environments in real time, a monitoring and analytics stack was deployed.
    • Monitoring: Tenant environments have their own Prometheus instances, securely scraped over mutual TLS (mTLS) by a SaaS Prometheus deployment.
    • Logging: Logs are shipped from the tenant environment with Logstash or Filebeat to a central Elasticsearch cluster and presented with Kibana.
  • Scalable Container Orchestration and Storage: Amazon EKS is used for hosting tenant workloads and observability tools. It uses AWS EFS and EBS for storage and AWS S3 for long-term, cold storage.
  • Robust Security and Isolation: The SaaS solution and each tenant environment are hosted in separate AWS accounts to enforce zero-trust security and isolation, as well as compliance with regulations. Microservices and observability traffic is secured via mTLS using NGINX ingress controllers.
  • DevSecOps Best Practices: Infrastructure code is version-controlled, and services are upgraded seamlessly with zero downtime using rolling upgrades, while secrets are stored securely using AWS Secrets Manager.

Through the use of these out-of-the-box cloud services and automation tools, Crest Data built a robust system that enables highly distributed, customer-managed deployments and reduces complexity.

Outcomes

The deployment of the automated XDR platform on AWS resulted in key operational and strategic outcomes for the customer:

  • Drastic Reduction in Deployment Time: By moving away from manual deployment and introducing a zero-touch automation model, the time to deploy new tenant environments was shortened from many hours to less than 30 minutes. This reduced customer onboarding time and improved engineering productivity.
  • Substantial Cost Savings: By moving to scalable, AWS-native services (such as Amazon EKS, EBS, and S3) and using self-hosted GitHub Actions runners, the customer was able to reduce its infrastructure and maintenance costs by over 35%.
  • Accelerated Incident Response: The federated observability platform enabled a single pane of glass view across real-time data for 70-80% faster incident detection and response times. It also successfully lowered the Mean Time to Resolution (MTTR) in all distributed environments.
  • Scalable and Secure Growth: Using Infrastructure as Code (IaC) and GitOps practices allowed for fully automated and repeatable tenant onboarding. This helped the customer deploy their XDR solution across a variety of third-party cloud environments, with no compromise to zero-trust security and data isolation.

About Crest Data

Crest Data is a data and AI-driven technology solutions provider for enterprises and technology innovators across cybersecurity, CloudOps, and cloud security, helping them move faster and more securely. The company has expertise in cloud, cybersecurity, and observability, offering services in CloudOps, XDR monitoring, and Cloud Security Operations to ensure robust, secure, and scalable environments. Crest Data uses AI-powered engineering and managed services to ensure optimal infrastructure, improved threat detection, and effortless cloud migration for enterprises worldwide.