AWS Serverless solution for Network Traffic Analysis

Problem

US national wireless service provider had large amount of daily data comming from a cable modem termination system equipment. The client had two-step requirement for the network traffic analyses:

  • Analyze if daily data is valuable for business custom analyses,
  • If it is, propose solution for data extraction and make it available for further ETL processes.

Solution

InterWorks’ analyses and development team proposed and built two-step solution, providing:

  • First phase: Information that data is valuable and has significant meaning for networks traffic analyses on daily, monthly and yearly levels.
  • Second phase: Optimized and secured data transfer serverless solution that provides readable data on Redshift cluster for further ETL analyses.

Our solution is based on Amazon Web Services serverless Lambda functions that are processing incoming IPDR data files. Incoming files are saved in data lake and reside in several S3 buckets, based on different processing stages providing long term storage of network data. Processed data is available on Redshift as warehouse suitable database for further ETL analyses. Solution is built under custom secured VPC, deployed in separate public, private and data subnets. Assigned execution roles and policies have minimum required permissions. Using CloudFormation templates the solution can be quickly deployable to different AWS accounts or as different environments in the same AWS account.

Benefits

  • The client gets a low cost reliable solution running entirely on AWS infrastructure making it fully scalable and upgradable at any moment.
  • Easy integration with third party systems allowing usage of various notification and monitoring tools that provide efficient support process.

Technologies

AWS Lambda functions, AWS CloudFormation, AWS Redshift, AWS DynamoDB, AWS SNS, Node.js. 

Industries

Telecommunication…

Another Case Study?