Team Lead @ From June 2014 to Present (1 year 5 months) Sr Engineer @ Working on the high-throughput, low latency data pipeline to ingest and index terabytes of logs/day -- Kafka + Storm + ElasticSearch
http://www.loggly.com/behind-the-screens/
https://www.youtube.com/watch?v=LpNbjXFPyZ0
* Built common Kafka consumer framework used throughout the pipeline.
* Built high performance, fault tolerant indexing consumer to index data to Elasticsearch(ES). Wrote test bed for measuring ES performance. Single cluster can index 100k events per second
* Developed Storm modules for parsing and classifying logs. Performance tuning of Storm
https://www.loggly.com/what-we-learned-about-scaling-with-apache-storm/
* Implemented distributed, multi-tenant log archiving pipeline to archive customer logs to S3
* Worked closely with operations to provide deployment guides and resolved system outages From March 2013 to June 2014 (1 year 4 months) San FranciscoPrincipal Member of Technical Staff @ * Technical lead for supporting multi-tenancy features in Oracle BI product.
* Implemented role based access control for Oracle BI
* Created re-usable and generic C++ components like LRU Memory Cache and Client State Store. Scalability and concurrency optimizations in C++ server to enable maximum resource sharing. From November 2006 to February 2013 (6 years 4 months) Software Engineer @ As part of a three member team, developed a multi-tiered J2EE analytics application for retail client “Stop and Shop”. Used by 550 executives, this web-based solution provides real time actionable data, key products information and alerts on mobile devices From June 2005 to October 2006 (1 year 5 months) Research Assistant (Bioinformatics Lab) @ Implemented a hybrid clustering scheme in R which improved run-time and produced better results. Published a conference paper with the improved findings. From January 2004 to May 2005 (1 year 5 months) Summer Internship @ Implemented algorithms to cluster product lines based on sales and demographics, extracted association rules between product lines, forecasted production using time series analysis. From January 2003 to June 2003 (6 months)
MS @ Boston University From September 2003 to 2005 BE @ B. M. S. College of Engineering From 1999 to 2003 Suyog Rao is skilled in: Java, C++, Business Intelligence, Cloud Computing, SQL, Big Data, Ruby on Rails, Web Applications, ElasticSearch, Kafka, Storm, Scalability, JavaScript, Algorithms, Distributed Systems, Apache Storm, Software Development, Apache Kafka, Amazon Web Services...