Successfully led a project to begin consolidating all company data onto HDInsight (Hortonworks Hadoop) and Data Lake Store with Azure. I built, configured, automated, and documented the HDInsight instances and storage provisions and wrote Hive scripts to prove out ETL methods. During the project, I discovered several issues and bugs and assisted Microsoft support in documenting and reproducing the issues for Hortonworks. I worked around the bugs and Hive's SQL implementation, ingesting data into efficiently partitioned and bucketed ORC files to reduce storage consumption and increase performance. While I worked with cluster configuration, benchmarking, and modelling, a fellow data engineer focused on automating ETL directly from our production database using Azure Data Factory. The project proved successful, however our company was undergoing a round of re-capitalization which led to our acquisition--essentially killing this project and all others which were in flight at the time.
On a project for Harley Davidson, I built a visualization and reporting solution for SendGrid email metrics. This involved streaming email status messages as JSON from the SendGrid web hook to an AWS Lambda function. The Lambda function would perform some transformation of the data and send it to a SQL Server instance on EC2/S3. I used a job on SQL Server to create models that made it easy for users to query and filter. I developed the visualization solution in Power BI which included: dashboard, data tables, and graphs that were all tied to user-defined filters. This let HD dealers access a history of email click-through statistics to help analyze customer engagement.
I built an analytics API for survey data; making it possible for applications and BI tools to easily visualize metrics. Taking a process where data scientists would ingest and manipulate survey data in R, I built a solution in Cisco Information Server that would publish that survey data as an API from a SQL Server database instead. Using the R scripts as a "template" for the API parameters, I worked with data scientists to clarify requirements and verify my models via the API. Added Value intended to use this system to power metrics on the Harley Davidson (HD) dealership portal.
Converted 3 terabytes of account servicing and transaction data to a completely new data model going from a monolithic database with hundreds of tables to 15 domain-based databases--supporting a full application stack rewrite. I worked with the business, partners, and development teams to support a hard cut over on a strict deadline. I wrote most of the conversion ETL in TSQL/SSIS while leading a team of developers and analysts assisting me on other aspects of the project—both offshore and in-house.
Re-engineered a Green Dot API implementation that supports over a billion dollars per year in transactions. The system was rebuilt at one point, by another developer, to support a new API that Green Dot uses to interact with our systems to verify and approve cash loads from Wal-Mart. The implementation was not designed properly, exhibiting serious bugs that would prevent cardholder funds from being credited to their account or reconciled with the bank. I documented the root causes and redesigned the data flow to support the required transaction resiliency. Lacking a formal project management team, I recruited another developer to implement and deploy the changes with me--successfully resolving the issues.
Reduced downtime and outages related to application performance and stability issues for major customer facing and customer service applications and websites. A ground-floor member of a new Application Support group, I quickly learned a large, multi-faceted system and drove improvements to the software and database code. Writing code and submitting detailed suggestions to the application development team, working with them to develop, enhance, test, and verify changes and document data flows. Working alongside business users that would report problems, I identified solutions and followed through to see that the solutions were implemented, maintaining communication with the users to ensure they were satisfied. After these improvements and the introduction of extensive business logic alerting, a 66% reduction in down-time year over year was realized, saving up to a million dollars in call center expenses.
Successfully integrated a third-party application to provide a real-time security badge printing and tracking solution for four college campuses. I took a single user, Microsoft Access based, solution suffering from access contention and data loss to a true multi-user, transactional solution on Microsoft SQL Server. Implementing the software upgrade, converting data, and writing custom software to support ERP data integration and reporting I eliminated contention and data loss while greatly enhancing customer service and creating a pleasant workflow for the users. I created documentation that enabled help desk technicians to support the software after my departure.
Things you might want me in the room for:
I'm always looking for ways to answer emerging business needs in ways that let the technology teams sleep at night and enjoy their work. I try to hold myself and others to a higher standard--keeping in mind that we are all human with diverse needs and abilities. I can say with confidence that: people like working with me, I always try to keep it real, and I’m not afraid to execute.