Dan is available for hire
Hire DanDan Larson
Verified Expert in Engineering
Data Engineer and Developer
Location
Portsmouth, Saint John Parish, Dominica
Toptal Member Since
September 29, 2023
Dan is a developer and technical lead with 20+ years of experience developing applications and data environments. He specializes and is certified in Profisee master data management (MDM). His robust experience includes creating financial applications and government military websites and developing data lakes and cloud warehouses. Dan connects with clients to discover how technology might best help them and loves learning new software and tools with every engagement.
Portfolio
Global Food & Agriculture Company
Azure Databricks, Python, SQL, Spark SQL, Azure Data Lake, Data Warehousing...
US Financial Services Company
Profisee MDM, MDM, Azure, SQL, Master Data Management (MDM), Data Migration...
Technology Consulting Firm
C#, ASP.NET, JavaScript, APIs, SQL, Web API, Code Review, Data Engineering...
Experience
Availability
Full-time
Preferred Environment
Microsoft SQL Server, C#, Azure Databricks, Angular, Python
The most amazing...
...project I've completed is a metadata repository that computes database metadata and lineage, displaying it in an easily searchable web interface.
Work Experience
Information Management Engineer, Technical Lead, MDM Architect
2016 - 2023
Global Food & Agriculture Company
- Led the technical implementation of a large-scale enterprise cloud data warehouse using Azure Databricks. Designed and implemented a metadata-driven environment with robust automation between data zones.
- Created automation for job scheduling and monitoring, allowing for meaningful and timely reporting of processing failures. Used ActiveX Data Objects for object and code review and deployment across environments.
- Headed the technical implementation of an enterprise master data solution using Profisee MDM tools.
- Architected a microservice and application layer for master data integrations, including APIs, Azure Service Bus messaging, and custom workflows for data authoring.
- Oversaw developers while creating a reusable codebase for future data-domain additions.
- Built data developer tools, including searchable data lineage and job incident history, combining BMC Control-M automation data with ticket data to show a record of errors for a given job, SSAS, SQL Server metadata, and VLTrader job or history data.
- Drove development efforts for a team working on a C# Web API or Angular intranet site. Conducted code reviews and created a deployment management process.
- Developed a process to crawl integration packages and database objects to display data lineage.
- Supported development and production for an organization with over 150 SQL Server instances, including writing and debugging SQL procedures and views, providing data for ad-hoc reporting, and performance-tuning various SQL objects.
Technologies: Azure Databricks, Python, SQL, Spark SQL, Azure Data Lake, Data Warehousing, ActiveX, Master Data Management (MDM), Microservices, API Integration, Data Integration, Azure Service Bus, Control-M, Automated Data Flows, Data Processing Automation, SQL Server Analysis Services (SSAS), SQL Server DBA, Web API, Debugging, Data Engineering, Spark, PySpark, Databricks, GitHub, Data Analytics, CI/CD Pipelines, Node.js, Big Data, Message Queues, Big Data Architecture, Data Cleaning, Data Aggregation, Data Science, Data Pipelines, Large Data Sets, Data Lakes, Data Modeling, Leadership, Microsoft Excel
MDM Technical Lead & Product Owner
2021 - 2022
US Financial Services Company
- Led the migration of an enterprise Profisee master data system to Azure hosting and organized the MDM code repository and deployment strategy.
- Created automated integrations between the company master data system and other data sources like Salesforce and Snowflake.
- Developed workflows to automate and streamline data steward review of changes.
- Trained data stewards and data governance on usage and practices.
Technologies: Profisee MDM, MDM, Azure, SQL, Master Data Management (MDM), Data Migration, Database Migration, Data Processing Automation, Automated Data Flows, Data Engineering, Data Analytics, CI/CD Pipelines, Big Data, Message Queues, Big Data Architecture, Data Cleaning, Data Aggregation, Data Science, Data Pipelines, Large Data Sets, Data Lakes, Data Modeling, Leadership, Microsoft Excel
Senior Programmer & Analyst
2015 - 2017
Technology Consulting Firm
- Developed a Web API and JavaScript package to simplify creating search pages. Custom classes allow for rapidly creating a full-function search grid with a predictive text search box and record editor.
- Created new data import mechanisms for page-side data or file processing with asynchronous upload to the server. Features allow users to see progress as a file is parsed, interact with errors, and dynamically modify records before server processing.
- Mentored junior developers and established code reviews and training sessions to maximize growth.
Technologies: C#, ASP.NET, JavaScript, APIs, SQL, Web API, Code Review, Data Engineering, GitHub, Data Analytics, CI/CD Pipelines, Amazon Web Services (AWS), Docker, Node.js, NumPy, Message Queues, Data Cleaning, Data Aggregation, Data Pipelines, Data Modeling, Leadership, Microsoft Excel, Excel VBA, Visual Basic for Applications (VBA)
Senior Software Developer
2014 - 2015
Medical GPO
- Supported, maintained, and upgraded the model-view-controller applications, CMS-400 company website, online learning management system, and SQL Server Reporting Services for sales volume.
- Handled database development and administration, including writing stored procedures and maintaining SQL Server Integration Services (SSIS).
- Oversaw data maintenance, backup plans, disaster recovery planning, and upgrades.
Technologies: C#, Web MVC, SQL, Content Management Systems (CMS), Learning Management Systems (LMS), SSRS Reports, SQL Server Reporting Services (SSRS), Database Development, Database Administration (DBA), SQL Server Integration Services (SSIS), Backups, Disaster Recovery Plans (DRP), Data Engineering, Data Cleaning, Data Aggregation, Leadership, Microsoft Excel, Excel VBA, Visual Basic for Applications (VBA)
Consulting Software Engineer
2003 - 2013
Independent Consulting
- Created multiple ASP.NET websites, including a mobile-friendly sales quoting CRM tool, customer access portal, service call tracker, installation status tool, and other reporting unavailable in the company ERP system.
- Implemented commission accounting systems that process millions of dollars of commissions annually using SQL Server databases, .NET applications, Windows desktop and service applications, and ASP.NET websites and web services.
- Developed automated data feed processing for over 90% of transactions. Designed customizable reporting and auditing systems to allow staff to complete audits quickly.
- Constructed a highly featured agent website with features to review statements, search for details, request status updates on cases, and track leads and appointments.
- Authored a highly featured PDF-generating system and reporting API to allow for quick report generation.
- Built a tracking system to monitor customer systems and create alerts for various system failures.
Technologies: C#, SQL, ASP.NET, Customer Relationship Management (CRM), Enterprise Resource Planning (ERP), SQL Server DBA, .NET, Windows, APIs, Microsoft Excel, Excel VBA, Visual Basic for Applications (VBA)
Experience
Enterprise Master Data
Implemented an enterprise MDM solution using Profisee tools.
I created ETL processes to pull data from more than 10 source systems of record into the MDM solution and developed a matching and golden record creation process. I also created workflow processes for master data authoring and review and APIs and service bus interfaces to allow microservice and subscription updates to the master data.
I created ETL processes to pull data from more than 10 source systems of record into the MDM solution and developed a matching and golden record creation process. I also created workflow processes for master data authoring and review and APIs and service bus interfaces to allow microservice and subscription updates to the master data.
Data Command Center (DCC)
Over several years, I led a team in developing a comprehensive data tools solution, including metadata from databases and user-maintained data like business names and data owners, data lineage, and troubleshooting data—combining Control-M job automation with database objects and service ticket data.
This system is a combination of back-end processes that crawl over 50 database servers and several integration servers to calculate the raw metadata and lineage data—with SQL server database and analytics, IBM Db2, and SSIS integrations—and a C# Web API or Angular website for user search, reports, and visualizations.
This tool saves support teams hundreds of critical response hours per year by quickly allowing users to search for data objects of any kind and see the upstream and downstream lineage associated, the automation jobs attached to the various objects, and the recent history of failures and resolutions. I also created a user-browsable flowchart interface using vis.js for data lineage display and browsing and flexible searches for the other kinds of data and relationships available within the DCC.
This system is a combination of back-end processes that crawl over 50 database servers and several integration servers to calculate the raw metadata and lineage data—with SQL server database and analytics, IBM Db2, and SSIS integrations—and a C# Web API or Angular website for user search, reports, and visualizations.
This tool saves support teams hundreds of critical response hours per year by quickly allowing users to search for data objects of any kind and see the upstream and downstream lineage associated, the automation jobs attached to the various objects, and the recent history of failures and resolutions. I also created a user-browsable flowchart interface using vis.js for data lineage display and browsing and flexible searches for the other kinds of data and relationships available within the DCC.
Skills
Languages
C#, SQL, Excel VBA, Visual Basic for Applications (VBA), Python, JavaScript
Tools
Microsoft Excel, GitHub, Spark SQL, Control-M
Storage
Microsoft SQL Server, Data Pipelines, Data Lakes, SQL Server Integration Services (SSIS), Master Data Management (MDM), Data Integration, SQL Server Analysis Services (SSAS), SQL Server DBA, Database Migration, SQL Server Reporting Services (SSRS), Database Administration (DBA)
Other
Profisee MDM, Data Engineering, APIs, Data Analytics, CI/CD Pipelines, Big Data, Message Queues, Big Data Architecture, Data Cleaning, Data Aggregation, Large Data Sets, Data Modeling, Leadership, Azure Databricks, MDM, Azure Data Lake, Web MVC, Data Warehousing, API Integration, Azure Service Bus, Automated Data Flows, Data Processing Automation, Debugging, Data Migration, Code Review, Content Management Systems (CMS), Learning Management Systems (LMS), SSRS Reports, Backups, Disaster Recovery Plans (DRP), Customer Relationship Management (CRM), Enterprise Resource Planning (ERP)
Frameworks
Spark, Angular, ASP.NET, Windows PowerShell, ActiveX, .NET
Libraries/APIs
PySpark, Node.js, Web API, NumPy
Paradigms
Microservices, ETL, Data Science, Database Development
Platforms
Azure, Databricks, Windows, Amazon Web Services (AWS), Docker
Certifications
JANUARY 2022 - PRESENT
Master Data Developer
Profisee
JANUARY 2022 - PRESENT
Master Data Architect
Profisee
JANUARY 2021 - PRESENT
Master Data System Administrator
Profisee
Collaboration That Works
How to Work with Toptal
Toptal matches you directly with global industry experts from our network in hours—not weeks or months.
1
Share your needs
Discuss your requirements and refine your scope in a call with a Toptal domain expert.
2
Choose your talent
Get a short list of expertly matched talent within 24 hours to review, interview, and choose from.
3
Start your risk-free talent trial
Work with your chosen talent on a trial basis for up to two weeks. Pay only if you decide to hire them.
Top talent is in high demand.
Start hiring