Senior ETL Developer (Extract, Transfer, Load)
Enso Recruitment are excited to announce a recent exclusive partnership with Nisos, the leading US-based Threat Intelligence Software company who are at the early stages of setting up their new operation here in Belfast!
Nisos is is a leading Threat Intelligence Company who are on a mission to build a team here in Belfast who will be responsible for the design and development of a new range of products to be used in the threat intelligence industry.
They are recruiting for a number of roles that will help with various product developments they have in the pipeline while helping to support the wider global team as they grow.
Their tech-enabled services enable some of the world’s largest companies to defend themselves against and more effectively respond to advanced cyber-attacks, disinformation, and abuse of digital platforms.
This is an opportunity to join a well-established and respected organisation during a period of fantastic growth. Although well established in the US, you will be joining a team here in Belfast who are currently in 'Start-up' mode as they start to prepare for various greenfield projects over the coming months.
After being established around 6 years ago, our client's team has grown to about 60 (In the US) and they have plans to add a further 6-10 employees to their Belfast site in the coming months.
Nisos is hiring a senior developer with expertise in data extraction, transformation, and loading (ETL) to lead the effort to ingest a wide array of data into a common model that you help design, and a pipeline that you help build.
Ideally, you will have a breadth of experience in data, ETL tools, and storage. You've unpacked databases in foreign languages saved as zip files inside tar files inside 7z files (Russian nesting data, if you will). You're just as comfortable with outer joins as you are with designing NoSQL schemas. As for ETL pipeline tools (Penatho, NiFi, Luigi, AWS Glue, Snowflake), you've tried them all. For the ones you haven't, you know someone you can ask. You take pride in building robust, fault tolerant code, but you also know when to say, "that's good enough."
Together with colleagues in the US and Northern Ireland, you'll design a data model, select ETL pipeline tools, and fill that pipeline with data goodness. As the team grows, we will rely on you to mentor and train less experienced ETLers.
What Are The Responsibilities?
- Evaluate and select ETL pipeline tools (e.g. Penatho, NiFi, Luigi, AWS Glue, Snowflake)
- Evaluate and select storage tools (e.g. Postgres, Snowflake, DynamoDB, Elastic). Bonus for experience with graph databases
- Design and implement a flexible schema for capturing common data elements
- Design and implement an ETL process for transforming static and dynamic data sets
- Instruct developers ETL of dynamically acquired data to common model
- Mentor junior developers into an ETL developer
- Light system administration of ETL and data systems as needed
Who You Are
- Proficient in one or more scripting languages (e.g. Python, PERL, Ruby). Bonus if you can ETShell (e.g. cut, tr, sed, awk, jq)
- Proficient in at least one and experience with more than one ETL platform (e.g Pentaho, Apache NiFi, Streamsets, Snowflake, AWS Glue, Luigi)
- Proficient in data query languages (e.g. SQL, NoSQL, (JSON|X)PATH, regex, Cypher)
- Competent in software development tools (e.g. IDEs, git, git flow) and methods (e.g. modularity, code reuse, design patterns)
- Competent with AWS including S3, SQS, DynamoDB, Amazon RDS
- Proficient using Mac OS and Linux
Initially, you will work remotely. When conditions permit, our client will open a physical office in Belfast to which the whole team will have the ability to work a hybrid schedule with time in the office and remote.
Travel is not necessary, but you will be given the opportunity to travel to the United States once or twice per year for short periods to meet with the wider company.
Full package to be finalised in the coming weeks.
If you are interested in hearing more about this position, feel free to reach out to Christian Campbell – Principal Recruitment Consultant at Enso: firstname.lastname@example.org