Teradata Think Big Senior Data Engineer in Germany

Think Big Senior Data Engineer

Description

Primary Responsibilities:

As a Senior Data Engineer I, you will provide technical leadership to clients in a team that designs and develops path-breaking large scale cluster data processing systems. You will mentor sophisticated organizations on large scale data and analytics and work with client teams to deliver results.

Additionally, as a senior member of our Consulting team, you will help Think Big, A Teradata Company establish thought leadership in the big data space by contributing white papers, technical commentary and representing our company at industry conferences.

Secondary Responsibilities:

Design and develop code, scripts and data pipelines that leverage structured and unstructured data integrated from multiple sources. Software installation and configuration. Participate in and help lead requirements and design workshops with our clients. Develop project deliverable documentation. Lead small teams of developers and coordinate development activities. Mentor junior members of the team in software development best practices. Other duties as assigned.

Qualifications

Job Qualifications:

  • Proven expertise in production software development

  • 7 years of experience programming in Java, Python, SQL, or C/C

  • Proficient in SQL, NoSQL, relational database design and methods for efficiently retrieving data

  • Strong analytical skills

  • Creative problem solver

  • Excellent verbal and written communications skills in German and English

  • Strong team player capable of working in a demanding start-up environment

  • Experience building complex and non-interactive systems (batch, distributed, etc.)

Preferred Knowledge, Skills and Abilities:

  • Prior consulting experience

  • Experience with Hadoop, Hive, Pig, Avro, Thrift, Protobufs and JMS: ActiveMQ, RabbitMQ, JBoss, etc.

  • Dynamic and/or functional languages (e.g., Python, Ruby, Scala, Clojure)

  • Experience designing and tuning high performance systems

  • Prior experience with data warehousing and business intelligence systems

  • Professional or academic background that includes mathematics, statistics, machine learning and data mining

  • Linux expertise

  • Prior work and/or research experience with unstructured data and data modeling

  • Familiarity with different development methodologies (e.g., agile, waterfall, XP, scrum, etc.)

  • Broader experience with spring ecosystem including spring-batch, spring-mvc, and spring-hadoop

  • Standards-based REST implementation

  • Configure a Jenkins build, create/update a Jira ticket, enable Automated Tests in gradle/maven build

  • Vagrant, Docker

  • Familiar with OSI stack and proper use of HTTP verbs etc.

  • Hive Tuning, Hive physical design (file formats, compression, partitioning, bucketing), Hive DSS Queries, Hive for Data Science, Hive for running R/Python (Streaming), Hive Transactions, Hive-HBase

  • Knows how to tune a job including parameters, more efficient API calls. Understands Spark SWL. Understands Spark Streaming and can transform a DStream.

  • Ability to write advanced UDFs, Serdes, input-loaders, log analysis, how the logical operators map to the lower, level physical implementation

  • Setup and leverage output from Ganglia, Nagios, Ambari, Cloudera Manager, etc.

  • Able to link Kerberos KDC to a backing LDAP or Active Directory authentication provider

  • Advanced understanding of access-driven key design, appropriate denomalization, use of co-locating records of differing schemas in a single table etc.

  • Understanding of best practices for Hive schemas. Denormalization, partitioning and bucketing, file formats.

  • Able to create, write to and read from Kafka topic. Understanding of key partitioning (just how it works), able to maintain an offset in the topic for consistent reading

Job Abilities:

Must be able to sit for long periods of time working on computers. Must be able to travel to client sites at least 50% of the time. Must be able to interact and communicate with the client in meetings. Must be able to write programming code in applicable languages. Must be able to write project documentation in English.

Education:

Bachelor's Degree or foreign equivalent in Computer Science or related technical field followed by six (6) years of progressively responsible professional experience programming in Java, Python or C/C . Experience with production software development lifecycle. Experience with Linux, SQL, relational database design and methods for efficiently retrieving data. Experience building complex and non-interactive systems (batch, distributed, etc.).

OR

Master's Degree or foreign equivalent in Computer Science or related technical field. Four (4) years of experience programming in Java, Python or C/C . Experience with production software development lifecycle. Experience with Linux, SQL, relational database design and methods for efficiently retrieving data. Experience building complex and non-interactive systems (batch, distributed, etc.).

Employer will accept any suitable combination of education, training, or experience.

Job: Services/Consulting

Schedule: Full-time

Primary Location: Europe, Middle East & Africa-Germany

Organization: TB Services International

Req ID: 168391