IBM Data Engineer

Introduction
IBM Systems Expert Labs is a Professional Services Team under IBM Technology Expert Labs.
IBM Technology Expert Labs has the proven expertise to help leaders plan design and implement the essential IT infrastructure for what comes next. We offer a wide range of IT infrastructure services for IBM Power Systems™ IBM Z® and IBM LinuxONE™ systems and IBM Storage & Software Defined Infrastructure. Technology Expert Labs – Systems’ consultants perform IT infrastructure services for clients on site helping them solve business challenges gain new skills and discover best practices. Expert Lab has a global presence and can deploy its consultants around the world.
• As a Storage Consultant the role involves providing specialised storage services around the world. The Expert Labs specialist is a highly skilled expert with a deep knowledge of IBM’s Storage Software portfolio and hardware products
• The Expert Labs consultant assume overall technical responsibility for the success of solution construction implementation and system integration in a technology industry or business specialty
• You deliver new and complex high-quality solutions to clients in response to varying business requirements
• You have deep skill and strong hands-on skills in ONE or MORE of the following subject matter areas: Enterprise Storage IBM DS8000 IBM Flash System Products SAN Networking and Tape products Spectrum Protect Spectrum Scale
• Exposure to Containers and Kubernetes technology
• Should be ready to travel (once or twice a month)

Your Role and Responsibilities
As an IBM data engineer you will be responsible for designing constructing installing testing and maintaining highly scalable data management systems. You will work closely with data architects data scientists and business analysts to understand data requirements design data models and implement solutions that optimize data ingestion storage and processing.
Key Responsibilities:
1. Data Pipeline Development: Develop and maintain data pipelines to ingest transform and load structured and unstructured data from various sources into data storage systems such as data lakes data warehouses and databases.
2. Data Modeling: Design and implement efficient data models to support analytical and operational needs ensuring data integrity accuracy and consistency across different data sets.
3. Data Integration: Integrate data from multiple sources including internal databases external APIs third-party data providers and streaming data sources using appropriate tools and technologies.
4. Data Processing: Implement data processing workflows using tools like Apache Spark Hadoop or IBM Cloud Pak for Data to perform ETL (Extract Transform Load) operations data cleansing and data enrichment.
5. Database Management: Manage and optimize databases including relational databases (e.g. IBM Db2 PostgreSQL) and NoSQL databases (e.g. MongoDB Cassandra) to ensure high performance availability and scalability.
6. Data Quality Assurance: Develop and implement data quality checks validation rules and monitoring processes to ensure data accuracy completeness and consistency over time.
7. Performance Tuning: Optimize data pipelines queries and processes for performance scalability and efficiency leveraging techniques such as indexing partitioning and caching.
8. Collaboration: Collaborate with cross-functional teams including data scientists business analysts software engineers and stakeholders to understand data requirements prioritize tasks and deliver data-driven solutions.
9. Documentation: Create and maintain documentation for data pipelines data models data dictionaries and technical specifications to ensure transparency repeatability and knowledge sharing within the team.

Required Technical and Professional Expertise
1. Bachelor’s or Master’s degree in computer science information technology data engineering or a related field.
2. Proficiency in programming languages such as Python Java Scala or SQL for data manipulation scripting and automation.
3. Experience with data pipeline orchestration tools like Apache Airflow IBM DataStage or Informatica.
4. Strong knowledge of database management systems (DBMS) data warehousing concepts and data modeling techniques.
5. Familiarity with big data technologies such as Apache Hadoop Apache Spark Kafka and distributed computing frameworks.
6. Experience with cloud platforms like IBM Cloud AWS Azure or Google Cloud Platform for data storage processing and analytics.
7. Excellent problem-solving skills analytical thinking and attention to detail in designing and troubleshooting data solutions.
8. Effective communication skills to collaborate with diverse teams present technical findings and explain complex concepts to non-technical stakeholders.
9. Ability to work independently prioritize tasks and adapt to changing project requirements in a dynamic environment.

Preferred Technical and Professional Expertise
1. Certification in data engineering or related technologies (e.g. IBM Certified Data Engineer AWS Certified Data Analytics – Specialty).
2. Experience with data visualization tools like Tableau Power BI or IBM Cognos for creating interactive dashboards and reports.
3. Knowledge of machine learning concepts data science workflows and statistical analysis techniques.
4. Familiarity with agile development methodologies DevOps practices and version control systems (e.g. Git).
5. Experience with containerization technologies such as Docker and orchestration tools like Kubernetes for deploying and managing data applications.