Designing and implementing scalable data pipelines to collect, process, transform, and store large volumes of structured and unstructured data.
Developing efficient ETL (Extract, Transform, Load) processes to ensure the availability of clean and accurate data for analysis.
Building and maintaining data warehouses or data lakes to enable easy access to structured datasets for reporting and analytics purposes.
Collaborating with cross-functional teams to identify opportunities for improving data quality, reliability, performance, and efficiency.
Implementing appropriate security measures to protect sensitive data from unauthorized access or breaches.
Monitoring system performance, identifying bottlenecks or issues, and implementing optimizations to ensure high availability and scalability.
Conducting thorough testing of developed solutions to ensure they meet functional requirements and performance expectations.
Mentoring junior team members by providing guidance on best practices in software development, database design, and data engineering techniques.
Staying up-to-date with emerging technologies in the field of big data processing, cloud computing, distributed systems, etc., and evaluating their potential applications in the organization's context.
Collaborating with stakeholders to understand their business needs and translating them into technical requirements.
To succeed in this role
Bachelor's or Master's degree in Computer Science, Engineering or a related field.
Proven experience as a Data Engineer or similar role with a focus on building scalable data processing systems.
Strong programming skills in languages like Python / Java / Scala along with proficiency in SQL for querying databases.
Experience working with big data technologies such as Hadoop ecosystem (HDFS, MapReduce), Apache Spark / PySpark for distributed computing.
Proficiency in working with relational databases (e.g., MySQL) as well as NoSQL databases (e.g., MongoDB).
Familiarity with cloud platforms like GCP / AWS / Azure for deploying scalable infrastructure using services like S3, EC2 instances etc.
Knowledge of containerization technologies like Docker / Kubernetes is desirable but not mandatory.
Strong problem-solving skills with an ability to analyze complex datasets efficiently.
Excellent communication skills to effectively collaborate with cross-functional teams.
Candidate should have a strong understanding of data platforms, including architecture, components, and functionalities.
Candidate should be proficient in data integration, storage, processing, analytics, and visualization technologies.
Knowledge of programming languages like SQL, Python, or R is also beneficial.
为此搜索创建职位提醒
Engineer • Hong Kong HK, HK
相关职位
Data Center Infrastructure Engineer, Data Center Operations
Amazon Data Services Hong Kong LimitedHong Kong, Hong Kong SAR, HKG
AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure.In other words, we’re the people who keep the cloud running.We support all AWS data c...展示更多上次更新时间:30 天前
Data Security Engineer (req752)
KLN Logistics (Hong Kong) LimitedHong Kong
Develop and implement data security policies, procedures, and controls to protect sensitive information and in accordance with applicable regulations (e.
Classify and secure data based on its sensit...展示更多上次更新时间:30 天前
Data Center Infrastructure Engineer, Data Center Operations
Amazon Data Services Hong Kong Limited - G43Hong Kong, Hong Kong SAR, HKG
AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure.In other words, we’re the people who keep the cloud running.We support all AWS data c...展示更多上次更新时间:30 天前
Contract Senior Data Engineer
Morgan McKinleyHong Kong
Responsible for generating ad-hoc reports for marketing (10 different country) fulfilment.Responsible for creating and maintaining Power BI datasets, dashboards and manage access for stakeholders w...展示更多最后更新时间: 12天前
FS Institution - Data Engineer (Centre of Excellence)
HaysHong Kong
Our client is one of the largest FS Institution who is seeking a skilled Data Engineer to help build and optimize their next-generation Data and Analytics Platform for Centre of Excellence.Build Sc...展示更多最后更新时间: 12天前
Market Data Production Support Engineer, Associate - Technology
Morgan StanleyHong Kong
Working alongside the existing team members on a shift basis.Monitoring and incident management on a daily basis.Daily Ready For Business cover and check outs.
End Of Day cover and check outs.Manage...展示更多最后更新时间: 6天前
Python Engineer – Quant Data Infrastructure (Systematic Trading)
Ashford Benjamin LtdHong Kong, Hong Kong
B Posted byRecruiterMy client, a top-tier systematic trading firm, is looking for a Python-focused engineer to build and optimize the data pipelines that power its quantitative research.This role s...展示更多最后更新时间: 17天前
Data Engineer
Tower Research CapitalHong Kong
Collaborating with portfolio managers, researchers and traders to understand their data requirements and to find data solutions.
Onboarding and managing vendor datasets used extensively in Tower tra...展示更多最后更新时间: 17天前
[LPS] Data Engineer
LenovoHong Kong, Hong Kong
Translate Data Pipeline requirements into data pipeline design, guide and direct the design by working closely with a range of stakeholders including the rest of the architecture team, external dev...展示更多最后更新时间: 17天前
Data Engineer / Database Administrator (Up to 65K)
ManpowerHong Kong
Build and maintain a reliable, high-performance data storage system for genomic data.Create clear system designs, including data models and flow diagrams.
Recommend suitable storage solutions (e.Ide...展示更多最后更新时间: 4天前
(Senior) Data Engineer
Crypto.comHong Kong, Other / Non-US, Hong Kong,
Quick Apply
Build a universal data platform, meet real-time / offline computing and storage requirements;.Define the data pipeline scheme according to the demand scenarios, and deliver it by multiple methods bas...展示更多最后更新时间: 6天前
Full Stack Software Engineer - Systematic Data Platform.
Millennium ManagementHong Kong, Hong Kong
Full Stack Software Engineer - Systematic Data Platform.We are building a world class systematic data platform which will power next generation of our systematic portfolio engines.We are looking fo...展示更多最后更新时间: 17天前
Data Engineer
Captar PartnersHong Kong
Our client is a cutting-edge retail innovation company, they provide cutting-edge data solutions, cloud infrastructure, and AI-powered analytics to optimize operations and enhance customer experien...展示更多最后更新时间: 17天前
新!
Senior Data Engineer
Talent Fields Hong Kong Company LimitedHong Kong
Job Description of Senior Data Engineer.We are actively seeking a highly skilled Senior Data Engineer to join our dynamic team.
As a Senior Data Engineer, you will play a pivotal role in driving our...展示更多最后更新时间: 20小时前
Senior Consultant, Data Engineer, Technology Enablement
KPMGHong Kong
As a Data Engineer, you will build the enterprise-grade data solution to execute clients' vision.Understand the current state process of the clients and bring it to future state improvement and sol...展示更多最后更新时间: 7天前
Data Engineer
OKXHong Kong
Design and build resilient and efficient data pipelines for both batch and real-time streaming data.Architect and design data infrastructure on cloud using industry standard tools.Execute projects ...展示更多最后更新时间: 9天前
Data Engineer | Singapore or Hong Kong
Oxford KnightHong Kong
High-frequency prop trading firm with offices worldwide looking for a talented Data Engineer to design, implement and maintain the infrastructure which underpins the firm’s data management processe...展示更多上次更新时间:30 天前
Data Engineer.
PCCWHong Kong HK, HK
Translate Data Pipeline requirements into data pipeline design, guide and direct the design by working closely with a range of stakeholders including the rest of the architecture team, external dev...展示更多上次更新时间:30 天前