1. Develop and manage data pipeline & data cleansing : Develop and manage design and flow of data storage, collection, processing and cleansing of data in accordance with the existing data warehouse architecture so that data can be analyzed and used by users
2. Monitoring & improvement data pipeline : Routinely monitor, manage data pipeline and schedule so that operational processes run optimally in terms of performance and cost
3. Service request fulfillment & troubleshooting incident or case : Perform request fulfillment and troubleshooting of incidents that occur in the scope of work in accordance with the SLA and create PICAPA documentations
4. Documentation Create and manage technical documentation (Technical Specification Document) data Pipeline, Schedule and Data Catalog to ensure the documentation can be used as a standardization reference.
5. Ensure compliance with regulatory requirements : Establish and maintain every process to ensure compliance with privacy policies and data security.Develop and implement procedures (SOP) to ensure data retention and disposal requirements are in accordance with applicable agreements.
Qualification:
1. Berpengalaman dalam penggunaan Bahasa pemrograman SQL dan Python.
2. Berpengalaman dalam ETL Tools (SSIS, Pentaho, Talend, Airflow)
3. Berpengalaman dalam penggunaan Database (SQL Server, PostgreSQL, Oracle, Mysql)
4. Berpengalaman dalam Machine learning.
5. Cloud GCP (Bigquery, Dataflow, Data Fusion), Cloud AWS.
3. Streaming Pipeline (Kafka, RabbitMq, Pub Sub, CDC, Debezium).
4. dbt, prefect, airflow.
Jenis Pekerjaan: Kontrak
Panjang kontrak: 3-12 bulan
Pengalaman:
- Data Engineer: 1 tahun (Diutamakan)
