DATA部門 Data Engineer

Insight & Data Tokyo, Japan


Position at Capgemini Japan


·        Collaborate with users and project team on requirements and expectations

·        Analyze business requirements, create proper data solutions that meets functional and non-functional requirements

·        Define the data architecture, structure, storage, security model of data in HDFS, Hive, HBase, Solr, RestAPI

·        Design and implement highly scalable data pipelines in Scala/Spark/Java for processing of extreme large amount of data

·        Create and maintain reusable frameworks for data ingestion, validation, normalization, and transformation

·        Automate Unit testing and release with CI/CD pipeline

·        Create scripts for operations, releases, data migrations, data validations

·        Maintain data lineage information

·        Provide technical support for releases, issues, and incidents



·        Solid knowledge and experience of Hadoop ecosystem: HDFS, Hive, HBase, Phoenix, Solr, Oozie, Impala, Knox, Zookeeper, YARN, Spark, Livy, HUE, Zeppelin

·        Strong programming skills in Scala and Spark

·        Strong capabilities in designing Database schemas on data platform

·        Good knowledge of Oracle database, MS-SQL Server, MongoDB, Kerberos, Linux, networking

·        Proficient in SQL, shell scripting, git, unit testing, CI/CD tools

·        Understanding of design principles, coding standards and best practices

·        Good problem-solving skills

·        Good data analytics skills

·        Excellent communication, presentation, and interpersonal skills


·        Understanding of REST API, micro-services, ETL, and Salesforce is preferred

·        Hand-on experience with Cloud services is preferred

·        Understanding of streaming data processing and change data capture is preferred

·        Experience with BI tools is preferred


·        Japanese: intermediate

·        English: intermediate



キャップジェミニについての詳細は、 をご覧ください。
Get the Future You Want - 望む未来を手に入れよう