Big Data Architect
Posted a week ago by MBR Partners
We are headhunting for an AiOps Architect and I would like to run the role past you to see if you have any interest or could recommend someone to me.
Our client is a mid-sized leader in AiOps Telecoms software products related to network management/service assurance.
The role involves designing and supporting the implementation of our client's Cloud Native Big Data Analytics infrastructure layer. This infrastructure serves as the backbone for supporting AI/ML workflows, enabling both real-time and batch processing for
training and inference purposes. The typical data throughput to support is in
the range of-based analysis on continuous streams of Terabytes of data per
hour.
We are looking for someone with the following background..
o Software development background, with major experience in
intense:
▪ back-end data processing,
▪ data lakehouse,
o Hands-on experience of Big Data opensource technologies such as:
▪ Apache Airflow,
▪ Apache Kafka,
▪ Apache Pekko,
▪ Apache Spark & Spark Structured Streaming,
▪ Delta Lake,
▪ AWS Athena, Trino,
▪ MongoDB,
▪ AWS S3, MinIO S3
o Proven successful hand-on experience of:
▪ setting up data governance tooling and processes
(schema registry, data lineage control) and data access
control
▪ setting up data pipelines for model training and inference
▪ Kubernetes or Openshift in context of Big Data analytics,
▪ AWS services In the context of both on-premise and SaaS Telco Service Assurance product development, your role is to: o Lead the general vision for a coherent and future-proof Big Data Analytics for AI/ML framework o Be an innovation force to leverage GenAI technologies to facilitate and accelerate the delivery of the big data jobs Version 2.0 | June 2024 o Build and validate the high-level design of the necessary components supporting the Roadmap requirements, including Test Strategy and Performance and Scalability validation o Build the Project Design (work packages, skillsets, efforts, dependencies, resourcing) accompanying the architecture for every project iteration o Communicate the architecture to the development teams and accompany the development process through periodic detailed reviews, ensuring architecture is understood and respected o Produce quarterly Technology Watch reports on the latest Big Data Analytics landscape o Setup rigorous selection process and support selection of appropriate 3rd-party components, supporting the Big Data Analytics stack o Educate/evangelize the development teams on Analytics design best practices o Review low-level designs and critical code changes produced by the development teams o Support all Big Data Analytics-related technical questions in context of RFI/RFQ responses o Provide feasibility study / high-level costings in response to 3 to 5 year roadmap high-level strategy
Our client is a mid-sized leader in AiOps Telecoms software products related to network management/service assurance.
The role involves designing and supporting the implementation of our client's Cloud Native Big Data Analytics infrastructure layer. This infrastructure serves as the backbone for supporting AI/ML workflows, enabling both real-time and batch processing for
training and inference purposes. The typical data throughput to support is in
the range of-based analysis on continuous streams of Terabytes of data per
hour.
We are looking for someone with the following background..
o Software development background, with major experience in
intense:
▪ back-end data processing,
▪ data lakehouse,
o Hands-on experience of Big Data opensource technologies such as:
▪ Apache Airflow,
▪ Apache Kafka,
▪ Apache Pekko,
▪ Apache Spark & Spark Structured Streaming,
▪ Delta Lake,
▪ AWS Athena, Trino,
▪ MongoDB,
▪ AWS S3, MinIO S3
o Proven successful hand-on experience of:
▪ setting up data governance tooling and processes
(schema registry, data lineage control) and data access
control
▪ setting up data pipelines for model training and inference
▪ Kubernetes or Openshift in context of Big Data analytics,
▪ AWS services In the context of both on-premise and SaaS Telco Service Assurance product development, your role is to: o Lead the general vision for a coherent and future-proof Big Data Analytics for AI/ML framework o Be an innovation force to leverage GenAI technologies to facilitate and accelerate the delivery of the big data jobs Version 2.0 | June 2024 o Build and validate the high-level design of the necessary components supporting the Roadmap requirements, including Test Strategy and Performance and Scalability validation o Build the Project Design (work packages, skillsets, efforts, dependencies, resourcing) accompanying the architecture for every project iteration o Communicate the architecture to the development teams and accompany the development process through periodic detailed reviews, ensuring architecture is understood and respected o Produce quarterly Technology Watch reports on the latest Big Data Analytics landscape o Setup rigorous selection process and support selection of appropriate 3rd-party components, supporting the Big Data Analytics stack o Educate/evangelize the development teams on Analytics design best practices o Review low-level designs and critical code changes produced by the development teams o Support all Big Data Analytics-related technical questions in context of RFI/RFQ responses o Provide feasibility study / high-level costings in response to 3 to 5 year roadmap high-level strategy
- Type:
- Permanent
- Contract Length:
- N/A
- Contact Name:
- Login or register to view
- Job Reference:
- db_0ea24938eb67e6b6231a920405936200__1706
- Job ID:
- 221966816
Remember: You should never send money to a prospective employer or disclose any financial information. Should you encounter any job listings requesting payments or financial details, please reach out to us immediately. For further guidance, visit jobsaware.co.uk.
