Role: DataStage Administrator for On-premises to AWS Migration (6-12 years' experience)
Location: Miami, FL (Hybrid)- Consultant should be in FL or willing to relocate from Day-1
Note: This is not a DataStage Developer Kindly look for the matching candidates and do provide the suitable candidates
Visa: Any Visa except H1B Transfer (All consultants should be on W2)
Project Overview:
Objective: To migrate an existing on-premises IBM InfoSphere DataStage environment to the cloud (AWS), ensuring seamless data integration, transformation, and ETL (Extract, Transform, Load) processes for telecom-related data.
Roles & Responsibilities of a Datastage Administrator in this Migration Project:
Pre-Migration Tasks:
Conduct a comprehensive audit of the current Datastage setup.
Identify and document all data flows and dependencies, particularly those related to the telecom domain (e.g., call data records, billing data, network performance metrics).
Work with other stakeholders (e.g., network engineers, telecom domain experts) to ensure all use cases are understood.
Choose appropriate AWS services for Datastage components, such as:
EC2 for computing
S3 for data storage
RDS/Aurora or Redshift for databases and analytics
Lambda and Glue for serverless processing
Migration Planning:
Create a migration strategy that includes timelines, resource allocation, and risk mitigation steps.
Develop a cutover plan to switch from on-premises to AWS with minimal service disruption.
Identify the migration method: lift-and-shift vs. re-architecture.
Coordinate with telecom domain architects for alignment with business requirements (e.g., billing cycles, data consistency, SLAs).
Infrastructure Setup on AWS:
Provision AWS resources (e.g., EC2 instances with suitable specifications, storage in S3, RDS for databases).
Configure IAM roles for access control and security.
Set up AWS Direct Connect or VPN if private network connectivity is required between on-prem and cloud.
Implement data security and compliance measures in line with telecom industry regulations (e.g., GDPR, HIPAA).
Datastage Migration Execution:
Migrate Datastage ETL jobs, sequences, and configurations to the cloud.
Ensure that existing custom scripts and connectors work in the cloud environment.
Adjust job parallelism and resource allocation to leverage AWS scalability features like Auto Scaling.
Address any issues with file transfer protocols (e.g., FTP, SFTP) between on-prem and AWS.
Testing & Validation:
Unit testing and integration testing of Datastage jobs post-migration.
Verify data accuracy and completeness for telecom datasets.
Test data transformation rules and ensure that they function correctly in the AWS environment.
Conduct performance testing to verify that the new AWS infrastructure can handle the data loads and workloads typical of the telecom domain.
Post-Migration Support:
Provide ongoing monitoring of Datastage jobs and AWS resources using CloudWatch, Datadog, or similar.
Troubleshoot performance issues (e.g., memory, CPU usage) and optimize configurations.
Fine-tune resource allocation to ensure cost efficiency without sacrificing performance.
Work with the security team to ensure data protection and compliance.
Documentation:
Document the new cloud-based Datastage environment and any customizations made.
Update disaster recovery plans to account for cloud-based resources.
Training & Knowledge Transfer:
Conduct training sessions for the internal team on the new AWS environment, tools, and best practices.
Ensure that knowledge is transferred for long-term management of the Datastage system in the cloud.