I am currently doing very similar role for other client based on hourly rate and available immediately. Below is my summary.
I have extensive Big Data experinece, and designed and delivered a metadata-driven data ingestion have extensive Big data experience and develped a number of PySpark/Scala framework which ingests data from various data sources to Data Hub (HDFS) then integrates, transforms, and publishes to target sources including Kafka, RDBMS (Teradata, Oracle, and SQL Server) and SFTP, etc. Technologies used for the project including AWS, EMR, Glue, Redshift, S3, Python, Spark, Spark SQL, Hadoop, HDFS, Hive, Cassandra, Hbase, Kafka, NIFI, and Atlas.
I created Scala/PySpark based framework which ingests data from customer rating bureau including Equifax, Illion, and Experian. Designed the entire JSON/XML explosion pattern which involves multi-level JSON/XML explosion and normalized table creation in HDFS platform using Scala/PySpark, Hive, SparkSQL, and Hbase. Created entire downstream conceptual, logical, and physical data models for downstream users including credit risk analysts and data scientists