Фільтрувати

Мої недавні пошуки
Фільтрувати по:
Бюджет
до
до
до
Тип
Навички
Мови
    Статус роботи
    1,746 pyspark dataframe робіт знайдено, ціни вказані в USD

    I'm in need of a Machine Learning Engineer who can migrate our existing notebooks from RStudio and PySpark to AWS Sagemaker. Your task will be to: - Understand two models I have running locally. One is a Rstudio logistic regression model, and the other is a pySpark XGboost also running on local. - Migrate These two models to AWS SAGEMAKER. Data will be on S3 -Prepare models to run on sagemaker totally, so that we can do training and testing 100% on sagemaker.-Models are already running on a local computer, but I need to move them to Sagemaker 100%. Data is on S3 already. -You need to configure and prepare Sagemaker from end to end, and teach me how you did it, since I need to replicate it in another system. -I will give you the data and access to AWS Ideal Skills and...

    $224 (Avg Bid)
    $224 Сер. заявка
    13 заявки

    The Data Engineer contractor role will be a project based role focused on migrating data pipelines from legacy infrastructure and frameworks such as Scalding to more modern infrastructure we support such as Spark Scala. This role will be responsible for: Analyzing existing data pipelines to understand their architecture, dependenci...Requirements The ideal candidate is a Data Engineer with considerable experience in migrations and Big Data frameworks. Must-Haves Scala programming language expertise Spark framework expertise Experience working with BigQuery Familiarity scheduling jobs in Airflow Fluency with Google Cloud Platform, in particular GCS and Dataproc Python programming language fluency Scalding framework fluency Pyspark framework fluency Dataflow(Apache Beam) framewor...

    $303 (Avg Bid)
    $303 Сер. заявка
    8 заявки

    Hi, Please apply only individual. Agency can apply but budget should not be more than mentioned. Role : GCP Engineer (OTP) Exp : 7 + yrs SHIFT: IST Cloud Storage Buckets, BigQuery (SQL, Data Transformations and movement) Airflow (python, DAGs), DBT IAM Policies PyCharm Databricks (pySpark), Azure DevOps Clear and confident communication

    $1424 (Avg Bid)
    $1424 Сер. заявка
    8 заявки

    I'm in need of a Machine Learning Engineer who can migrate our existing notebooks from RStudio and PySpark to AWS Sagemaker. Your task will be to: - Understand two models I have running locally. One is a Rstudio logistic regression model, and the other is a pySpark XGboost also running on local. - Migrate These two models to AWS SAGEMAKER. Data will be on S3 -Prepare models to run on sagemaker totally, so that we can do training and testing 100% on sagemaker.-Models are already running on a local computer, but I need to move them to Sagemaker 100%. Data is on S3 already. -You need to configure and prepare Sagemaker from end to end, and teach me how you did it, since I need to replicate it in another system. -I will give you the data and access to AWS Ideal Skills and...

    $400 (Avg Bid)
    Угода про нерозголошення
    $400 Сер. заявка
    3 заявки

    The Data Engineer contractor role will be a project based role focused on migrating data pipelines from legacy infrastructure and frameworks such as Scalding to more modern infrastructure we support such as Spark Scala. This role will be responsible for: Analyzing existing data pipelines to understand their architecture, dependenci...Requirements The ideal candidate is a Data Engineer with considerable experience in migrations and Big Data frameworks. Must-Haves Scala programming language expertise Spark framework expertise Experience working with BigQuery Familiarity scheduling jobs in Airflow Fluency with Google Cloud Platform, in particular GCS and Dataproc Python programming language fluency Scalding framework fluency Pyspark framework fluency Dataflow(Apache Beam) framewor...

    $305 (Avg Bid)
    $305 Сер. заявка
    10 заявки

    I'm looking for a programmer proficient in either R or Python, who has worked with the Interactive Brokers API before. The primary goal of this project is to create a list of functions that can be used to establish a connection with IBKR, source data from it, and place order...using API, data sourcing order placement, fectching live positions and sourcing market depth for given stocks - Transform the output in desired format (preferably in R) Ideal skills/experience: - Proficiency in either R or Python - Previous experience working with the Interactive Brokers API - Strong understanding of stock market data and trading The final output should be in the form of a dataframe or list of data frames, making it easy to use in R. Prior experience with financial data or trading platf...

    $124 (Avg Bid)
    $124 Сер. заявка
    7 заявки

    I am looking for a dedicated specialist well-versed in using Databricks and PySpark for data processing tasks, with a primary focus on data transformation. With the provision of JSON format files, you will perform following tasks: - Carry out complex data transformations - Implement unique algorithms to ensure efficient data processing - Test results against required benchmarks Ideal Skills: - Proficient in Databricks and PySpark. - Must possess a solid background in data transformation. - Experience handling large JSON datasets. The end goal is to achieve seamless data transformation leveraging the power of Databricks and PySpark, enhancing our ability to make informed business decisions. Please provide your completed projects, and the strategies you've used ...

    $46 / hr (Avg Bid)
    $46 / hr Сер. заявка
    29 заявки
    AWS Data Engineer- 7+ Yrs(India) Закінчився left

    ...functions to handle data quality and validation. -Should have good understanding on S3,Cloud Formation, Cloud Watch, Service Catalog and IAM Roles -Perform data validation and ensure data accuracy and completeness by creating automated tests and implementing data validation processes. -Should have good knowledge about Tableau, with creating Tableau Published Datasets and managing access. -Write PySpark scripts to process data and perform transformations.(Good to have) -Run Spark jobs on AWS EMR cluster using Airflow DAGs.(Good to have)...

    $1487 (Avg Bid)
    $1487 Сер. заявка
    22 заявки
    Experienced Data Engineer Needed Закінчився left

    ...Stay current with new technology options and vendor products, evaluating which ones would be a good fit for the company Troubleshoot the system and solve problems across all platform and application domains Oversee pre-production acceptance testing to ensure the high quality of a company’s services and products Skill Sets: Strong development experience in AWS Step Functions, Glue, Python, S3, Pyspark Good understanding of data warehousing, Large-scale data management issues, and concepts. Good experience in Data Analytics & Reporting and Modernization project Expertise in at least one high-level programming language such as Java, Python Skills for developing, deploying & debugging cloud applications Skills in AWS API, CLI and SDKs for writing applications Knowledge...

    $718 (Avg Bid)
    $718 Сер. заявка
    27 заявки
    PySpark Debugging Specialist... Закінчився left

    I am in need of a proficient PySpark coder to aid in debugging errors present within my current code. The main focus of this project is optimization and troubleshooting. Unfortunately, I can't specify the type of errors– I need a professional to help identify and rectify them. If you are an experienced PySpark coder with a keen eye for bug identification and problem solving, I'd appreciate your expertise.

    $7 - $20
    Прихований
    $7 - $20
    10 заявки
    PySpark Debugging Specialist Закінчився left

    I am in need of a proficient PySpark coder to aid in debugging errors present within my current code. The main focus of this project is optimization and troubleshooting. Unfortunately, I can't specify the type of errors– I need a professional to help identify and rectify them. If you are an experienced PySpark coder with a keen eye for bug identification and problem solving, I'd appreciate your expertise.

    $7 - $20
    Прихований
    $7 - $20
    6 заявки

    I'm searching for a PySpark expert who can provide assistance on optimizing and debugging current PySpark scripts. I am specifically focused on PySpark, so expertise in this area is crucial for the successful completion of this project. Key Responsibilities: - Optimizing PySpark scripts to improve efficiency - Debugging current PySpark scripts to resolve existing issues Ideal Candidate: - Proficient with PySpark - Experience in big data management, data ingestion, processing, analysis, visualization, and reporting - Strong problem-solving skills to identify and resolve issues effectively - Knowledgeable in performance tuning within PySpark.

    $102 (Avg Bid)
    $102 Сер. заявка
    65 заявки

    I'm looking for a skilled freelancer to create a Spark script that transfers data from a Hive metastore to an S3 bucket. The goal of this project is to enable backup and recovery. Skills and Experience: - Proficiency in Spark and Hive - Extensive experience with S3 buckets - Understanding of data backup strategies Project Details: - The script needs to read the schema and perform metadata transfer for selected schema to s3 bucket. - Only bid if you have work experience with spark, hive, s3 - 4 schemas needs to be migrated - I have already got access to s3 configured - I have local instance of netapp s3 available and bucket created. - Server is Ubuntu

    $98 (Avg Bid)
    $98 Сер. заявка
    10 заявки

    I am looking for an experienced data analyst who is well-versed in PySpark to clean up a medium-sized dataset in a CSV file format. The file contains between 10k-100k rows, and your primary role will be to: - Remove duplicate data entries - Deduplicate the dataset - Handle missing values - Aggregate the resultant data Your proficiency in using PySpark to automate these processes efficiently will be critical to the success of this project. Therefore, prior experience in handling and cleaning similar large datasets would be beneficial. Please note, this project requires precision, meticulousness, and a good understanding of data aggregation principles.

    $25 (Avg Bid)
    $25 Сер. заявка
    9 заявки
    Pyspark Data Cleaning and Sorting Закінчився left

    This vital task entails cleaning and sorting two CSV files of approximately 100,000 rows and second one of about 1.5million rows using pyspark (Python) in Jupyter Notebook(s). The project consists of several key tasks: Read in both datasets and then: - Standardizing data to ensure consistency - Removal of duplicate entries - Filtering columns we need - Handling and filling missing values - Aggregating data on certain groupings as output Important requirement: I also need unit tests to be written for the code at the end. Ideal Skills: Candidates applying for this project should be adept with Pyspark in Python and have experience in data cleaning and manipulation. Experience with working on datasets of similar size would also be preferable. Attention to detail in ensuring ...

    $181 (Avg Bid)
    $181 Сер. заявка
    57 заявки

    Seeking an experienced data scientist to assist in optimizing my synthetic data generator, specific...assist in optimizing my synthetic data generator, specifically by dividing larger datasets into smaller subsets. Expectations are as follows: - Integrate the capability of dividing data into subsets: you will need to specify both the number and size of these subsets. - Implement parallel processing using multi-threading to process these subsets concurrently. - Store the resulting data in a pandas dataframe for further analysis. The dataset involved is rather extensive, containing around 1M records, hence efficiency and good understanding of high-volume data processing is key. Skills in working with Genrocket, Python, Pandas, and multi-threading are highly desirable for success in ...

    $14 (Avg Bid)
    $14 Сер. заявка
    3 заявки

    I'm seeking an experienced Data Engineer with proficiency in SQL and PySpark. Key Responsibilities: - Develop and optimize our ETL processes. - Enhance our data pipeline for smoother operations. The ideal candidate should deliver efficient extraction, transformation, and loading of data, which is critical to our project's success. Skills and Experience: - Proficient in SQL and PySpark - Proven experience in ETL process development - Previous experience in data pipeline optimization Your expertise will significantly improve our data management systems, and your ability to deliver effectively and promptly will be highly appreciated.

    $92 (Avg Bid)
    $92 Сер. заявка
    17 заявки

    - Conversion of the entire Python code into PySpark. Skills and experience required: - Proficient knowledge in Python.

    $26 (Avg Bid)
    $26 Сер. заявка
    26 заявки

    My project requires detailed scraping of the 'Last 21 Days' form from the Irish Racing website for all trainers. I'm interested in gathering specific information from each line item in the form. Not only should you efficiently loop through the website's entire trainer database, you'll also need to provide a neat and organized dataframe in a CSV file format. The Ideal candidate should: - Be proficient in Python for web scraping - Able to handle large amounts of data scraping and manipulation professionally - Show a strong understanding in parsing CSV files Knowing the horse racing field is a plus, but not mandatory.

    $40 (Avg Bid)
    $40 Сер. заявка
    16 заявки

    Будь ласка, зареєструйтесь або увійдіть в систему для перегляду деталей.

    Прихований Угода про нерозголошення
    Python Developer for Data Comparison Закінчився left

    ...competent in either PySpark or RDD, using Python to create versatile code fitting for several scenarios. Your main task will be to write code to compare rows using Python in line with the clear set of rules I provide. These rules are detailed in an attached Word document and are based on comparisons encompassing specific columns, presence or absence of particular data, and multiple criteria comparisons. The expected output is a reversal logic for claim_opened_timestamp_utc. I need output that are in right side. I need either in pyspark or in rdd to compare rows. spark - spark-3.3.0-bin-hadoop3 py4j-0.10.9.5 I am using I need your support till I execute it in my office computer I need it in 3 days. Ideal Skills and Experience: - Proficiency in Python - Experience with...

    $156 (Avg Bid)
    Терміновий
    $156 Сер. заявка
    16 заявки

    I'm beginer user of Azure Databricks and Pyspark. I'm looking to boost my skills to the next level and need an expert to guide me through advanced techniques. Ideal freelancers should have vast experience and profound knowledge in data manipulation using Pyspark, Azure Databricks, data pipeline construction, and data analysis and visualization. If you've previously tutored or mentored in these areas, it'll be a plus.

    $12 / hr (Avg Bid)
    $12 / hr Сер. заявка
    4 заявки

    I need complete 2 small projects done. The data needs to be pulled from API using python. The pulled data needs to be unnested, then transformed to answer some insights with medallion architecture. Here, you need to showcase SCD-type 2 ingestions, incremental joins,...to be pulled from API using python. The pulled data needs to be unnested, then transformed to answer some insights with medallion architecture. Here, you need to showcase SCD-type 2 ingestions, incremental joins, managing PII information, aggregation. Final deliverable needed for 1st project (databricks): Data model designed and architecture overview Notebooks of transformations in Python and PySpark/Spark Scala Final deliverable needed for 2nd project (dbt): Data model designed and architecture overview dbt sql and ...

    $276 (Avg Bid)
    $276 Сер. заявка
    16 заявки

    Looking for someone with good skills in Airflow, Pyspark and SQL.

    $251 (Avg Bid)
    $251 Сер. заявка
    13 заявки
    "Python Data Pipeline Forecasting" Закінчився left

    I am looking for a skilled professional in Python, with a comprehensive understanding of PySpark, Databricks, and GCP. A primary focus of the project is to build a data pipeline and apply time series forecasting techniques for revenue projection, using historical sales data. Key tasks will include: - Constructing a robust data pipeline using Python, PySpark, and Databricks. - Applying time series forecasting to produce revenue predictions. - Using Mean Squared Error (MSE) to measure model accuracy. The ideal candidate for this project would have: - Proven experience with Python, PySpark, Databricks, and GCP. - Expertise in time series forecasting models. - Practical understanding and use of Mean Squared Error (MSE) for model accuracy. - Experience with large scale ...

    $11 / hr (Avg Bid)
    $11 / hr Сер. заявка
    14 заявки

    I am looking to develop a sophisticated and efficient data pipeline for revenue forecasting. This pipeline will be implemented using Python, pyspark, databrics, and gcp Big Data. Here is what you need to know about this task: - Data Source: The data originates from Google Cloud Platform's Big Data service. As such, the freelancer should have solid experience and understanding of working with Big Data services on GCP. - Data Update Frequency: The frequency of data updates will be confirmed during the project, but suffice to say frequency could be high. Prior experience with real-time or near-real-time data processing will be highly beneficial. - Performance Metrics: The key performance metric I'm focusing on is data processing speed. The freelancer should have a strong kn...

    $18 / hr (Avg Bid)
    $18 / hr Сер. заявка
    13 заявки

    I'm in need of a specialist, ideally with experience in data science, Python, PySpark, and Databricks, to undertake a project encompassing data pipeline creation, time series forecasting and revenue forecasting. #### Goal: * Be able to extract data from GCP BigData efficiently. * Develop a data pipeline to automate this process. * Implement time series forecasting techniques on the extracted data. * Use the time series forecasting models for accurate revenue forecasting. #### Deadline: * The project needs to be completed ASAP, hence a freelancer with a good turnaround time is preferred. #### Key Skill Sets: * Data Science * Python, PySpark, Databricks * BigData on GCP * Time series forecasting * Revenue forecasting * Data Extraction and Automation Qualification in...

    $18 / hr (Avg Bid)
    $18 / hr Сер. заявка
    15 заявки

    I am looking for a developer to create an AWS Glue and Pyspark script that will strengthen the data management of my project. The task involves moving more than 100GB of text data from a MySQL RDS table to my S3 storage account, on a weekly basis. Additionally, the procured data needs to be written on parquet files, for easy referencing. The developer will also need to send scripts to deploy the AWS Glue pipelines on Terraform, fitting all parameters. Skilled expertise in AWS Glue, PySpark, Terraform, MySQL and experience in handling large data is required. There is no compromise on the quality and completion timeline. Effective performance on this project will open doors to more work opportunities on my various projects.

    $41 (Avg Bid)
    $41 Сер. заявка
    15 заявки

    I am seeking a skilled professional proficient in managing big data tasks with Hadoop, Hive, and PySpark. The primary aim of this project involves processing and analyzing structured data. Key Tasks: - Implementing Hadoop, Hive, and PySpark for my project to analyze large volumes of structured data. - Use Hive and PySpark for sophisticated data analysis and processing techniques. Ideal Skills: - Proficiency in Hadoop ecosystem - Experience with Hive and PySpark - Strong background in working with structured data - Expertise in big data processing and data analysis - Excellent problem-solving and communication skills Deliverables: - Converting raw data into useful information using Hive and Visualizing the results of queries into the graphical representation...

    $17 / hr (Avg Bid)
    $17 / hr Сер. заявка
    15 заявки

    ...currently searching for an experienced AWS Glue expert, proficient in PYsPARK with data frames and Kafka development. The ideal candidate will have: • Expertise in data frame manipulation. • Experience with Kafka integration. • Strong PYsPARK development skills. The purpose of this project is data integration, and we will be primarily processing data from structured databases. The selected freelancer should be able to work with these databases seamlessly, ensuring efficient and effective data integration using AWS Glue. The required work would involve converting structured databases to fit into a data pipeline, setting up data processing, and integrating APIs using Kafka. This project requires a strong background in AWS Glue, PYSPARK, data frame ...

    $235 (Avg Bid)
    $235 Сер. заявка
    24 заявки

    I'm seeking assistance to develop a Python-based solution utilizing PySpark for efficient data processing using the Chord Protocol. This project demands an intermediate level of expertise in Apache Spark or PySpark, combining distributed computing knowledge with specific focus on Python programming. Key Requirements: - Proficiency in Python programming and PySpark framework. - Solid understanding of the Chord Protocol and its application in data processing. - Capable of implementing robust data processing solutions in a distributed environment. Ideal Skills and Experience: - Intermediate to advanced knowledge in Apache Spark or PySpark. - Experience in implementing distributed file sharing or data processing systems. - Familiarity with network communicati...

    $545 (Avg Bid)
    $545 Сер. заявка
    38 заявки

    ...Professional with strong expertise in Pyspark for a multi-faceted project. Your responsibilities will extend to but not limited to: - Data analysis: You'll be working with diverse datasets including customer data, sales data and sensor data. Your role will involve deciphering this data, identifying key patterns and drawing out impactful insights. - Data processing: A major part of this role will be processing the mentioned datasets, and preparing them effectively for analysis. - Performance optimization: The ultimate aim is to enhance our customer targeting, boost sales revenue and identify patterns in sensor data. Utilizing your skills to optimize performance in these sectors will be highly appreciated. The ideal candidate will be skilled in Hadoop and Pyspark wi...

    $463 (Avg Bid)
    $463 Сер. заявка
    25 заявки

    I would like a Python scri...run on Colab that retrieves: 1. Information about games from the following leagues: England (Premier League, Championship), France (Ligue 1, Ligue 2), Germany (Bundesliga, Bundesliga 2), Italy (Serie A, Serie B), Spain (La Liga, La Liga 2). 2. The information needed is for the past season (2022-2023) and the current season (2023-2024), as well as games scheduled for today and tomorrow. 3. The dataframe should contain the following columns: season, date, country, league, home_team, away_team, home_goals (FT), away_goals (FT), home_goals (HT), away_goals (HT), home_odds, draw_odds, away_odds (FT and HT), odds_O0.5, odds_U0.5, odds_O1.5, odds_U1.5, odds_O2.5, odds_U2.5 (FT and HT), BTTS (FT and HT), AH (FT and HT). 4. Odds from Bookmakers (Pinnacle, Betfair...

    $147 (Avg Bid)
    $147 Сер. заявка
    55 заявки

    ...demands comprehensive data cleaning using the Pandas dataframe. The freelancer awarded this job needs to be proficient in Python, especially with the Pandas library. Their resourcefulness in data manipulation and cleaning techniques will be paramount. Specific Tasks: - Handling missing data: You ought to remove rows with missing data from the dataframe. It's critical to ensure that the dataset is left with complete entries only. - Removing duplicates: Duplicate entries should be identified and eradicated, ensuring that each data entry is unique and valid. - Adjusting data types: Depending on the necessity of specific analytical goals, you will have to adjust the types of data in the dataframe. - Data standardisation: Data in the dataframe needs to b...

    $20 / hr (Avg Bid)
    $20 / hr Сер. заявка
    110 заявки
    Create Python-Databricks Script Закінчився left

    Creating python-Databricks Script, covering below points and present in this yt video- : 1. Securely connect to SAP 2. Load Table to Dataframe 3. Load CDS View to Dataframe 4. Load HANA Object to Dataframe 5. Load ODP Extractor to Dataframe in Delta Mode 6. Load Table and write to Datalake storage 7. As is setup and steps in Video It will not cover any Data Analysis, Prediction, or Identifying trends from data. Some basic steps that cover the above points.

    $280 (Avg Bid)
    $280 Сер. заявка
    1 заявки
    Amazon review Закінчився left

    I need to impro...()) (("span", {"data-hook": "review-body"}).()) (("span", {"data-hook": "review-date"}).()) (("i", {"data-hook": "review-star-rating"}).()) # Crea un DataFrame con i dati raccolti df_reviews = ({ "ID": review_ids, "Title": review_titles, "Review": review_texts, "Date": review_dates, "Rating": review_ratings }) # Salva il DataFrame in un file Excel df_reviews.to_excel("", index=False) print("Recensioni estratte e salvate con successo.") Improvements: Using an excel where there are the asins to elaborate Select number of review of each product Select review range (f...

    $25 (Avg Bid)
    $25 Сер. заявка
    15 заявки
    PostgreSQL Data Transformation in AWS Закінчився left

    Build a glue etl using pyspark to transfer data from mysql to postgres. facing challenges in column mappings between the 2 sources, the target database has datatypes enums and text arrays. should solve the erros in column mappings Should have prior experience ingesting data into postgres enum datatype

    $22 / hr (Avg Bid)
    $22 / hr Сер. заявка
    54 заявки
    Instagram Data Scraper -- 2 Закінчився left

    Objective: Develop a Python script that retrieves publicly available information from specified Instagram accounts and stores the data in a structured pandas table, while also creating public URLs to access the creative. Input: Instagram account username (no login credentials required). Output: A pandas dataframe with the following columns populated with data from each Instagram post in the account: - Account Username - Account Followers Count - Audio Track (if applicable) - Post URL - Date of Post - Links to Media in Post (Images/Videos) – you should have a system that downloads each creative in a post and uploads it to a public link I can access and download from. - Post Type (Static, Carousel, Video, Other) - Post Caption - Tagged Accounts - Number of Likes - Number of Co...

    $49 (Avg Bid)
    $49 Сер. заявка
    28 заявки

    I have a csv dataframe with OHLC data. I want to plot this data plus some additional lines that came from a list. I want to use plotly. I have a script that plots the candle data, just need to add the additional lines.

    $19 (Avg Bid)
    $19 Сер. заявка
    22 заявки

    I am in need of an experienced data engineer with specific expertise in PySpark. This project involves the integration and migration of data from structured databases currently housed in AWS. Here's a rundown of your key responsibilities: - Data integration from various existing structured databases - Migration of the combined data to a single, more efficacious database Ideal Candidate: - Proven experience in data migration and integration projects - Expertise in PySpark is indispensable - Proficiency in manipulating AWS databases - A solid understanding of structured databases and various data formats is mandatory This project is more than just technical skills- I'm looking for someone who can understand the bigger picture and contribute to the overarching str...

    $661 (Avg Bid)
    $661 Сер. заявка
    13 заявки

    I'm looking for a professional with a strong understanding of PySpark to help transform a dataframe into JSON following a specific schema. This project's main task is data transformation to aid in data interchange. The project requires: - Expertise in PySpark - Proficiency in data transformation techniques - Specific experience in data aggregation For the transformation, I require the application of an aggregation method. In this case, we will be sorting the data. It's crucial that you are skilled in various aggregation methods, especially sorting. Your knowledge in handling critical PySpark operations is crucial for this job's success. Experience in similar projects will be highly regarded.

    $24 (Avg Bid)
    $24 Сер. заявка
    18 заявки

    Looking for an expert Azure Data Engineer to assist with multiple tasks. Your responsibilities will include: - Implementing and managing Azure Data Lake and Data Ingestion. - Developing visual reports...platforms to achieve three main objectives: - Perform sophisticated data analysis and visualization. - Enable advanced data integration and transformation. - Build custom applications to meet specific needs. Candidates should have an advanced understanding of Azure Data Lake, Power BI, and Powerapps, bringing a minimum of 6 years experience as Databricks. Proficiency in Python, SQL, PostGre SQL, and Pyspark is also required. Knowledge of GitHub and the CI/CD Process will be beneficial for this role. If you have the skills and expertise needed for this project, I'd love to...

    $34 / hr (Avg Bid)
    $34 / hr Сер. заявка
    28 заявки

    ...need to be pushed swiftly to Elasticsearch using Pyspark. Your expertise will help push all data columns from this file into Elasticsearch, establishing a more actionable access to a significant amount of data. Given the project's urgency, I'm expecting a rapid, reliable transition. While the structure for the documents remains undecided due to the project's intricacies, I'm open to suggestions that will make this process more efficient and effective. Anyone with experience in Pyspark, Elasticsearch, and vast data manipulation will have a substantial edge on this project, as these skills are highly necessary for success. A strong understanding of different data structures is also a plus. • Leading Skills Required: Proficiency in Pyspark ...

    $10 / hr (Avg Bid)
    $10 / hr Сер. заявка
    3 заявки
    Pyspark data engineering training Закінчився left

    ...Title: Pyspark Data Engineering Training Overview: I am a beginner/intermediate in Pyspark and I am looking for a training program that focuses on data processing. I prefer one on one and written guides as the format for the training. Skills and Experience Required: - Strong expertise in Pyspark and data engineering - Excellent knowledge of data processing techniques - Experience in creating and optimizing data pipelines - Familiarity with data manipulation and transformation using Pyspark - Ability to explain complex concepts in a clear and concise manner through written guides - Understanding of best practices for data processing in Pyspark Training Topics: The training should primarily focus on data processing. The following topics should be cov...

    $23 / hr (Avg Bid)
    $23 / hr Сер. заявка
    72 заявки
    Databricks pyspark python training Закінчився left

    ...training is expected to be spread across multiple days. The trainer must have the capability to provide an understanding of the major concepts and components of Apache Spark, with a focus on how to use Databricks and the Pyspark API to manipulate and visualize data. As the training progresses, the instructor should be able to explain how to develop applications using Pyspark and articulate different approaches that a data scientist would use to evaluate and test their models. The instructor should also be able to educate the users on how to deploy and maintain Pyspark applications and how to provide feedback and questions in order to improve their performance. We expect the trainer to be readily available to answer any questions and guide the users along the w...

    $101 (Avg Bid)
    $101 Сер. заявка
    75 заявки
    Databricks Python Pyspark training Закінчився left

    I am seeking an expert in the field to provide remote training in the use of Databricks and Python with PySpark. This is important for developing data processing applications with a high degree of efficiency. The training should cover areas such as data wrangling, machine learning, and Spark streaming. In order to be successful, attendees must be well-versed in Databricks, Python and PySpark, as these skills will be essential for completing the course. The course should provide a good understanding of the concepts and practical application of these tools. This training will give attendees the skills they need to analyse and manipulate large datasets, develop effective data processing pipelines, design powerful machine learning models and build reliable applications that use...

    $108 (Avg Bid)
    $108 Сер. заявка
    74 заявки

    ...S3, and RDS; Azure services; and Pyspark data processing and transformations. Essential Skills: - Proficient in AWS, specifically on EC2, S3, RDS with strong understanding of data storage and retrieval. - Expert in Azure services such as Azure SQL Database and Blob Storage. - Highly experienced in writing efficient data transformations using Pyspark. Ideal Experience: - Minimum 7 years in the field with solid experience in technical interviews and coaching. Your task will be to provide actionable insights, best practices, and expert advice to nail my upcoming technical interview. Having been on the other side of the interview table would be an added advantage. - Proven track record of performing successful data processing and transformations using Pyspark. - Prev...

    $15 / hr (Avg Bid)
    $15 / hr Сер. заявка
    8 заявки

    Experienced Python + SQL +AWS +AZURE data engineer (7+ years) for evening IST timings. For guiding in interview preparation specially for data engineering. Tasks: Should have good knowledge of pyspark, sql, pandas Should have written multiple ETL pipeline in aws and azure. Note: The freelancer must be available during evening ist timings.

    $10 / hr (Avg Bid)
    $10 / hr Сер. заявка
    12 заявки

    ...structured data such as SQL databases. Skills and experience required: - Expertise in AWS migration, specifically from another cloud provider - Strong knowledge and experience with structured data, particularly SQL databases - Familiarity with AWS Glue and Athena for data processing and analysis - Ability to work with a combination of different AWS services for optimal performance and efficiency Pyspark ,sql,python Cdk Typescript Aws glue ,Emr and andes Currently Migrating from teradata to aws. Responsibilities: - Migrate data from another cloud provider to AWS, ensuring a smooth transition and minimal downtime - Design and develop applications that utilize AWS Glue and Athena for data processing and analysis - Optimize data storage and retrieval using AWS S3 and R...

    $9 / hr (Avg Bid)
    $9 / hr Сер. заявка
    14 заявки

    I have an NBA Prediction code in python. Task: 1.) Copy all the python codes to a Jupyter notebook and debug/run it successfully 2.) Output the predictions in a dataframe Notes: 1.) The code already runs but it's easier for me to run it via Jupyter 2.) Important inputs are the dates 2a) Historical data date 2b) Current predictions date. Deadline: 12 hrs Budget: $20

    $19 (Avg Bid)
    $19 Сер. заявка
    8 заявки