PySpark-Tutorial provides basic algorithms using PySpark

Our style guide for writing readable and maintainable PySpark code.

All updated cheat sheets regarding data science, data analysis provided by Datacamp are here. These cheat sheets cover quick reads on Machine Learning, Deep Learning, Python, R, SQL and more. Perfect cheat sheets when you want to revise some topics in less time.

List of useful commands for Pyspark

This code demonstrates how to integrate PySpark with datasets and perform simple data transformations. It loads a sample dataset using PySpark's built-in functionalities or reads data from external sources and converts it into a PySpark DataFrame for distributed processing and manipulation.

Project based on application of azure databricks

This notebook contains the usage of Pyspark to build machine learning classifiers (note that almost ml_algorithm supported by Pyspark are used in this notebook)

This repository contains the Notes for Pyspark

Analyzing 7 million Amazon's customer reviews using Apache Spark. Text mining and visualization on reviews, ratings, and helpfulness of 50+ product categories.

Generate a synthetic dataset with one million records of employee information from a fictional company, load it into a PostgreSQL database, create analytical reports using PySpark and large-scale data analysis techniques, and implement machine learning models to predict trends in hiring and layoffs on a monthly and yearly basis.

This notebook performs EDA over a movie ratings dataset via pyspark sql.

Clustering vs Classification

Batch Processing using Apache Spark and Python for data exploration

This script builds a linear regression model using PySpark to predict student admissions at Unicorn University.

This repository is part of my journey to learn **PySpark**, the Python API for Apache Spark. I explored the fundamentals of distributed data processing using Spark and practiced with real-world data transformation and querying use cases.

Inventory value is also important for determining a company's liquidity, or its ability to meet its short-term financial obligations. A high inventory value can indicate that a company has too much money tied up in inventory, which could make it difficult for the company to pay its bills.

Module 22 challenge: Using Google Colab to work on Big Data queries with PySpark SQL, parquet, and cache partitions

📚 Master PySpark in 18 days with structured lessons, hands-on tasks, and an end-to-end project, covering essential concepts and ML model training.