Project Description:
This project focuses on creating a brief but engaging introductory course on Data Science tailored for high school students. The aim is to showcase the versatility and power of the Wolfram Language and leverage it to demystify data science, making it an approachable, appealing and meaningful subject for the young learners. This project-based course is ideally structured for delivery in a 3-4 hour workshop format. The course not only aspires to ignite students’ interest with Data Science, but it also sets the groundwork for them to progress to more advanced Data Science courses available on Wolfram U. Future iteration would include a video lecture as part of a flipped classroom experience.
I was drawn to this project primarily because of my teaching background and my interest in exploring the effective integration of data science in high school curriculums. I’m on a journey of learning Data Science myself and I believe that teaching is an effective way to comprehend a concept. Thus, this project is a perfect blend of my interests and provides an opportunity for both my teaching and learning the subject simultaneously.
This project focuses on creating a brief but engaging introductory course on Data Science tailored for high school students. The aim is to showcase the versatility and power of the Wolfram Language and leverage it to demystify data science, making it an approachable, appealing and meaningful subject for the young learners. This project-based course is ideally structured for delivery in a 3-4 hour workshop format. The course not only aspires to ignite students’ interest with Data Science, but it also sets the groundwork for them to progress to more advanced Data Science courses available on Wolfram U. Future iteration would include a video lecture as part of a flipped classroom experience.
I was drawn to this project primarily because of my teaching background and my interest in exploring the effective integration of data science in high school curriculums. I’m on a journey of learning Data Science myself and I believe that teaching is an effective way to comprehend a concept. Thus, this project is a perfect blend of my interests and provides an opportunity for both my teaching and learning the subject simultaneously.
Abstract
Data Science is a fascinating field that utilizes scientific methods, processes, algorithms, and systems to extract knowledge and insights from both structured and unstructured data. It’s a discipline that involves a lot of creativity and critical thinking, and can be used to make data-driven decisions and predictions, making it an essential skill for the 21st century. This introductory course aims to familiarize high school students with data science using the Wolfram Language. The Wolfram Language is a high-level programming language developed by Wolfram Research. Known for its sophisticated computational capabilities, it is widely used in scientific research, engineering, data science, machine learning, and many other areas. In data science, the Wolfram Language stands out because of its intuitive syntax, massive inbuilt computation and knowledge engine, and extensive data visualization capabilities.
Data Science is a fascinating field that utilizes scientific methods, processes, algorithms, and systems to extract knowledge and insights from both structured and unstructured data. It’s a discipline that involves a lot of creativity and critical thinking, and can be used to make data-driven decisions and predictions, making it an essential skill for the 21st century. This introductory course aims to familiarize high school students with data science using the Wolfram Language. The Wolfram Language is a high-level programming language developed by Wolfram Research. Known for its sophisticated computational capabilities, it is widely used in scientific research, engineering, data science, machine learning, and many other areas. In data science, the Wolfram Language stands out because of its intuitive syntax, massive inbuilt computation and knowledge engine, and extensive data visualization capabilities.
Introduction to Data Science
Introduction to Data Science
Imagine being a detective but with a twist! Instead of solving crimes, you are solving puzzles hidden within vast amounts of information, also known as data. Welcome to the thrilling world of Data Science! 🤗
Data science is a hot topic these days and for good reason. It’s a unique field that uses methods from statistics and computer science to draw insights from data. These insights can help answer complex questions and inform decision-making in almost any area you can think of: sports, healthcare, finance, entertainment, and even social media. It’s also used in music platforms like Spotify to recommend songs, in hospitals to predict diseases, in businesses to understand customer behavior, and so much more.
So, if you love solving puzzles, are curious about the world, and have a flair for math and computers, data science could be your calling! It’s a field that turns raw data into stories and knowledge, helping us understand and navigate the world around us better. 🔎
Data science is a hot topic these days and for good reason. It’s a unique field that uses methods from statistics and computer science to draw insights from data. These insights can help answer complex questions and inform decision-making in almost any area you can think of: sports, healthcare, finance, entertainment, and even social media. It’s also used in music platforms like Spotify to recommend songs, in hospitals to predict diseases, in businesses to understand customer behavior, and so much more.
So, if you love solving puzzles, are curious about the world, and have a flair for math and computers, data science could be your calling! It’s a field that turns raw data into stories and knowledge, helping us understand and navigate the world around us better. 🔎
Why Should I Care About Data Science?
Why Should I Care About Data Science?
We live in the era of big data. Every day, we generate an incredible amount of digital data — from social media posts and digital photos to GPS signals and medical records. This flood of data is a gold mine of insights and information, but only if we know how to tap into it. That’s where data science comes in. Data science allows us to process and analyze these vast amounts of data to discover patterns, make predictions, and drive decisions.
Where Can I See Data Science in Action?
Where Can I See Data Science in Action?
You don’t need to look too far to see data science at work:
◼
Google Search Engine: Google uses data science algorithms to understand your search query, trawl through billions of webpages, and return relevant results in a fraction of a second.
◼
Weather Forecasting: Data scientists use weather data from satellites, weather stations, and radars to predict weather patterns.
◼
Cybersecurity: Banks use data science to catch sneaky fraudsters trying to swipe your cash. It’s like a digital “Cops and Robbers” game!
◼
Traffic and City Planning: Data science helps analyze traffic patterns to optimize signal timings and suggest fastest routes in navigation apps.
◼
E-commerce: Ever noticed how Amazon seems to read your mind, suggesting just the thing you didn’t know you needed? That’s not telepathy 😶🌫️, that’s data science! It’s used to personalize your shopping experience and predict what products you might like. (Ever wondered how Amazon knows you’re out of toilet paper before you do? That’s data science hard at work, making sure you’re never stranded in your time of need).
◼
Astronomy: Scientists use data science to sift through enormous amounts of space data, detecting new planets and understanding cosmic phenomena 👽.
◼
Social Media: Ever noticed how you keep running into cat videos on your feed? That’s not a coincidence, it’s data science saying, “Hey, you look like a cat person!” 😸. Facebook, Instagram, Twitter, and others use data science to decide what appears in your feed and who to suggest as friends. So when you come across a post you really like, remember to give a nod to data science.
◼
Education: Data science in schools is like that one teacher who just gets you. It spots the troublemakers (I mean, academically speaking) and turns them into model students. Schools and colleges use data science to understand how students learn and identify those who might need extra help.
So buckle up and get ready for an amazing journey through the land of data. Our ride? The powerful and efficient Wolfram Language! You might encounter a few bumps on the road, but remember - no great adventure ever started with a smooth ride. Onward, data detectives! 🚀
YOUR TURN
YOUR TURN
Please attempt to respond to these questions using your own perspective. Strive for originality in your thought process.
Data Definition
◼
How would you define data in your own words?
◼
Can you give examples of data in everyday life?
Importance of Data
◼
Why do you think data is valuable in today’s digital age?
◼
How can data be used to solve real-world problems?
Data Science
◼
How would you explain data science to a friend?
◼
Why is data science considered an interdisciplinary field?
Data Collection
◼
What are some methods of data collection you can think of?
◼
How is data organized and stored for analysis?
Data Analysis
◼
What does it mean to analyze data?
◼
Can you think of a situation where data analysis could be useful?
Role of Statistics in Data Science
◼
Why is statistics important in the field of data science?
◼
How do data scientists use statistical methods to analyze data?
Role of Programming in Data Science
◼
Why do data scientists need to know how to program?
◼
How do programming languages like Python or Wolfram Language aid in data analysis?
Data Visualization in Data Science
◼
Why is data visualization important in data science?
◼
Can you think of any examples where a visualization made understanding data easier?
Ethical Considerations in Data Science
◼
Why is it important to consider ethics when working with data?
◼
Can you think of any potential ethical issues related to data collection or use?
Introduction to Wolfram Language
Introduction to Wolfram Language
The Wolfram Language, created by Stephen Wolfram and introduced in 1988 as a part of the Mathematica system, has grown into a highly sophisticated and comprehensive tool for computational intelligence. It is a high-level, general-purpose programming language known for its advanced computational capabilities and functional programming syntax.
The Wolfram Language is especially noteworthy for its built-in computational knowledge base and automated algorithm selection, which are capable of handling tasks ranging from simple calculations to complex computations and data analysis. Its vast array of pre-built functions for various tasks and ease of integration with Wolfram Alpha’s computational knowledge make it a powerful tool for learning, exploration, and application in diverse fields.
The Wolfram Language, with its high-level abstractions and built-in computational intelligence, provides an excellent platform for both beginners and seasoned professionals in data science.
The Wolfram Language is especially noteworthy for its built-in computational knowledge base and automated algorithm selection, which are capable of handling tasks ranging from simple calculations to complex computations and data analysis. Its vast array of pre-built functions for various tasks and ease of integration with Wolfram Alpha’s computational knowledge make it a powerful tool for learning, exploration, and application in diverse fields.
The Wolfram Language, with its high-level abstractions and built-in computational intelligence, provides an excellent platform for both beginners and seasoned professionals in data science.
The Wolfram Language — An Overview
The Wolfram Language — An Overview
◼
The Wolfram Language is a knowledge-based language, meaning it has a vast amount of knowledge about how to do computations and about the world built into it
◼
The Wolfram Language is based on a fully symbolic language, and has the world’s largest collection of algorithms and data built directly into it
◼
The Wolfram Language has a higher-level and more integrated philosophy than Python
◼
The Wolfram Language can be used completely interactively, even on the web, so you can type something in and immediately get a result
Shift+Enter
Shift+Enter
In the Wolfram Language, Shift+Enter is the command used to execute or run the code that you’ve written. After typing out your code in a cell, you press the Shift key and the Enter key at the same time, and the Wolfram system will process your code and return an output.
Capitalization and Use of Square Brackets
Capitalization and Use of Square Brackets
Square brackets in the Wolfram Language are used for function calls. When you want to use a function, you write the name of the function followed by square brackets, and inside these brackets, you put the arguments for the function. The built-in functions within Wolfram Language always start with capital letter. Example Print, Import, Sin[x], Integrate etc.
Functional Programming:
Functional Programming:
In Wolfram Language, everything is a symbolic expression. This includes numbers, variables, functions, graphics, etc . The language primarily follows the principles of functional programming, which means that most operations are carried out by applying functions to values . For example, to calculate the square of 5, you'd execute Square[5].
Built-in Functions:
Built-in Functions:
One of the key features of the Wolfram Language is its extensive collection of built-in functions. These functions cover a wide range of areas, from mathematical operations like Plus, Minus, Divide, and Multiply, to more specialized functions like Plot for generating graphs, Solve for solving equations, and Table for creating tables of values.
The Wolfram Language provides a wide variety of built-in functions that can be useful for data science. Here are some important ones:
The Wolfram Language provides a wide variety of built-in functions that can be useful for data science. Here are some important ones:
Data Manipulation
Data Manipulation
Select, Sort, Join, DeleteDuplicates, Union, Intersection, Complement
These functions allow you to manipulate lists and other collections of data.
Statistical Functions
Statistical Functions
Mean, Median, StandardDeviation, Variance, Correlation, Covariance, Histogram, PDF, CDF
These functions perform statistical calculations and create statistical visualizations.
Data Visualization
Data Visualization
ListPlot, BarChart, PieChart, Histogram, BoxWhiskerChart, Plot3D, ContourPlot
These functions create various types of data visualizations.
Machine Learning Functions
Machine Learning Functions
Classify, Predict, NetTrain, NetModel
These functions provide machine learning capabilities, including classification, prediction, and neural networks.
Graphs and Plots
Graphs and Plots
Plot, ListPlot, Plot3D, ListLinePlot, Graph, NumberLinePlot, BarChart, PieChart, Histogram
These functions provide capabilities for working with graphs and networks.
Text and String Manipulation Functions
Text and String Manipulation Functions
StringJoin, StringSplit, StringReplace, ToLowerCase, ToUpperCase, RegularExpression
These functions allow you to manipulate text and strings.
Geocomputation Functions
Geocomputation Functions
GeoDistance, GeoPosition, GeoGraphics, GeoIdentify, GeoNearest
These functions provide capabilities for working with geographical data.
Lists and Association:
Lists and Association:
Lists are a fundamental data structure in Wolfram Language and are represented as a sequence of items enclosed in {} . For example, {1, 2, 3} is a list of numbers . You can perform operations on lists. For example, Plus[{1, 2}, {3, 4}] gives {4, 6}.
Association
Association
In the Wolfram Language, an Association is a built-in data structure for representing a collection of key-value pairs. Associations are used to store data in a way that enables efficient lookup, insertion, and removal of items.
Each item in an association is a rule that maps from a key to a value. The keys can be any expression (for example, numbers, strings, or other associations), and they are typically unique within a single association. Here’s an example of an association that represents a person:
Each item in an association is a rule that maps from a key to a value. The keys can be any expression (for example, numbers, strings, or other associations), and they are typically unique within a single association. Here’s an example of an association that represents a person:
In[]:=
person = <|"name" -> "Khan", "age" -> 30, "city" -> "Arusha"|>;
You can access values in an association using their keys:
In[]:=
person["name"] (*Returns "Alice"*)
Out[]=
Khan
In[]:=
Keys[person]
Associations in Wolfram Language are powerful and flexible, and they are used in many built-in functions and data types.
Symbolic Computations:
Symbolic Computations:
In the Wolfram Language, computations are done symbolically whenever possible . This means that expressions are kept in symbolic form unless a numerical value is explicitly requested . For example, Sqrt[2] remains as Sqrt[2] unless you ask for a numerical approximation with N[Sqrt[2]].
Accessing External Data :
Accessing External Data :
Wolfram Language includes functions for accessing external data . For example, FinancialData can be used to get stock prices, WeatherData to get weather information, and CountryData to get data about countries.
Accessing external data in the Wolfram Language can be achieved in various ways depending on the source and type of the data. Here are a few examples:
Accessing external data in the Wolfram Language can be achieved in various ways depending on the source and type of the data. Here are a few examples:
File-Based Data:
File-Based Data:
If you have a data file (like a CSV, XLSX, or JSON file), you can use the Import function to read the data. For example:
Web - Based Data:
Web - Based Data:
If your data is hosted on the web, you can use URLRead, URLFetch, Import, etc . For example, to fetch data from a URL:
Or, if you want to get the HTML of a web page:
The Titanic dataset is a popular dataset for machine learning and data analysis, and it is readily available from the Wolfram Data Repository. Here’s how you can access it using the Wolfram Language:
Knowledge Based
Knowledge Based
It means that we can use it to obtain data about the world. Using natural language input (by typing Ctrl + = ) is the best way to access such data. For example, let’s obtain the population of Tanzania:
The Wolfram Language has a rich and extensive set of features, you should be able to find and learn about the functions that are relevant to the problem you' re trying to solve. You can access detailed information on:
Your Turn
Your Turn
1
.Find 2+2
2
.Execute 2(3-8)^1.5
3
.Plot the graph of sin(x) from 0 to 2π. Hint:
4
.Print the Text “I’am Happy and I know it”
5
.Execute the following:
RGBColor[0,0,0]
RGBColor[1,0,0]
RGBColor[0,1,0]
RGBColor[0,0,1]
RGBColor[0,0,0]
RGBColor[1,0,0]
RGBColor[0,1,0]
RGBColor[0,0,1]
6
.Define a variable x with the value 10, and a variable y with the value 20. Calculate the Sum, Difference, Product, and Quotient of x and y.
7
.Create a list of the first five prime numbers. Then find the length of the list and the third element in the list.
Hint:
Hint:
Data Collection and Handling
Data Collection and Handling
Data collection is the process of gathering information or facts from various sources. It involves collecting different types of data such as numbers, text, images, or videos that are relevant to a specific topic or problem.
Data cleaning, on the other hand, is the process of preparing and organizing the collected data to ensure its accuracy, consistency, and completeness. It involves identifying and correcting errors, removing duplicate or irrelevant information, handling missing data, and transforming the data into a consistent format.
In simpler terms, data collection is like gathering puzzle pieces, while data cleaning is like putting those puzzle pieces together, making sure they fit properly and are in good condition before using them to solve a bigger puzzle or answer a question. In Wolfram Language, data collection and cleaning can be achieved using various built-in functions and capabilities. Here’s a short explanation of how it can be done:
Data cleaning, on the other hand, is the process of preparing and organizing the collected data to ensure its accuracy, consistency, and completeness. It involves identifying and correcting errors, removing duplicate or irrelevant information, handling missing data, and transforming the data into a consistent format.
In simpler terms, data collection is like gathering puzzle pieces, while data cleaning is like putting those puzzle pieces together, making sure they fit properly and are in good condition before using them to solve a bigger puzzle or answer a question. In Wolfram Language, data collection and cleaning can be achieved using various built-in functions and capabilities. Here’s a short explanation of how it can be done:
Data Collection:
Data Collection:
1. Importing Data: Wolfram Language provides functions like Import to bring in data from various file formats such as CSV, Excel, JSON, or databases.
2. Web Scraping: With functions like URLFetch or Import, you can extract data from websites or web APIs, allowing you to collect relevant information.
2. Web Scraping: With functions like URLFetch or Import, you can extract data from websites or web APIs, allowing you to collect relevant information.
Data Cleaning:
Data Cleaning:
1. Handling Missing Data: Wolfram Language offers functions like DeleteMissing or MissingDataRules to handle missing values in your dataset.
2. Removing Duplicates: You can use functions like DeleteDuplicates to eliminate duplicate entries or DeleteDuplicatesBy to remove duplicates based on specific criteria.
3. Data Transformation: Wolfram Language provides powerful functions like Map, Apply, or Transpose to transform, reshape, or reformat your data as per your requirements.
4. Filtering and Sorting: You can use functions like Select, Cases, or SortBy to filter and extract specific subsets of data based on certain conditions or criteria.
5. Data Manipulation: With a wide range of functions such as Replace, Part, or Merge, you can modify or combine different parts of your dataset, allowing you to clean and organize your data effectively.
These are just a few examples of how data collection and cleaning can be accomplished using Wolfram Language. The language provides a rich set of functions and capabilities that can be utilized to perform more complex data manipulation and cleaning tasks as well.
2. Removing Duplicates: You can use functions like DeleteDuplicates to eliminate duplicate entries or DeleteDuplicatesBy to remove duplicates based on specific criteria.
3. Data Transformation: Wolfram Language provides powerful functions like Map, Apply, or Transpose to transform, reshape, or reformat your data as per your requirements.
4. Filtering and Sorting: You can use functions like Select, Cases, or SortBy to filter and extract specific subsets of data based on certain conditions or criteria.
5. Data Manipulation: With a wide range of functions such as Replace, Part, or Merge, you can modify or combine different parts of your dataset, allowing you to clean and organize your data effectively.
These are just a few examples of how data collection and cleaning can be accomplished using Wolfram Language. The language provides a rich set of functions and capabilities that can be utilized to perform more complex data manipulation and cleaning tasks as well.
USING ENTITIES:
USING ENTITIES:
One of the most significant hurdles in data-centric tasks is FINDING raw data and converting it into a computable format. Wolfram Research has incorporated a deeply integrated FORMALISM that facilitates easy access to structured data. This object is known as an Entity. Using Entity, thousands of properties can be directly accessed from various entity types including people, countries, and planets, etc.
Moreover, the issue of data access and discovery is resolved with advanced natural language processing techniques. This enable us to input free-form expressions requesting any type of data and receive a precise Wolfram Language™ representation suitable for evaluation. These entities enable robust data analysis across a wide range of domains.
Use Ctrl+= and type in the desired entity, then press Enter to convert this free-form expression into a Wolfram Language one. For example, take the country entity corresponding to the Tanzania.
Moreover, the issue of data access and discovery is resolved with advanced natural language processing techniques. This enable us to input free-form expressions requesting any type of data and receive a precise Wolfram Language™ representation suitable for evaluation. These entities enable robust data analysis across a wide range of domains.
Use Ctrl+= and type in the desired entity, then press Enter to convert this free-form expression into a Wolfram Language one. For example, take the country entity corresponding to the Tanzania.
Input the Wolfram Language expression for the country “Tanzania”:
Entity[“Country”,”Tanzania”]
Entity[“Country”,”Tanzania”]
Each entity comes with a lots of properties over a wide variety of areas. This is built-in knowledge in the Wolfram Language.
Call the properties of the Tanzania country entity
Entity[“Country”,”Tanzania”][“Properties”]
Find the population of the country
Entity[“Country”,”Tanzania”][“Population”]
YOUR TURN
YOUR TURN
Statistics and Probability for DS
Statistics and Probability for DS
Statistics plays a crucial role in Data Science. It helps to process complex data, make inferences and informed decisions, and validate models. Wolfram Language has a rich set of built-in statistical functions that can be used to perform statistical analysis.
Basic Descriptive Statistics
Descriptive statistics provide basic measures that summarize and represent a dataset. Some of these measures include the mean (average), median (middle value), mode (most common value), variance (spread of data), and standard deviation (average distance of data from the mean).
In Wolfram Language, you can compute these using the Mean, Median, Commonest, Variance, and StandardDeviation functions respectively. For example:
Basic Descriptive Statistics
Descriptive statistics provide basic measures that summarize and represent a dataset. Some of these measures include the mean (average), median (middle value), mode (most common value), variance (spread of data), and standard deviation (average distance of data from the mean).
In Wolfram Language, you can compute these using the Mean, Median, Commonest, Variance, and StandardDeviation functions respectively. For example:
Probability Distributions
Wolfram Language supports a wide variety of probability distributions like Normal Distribution (NormalDistribution), Binomial Distribution (BinomialDistribution), Poisson Distribution (PoissonDistribution), and many others.
You can use these to compute probabilities, generate random variables, or plot the probability density function. For example:
Wolfram Language supports a wide variety of probability distributions like Normal Distribution (NormalDistribution), Binomial Distribution (BinomialDistribution), Poisson Distribution (PoissonDistribution), and many others.
You can use these to compute probabilities, generate random variables, or plot the probability density function. For example:
Statistical Testing
Statistical testing is used to make inferences in data. Some common statistical tests are t-test (TTest), chi-square test (ChiSquareTest), and ANOVA (To use ANOVA, you first need to load the Analysis of Variance Package using Needs[“ANOVA`”]), which are used to compare means, test for independence, and compare variances respectively.
Statistical testing is used to make inferences in data. Some common statistical tests are t-test (TTest), chi-square test (ChiSquareTest), and ANOVA (To use ANOVA, you first need to load the Analysis of Variance Package using Needs[“ANOVA`”]), which are used to compare means, test for independence, and compare variances respectively.
Regression Analysis
Wolfram Language supports various types of regression analysis, such as linear regression (Fit or LinearModelFit), logistic regression (LogitModelFit), and others. These functions create models that can predict one variable based on others.
Wolfram Language supports various types of regression analysis, such as linear regression (Fit or LinearModelFit), logistic regression (LogitModelFit), and others. These functions create models that can predict one variable based on others.
Distribution Fitting
The FindDistribution function in Wolfram Language can be used to automatically find the best distribution for a given dataset. This is useful when you want to understand the underlying probability distribution that fits your data. Here’s how you can use it:
The FindDistribution function in Wolfram Language can be used to automatically find the best distribution for a given dataset. This is useful when you want to understand the underlying probability distribution that fits your data. Here’s how you can use it:
The FindDistribution function returns a DataDistribution object, which can be used as a normal distribution for other computations, such as calculating probabilities, quantiles, etc.
Correlation and Covariance
Correlation measures the degree to which two variables are linearly related. The Correlation function can be used to compute the correlation:
Correlation measures the degree to which two variables are linearly related. The Correlation function can be used to compute the correlation:
Covariance, on the other hand, measures how much two random variables vary together . The Covariance function can be used to compute it :
Hypothesis Testing
Hypothesis testing is a statistical method that is used in making statistical decisions using experimental data. Hypothesis Testing is basically an assumption that we make about the population parameter. For example, you can perform a hypothesis test comparing the mean of a sample to a given value using the TTest function:
Hypothesis testing is a statistical method that is used in making statistical decisions using experimental data. Hypothesis Testing is basically an assumption that we make about the population parameter. For example, you can perform a hypothesis test comparing the mean of a sample to a given value using the TTest function:
HYPOTHESIS TESTING (Extension)
HYPOTHESIS TESTING (Extension)
HYPOTHESIS TESTING
Hypothesis testing is a core concept in statistics and is widely used in data science to make informed decisions based on data. A statistical hypothesis test is a method of making decisions using data, whether from a controlled experiment or an observational study.
In hypothesis testing, we have two statements of interest: the first is our actual test and is known as the Null Hypothesis (denoted as H0), while the other, called the Alternative Hypothesis (denoted as H1 or Ha), is the statement we want to switch to if our hypothesis is rejected.
The basic steps involved in performing a hypothesis test are:
-State the null hypothesis and the alternative hypothesis.
- Choose the significance level for the test.
- Collect and summarize the data.
- Perform the test.
- Make a decision based on the test result.
Wolfram Language has several built-in functions for performing various types of hypothesis tests. Here are some examples:
Hypothesis testing is a core concept in statistics and is widely used in data science to make informed decisions based on data. A statistical hypothesis test is a method of making decisions using data, whether from a controlled experiment or an observational study.
In hypothesis testing, we have two statements of interest: the first is our actual test and is known as the Null Hypothesis (denoted as H0), while the other, called the Alternative Hypothesis (denoted as H1 or Ha), is the statement we want to switch to if our hypothesis is rejected.
The basic steps involved in performing a hypothesis test are:
-State the null hypothesis and the alternative hypothesis.
- Choose the significance level for the test.
- Collect and summarize the data.
- Perform the test.
- Make a decision based on the test result.
Wolfram Language has several built-in functions for performing various types of hypothesis tests. Here are some examples:
One-Sample T-Test
Let’s assume you have a sample of 10 weights (in kilograms) of a certain species of apple:
And you want to test whether the average weight of this species of apple is 200 kg. Here, your null hypothesis would be that the mean weight is 200 kg, and your alternative hypothesis would be that the mean weight is not 200 kg.
In Wolfram Language, you can use the TTest function to perform this test:
In Wolfram Language, you can use the TTest function to perform this test:
This will return a string that contains the conclusion of the hypothesis test at a 5 % significance level . Two - Sample T - Test
Suppose now you have weights from another species of apple :
Suppose now you have weights from another species of apple :
And you want to test whether there is a significant difference between the weights of the two species of apples . The null hypothesis here would be that the mean weight of the two species is the same, while the alternative hypothesis would be that the means are different . The TTest function can also be used for this purpose :
This will perform a two-sample t-test and return the conclusion of the test.
Chi-Squared Test
The chi-squared test is used to determine if there is a significant association between two categorical variables. Suppose we have the following data of observed frequencies of smokers and non-smokers who exercise and don’t exercise:
Where the first row represents smokers and the second row represents non-smokers. The first column represents people who exercise, and the second column represents people who don’t exercise. We want to test whether smoking is independent of exercising.
The PearsonChiSquareTest function can be used for this:
The PearsonChiSquareTest function can be used for this:
This performs a chi-squared test for independence and returns the conclusion. Please note that conclusions of these tests are based on a significance level of 5%, and they assume that the data satisfy the assumptions of the tests. Always make sure to check the assumptions before performing the tests, and interpret the results in the context of the problem.
The above examples demonstrate the capability of Wolfram Language in statistics and data science . As a data scientist, you can use these functions to perform complex analyses, make predictions, and derive insights from data.
These examples illustrate some of the capabilities of the Wolfram Language for performing sophisticated statistical analyses. The Wolfram Language provides many more statistical functions, covering virtually all areas of modern statistics.
These examples illustrate some of the capabilities of the Wolfram Language for performing sophisticated statistical analyses. The Wolfram Language provides many more statistical functions, covering virtually all areas of modern statistics.
Data Exploration and Visualization
Data Exploration and Visualization
In this Section, we will explore some data visualization functions available in the Wolfram Language and use real-life data to apply them. Data exploration is a crucial step in any data analysis or data science project. It involves understanding the structure, properties, and patterns in the data. This often involves looking at descriptive statistics, visualizing the data using various types of plots, and sometimes computing correlations or other summary measures.
Categorical (or qualitative) variable: a variable that identifies a class or category
Numerical (or quantitative) variable: a variable that specifies a quantity or amount
Numerical (or quantitative) variable: a variable that specifies a quantity or amount
Visualizing data is one of the most powerful tools for data exploration, as it allows you to immediately see patterns, outliers, and trends in your data. Wolfram Language provides several functions for creating different types of plots.
ListPlot: This function can be used to create a simple scatter plot. It’s often used for visualizing a sequence of numbers, or pairs of numbers.
For plotting pairs of numbers:
Histogram: This function provides a visualization of the distribution of a set of continuous data points by dividing the data into bins.
BarChart: If you have categorical data, you might want to use a bar chart to show the count of each category.
PieChart: This function is useful for showing the proportion of different categories in your data.
BoxWhiskerChart: This type of chart is useful for showing the median, quartiles, and possible outliers in your data.
Plot3D, ListPointPlot3D : These functions are useful for visualizing data that has three variables .
EXAMPLES
EXAMPLES
1. Visualizing the Digits of Pi
1. Visualizing the Digits of Pi
The digits of pi are seemingly random, but we can visualize them to see any potential patterns. In the following example, we’ll plot the first 100 digits of pi in a spiral format.
First, let’s generate the digits:
First, let’s generate the digits:
This gives us a list of the first 100 digits of pi. Now, let’s create the spiral visualization:
Each point on the plot represents a digit, and they are connected in the order they appear in pi. The distance from the center represents the digit’s value.
2. Visualizing Music
2. Visualizing Music
Wolfram Language can also import and analyze sound data. Let’s create a spectrogram of a piece of music:
The spectrogram represents the frequency content of the music over time . The x - axis is time, the y - axis is frequency, and the color represents the intensity of a particular frequency at a particular time .
3. The Mandelbrot Set
3. The Mandelbrot Set
The Mandelbrot set is one of the most famous fractals and can be visualized using complex numbers . In the following example, we' ll generate a plot of the Mandelbrot set :
The plot shows the points in the complex plane that belong to the Mandelbrot set . The colors represent the rate at which points escape to infinity - points inside the set are colored black, and points outside the set are colored according to how quickly they escape to infinity .
4. Visualizing Earth Elevation Data
4. Visualizing Earth Elevation Data
Wolfram Language can access a vast amount of real-world data, including elevation data for the Earth’s surface. Let’s create a relief plot of the elevations in a region around Mount Everest:
The ReliefPlot shows a 2D representation of the 3D elevation data. The colors represent different elevations, with darker colors representing lower elevations and lighter colors representing higher elevations.
5. Visualizing the Golden Ratio
5. Visualizing the Golden Ratio
Golden Ratio is a mathematical constant that appears frequently in nature, architecture, and art. It is often represented by the Greek letter phi (Φ) and is approximately equal to 1.6180339887.
Problem: Plot the first 100 terms of the Fibonacci sequence divided by their preceding term. The ratio should approach the Golden Ratio.
Problem: Plot the first 100 terms of the Fibonacci sequence divided by their preceding term. The ratio should approach the Golden Ratio.
In this plot, the blue line represents the ratio of consecutive Fibonacci numbers, and the red line represents the Golden Ratio. As the term number increases, the ratio of consecutive Fibonacci numbers approaches the Golden Ratio.
6. Visualizing Earthquakes
6. Visualizing Earthquakes
Wolfram Language has access to a variety of real-world data, including seismic activity data.
Problem: Plot the locations of all earthquakes that occurred in the last 30 days on a world map.
Problem: Plot the locations of all earthquakes that occurred in the last 30 days on a world map.
In this plot, each red point represents the location of an earthquake that occurred in the last 30 days.
These examples show how Wolfram Language can be used to create a wide variety of data visualizations, from mathematical constructs to real - world data . The language provides many more functions for creating and customizing visualizations to suit your needs .
YOUR TURN
YOUR TURN
Plot the GDP of East African Countries with both PieChart and BarChart. Discuss which graph is more suitable and why.
Machine Learning
Machine Learning
Machine Learning is a subset of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. In simpler terms, it’s a method of data analysis that allows computers to “learn” from data and make predictions or decisions without being explicitly programmed to do so.
Machine learning algorithms find natural patterns in data that generate insight and help make better decisions and predictions. They are used every day to make critical decisions in medical diagnosis, stock trading, energy load forecasting, and more.
Machine learning algorithms find natural patterns in data that generate insight and help make better decisions and predictions. They are used every day to make critical decisions in medical diagnosis, stock trading, energy load forecasting, and more.
Here are some examples:
1. Email Filtering: You may notice that your email account filters out spam or promotional emails and sends them to different folders automatically. This is a basic example of machine learning. The email service has learned from past data, such as which emails you delete or mark as spam, to predict and classify new incoming emails.
2. Recommendation Systems: When you shop online, the website often recommends products you might like. These recommendations are made by machine learning algorithms that have learned your preferences from your past activities on the website, such as products viewed, purchased, or searched.
3. Fraud Detection: Credit card companies use machine learning to detect fraudulent transactions. They build a model based on past transactions of each customer. When a new transaction comes in, they compare it with the model, and if it deviates significantly, it is flagged as potentially fraudulent.
4. Speech Recognition: When you ask Siri, Google Assistant, or Alexa a question, machine learning is at work. These systems have been trained on vast amounts of speech data and can convert your spoken words into written text, and understand what you’re asking.
In each of these examples, a machine learning model was trained on a large amount of data. “Training” involves inputting a lot of data into the model and allowing the model to adjust its internal parameters to best represent the data. Once the model is trained, it can be used to make predictions on new, unseen data.
The Wolfram Language includes a high-level framework for machine learning, which is simple to use, even for beginners.
1. Email Filtering: You may notice that your email account filters out spam or promotional emails and sends them to different folders automatically. This is a basic example of machine learning. The email service has learned from past data, such as which emails you delete or mark as spam, to predict and classify new incoming emails.
2. Recommendation Systems: When you shop online, the website often recommends products you might like. These recommendations are made by machine learning algorithms that have learned your preferences from your past activities on the website, such as products viewed, purchased, or searched.
3. Fraud Detection: Credit card companies use machine learning to detect fraudulent transactions. They build a model based on past transactions of each customer. When a new transaction comes in, they compare it with the model, and if it deviates significantly, it is flagged as potentially fraudulent.
4. Speech Recognition: When you ask Siri, Google Assistant, or Alexa a question, machine learning is at work. These systems have been trained on vast amounts of speech data and can convert your spoken words into written text, and understand what you’re asking.
In each of these examples, a machine learning model was trained on a large amount of data. “Training” involves inputting a lot of data into the model and allowing the model to adjust its internal parameters to best represent the data. Once the model is trained, it can be used to make predictions on new, unseen data.
The Wolfram Language includes a high-level framework for machine learning, which is simple to use, even for beginners.
Let’s see it in action:
Classification:
Classification:
The Classify function in the Wolfram Language is a high-level function used for creating a machine learning classifier. A classifier is a predictive model used to assign new data points to one of a set of categories based on their features.
Here’s the general syntax of the Classify function:
Classify[training]
In this function, training is an association where the keys are the classes and the values are the examples of each class. The examples can be given as a list of elements or as a single element.
Here’s the general syntax of the Classify function:
Classify[training]
In this function, training is an association where the keys are the classes and the values are the examples of each class. The examples can be given as a list of elements or as a single element.
In this example, we’re creating a classifier that categorizes text as either related to cats or dogs based on two examples of sounds each animal makes. Once we have the classifier, we can use it to classify new data:
Let’s find c[“Meaw], c[“Meaow”], c[“moof”], c[bork]
Let’s find c[“Meaw], c[“Meaow”], c[“moof”], c[bork]
This returns “Cats”, since “meow” is a sound associated with cats in our training data.
Classify can work with all sorts of data, not just text. For instance, you could classify images, numerical data, or any other kind of data for which you can provide examples.The Wolfram Language’s Classify function uses a variety of advanced machine learning algorithms to create the classifier, automatically choosing the best one for your data. It’s a powerful tool for many kinds of machine learning tasks.
Classify can work with all sorts of data, not just text. For instance, you could classify images, numerical data, or any other kind of data for which you can provide examples.The Wolfram Language’s Classify function uses a variety of advanced machine learning algorithms to create the classifier, automatically choosing the best one for your data. It’s a powerful tool for many kinds of machine learning tasks.
Text Sentiment Analysis:
We could use Classify to create a basic sentiment analysis model that classifies text as “Positive”, “Negative”, or “Neutral”.
We can classify a new piece of text as follows:
Prediction:
Prediction:
The Predict function is used to train a regression model, which predicts a number. The Predict function in the Wolfram Language is a high-level function used for training predictive models. A predictive model is a model that’s used to predict future or unseen data based on the relationship between the variables in the training data.
The general syntax of the Predict function is:
The general syntax of the Predict function is:
In this function, training is a list of rules or an association that maps inputs to outputs. Each input-output pair is an example from which the function can learn.
Here’s a simple example of using Predict:
Here’s a simple example of using Predict:
This will return a number close to 12, since the predictor has learned to approximate the function f(x) = 2x.
The Wolfram Language’ s Predict function uses a variety of advanced machine learning algorithms to create the predictor, automatically choosing the best one for your data . It’ s a powerful tool for many kinds of machine learning tasks .
The Wolfram Language’ s Predict function uses a variety of advanced machine learning algorithms to create the predictor, automatically choosing the best one for your data . It’ s a powerful tool for many kinds of machine learning tasks .
Image Identification
Image Identification
Wolfram Language also provides high-level functions for specific machine learning tasks. For example, ImageIdentify can identify the content of an image.
LanguageIdentify:
LanguageIdentify:
If you want more control, NetTrain and NetModel functions are available to build and train deep learning models from scratch or modify pre-trained models.
Final Projects
Final Projects
YOUR TURN – PROJECTS
YOUR TURN – PROJECTS
1. Climate Change Analysis
1. Climate Change Analysis
Use historical weather data to analyze climate change in a East Africa. This can be done using the built-in WeatherData function in the Wolfram Language. You can create plots to visualize the change in temperature over time.
Follow-up Questions:
◼
How has the average temperature changed over the last 100 years?
◼
Can you identify any trends or patterns in the data?
◼
How can you use this data to predict future temperatures?
2. Population Growth Visualization:
2. Population Growth Visualization:
Use Wolfram’s CountryData function to get information about population growth in 5 different countries of your choice. You can create plots to visualize this growth and compare different countries.
Follow-up Questions:
◼
Which countries have the highest population growth?
◼
How has the population of your country changed over time?
◼
What factors might contribute to changes in population growth?
3. Visualizing Music Data:
3. Visualizing Music Data:
Wolfram Language can analyze MIDI files. Select several pieces from various genres and create visualizations of attributes such as note distribution, key, and tempo. Explore midi file of built-in sounds in WL.
Follow-up Questions:
◼
What patterns can you find in different genres of music?
◼
How does tempo and key vary across genres?
4. Astronomy Data Visualization
4. Astronomy Data Visualization
Use the PlanetaryMoonData and PlanetData functions in the Wolfram Language to get information about different celestial bodies. You can then create plots to visualize this data.
Follow-up Questions:
◼
How do the sizes of the planets compare to each other?
◼
How do the distances of the moons from their planets correlate with their sizes?
◼
How does the presence of moons relate to the size of the planet?
6. Stock Market Analysis:
6. Stock Market Analysis:
Use the built-in FinancialData function to retrieve historical data for a specific stock. You can plot this data over time, calculate basic statistics like the mean and standard deviation, and analyze trends.
Follow-up Questions:
◼
How has the price of the stock changed over time?
◼
What patterns can you identify in the data?
◼
Can you relate these patterns to any historical events?
7. Visualizing Earthquakes:
7. Visualizing Earthquakes:
Use the built-in EarthquakeData function to access information about past earthquakes. You can plot these earthquakes on a map, possibly using GeoGraphics, and analyze their distribution.
Follow-up Questions:
◼
Where do earthquakes most frequently occur?
◼
Can you find a correlation between the depth and magnitude of earthquakes?
◼
Can you identify any trends over time?
8. Digital Image Processing:
8. Digital Image Processing:
Use the built-in Image Processing functions to manipulate image of yours. You should apply filters, adjust colors, blur, and create effects.
Follow-up Questions:
◼
What effects can you achieve by combining different filters?
◼
Can you write a function to automatically enhance an image
9. COVID - 19 Data Dashboard:
9. COVID - 19 Data Dashboard:
Create an interactive dashboard that visualizes COVID-19 statistics such as cases, deaths, and vaccination rates. Use Wolfram Language’ s visualization functions to display data in charts, maps, and tables.
10. Population Pyramid:
10. Population Pyramid:
Generate a population pyramid using demographic data to illustrate the distribution of age and gender within a population. Wolfram Language’ s bar chart or histogram functions can be used to create the pyramid visualization .
11. World Map with Country Data:
11. World Map with Country Data:
Create a world map visualization with different countries shaded based on specific data such as GDP, population, or life expectancy. Wolfram Language’ s built-in geographic visualization functions can be utilized for this project.
12. Time Series Line Chart:
12. Time Series Line Chart:
Plot a time series line chart to visualize trends and patterns in data over time. You can choose various datasets such as stock prices, temperature fluctuations, or social media trends to create informative visualizations.
13. Word Cloud:
13. Word Cloud:
Generate a word cloud visualization to display the most frequently occurring words in a text document or dataset. Wolfram Language’ s word cloud functions can be used to create visually appealing and informative word clouds.
Concluding Remarks
Concluding Remarks
Wolfram language, with its broad capabilities and user-friendly nature, has proven to be an excellent tool for teaching and learning. It has made the complex tasks of Data Science approachable and the abstract concepts tangible. The discipline of data science is ever more important in our contemporary world. With the increasing digitization and the explosion of data in various fields, there is a growing need for individuals who can navigate, analyze, and draw meaningful insights from this wealth of information. This need is not merely a passing trend but a requirement for the future. Through mastering data science, you are preparing yourselves to be the future leaders, decision-makers, and innovators in this data-driven world. A particular highlight in this course has been the exploration of machine learning. It’s an area full of potential, capable of transforming the way we interact with data and make predictions. I am eager to expand on it in future courses.
There are virtually no limits to what you can do with the Wolfram Language. Its functionality extends far beyond what we’ve been able to cover in this course.
Let’s continue to explore, learn, and innovate. The future awaits, and it is brighter and beautiful because of the young minds able to make informed and data-driven decisions.
There are virtually no limits to what you can do with the Wolfram Language. Its functionality extends far beyond what we’ve been able to cover in this course.
Let’s continue to explore, learn, and innovate. The future awaits, and it is brighter and beautiful because of the young minds able to make informed and data-driven decisions.
Acknowledgement
Acknowledgement
Firstly, I would like to express my profound gratitude to John McNally. His diligent follow-ups and constant support throughout the work have been invaluable. His expertise in coding was instrumental in bringing the practical aspects of the course to life, and in maintaining the direction and focus of the course. Next, my heartfelt thanks go to Paul Abbott. Paul Abbott, his readiness to lend a hand and share his wealth of ideas has been greatly appreciated.
I must credit Stephen Wolfram, the central figure of the Wolfram Summer School. His pioneering work in developing the WL has made it possible for countless individuals to delve into data science and computational thinking in a meaningful way. His visionary thinking is the bedrock upon which this course was built, and for that, I am extremely grateful.
Lastly, but importantly, I extend my heartfelt thanks to all the dedicated staff members who have tirelessly worked behind the scenes. Their collective efforts and commitment have turned this summer school program into a success. Their support has been instrumental, and I am truly grateful for the opportunity to work with such dedicated and talented individuals.
I must credit Stephen Wolfram, the central figure of the Wolfram Summer School. His pioneering work in developing the WL has made it possible for countless individuals to delve into data science and computational thinking in a meaningful way. His visionary thinking is the bedrock upon which this course was built, and for that, I am extremely grateful.
Lastly, but importantly, I extend my heartfelt thanks to all the dedicated staff members who have tirelessly worked behind the scenes. Their collective efforts and commitment have turned this summer school program into a success. Their support has been instrumental, and I am truly grateful for the opportunity to work with such dedicated and talented individuals.
References
References
1. Wolfram, S. (2019). An elementary introduction to the Wolfram language. Wolfram Media.
2. Wolfram Language & System Documentation Center. (n.d.). https://reference.wolfram.com/language/
3. Introduction to Statistics, Interactive online video course: Wolfram U. (n.d.). https://www.wolfram.com/wolfram-u/courses/mathematics/introduction-to-statistics/
4. Multiparadigm Data Science, Interactive Online Video Course: Wolfram U. (n.d.). Multiparadigm Data Science, Interactive Online Video Course: Wolfram U. https://www.wolfram.com/wolfram-u/courses/data-science/multiparadigm-data-science/
5. Visual Explorations in Data Science: Wolfram U Course. (n.d.). Visual Explorations in Data Science: Wolfram U Course. https://www.wolfram.com/wolfram-u/courses/data-science/visual-explorations-in-data-science/
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
Hi Data: a data science primer for high school students with Wolfram Language
by Hasan Khan
Wolfram Community, STAFF PICKS, July 12, 2023
https://community.wolfram.com/groups/-/m/t/2958795
by Hasan Khan
Wolfram Community, STAFF PICKS, July 12, 2023
https://community.wolfram.com/groups/-/m/t/2958795