On improving the question-answer database of Elementary Introduction to Wolfram Language
On improving the question-answer database of Elementary Introduction to Wolfram Language
Tuseeta Banerjee
Wolfram Research
Elementary Introduction to Wolfram Language (EIWL) is perhaps one of the best, concise and one-stop and free introductions to the Wolfram Language. It not only introduces Wolfram Language, but can also be considered a tool to improve one’s existing coding skills. Any individual, with a programming or non-programming background can read this book for free, and get at par with programming in Wolfram Language. The chapters of this book do an excellent job of capturing the richness and variety of Wolfram Language and the interface of submitting answers with automatic grading is also quite attractive. Students at WSS over the past few years solve various problems from EIWL before moving on to Wolfram Challenges questions. However, the answers to the problems in the book are not unique, and that issue is already mentioned in the book. A part of this project is to create alternate versions of answers to challenges that do not currently exist in the database of answers, so that they may be helpful in improving the learning experience of thousands of students online who use this book judiciously to improve their coding skills.
A second aspect of the project is to identify questions that have ambiguity in them and may be subject to multiple interpretations from the student. From an educational stand point, the question in turn does not become a metric of the knowledge of the student, but the ability of the student to actually “guess” what the question-maker asks for. A part of this project also addresses some of these questions.
A second aspect of the project is to identify questions that have ambiguity in them and may be subject to multiple interpretations from the student. From an educational stand point, the question in turn does not become a metric of the knowledge of the student, but the ability of the student to actually “guess” what the question-maker asks for. A part of this project also addresses some of these questions.
Section 1: Database questions
Section 1: Database questions
This section deals with all the internal database questions, which might be updated regularly and hence the answers do not match to the existing answers.
1. Count the number of words in WordList[] that start with “q”
I am not sure if the WordList[] is updated which results in the incorrectness of the following two answers. However, I verified the solution with two very different approaches with String (and they both confirmed same answers).
In[]:=
(Counts@StringTake[ToLowerCase@WordList[],1])["q"]
Out[]=
194
In[]:=
Length@Select[WordList[],StringMatchQ[#,"q"~~___]&]
Out[]=
194
2. Make a line plot of the lengths of the first 1000 words in WordList[]
In[]:=
ListLinePlot@StringLength@(WordList[][[1;;1000]])
Out[]=
Plausible reason: WordList has been updated.
3. Plot together the historical frequencies of the words “science” and “technology
I have no idea if the WordFrequencyData for these words got updated, but they don’t seem to match. I even added PlotLegends to match the Expected Output, but it still did not work.
In[]:=
DateListPlot[{WordFrequencyData["technology","TimeSeries"],WordFrequencyData["science","TimeSeries"]}]
Out[]=
4. Make a labeled bar chart of the number of times each possible first letter occurs in words from WordList[].
In[]:=
BarChart[Counts@StringTake[ToLowerCase@WordList[],1],ChartLabelsAlphabet[]]
Out[]=
5. Make an association giving the 5 most common first letters of words in WordList[] and their counts.
In[]:=
(ReverseSort@Counts@StringTake[ToLowerCase@WordList[],1])[[;;5]]
Out[]=
s4502,c3694,p3168,d2436,a2397
6. Find the 10 most common words in ExampleData[{“Text”,”AliceInWonderland”}]
In[]:=
Keys@((ReverseSort@Counts@TextWords@ToLowerCase@ExampleData[{"Text","AliceInWonderland"}])[[1;;10]])
Out[]=
{the,and,a,to,she,of,it,was,in,alice}
7. Find the longest 10 words in WordList[].
In[]:=
(ReverseSort@AssociationThread[WordList[]->StringLength/@WordList[]])[[1;;10]]
Out[]=
electroencephalographic23,electroencephalograph21,uncharacteristically20,magnetohydrodynamics20,internationalization20,electroencephalogram20,counterrevolutionary20,compartmentalization20,buckminsterfullerene20,straightforwardness19
8. Correct the program Counts[StringTake[#,2]&/@WordList[]]
In[]:=
Counts[StringTake[#,2]&/@Select[WordList[],StringLength[#]>2&]]
Section 2: Ambiguous questions
Section 2: Ambiguous questions
1. Create a sound from the letters in CABBAGE, each playing for 0.3 seconds sounding like a guitar
This question is ambiguous and I kept guessing what I need to do with the letters of the word “CABBAGE”, LetterNumber made most sense, but apparently it is not what the question wants.
In[]:=
Sound[Table[SoundNote[n,0.3,"Guitar"],{n,LetterNumber["cabbage"]}]]
Out[]=
2. Display 100 polygons with side length 10 and opacity [0.5] and random choices of colors, sides between 3 and 8, and integer coordinates up to 100
The confusion to this question is about the words side length, are they the radius of the polygon or the side of the polygon, because although they are related, they are not the same.
In[]:=
Graphics@Table[{RandomColor[],Opacity[0.5],RegularPolygon[{RandomInteger[100],RandomInteger[100]},10,RandomInteger[{3,8}]]},{i,100}]
Out[]=
3. Make a histogram of sentence length in the Wikipedia article on computers.
This is an ambiguous question: Is Sentence Length = # of words? # of characters? From the histogram of expected output it seems character Length. I tried both none of them were accepted.
In[]:=
Histogram@(Length/@(TextWords/@TextCases[WikipediaData["computers"],"Sentence"]))
Out[]=
In[]:=
Histogram[StringLength/@TextCases[WikipediaData["computers"],"Sentence"]]
Out[]=
Section 3: Entity questions
Section 3: Entity questions
1. Get an image of an elephant
A simple answer from making a query to W|A resulted in the image, which seems correct, but is unacceptable by the database.
2. Compute the height of the Empire State Building divided by the height of the Great Pyramid
A simple answer from making a query to W|A resulted in the image, which seems correct, but is unacceptable by the database. There are multiple pyramids, so specifying which one of them would help clear the ambiguity here.
3. Find the dominant colors of an image collage of the flag images of all countries in Europe
I seem to not be getting the correct answers for any of the ImageCollage-s on the images of flags of countries in Europe, if I do either the W|A query input or the hard-coded Entity way of programming. Not sure where the answer database fails to match mine. I have not compared the answers to the already existing answers (because that is not how a student is expected to approach the problem). I believe it is in the way the question is structured. My guess is that it wants us to query for countries in European Union, and not the countries in continent Europe, but there remains an ambiguity in the question.
4. Make Pie Chart of the GDP of countries of Europe
I seem to not be getting the correct answers for any of the countries in Europe questions. My guess is that it wants us to query for countries in European Union, and not the countries in continent Europe, but there remains an ambiguity in the question. However, the pie chart looks exactly the same as the one given in Expected Answers, and thus I ran out of guesses as well!
5. Make ImageCollage of the flags of all countries in Europe. using the “FlagImage” property
I seem to not be getting the correct answers for any of the ImageCollage-s on the images of flags of countries in Europe, if I do either the W|A query input or the hard-coded Entity way of programming. Not sure where the answer database fails to match mine. I have not compared the answers to the already existing answers (because that is not how a student is expected to approach the problem). I believe it is in the way the question is structured. My guess is that it wants us to query for countries in European Union, and not the countries in continent Europe, but there remains an ambiguity in the question.
6. Get a list of distances to each planet using “DistanceFromEarth” property and convert all the results to light minutes
Not sure if the data is updated as a result of which the answers do not match.
7. Binarize each flag of Europe and make an image collage of the result.
I seem to not be getting the correct answers for any of the ImageCollage-s on the images of flags of countries in Europe, if I do either the W|A query input or the hard-coded Entity way of programming. Not sure where the answer database fails to match mine. I have not compared the answers to the already existing answers (because that is not how a student is expected to approach the problem). I believe it is in the way the question is structured. My guess is that it wants us to query for countries in European Union, and not the countries in continent Europe, but there remains an ambiguity in the question.
8. Make a pie chart of the GDP’s of the countries in the G5, labeling each wedge.
9. Make a pie chart of the GDP’s of the countries in the G5, giving a legend for each wedge.
10. Make a word cloud of countries in Central America with the names of countries are proportional to the lengths of the Wikipedia article about them.
11. Find universities that can be referred to as “U of X” where x is any letter of the alphabet.
They are the same answers.
Section 4: Manipulate questions
Section 4: Manipulate questions
11. Make a Manipulate to rotate an image of a cat between 0° and 180°
This has been observed in many Manipulate question. Many a times, the step size is not mentioned, as a result of which there can be ambiguities, do you want continuous step sizes or in some size differences. This is just an example
Section 5: Apparently correct answers
Section 5: Apparently correct answers
1. Find the distance from 0 latitude 0 longitude to Eiffel Tower
This answer is actually correct and matches the answer provided but the answer is not accepted.
2.Make a 12x12 multiplication table with white type on black background
This grid did not actually evaluate properly in the cloud and the answer did not match the existing answers.
3. Flatten to generate a table from 1 to 50 where even numbers are colored red.
This look right, and use the functions also mentioned in the question
4. Make a 3D plot of heights generated from Mod[i,j] with i and j going upto 100
This plot look very similar to the Expected Output but it was not accepted.
5. Generate a 3D list plot using the image data from a binarized size-200 letter “W” as heights.
This plot look very similar to the Expected Output but it was not accepted.
6. Make a graphics of 1000-step 2D random walk which starts at {0,0}, and in which at each step a pair of random numbers between -1 and 1 are added to the coordinates
The output look similar to the Expected Output.
7. Make a graphics of 1000 - step 3 D random walk which starts at {0, 0,0}, and in which at each step a pair of random numbers between - 1 and 1 are added to the coordinates
8. Make a LinePlot that takes lengths of Roman numerals up to 100 that PlotRange would be sufficient for all numerals up to 1000
This answer actually matches if I explicitly say Range {0,12}, although we do not need to go upto 12, and PlotRange -> All/Automatic were sufficient.
9. Start with a size-50 “A”, then make a list of nestedly applying frame and random rotations 5 times
I am not sure about why not a single approach was accepted. I tried various approaches and the output look similar to the Expected Output
10. Generate a graph with 200 connections, each between nodes with numbers chosen randomly between 0 and 100
This answer is very similar to a previously accepted answer and indeed we make 200 connections randomly between nodes numbered from 0 to 100
11. Generate a plot that shows communities in a random graph with nodes numbered between 0 and 100, and 200 connections
12. Generate a graph obtained by nestedly finding bordering countries starting from the United States, and going 4 iterations.
13. Make a list of all the different strings that can be obtained by permuting the characters in “hello”
14. Make an array plot from the numbers of the letters in the first 1000 characters in the Wikipedia article on “Computers”, with 30 letters per row.
15. Riffle periods in the string “UNCLE” to make “U.N.C.L.E.”
16. Make a list of array plots, each 50 rows high and with image size 50, of successive possible 8-tuples of 0 and 1.
17. Generate 1000 random sequences of 4 letters, and make a list of ones that appear in WordList[]
18. Make a word cloud of country names in the Wikipedia article on “gunpowder”.
19. Find all nouns in “She sells seashells by the sea shore”
20. Use TextCases to find the number of nouns, verbs and adjectives in the first 1000 characters of the Wikipedia article on computers.
56. Generate 10 random strings made of 5 letters, in which consonants (non-vowels) alternate with vowels (aeiou).
21. Define a function evenodd that gives Black if its argument is even and white otherwise, but gives red, if the argument is o.
22. Find the 5 longest integer names for integers up to 100.
23. find the 5 English names for integers up to 100 that have the largest number of “e”s in them.
Section 6: Time and day related questions
Section 6: Time and day related questions
There is a common pattern for these questions, they are hard to verify with the expected output, and follow a pattern for next few days (some mention today, some not, in that case the length of the output had to be compared), and some discrepancies are just hard to verify.
1. Compute the number of days since January 1 1900
This answer is hard to verify with the Expected Answer (the Expected Output for such questions need to be updated on a daily basis). Also, the queries from W|A actually match the code as well. Not sure why it is not accepted.
2. Find the length of daylight today by subtracting today’s sunrise form today’s sunset
I have tried this question in different ways, however, I never got it to accept.
3. Generate list of icons for the moon phases from today until 10 days from now
Many of these questions refer to from today until 10 days from now, and often require which days to be included. This particular answer never got accepted and I did not expect the Expected Output to match either
4. How many weeks have elapsed since January 1, 1900?
Again the coded interpretation match the W|A query but none match the answer in the database. Not sure if it a Round/Floor error, which seem to be the case. There is some ambiguity here.
5. Show in a column the times of the sunrise for the next 10 days starting today
Many of these questions refer to from today until n days from today and often require which days to be included. I often match the number of outputs in the Expected answer but this case it never got accepted.
6. Make a Manipulate of the icon for the moon phase over the next 15 days
Many of these questions refer to from today until n days from now, and often require which days to be included. I often match the number of outputs in the Expected answer but this case it never got accepted.
7. Make a plot of the air temperature in Paris over the past week with the plot range restricted between 50 and 80 degrees
This question was confusing, because a similar question from some previous chapters was accepted. The only difference was the PlotRange which was implemented, but still the answer was not accepted.
Similar question was the temperature of Eiffel Tower over the past week
Section 7: Geo-visualization questions
Section 7: Geo-visualization questions
Questions with geo-maps need to specify the visualizations type. (GeoGraphics, GeoListPlot...) and accept wider styling rules
1. Use /@ to draw separate maps of each country in the G5.
I believe in such “Map” questions there are multiple ways of plotting countries. We could specify exactly which plot is accepted or accept all forms of visualizing a country in the map.
2. Make a list of maps of Central America, highlighting each country in turn.
I believe in such “Map” questions there are multiple ways of plotting countries. We could specify exactly which plot is accepted or accept all forms of visualizing a country in the map.
Section 8: Simpler form questions
Section 8: Simpler form questions
Simpler form questions simply accepts the same form, which is unacceptable
1. Find simpler form for Select[IntegerDigits[Range[100,999]],First[#]Last[#]&]
Both of them are accepted
Section 9: Cloud deployment questions
Section 9: Cloud deployment questions
1. Deploy to the web a map that shows the current inferred location of the user.
I think the Here is not accepted here
2. Deploy a form on the web that takes a number x and returns x^x
For the next set of answers, the Interpreter expects “Number”, “String”, but not in quotes form works too
3. Deploy a form on the web that takes two numbers x and y and returns x^y
For the next set of answers, the Interpreter expects “Number”, “String”, but not in quotes form works too
4. Deploy a form page on the web that takes a string and repeatedly gives a reversed version at size 50.
For the next set of answers, the Interpreter expects “Number”, “String”, but not in quotes form works too
5. Deploy a form on the web that takes the topic of a Wikipedia page and gives a word cloud for the page
For the next set o answers, the Interpreter expects “Number”, “String”, but not in quotes form works too
6. Deploy a form on the web that takes an integer n and repeatedly generates a picture of a polygon with a random color and n sides.
For the next set o answers, the Interpreter expects “Number”, “String”, but not in quotes form works too
7. Make an image collage of disks with the dominant colors on https://www.google.com
Section 10: SendMail questions
Section 10: SendMail questions
None of them worked, however, I received the correct email with correct attachment
1 Send yourself mail with an icon of the current phase of the moon
2. Send yourself mail with a map of your current location
Section 11: Planet dataset questions
Section 11: Planet dataset questions
None of the planet dataset questions are accepted, and I believe it is because it does not expect the data to be obtained from ExampleData[{“Dataset”,”Planets”}]
1. Make a word cloud of the planets, with weights determined by their number of moons.
2. Make a bar chart of the number of moons for each planet
3. Make a dataset of the masses of the planets, sorted by their number of moons.
4. Make a dataset of planets and the mass of each one’s most massive moon.
At first I thought it did not make sense to have Mercury and Venus and so I just cut them out. But then I saw it was acceptable
5. Make a dataset of planets and the median mass of each one’s moons
6. For each planet, make a list of moons larger in mass than 0.0001 Earth masses.
Section 12: Dataset questions
Section 12: Dataset questions
None of the planet dataset questions are accepted, and I believe it is because it does not expect the data to be obtained from ExampleData[{“Dataset”,”Planets”}]
1. Find a dataset of the 5 largest observed altitudes in the Fireballs and Bolides dataset.
In[]:=
fireballs = ResourceData["Fireballs and Bolides"]
2. Find a dataset of the 5 largest observed altitudes in the Fireballs and Bolides dataset.
In[]:=
fireballs[TakeLargestBy[#Altitude &, 5]][All, "Altitude"]
3. Plot the nearest cities for the first 10 entries in the Fireballs and Bolides dataset, labeling each city.
In[]:=
GeoListPlot[ResourceData["Fireballs and Bolides"][All, "NearestCity"][[1 ;; 10]], GeoLabels -> True]
4. Plot the nearest cities for the entries with largest altitudes in the Fireballs and Bolides dataset, labeling each city.
In[]:=
GeoListPlot[ResourceData["Fireballs and Bolides"][TakeLargestBy[#Altitude &, 10]][All, "NearestCity"], GeoLabels -> True]
Section 13: NestWhileList or FixedPointList
Section 13: NestWhileList or FixedPointList
1. Use grid to show the sorting process in this section for {4,5,1,3,2}.
2. Use ArrayPlot to show the sorting process in this section for a list of length 50, with successive steps going across teh page.
3. Implement Euclid’s algorithm fro GCD in which {a,b} is repeatedly replaced by {b, Mod[a,b]} until b is 0, and apply to algorithm 12345,54321
Summary of the observations
Summary of the observations
The project was an exploratory one to identify the common pattern where the answers were not accepted. The questions were answered and the pattern of why the answers are not accepted are analysed. None of the answers are matched to the existing answers, to be fair to the users. Some pointers as to why there might be a difference between are pointed below as seen in the sections
◼
Some of the answers use existing rich databases in WL, and some of the outputs of the answers depend if they are updated or not. Such Expected Output should be updated as the databases are updated. Or for such answers, the methodology to the answer should be matched.
◼
Ambiguous questions on sizes of polygon, word/character size definitions could be reformulated.
◼
W|A query based/Entity based questions often fail even with typing with the exact phrase in the question. It appears mostly for the ambiguity in definition of the countries in Europe and mentioning which Pyramid might also help
◼
Some of the questions with Manipulate need to have better specifications of the step-sizes.
◼
Questions with maps need to specify the visualizations type. (GeoGraphics, GeoListPlot...)
◼
There are some cloud submission errors, and time outs for Entities (which are understandable)
◼
Simpler form questions need to be re-checked. The complex forms are also acceptable
◼
Cloud Deployment with some Interpreter problems with FormPage/FormFunction
◼
SendMail questions are correct, but don’t get accepted
◼
Planet dataset can be obtained from ExampleData, which it is not recognised
◼
NestWhileList can also be accepted with FixedPointList.
Concluding remarks
Concluding remarks
The scope of this project goes well beyond what is discussed currently . An ongoing project in the machine learning team at Wolfram Research, is to come up with ways of classifying errors and utilize some of the error kinds to build more robust code - correction models beyond parenthesis or sequence matching . This would require more reliable data, which in turn depends on updating the database or existing rules of checking answers for books like EIWL, Wolfram Challenges, etc .
Keywords
Keywords
◼
Wolfram Language
◼
Elementary Introduction
◼
Programming
◼
User Experience
◼
Question Interpretations
Acknowledgment
Acknowledgment
First, I would like to acknowledge Erin Cherry and Jesse Friedman led for the inspiration. Furthermore, I would also like to thank some useful comments and valuable feedback I received from present and prior WSS students about their experience with EIWL which further motivated me to take a more close look into the end of chapter problem sets.
References
References