| 8,673,324 | Titles Scanned |
| 175,296 | contain "vocab" |
| 48,233 | contain "spelling" |
| 17,179 | contain "quadratic" |
| 6,307 | contain "scien" and "method" |
| 4,981 | contain "mitosis" |
| 4,032 | contain "gravity" |
| 2,607 | contain "credib" or "bias" or "media lit"* |
| 2,400 | contain "covalent" |
| 2,319 | contain "synonym" |
| 1,930 | contain "experiment" and "design/method" or "control" and "study/studies/experiment" |
| 1,831 | contain "antonym" |
| 1,688 | contain "homonym" or "homophone" or "homograph" |
| 1,045 | contain "fact" and "opinion" |
| 619 | contain "fallac" (for fallacy/fallacies/fallacious) |
| 126 | contain "pseudoscien" or "astrolog" or "horoscop" or "alt" and "medic" |
| 84 | contain "correlat" and "caus" |
ASSIGNMENTS & TESTS
How many assignments and tests contain certain keywords in the title in teachers’ grade books.
* Fact/Opinion is in one standard for three years. I counted that as 3 because the standard is very focused on that one skill (see below). Other standards span multiple years, but the skill is just part of a long list, so I did not multiple those. The count is just a rough proxy for priority, since weighting is very subjective.
How many assignments and tests contain certain keywords in the title in teachers’ grade books.
Data was scanned from the JupiterEd.com database for U.S. K-12 public schools (non-charter) over 3 years (fall 2018 - summer 2021).
Assignments and tests are a proxy for how often a skill is taught. Titles were the most consistent data available, since detailed instructions are in many different media and impractical to scan. However, teachers are not likely to use certain keywords in titles, like “correlation/causation”, “media literacy”, etc.; and “motive” is ambiguously for fictional characters. So the title is a useful proxy for only some keywords.
* Assessing Credibility/Bias is probably a requirement in many other assignments where it’s not in the title, so it is probably taught often; plus it is well represented in the standards.
Comparison keywords were chosen to show a few broad skills (like vocabulary) and very specific concepts (like antonyms).
How many tests and lessons contain certain keywords in the content/questions, weighted by number of students in class (e.g. 2 tests for 30 students = 60).
| 606,805 | Materials Scanned |
| 15,925,461 | Materials × Students (basis for data below) |
| 249,536 | contain "gravity" |
| 242,757 | contain "photosynth" |
| 103,122 | contain "experimental design" or "control group" or "controlled study/experim" or "placebo" or "random[ized] study/trial" or "sample size" or "blind[ed] study/experiment" |
| 86,003 | contain "covalent" |
| 70,628 | contain "scientific method" |
| 69,196 | contain "mitosis" |
| 69,046 | contain "control group" or "controlled study/experim" |
| 63,193 | contain "vaccine" and "vaccinat" |
| 18,122 | contain "pseudo-scien" or "non-scien" or "astrolog" or "horoscop" or "alternative medic" or "psychic" or "homeopath" |
| 15,501 | contain "experimental design" |
| 15,190 | contain "sample size" |
| 14,543 | contain "placebo" |
| 11,731 | contain "random[ized] study/trial" |
| 9,415 | contain "blind[ed] study/experiment" |
| 9,125 | contain "correlat" and "caus" |
Data was scanned from the JupiterEd.com database for US public schools using NGSS or a close version, from Fall 2018 to 5/5/22. This includes only tests and lessons that teachers wrote for the Jupiter LMS, not textbook assignments, PDFs, external sites/apps, etc.
Narrow science concepts are less likely to be in the assignment title, so this measure is probably a better representation for science topics, specifically keywords related to experimental design and pseudoscience.
STANDARDS
How many CCSS/NGSS objectives cite each skill
(out of 1479 total CCSS + 343 total NGSS)
| Standards | Qty | Grade | |
| Experimental Design | 1 | 5 | |
| Correlation/Causation | 1 | HS | |
| Motive | 1 | 8 | |
| Fact/Opinion* | 3 | 6-8 | |
| Fallacious Reasoning | 3 | 9-10 | |
| Evaluate Claims | 3 | 9-12 | |
| Gravity | 5 | 5-12 | |
| Credibility | 10 | 6-12 | |
| Quadratic Equations | 14 | 9-12 | |
| Vocabulary | 17 | 4-12 |
* Fact/Opinion is in one standard for three years. I counted that as 3 because the standard is very focused on that one skill (see below). Other standards span multiple years, but the skill is just part of a long list, so I did not multiple those. The count is just a rough proxy for priority, since weighting is very subjective.
NGSS Motion and Stability: Forces and Interactions: MS-PS2-5
Conduct an investigation and evaluate the experimental design to provide evidence that fields exist between objects exerting forces on each other even though the objects are not in contact. [Clarification Statement: Examples of this phenomenon could include the interactions of magnets, electrically-charged strips of tape, and electrically-charged pith balls. Examples of investigations could include first-hand experiences or simulations.] [Assessment Boundary: Assessment is limited to electric and magnetic fields, and is limited to qualitative evidence for the existence of fields.]
Conduct an investigation and evaluate the experimental design to provide evidence that fields exist between objects exerting forces on each other even though the objects are not in contact. [Clarification Statement: Examples of this phenomenon could include the interactions of magnets, electrically-charged strips of tape, and electrically-charged pith balls. Examples of investigations could include first-hand experiences or simulations.] [Assessment Boundary: Assessment is limited to electric and magnetic fields, and is limited to qualitative evidence for the existence of fields.]
This is not quite what I’m looking for e.g. to detect the flaw in someone’s claim that a certain diet works because of anecdotal evidence. Furthermore NGSS does not mention “control group”, “sample size”, “randomize”, “placebo”, “blind”, or any objective to distinguish “anecdotal evidence” or “pseudoscience”.
CCSS Math 9-12 Statistics & Probability: S-ID.C.9
Distinguish between correlation and causation.
CCSS applies only to math class. The Next Generation Science Standards does not directly mention correlation vs causation, but it does frequently cite “distinguishing between correlation and causation” in reference to a book: National Research Council 2012. A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas
CCSS Language Arts 6-8 History Literacy: RH.6-8.8
Distinguish among fact, opinion, and reasoned judgment in a text.
CCSS Language Arts 8 SL.8.2:
Analyze the purpose of information presented in diverse media and formats (e.g., visually, quantitatively, orally) and evaluate the motives (e.g., social, commercial, political) behind its presentation.
A couple writing standards include anticipating the reader’s biases, but none cites detecting a writer/speaker’s bias.
CCSS Language Arts 9-10: Informational Text: RI.9-10.8
Delineate and evaluate the argument and specific claims in a text, assessing whether the reasoning is valid and the evidence is relevant and sufficient; identify false statements and fallacious reasoning.
CCSS Language Arts 9-10: Writing: W.9-10.9b
Apply grades 9-10 Reading standards to literary nonfiction (e.g., Delineate and evaluate the argument and specific claims in a text, assessing whether the reasoning is valid and the evidence is relevant and sufficient; identify false statements and fallacious reasoning).
CCSS Language Arts 9-10: Speaking & Listening: SL.9-10.3
Evaluate a speaker’s point of view, reasoning, and use of evidence and rhetoric, identifying any fallacious reasoning or exaggerated or distorted evidence.
NGSS Ecosystems: Interactions, Energy, and Dynamics: HS-LS2-6
Evaluate the claims, evidence, and reasoning that the complex interactions in ecosystems maintain relatively consistent numbers and types of organisms in stable conditions, but changing conditions may result in a new ecosystem.
NGSS Biological Evolution: Unity and Diversity: HS-LS4-5
Evaluate the evidence supporting claims that changes in environmental conditions may result in (1) increases in the number of individuals of some species, (2) the emergence of new species over time, and (3) the extinction of other species.
NGSS Waves and Their Applications in Technologies for Information Transfer: HS-PS4-4.
Evaluate the validity and reliability of claims in published materials of the effects that different frequencies of electromagnetic radiation have when absorbed by matter.
CCSS Language Arts 6: W.6.1b: Support claim(s)...using credible sources...
CCSS Language Arts 7: W.7.1b: Support claim(s)...accurate, credible sources...
CCSS Language Arts 8: W.8.1b: Support claim(s)...accurate, credible sources...
CCSS Language Arts 6-8: Hist/Sci Writing: WHST.6-8.1b: ...using credible sources.
CCSS Language Arts 6: W.6.8: ...assess the credibility of each source...
CCSS Language Arts 7: W.7.8: ...assess the credibility and accuracy of each source...
CCSS Language Arts 8: W.8.8: ...assess the credibility and accuracy of each source...
CCSS Language Arts 6-8: Hist/Sci Writing: WHST.6-8.8: ...assess the credibility and accuracy...
CCSS Language Arts 9-10: SL.9-10.2: ...evaluating the credibility and accuracy of each source.
CCSS Language Arts 11-12: SL.11-12.2: ...evaluating the credibility and accuracy of each source...
CCSS Math 9-12 Number & Quantity N-CN.C.7 Solve quadratic equations...
CCSS Math 9-12 Number & Quantity N-CN.C.9 (+) ...true for quadratic polynomials.
CCSS Math 9-12 Algebra A-SSE.B.3a Factor a quadratic expression...
CCSS Math 9-12 Algebra A-SSE.B.3b Complete the square in a quadratic expression...
CCSS Math 9-12 Algebra A-CED.A.1 ...Include...linear and quadratic functions...
CCSS Math 9-12 Algebra A-REI.B.4 Solve quadratic equations...
CCSS Math 9-12 Algebra A-REI.B.4a ...transform any quadratic... Derive the quadratic formula...
CCSS Math 9-12 Algebra A-REI.B.4b Solve quadratic equations...quadratic formula...
CCSS Math 9-12 Algebra A-REI.C.7 Solve...quadratic equation...
CCSS Math 9-12 Functions F-IF.C.7a Graph linear and quadratic functions...
CCSS Math 9-12 Functions F-IF.C.8a ...factoring and completing the square in a quadratic function...
CCSS Math 9-12 Functions F-IF.C.9 ...a graph of one quadratic function...
CCSS Math 9-12 Functions F-LE.A.3 ...quantity increasing linearly, quadratically...
CCSS Math 9-12 Statistics & Probability S-ID.B.6a ...linear, quadratic, and exponential models.
CCSS Language Arts 4 W.4.2d vocabulary
CCSS Language Arts 5 W.5.2d vocabulary
CCSS Language Arts 6 W.6.2d vocabulary
CCSS Language Arts 6 L.6.6 vocabulary
CCSS Language Arts 7 W.7.2d vocabulary
CCSS Language Arts 7 L.7.6 vocabulary
CCSS Language Arts 8 W.8.2d vocabulary
CCSS Language Arts 8 L.8.6 vocabulary
CCSS Language Arts 9-10 W.9-10.2d vocabulary
CCSS Language Arts 9-10 L.9-10.6 vocabulary
CCSS Language Arts 11-12 W.11-12.2d vocabulary
CCSS Language Arts 11-12 L.11-12.6 vocabulary
CCSS Language Arts 6-8 History Literacy RH.6-8.4 vocabulary
CCSS Language Arts 6-8 History/Science Writing WHST.6-8.2d vocabulary
CCSS Language Arts 9-10 History Literacy RH.9-10.4 vocabulary
CCSS Language Arts 9-10 History/Science Writing WHST.9-10.2d vocabulary
CCSS Language Arts 11-12 History/Science Writing WHST.11-12.2d vocabulary
NGSS 5-PS2-1. gravitational force
NGSS MS-PS2-4. gravitational interactions
NGSS MS-ESS1-2. gravity
NGSS MS-ESS2-4. force of gravity
NGSS HS-PS2-4. gravitational forces
EXEMPLAR SCIENCE STANDARDS
A couple states focus more on scientific critical thinking than the NGSS does with these objectives:
Wisconsin
HS-ETS3-1. Ask questions to clarify an author’s motivation for promoting unscientific or falsified information on science topics (e.g. climate change, vaccines, GMOs, nuclear energy).
Florida
Adopted 2010
SC.8.N.2.1. Distinguish between scientific and pseudoscientific ideas.
SC.912.N.2.3. Identify examples of pseudoscience (such as astrology, phrenology) in society.
SC.3.N.1.5. Recognize that scientists question, discuss, and check each other's evidence and explanations.
SC.3.N.1.7. Explain that empirical evidence is information, such as observations or measurements, that is used to help validate explanations of natural phenomena.
SC.4.N.1.3. Explain that science does not always follow a rigidly defined method (“the scientific method”) but that science does involve the use of observations and empirical evidence.
SC.4.N.1.7. Recognize and explain that scientists base their explanations on evidence.
SC.5.N.2.1. Recognize and explain that science is grounded in empirical observations that are testable; explanation must always be linked with evidence.
SC.5.N.2.2. Recognize and explain that when scientific investigations are carried out, the evidence produced by those investigations should be replicable by others.
SC.6.N.2.2. Explain that scientific knowledge is durable because it is open to change as new evidence or interpretations are encountered.
SC.7.N.1.6. Explain that empirical evidence is the cumulative body of observations of a natural phenomenon on which scientific explanations are based.
SC.7.N.2.1. Identify an instance from the history of science in which scientific knowledge has changed when new evidence or new interpretations are encountered.
SC.7.N.3.1. Recognize and explain the difference between theories and laws and give several examples of scientific theories and the evidence that supports them.
SC.8.N.1.6. Understand that scientific investigations involve the collection of relevant empirical evidence, the use of logical reasoning, and the application of imagination in devising hypotheses, predictions, explanations and models to make sense of the collected evidence.
SC.912.N.3.1. Explain that a scientific theory is the culmination of many scientific investigations drawing together all the current evidence concerning a substantial range of phenomena; thus, a scientific theory represents the most powerful explanation scientists have to offer.