- Autocorrelation, serial correlation, bang-bang duplicate, or pseudo-replications are when a piece of data follows another without new measures
- There is a difference between having questions to be solved by a database and poking around a database looking for interesting answers to question
- Early screening boosts “survival time” in useless ways
- Adjust measures to avoid errors rather than modeling them away
- “Guide to Bad Data” by Chris Groskopf
- Values missing
- Zeros replace missing values
- Data missing that you know should be there
- Rows or values duplicated
- Total differ from aggregates
- Suspicious values present
- Spreadsheet have 65536 rows or 255 columns
- Margin-of-error to large or unknown
- Benford’s Law fails
- Too good to be true
- Fix bad names immediately
- Survivor bias: “Most medieval castles were made of wood”
Category: Notes
-
“Seeing With Fresh Eyes” by Edward Tufte
-
“Time Travel” by James Gleick
- Concepts of time travel started in 1914
- 1876 was the first record of a centennial celebration
- No one pondered the future or the past until the 20th century
- New (remote) lands filled the roles of time travel in fiction
- We should not think about time as the same as space
- We cannot freely move through time like we can through space
- “What the balloon does for space, the time machine does for time”
- Prior to clocks, time was relative based on the setting
- Newton has to define time so acceleration could happen
- “If the speed of light is fixed then time cannot be.”
- The Olympics replace starting pistols with speakers to ensure everyone can hear the signal at the same time
- “Souls cannot move that fast so one needs to wait”
- We are always looking into the past
- All time was local until the train
- Is the arrow of time because of entropy?
- “You cannot stir things apart”
- In many Latinx cultures, you look forward to the past and back for the future
- (This makes a lot of sense because you observe the past, in front of you, and cannot see the future coming behind you)
- Eternity is “outside of time”
- Nostalgia originally meant “homesick”, not wanting a different time
- In 1955, we redefined “a second” from a fraction of a day to an independent time frame
- A nano second is about the length light travels a foot
- Mental Time Travel let’s us project forward and back over time
- “Now” is generally 2-3 seconds lumped together by the brain
- We need at least 1/10th a second to accurately differentiate the order of two events
-
“Chaos: Making a new science” by James Gleick
- Science became really interesting when looking at weather
- Warm coffee just cools; Hot coffee has convection
- Modern scientist have iterated on the past but that assumes everything that came before was true
- “Sensitive dependance on initial conditions”
- Physics and maths divorced in the 30’s but started getting back together in the 60’s
- When studying systems, scientists tended to assume oscillation was around a steady state
- Measuring non-standard shapes requires an infinitely small measuring tool
- Fractals allow for infinite length in a finite space
- “Dimensions” are the number of identifiers needed to identify a single point
- At a distance, a ball of string is a point and so one-dimensional; up close it is a one-dimensional object arranged to occupy three dimensions
- All physical touching is described by fractals
- Turbulence is very poorly understood and may be unknowable
- The Manhattan project was really about turbulence
- Attractors bring stability, even if not predictability
- Poincare return map
- Cross sections of the transit to track movement over time
- In order to be universal, it has to have scale
- Universality was originally rejected from publication
- Flow = shape + change or motion + form
- Control the space so you can manipulate time
- https://en.wikipedia.org/wiki/Mandelbrot_set
- “In biology, randomness is death”
- https://en.wikipedia.org/wiki/Barnsley_fern
- Computer experimentation really let fractals take off
- https://en.m.wikipedia.org/wiki/Lorenz_system
- Random (spread out and undefined) vs chaos (deterministic and patterned)
- Chaos brings a wealth of information (because steady states can have their information compressed)
- A map needs to simplify the things it maps
- Giving mosquitos a photo burst at midnight will destroy their circadian signals
- Mode locking is when things lock into frequencies
- “Is it possible that mathematical pathology, i.e. chaos, is health and that mathematical health, which is the predictability and differentiability of this kind of structure is disease?”
- Snowflake’s non-equilibrium phenomenon
- They have 6 sides based on the stress of surface tension
-
“The Chaos Machine” by Max Fisher
- People who trust the system do not spend much time researching
- Intermittent Variable Reinforcement
- Abusers veer all over making them unpredictable and alluring
- Self-esteem is a gauge of how much we are accepted and valued by our peers
- Facebook subsidizes mobile data charges in developing countries
- Silicon Valley took “anti-social drop-out” to mean “genius”
- The core ideal is, “free, unfiltered speech.”
- While young male gamers?
- In the 80’s, the American video gaming industry collapsed
- Reagan cancelled regulation that prohibited advertising to children
- Toy companies started pushing gender self-engagement
- Japanese game companies picked boys because parents spend more money on them
- Larger social groups tend to become more violent and strengthen hierarchy enforcement
- Twitter started as a text message rely service
- Morality is emotionally triggered, not rationally triggered
- Global shaming can easily spin out of control
- Domesticated animals have shorter tails, softer ears, shorter faces, and a star-shaped pattern on their forehead
- Communication allowed for the “tyranny of the cousins”
- Moral signaling scales with audience size
- Machine Learning built SPAM filters that were better than any human built filters
- ML super-powered YouTube’s algorithms
- Additionally, Google set a goal to 10x view in several years around the 2020 election
- Humans tend to accept things that are similar to things we have already accepted
- (This is why Slippery Slope fallacy gets so dangerous)
- Repetition makes things more believable (and comforting)
- Mashing together people who disagree does not boost tolerance
- You have to compel them to engage together
- Having user comments adds a social context to media that detaches our rationality
- People generally do not want to share misinformation, but want to be social; once they decide to share, they decide it must be true
- Motivation, attention, design
- Moral words overwhelm almost regardless of the context
- Morality is actually tied to what we thing our peers think
- We use a subset of people, social referents, as a short cut
- “If <insert name> hates it then it must be bad”
- “Everyone feels this way”
- YouTube recommendations really quickly go to the fringe and never come back (because these illicit the most responses)
- For the algorithms, watching is endorsing
- Terrorism is in part a theatrical display
- Citizen movement has dropped as Social Media raised
- “We’re a society, not just a market.”
-
“Making Sense of Chaos” by J. Doyne Farmer
- Complexity Economics shift from “as if” models to “as is”
- Models should fit the facts with plausible inputs
- ‘Complicated’ (many moving parts) and ‘complex’ (outputs are not easily calculated) are not synonymous
- Modern economies are ecosystems
- Money is a contract, contracts live on balance sheets
- “What are the nodes? What are the communities?”
- Only 35% of improvements come from within the industry, the rest come from the supply chain
- Technology typically shifts unemployment between industries
- Risks are about known events, uncertainty is about the unknown
- Chaos pervades complicated systems but is rare in simple systems
- Data that look chaotic can have solid, short run predictions
- Games enter a static state when they are not competitive
- Leverage adds volatility that lead to crashes
- When pressure is applied to a fluid, it is stable. When extreme pressure is applied, turbulence occurs.