CRITICAL//Consensus

Links for 2024-05-28 AI: 1. Transformers Can Do Arithmetic with the Right Embeddings — Achieves up to 99% accuracy on 100 digit addition problems by training on only 20 digit numbers with a single GPU for one day. https://github.com/mcleish7/arithmetic 2. Machine Learning and Information Theory Concepts towards an AI Mathematician. ~ Yoshua Bengio and Nikolay Malkin. [PDF] https://www.ams.org/journals/bull/2024-61-03/S0273-0979-2024-01839-4/S0273-0979-2024-01839-4.pdf 3. Mathematics as a Translation Task - the Importance of Training Distributions https://www.youtube.com/watch?v=k9xLg-3WfG8 4. HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models https://arxiv.org/abs/2405.14831 5. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models https://arxiv.org/abs/2405.17428 6. MoEUT: Mixture-of-Experts Universal Transformers https://arxiv.org/abs/2405.16039 7. IDEA Research unveiled Grounding DINO 1.5. It's a new set of AI models that can accurately detect and ID objects in images and videos in real time. https://github.com/IDEA-Research/Grounding-DINO-1.5-API 8. Artificial intelligence is far more efficient at producing content than human beings, as far as carbon emissions go. https://www.nature.com/articles/s41598-024-54271-x 9. The Zhousidun dataset: A Chinese-originated dataset that exhaustively labels critical components of American and Allied destroyers. https://arxiv.org/abs/2405.12167v1 10. Some falsifiable predictions about AI from an economist: “What about the share of tasks that will be affected by AI and related technologies? Using numbers from recent studies, I estimate this to be around 4.6%, implying that AI will increase TFP by only 0.66% over ten years, or by 0.06% annually. Of course, since AI will also drive an investment boom, the increase in GDP growth could be a little larger, perhaps in the 1-1.5% range.” https://www.project-syndicate.org/commentary/ai-productivity-boom-forecasts-countered-by-theory-and-data-by-daron-acemoglu-2024-05 [no paywall: https://archive.is/1fYkJ] 11. Stanford discusses the dark arts of GPU programming https://hazyresearch.stanford.edu/blog/2024-05-12-tk Miscellaneous: 1. Truthseeking is the ground in which other principles grow https://www.lesswrong.com/posts/kbnJHpapusMJZb6Gs/truthseeking-is-the-ground-in-which-other-principles 2. Paper Tiger? Chinese Science and Home Bias in Citations https://www.nber.org/papers/w32468 3. Local exposure to poor individuals reduces support for redistribution among the well-off. https://www.cambridge.org/core/journals/british-journal-of-political-science/article/unequal-and-unsupportive-exposure-to-poor-people-weakens-support-for-redistribution-among-the-rich/FC32FA59B3C5525A178C7012859F95D8 4. Females mount stronger immune responses to many pathogens, they awaken more frequently at night, they express greater concern about physically dangerous stimuli, they exert more effort to avoid social conflicts, they exhibit a personality style more focused on life’s dangers, they react to threats with greater fear, disgust and sadness and they develop more threat-based clinical conditions than males. https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/abs/selfprotection-as-an-adaptive-female-strategy/596ABFD521F358F1E8797C542084679B 5. Exclusive Interview: Ukrainian Sniper's Arsenal Revealed https://www.youtube.com/watch?v=KEgLrrrs5ZQ

National EMERGENCY we NEED more METH!

Links for 2024-05-27 AI: 1. Meta presents Automatic Data Curation for Self-Supervised Learning: Features trained on their automatically curated datasets outperform ones trained on manually curated data https://arxiv.org/abs/2405.15613 2. Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization — “…we demonstrate that for a challenging reasoning task with a large search space, GPT-4-Turbo and Gemini-1.5-Pro based on non-parametric memory fail badly regardless of prompting styles or retrieval augmentation, while a fully grokked transformer can achieve near-perfect accuracy, showcasing the power of parametric memory for complex reasoning.” https://arxiv.org/abs/2405.15071 3. AutoCoder, the first Large Language Model to surpass GPT-4 Turbo (April 2024) and GPT-4o in pass@1 on the Human Eval benchmark test (90.9% vs. 90.2%). https://arxiv.org/abs/2405.14906 4. Luban: Building Open-Ended Creative Agents via Autonomous Embodied Verification https://arxiv.org/abs/2405.15414 5. iVideoGPT: Interactive VideoGPTs are Scalable World Models https://arxiv.org/abs/2405.15223 6. Look Once to Hear: Target Speech Hearing with Noisy Examples. Look once to hear is an intelligent hearable system where users choose to hear a target speaker by just looking at them for a few seconds. https://github.com/vb000/LookOnceToHear 7. Top VC Kai-Fu Lee says his prediction that AI will displace 50% of jobs by 2027 is ‘uncannily accurate’ https://fortune.com/2024/05/25/ai-job-displacement-forecast-50-percent-2027-kai-fu-lee-chatgpt-openai/ [Unlikely] 8. How long until AGI? Elon Musk says next year. https://x.com/elonmusk/status/1794240517875920935 [Very unlikely] 9. Musk AI startup raises $6B https://x.ai/blog/series-b Evolution: 1. Bats and dolphins evolved echolocation in the same way (down to the molecular level). https://www.science.org/content/article/bats-and-dolphins-evolved-echolocation-same-way [no paywall: https://archive.is/0nmcB] 2. Unlike humans, octopuses have evolved eyes without a blind spot. https://www.pbs.org/wgbh/evolution/change/grand/page05.html 3. Depression, schizophrenia and bipolar disorder linked with ancient viral DNA in our genome – new research https://theconversation.com/depression-schizophrenia-and-bipolar-disorder-linked-with-ancient-viral-dna-in-our-genome-new-research-230490 Computer Science: 1. A tiny raycasting engine and city generator that fits in a standalone 256 byte html file. https://frankforce.com/city-in-a-bottle-a-256-byte-raycasting-system/ 2. “So here's a story of, by far, the weirdest bug I've encountered in my CS career.” https://x.com/CupiaBart/status/1793930355617259811

Bird Intelligence: 1. "Among others, bird cognition encompasses abilities such as delay of gratification, mental time travel, reasoning, metacognition, mirror self-recognition, theory of mind, and third-party intervention." http://cell.com/trends/cognitive-sciences/abstract/S1364-6613(16)00042-5 2. African grey can perform some cognitive tasks at levels beyond that of 5-year-old humans. https://news.harvard.edu/gazette/story/2019/02/harvard-study-shows-parrots-can-pass-classic-test-of-intelligence/ 3. Cognitive performance of four-months-old ravens may parallel adult apes https://phys.org/news/2020-12-cognitive-four-months-old-ravens-parallel-adult.html 4. New Caledonian crows plan for specific future tool use https://royalsocietypublishing.org/doi/10.1098/rspb.2020.1490 5. Crows keep special tools extra safe. Study suggest that crows have some concept of the relative 'value' of different tool types. https://www.mpg.de/18061418/1217-ornr-crows-keep-special-tools-extra-safe-987453-x 6. These crows have counting skills previously only seen in people https://www.nature.com/articles/d41586-024-01482-x 7. For the first time, research reveals crows use statistical logic https://arstechnica.com/science/2023/09/for-the-first-time-research-reveals-crows-use-statistical-logic/ 8. New Zealand birds are capable of statistical inference. https://www.science.org/content/article/new-zealand-birds-show-humanlike-ability-make-predictions 9. Study Finds That Crows Are So Intelligent They Understand the Concept of Zero https://mymodernmet.com/crows-understand-zero/ 10. Crows and magpies using anti-bird spikes to build nests, researchers find https://www.theguardian.com/science/2023/jul/11/crows-and-magpies-show-their-metal-by-using-anti-bird-spikes-to-build-nests 11. Brainiacs, not birdbrains: Crows possess higher intelligence and conscious processes long thought a primarily human attribute — "Research unveiled in Science finds that crows know what they know and can ponder the content of their own minds, a manifestation of higher intelligence and analytical thought long believed the sole province of humans and a few other higher mammals." https://www.statnews.com/2020/09/24/crows-possess-higher-intelligence-long-thought-primarily-human/ 12. How did birds get so smart? Part of the answer is that they've got primate-like numbers of neurons squeezed into their tiny bird brains. Another part is that avian neurons consume three times less glucose than mammalian neurons. https://doi.org/10.1016/j.cub.2022.07.070 13. “Corvids and parrots with brains of 5–20 g show cognitive properties similar to those of great apes like chimpanzees with brains of about 400 g…many associative pallial neurons, a pallial area that overtakes functions of the mammalian prefrontal cortex, dense dopaminergic innervation of associative pallial areas, and flexible neural fundaments of working memory…” https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(23)00281-4