Python Resources - Basic Python, ML, DataScience, BigData

send

https://github.com/ignani/PythonMLDataScience Have just uploaded some samples. Will keep adding as and when I get some free time. You can use it for learning. Let me know what view. 😊

Interesting paper bout reproducibility in AI/ML from Dr. Edward Raff is a Chief Scientist at Booz Allen Hamilton. He analyzed 255 papers, and successfully reproduce 162 from them. A 62% success rate is higher than many meta-analyses from other sciences, and I suspect my 62% number is lower than reality Interesting facts: 1. Having fewer equations per page makes a paper more reproducible. 2. Empirical papers may be more reproducible than theory-oriented papers. 3. Sharing code is not a panacea 4. Having detailed pseudo code is just as reproducible as having no pseudo code. 5. Creating simplified example problems do not appear to help with reproducibility. 6: Please, check your email (papers of people who answer on emails is more reproducible) https://thegradient.pub/independently-reproducible-machine-learning/ Leave a comment on how do you feel about this.

​​Exploring Transfer Learning with T5: the Text-To-Text Transfer Transformer tl;dr: - 11 billion parameters - encoder-decoder models generally outperformed “decoder-only” language models - fill-in-the-blank-style denoising objectives worked best; - the most important factor was the computational cost; - training on in-domain data can be beneficial but that pre-training on smaller datasets can lead to detrimental overfitting; - multitask learning could be close to competitive with a pre-train-then-fine-tune approach but requires carefully choosing how often the model is trained on each task The model can be fine-tuned on smaller labeled datasets, often resulting in (far) better performance than training on the labeled data alone. Present a large-scale empirical survey to determine which transfer learning techniques work best and apply these insights at scale to create a new model that we call the T5. Also, introduce a new open-source pre-training dataset, called the Colossal Clean Crawled Corpus (C4). The T5 model, pre-trained on C4, achieves SOTA results on many NLP benchmarks while being flexible enough to be fine-tuned to a variety of important downstream tasks. blog post: https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html paper: https://arxiv.org/abs/1910.10683 github (with pre-trained models): https://github.com/google-research/text-to-text-transfer-transformer colab notebook: https://colab.research.google.com/github/google-research/text-to-text-transfer-transformer/blob/master/notebooks/t5-trivia.ipynb #nlp #transformer #t5

In python 3.9, PEP-616 introduced str.removeprefix and str.removesuffix methods: 'abcd'.removeprefix('ab') # 'cd' 'abcd'.removeprefix('fg') # 'abcd' The implementation is simple (it's implemented on C, of course, but the idea is the same): def removeprefix(self: str, prefix: str) -> str: if self.startswith(prefix): return self[len(prefix):] return self

#DS #ML #NN #ebooks https://blog.paralleldots.com/data-science/24-best-and-free-books-to-understand-machine-learning/

​​Cross-Lingual Ability of Multilingual BERT: An Empirical Study to #ICLR2020 In this work, the authors provide a comprehensive study of the contribution of different components in multilingual #BERT (M-BERT) to its cross-lingual ability. They study the impact of linguistic properties of the languages, the architecture of the model, and the learning objectives. The experimental study is done in the context of three typologically different languages – #Spanish, #Hindi, & #Russian – & using two conceptually different #NLP tasks, textual entailment & #NER. Also, they construct a new corpus – Fake-English (#enfake), by shifting the Unicode of each character in English Wikipedia text by a large constant so that there is strictly no character overlap with any other Wikipedia text. And, in this work, they consider Fake-English as a different language. Among their key conclusions are the fact that the lexical overlap between languages plays a negligible role in the cross-lingual success, while the depth of the network is an integral part of it. paper: https://arxiv.org/abs/1912.07840