Deep Learning, Computer Vision and NLP

H2O LLM Studio A framework and no-code GUI for fine-tuning LLMs and no-code GUI designed for fine-tuning state-of-the-art large language models (LLMs). Documentation: https://h2oai.github.io/h2o-llmstudio/ https://github.com/h2oai/h2o-llmstudio

I work on a lot of NLP projects and it's starting to feel like I do more prompting than actual coding. LLMs might completely change the way we code as they become more integrated into our tools and workflows. Still doing a lot of stitching with tools like ChatGPT and Copilot but I expect more seamlessness as these tools provide more functionalities like function calling.

Kaiming He, inventor of ResNet, is leaving industry to join MIT faculty in 2024!! He’s one of the most impactful figures in deep learning.

How you can train Large Language Models? Large language models (LLMs) are gaining significant popularity due to their versatility in text generation, translation, and question-answering tasks. However, training these models can be resource-intensive and time-consuming. LLMs examples include 𝐆𝐏𝐓-3 and 𝐆𝐏𝐓-4 from 𝐎𝐩𝐞𝐧𝐀𝐈, 𝐋𝐋𝐚𝐌𝐀 from 𝐌𝐞𝐭𝐚, 𝐚𝐧𝐝 𝐏𝐚𝐋𝐌2 from 𝐆𝐨𝐨𝐠𝐥𝐞. Several LLM training frameworks have emerged to address this challenge, offering solutions to streamline and enhance the training process. Here are some of the most popular frameworks that help you to train and tuning LLMs Models: ✅ Deepspeed: An efficient deep learning optimization library that simplifies distributed training and inference, enabling easy and effective implementation. Examples: https://www.deepspeed.ai/ ✅ Megatron-DeepSpeed: A DeepSpeed version of NVIDIA's Megatron-LM, offering additional support for MoE model training, Curriculum Learning, 3D Parallelism, and other advanced features. Examples: https://huggingface.co/blog/bloom-megatron-deepspeed ✅ FairScale: A PyTorch extension library designed for high-performance and large-scale training, empowering researchers and practitioners to train models more efficiently. Example: https://fairscale.readthedocs.io/en/latest/tutorials/oss.html ✅ Megatron-LM: A research-focused framework dedicated to training transformer models at scale, facilitating ongoing exploration in the field. Examples:https://huggingface.co/blog/megatron-training ✅ Colossal-AI: A platform that aims to make large AI models more accessible, faster, and cost-effective, contributing to democratizing AI advancements. Examples: https://github.com/hpcaitech/ColossalAI/tree/main/examples ✅ BMTrain: An efficient training framework tailored for big models, enabling smoother and more effective training processes. Examples: https://github.com/OpenBMB/BMTrain ✅ Mesh TensorFlow: A framework simplifying model parallelism, making it easier to leverage distributed computing resources for training large models. Examples: https://github.com/tensorflow/mesh ✅ Max text: A performant and scalable Jax LLM framework designed to simplify the training process while maintaining high performance. Examples: https://github.com/EleutherAI/maxtext ✅ Alpa: A system specifically developed for training and serving large-scale neural networks, offering comprehensive support for training requirements. Examples: https://alpa.ai/opt ✅ GPT-NeoX: An implementation of model parallel autoregressive transformers on GPUs, built on the DeepSpeed library, providing enhanced training capabilities. Examples: https://blog.eleuther.ai/announcing-20b/ If you're interested in training LLMs, I encourage you to explore these frameworks. They can significantly simplify and optimize the training process, allowing you to achieve better results efficiently.

The source code for DragGAN has been released! 🔥🔥🔥 We can finally play with that marvel! ⮑ 🔗 GitHub repository: https://github.com/XingangPan/DragGAN

Text -> Video just got real. And it is all yours for taking! The most powerful video generation model is now an open-source model, try it here: huggingface.co/cerspense/zero…

After spending almost month with new hype of GenAI (text, LLM not image/video) these are my observations. Not in particular order and these are 'MY' observations on 'MY' tasks. Your conclusions will differ. 1. We need minimum 7B parameter models. Less than that performance of natural language understanding goes down drastically. More than this you need >24GB gpu. 2. Benchmarks are tricky ... some LLMs are good with some tasks while bad in others. Try to find model which works in your case the best. MPT-7B is still best for my usecases .. even better than Falcon-7B. 3. Prompts change with almost each model. You have to rework many times (There are some solutions around it .. trying to see if they work) 4. For finetuning you need at-least 1 gpu with >24 Gb vram .. 32 or 40 GB one good enough. 5. Finetuning just last few layers to speed up training/finetuning of LLM might not work out well (I tried!) 6. 8-bit, 4-bit model loading for VRAM saving works. For 7B model instead of 16gb, it takes ~10gb and <6gb respectively. BUT .. inference speed goes down drastically. (At-least I faced this issue). Performance also goes down in text understanding tasks. 7. Those like me who are trying to figure out LLM applications for your companies .. be aware for Licensing part. One model trained with other as reference and in case of llama you need original weights ... not a good idea to work in commerical setting. 8. There are 3 types of major LLMs types - basic(like gpt2/3), chat enabled, instruction enabled. Most of the time basic is not usable as it is .. unless you finetune it. Chat versions are the best versions. But most of the time they are not open-source. 9. Not everything needs to be solved with LLMs. Just do not force-fit any solution around LLM .. I have seen the same happening with Deep reinforcement learning some years back. Check this out -> https://lnkd.in/d2mxqhH9 10. I tried out but did not use langchains & vector-dbs. Never needed to ... simple python, embddings and efficient dot product worked for me. 11. LLMs need not have whole world knowledge .. we humans also do not have complete knowledge and still we survive bcz of adaptibility. They just need to know how to use knowledge. I think we can go super smaller in model size if we separate knowledge part somehow. 12. Simulating "thoughts" before answering and NOT just predicting one word after another might be the next wave of innovation.

Drowsiness Detection A simple Drowsiness Detection module for humans. Code : https://github.com/Niraj-Lunavat/Drowsiness-Detection

Advanced Computer Vision Very Interesting course on Advance computer vision covers Dallie, Diffusion Models. https://www.crcv.ucf.edu/courses/cap6412-spring-2023/schedule/ https://www.youtube.com/playlist?list=PLd3hlSJsX_In7qup928HaHmilugBGctuF

Google engineers offered 28 actionable tests for #machinelearning systems. 👇 Introducing 👉 The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction (2017). 👈 If #ml #training is like compilation, then ML testing shall be applied to both #data and code. 7 model tests 1⃣ 👉 Review model specs and version-control it. It makes training auditable and improve reproducibility. 2⃣ 👉 Ensure model loss is correlated with user engagement. 3⃣ 👉 Tune all hyperparameters. Grid search, Bayesian method whatever you use, tune all of them. 4⃣ 👉 Measure the impact of model staleness. The age-versus-quality curve shows what amount of staleness is tolerable. 5⃣ 👉 Test against a simpler model regularly to confirm the benefit more sophisticated techniques. 6⃣ 👉 Check the model quality is good across different data segment, e.g. user countries, movie genre etc. 7⃣ 👉 Test model inclusion by checking against the protected dimensions or enrich under-represented categories. 7 data tests 1⃣ 👉 Capture feature expectations in schema using statistics from data + domain knowledge + expectations. 2⃣ 👉 Use beneficial features only, e.g. training a set of models each with one feature removed. 3⃣ 👉 Avoid costly features. Cost includes running time, RAM as well as upstream work and instability. 4⃣ 👉 Adhere to feature requirements. If certain features can’t be used, enforce it programmatically. 5⃣ 👉 Set privacy controls. Budget enough time for new feature that depends on sensitive data. 6⃣ 👉 Add new features quickly. If conflicting with 5⃣ , privacy goes first. 7⃣ 👉 Test code for all input features. Bugs do exist in feature creation code. See 7 Infrastructure & 7 monitoring tests in paper. 👇 They interviewed 36 teams across Google and found 👉 Using a checklist helps avoid mistakes (like a surgeon would do). 👉 Data dependencies leads to outsourcing responsibility. Other teams’ validation may not validate your use case. 👉 A good framework promotes integration test which is not well adopted. 👉 Assess the assessment to better assess your system. https://research.google.com/pubs/archive/aad9f93b86b7addfea4c419b9100c6cdd26cacea.pdf

Adjust your mindset for Machine Learning with Mark Ryan (Google Manager) 🚀 Get a chance to WIN free copies of Deep Learning with Structured Data book worth $35.99!! To enter, share the LinkedIn post or Just comment your favorite part from this interview, Or you can also Retweet this tweet or just share your favorite part from this interview and tag us on Twitter. Watch Podcast: https://youtu.be/iKPWTRhSJ4o?t=102