Neural Black Magic

سلام دوستان اخیرا مقاله‌ای با ایده‌ای هوشمندانه تحت عنوان Byte Latent Transformer: Patches Scale Better Than Tokens توسط شرکت Meta معرفی شده که با ارائه یک روش قابل آموزش، مبتنی بر داده (Data-driven) و پویا (Dynamical) مدل‌های زبانی رو قادر می‌کنه به طور مستقیم به یادگیری در سطح Byte بپردازد. در این مقاله علاوه بر یک معماری جدید برای پردازش متن، روشی هوشمند برای Patch کردن Byte ها مبتنی بر آنتروپی تخمین ارائه شده. در این ویدئو توضیحات دقیق و جذابی از این مقاله ارائه کرده‌ایم🔥: https://youtu.be/tFRdrIG4Ztc?si=NMXC_-kMXoZRYqkk با دیدن این ویدئو بدون اتلاف وقت به طور کامل رویکرد این مقاله جدید رو به به‌ترین شکل فرا می‌گیرید. مقاله رو می‌تونید از اینجا مطالعه کنید. امیدوارم مفید واقع بشه❤️ @neuralblackmagic

Deep Learning Interviews: Hundreds of fully solved job interview questions from a wide range of key topics in AI Download Link: https://arxiv.org/abs/2201.00650 @neuralblackmagic

Machine Learning Insights: How Linear Classifiers Work? In this post, we investigate how Linear Classifiers, the simplest types of classifiers, learn to discriminate between different classes. As you can see, Linear Classifiers learn a global pattern of samples within each class label and learn how samples within each class generally look like. In the test time, the input samples are compared to the learned patterns via dot product and the class label corresponding to the highest score is assigned to the input sample. You can see the learned pattern by the linear classifier for class "0", "2", "3", and "5" clearly. Read and support our full blogpost here 🔥. follow us for more: @neuralblackmagic

Defining a Deep Fully-connected Neural Network with arbitrary depth and neurons in 4 lines of code using PyTorch🔥

import torch
import torch.nn as nn

x = torch.rand(32, 16) # input data

layers = [2**i for i in range(5, 5+10)] # [layer_1, layer_2, ..., layer_n]
layers.insert(0, x.shape[-1]) # [input_shape, layer_1, layer_2, ..., layer_n]
fcs = nn.ModuleList([nn.Linear(layers[i], layers[i+1]) for i in range(len(layers)-2)]) # defining fully-connected layers
for i in range(1, 2*len(fcs)-1, 2): # adding activation function to network
  fcs.insert(i, nn.ReLU())
model = nn.Sequential(*fcs) # model definition

print(model)
print("Output Shape: ", model(x).shape)

Check the output:

Sequential(
  (0): Linear(in_features=16, out_features=32, bias=True)
  (1): ReLU()
  (2): Linear(in_features=32, out_features=64, bias=True)
  (3): ReLU()
  (4): Linear(in_features=64, out_features=128, bias=True)
  (5): ReLU()
  (6): Linear(in_features=128, out_features=256, bias=True)
  (7): ReLU()
  (8): Linear(in_features=256, out_features=512, bias=True)
  (9): ReLU()
  (10): Linear(in_features=512, out_features=1024, bias=True)
  (11): ReLU()
  (12): Linear(in_features=1024, out_features=2048, bias=True)
  (13): ReLU()
  (14): Linear(in_features=2048, out_features=4096, bias=True)
  (15): ReLU()
  (16): Linear(in_features=4096, out_features=8192, bias=True)
)
Output Shape:  torch.Size([32, 8192])

Follow us for high quality tutorials on Machine Learning and Deep Learning: @neuralblackmagic

یلداتون خجسته 🍉❤️؛ به‌ترین‌ها رو براتون آرزومندم.

Federated Learning course By DeepLearning.AI YouTube good courses: CS-E4740 Federated Learning (Spring 23) https://www.youtube.com/playlist?list=PLrbn2dGrLJK8c6hCQXBVFoYsPXG-g_75c CS-E4740 Federated Learning (Spring 24) https://www.youtube.com/playlist?list=PLrbn2dGrLJK9TwK1TILxluPApQCLq1Tf5 @neuralblackmagic

📃 Hymba: A Hybrid-head Architecture for Small Language Models Hymba, a family of small language models featuring a hybrid-head parallel architecture that integrates transformer attention mechanisms with state space models (SSMs) for enhanced efficiency @neuralblackmagic

Optimization for Machine Learning Lecture Notes @neuralblackmagic

Are you interested in adding interpretability flavor to your Machine Learning models? Interpretable Machine Learning with Python: Learn to build interpretable high-performance models with hands-on real-world examples, Serg Masís. Interpretable Machine Learning: A Guide For Making Black Box Models Explainable, Christoph Molnar. @neuralblackmagic

ADOPT: Modified Adam Can Converge with Any β2 with the Optimal Rate Paper Github Code 💻

from adopt import ADOPT
#optimizer = Adam(model.parameters(), lr=1e-3)
optimizer = ADOPT(model.parameters(), lr=1e-3)

@neuralblackmagic

How Diffusion Models Work Short course Watch the course: https://www.deeplearning.ai/short-courses/how-diffusion-models-work/ Level 📈 Basic, Intermediate @neuralblackmagic

Stanford University 🏛 Reinforcement Learning 🤖🤘 Emma Brunskill 👩🏻‍🏫 Spring 2024: https://www.youtube.com/playlist?list=PLoROMvodv4rN4wG6Nk6sNpTEbuOSosZdX Winter 2019: https://www.youtube.com/playlist?list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u Level 📈 Intermediate Follow us: @neuralblackmagic

DeepLearning.AI new course🔥 Watch the course: https://www.deeplearning.ai/short-courses/llms-as-operating-systems-agent-memory/ LLMs as Operating Systems: Agent Memory Follow for more: https://t.me/NeuralBlackMagic

Foundations Of The Theory Of Probability by Andrey Nikolaevich Kolmogorov 🔥🔥🔥 Read the book Follow for more: https://t.me/NeuralBlackMagic

📃🖋 Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts Follow for more: https://t.me/NeuralBlackMagic

📃🖋 Diffusion Model Predictive Control A fresh awesome work on the intersection of Control Systems and Generarive Models by DeepMind 🔥 Follow for more: https://t.me/NeuralBlackMagic

📃 Were RNNs All We Needed? This paper introduces minLSTM and minGRU, minimal versions of LSTM and GRU with significanly fewer parameters that can be trained in fully parallel fashion. Follow for more: https://t.me/NeuralBlackMagic