Data Science & Machine Learning

Q.   How can outlier values be treated? A.  An outlier is an observation in a dataset that differs significantly from the rest of the data. This signifies that an outlier is much larger or smaller than the rest of the data. Given are some of the methods of treating the outliers: Trimming or removing the outlier, Quantile based flooring and capping, Mean/Median imputation. Q.   What is root cause analysis? A.  A root cause is a component that contributed to a nonconformance and should be eradicated permanently through process improvement. The root cause is the most fundamental problem—the most fundamental reason—that puts in motion the entire cause-and-effect chain that leads to the problem (s). Root cause analysis (RCA) is a word that refers to a variety of approaches, tools, and procedures used to identify the root causes of problems. Some RCA approaches are more directed toward uncovering actual root causes than others, while others are more general problem-solving procedures, and yet others just provide support for the root cause analysis core activity. Q.  What is bias and variance in Data Science? A.  The model's simplifying assumptions simplify the target function, making it easier to estimate. Bias is the difference between the Predicted Value and the Expected Value in its most basic form. Variance refers to how much the target function's estimate will fluctuate as a result of varied training data. In contrast to bias, variance occurs when the model takes into account the data's fluctuations, or noise. Q.  What is a confusion matrix? A.   A confusion matrix is a method of summarising a classification algorithm's performance. Calculating a confusion matrix can help you understand what your classification model is getting right and where it is going wrong. This gives us the following: "True positive" for event values that were successfully predicted. "False positive" for event values that were mistakenly predicted. For successfully anticipated no-event values, "true negative" is used. "False negative" for no-event values that were mistakenly predicted.

Data scientists spend 80% of their time working on the data. Books spend 80% of their time talking about algorithms. Today, there's a large gap between academia and reality. Between what they say is important, and what really is. Better data is better than better models.

1. How Are Weights Initialized in a Neural network? Ans: There are two methods here: we can either initialize the weights to zero or assign them randomly. Initializing all weights to 0: This makes your model similar to a linear model. All the neurons and every layer perform the same operation, giving the same output and making the deep net useless. Initializing all weights randomly: Here, the weights are assigned randomly by initializing them very close to 0. It gives better accuracy to the model since every neuron performs different computations. This is the most commonly used method. 2. What are the variants of Gradient descent? Ans: Stochastic Gradient Descent: We use only a single training example for calculation of gradient and update parameters. Batch Gradient Descent: We calculate the gradient for the whole dataset and perform the update at each iteration. Mini-batch Gradient Descent: It’s one of the most popular optimization algorithms. It’s a variant of Stochastic Gradient Descent and here instead of single training example, mini-batch of samples is used. 3. What are the feature selection methods used to select the right variables? Ans: There are two main methods for feature selection: Filter Methods This involves: • Linear discrimination analysis • ANOVA • Chi-Square The best analogy for selecting features is "bad data in, bad answer out." When we're limiting or selecting the features, it's all about selecting the useful feature. Wrapper Methods This involves: • Forward Selection: We test one feature at a time and keep adding them until we get a good fit • Backward Selection: We test all the features and start removing them to see what works better • Recursive Feature Elimination: Recursively looks through all the different features and how they pair together. Wrapper methods are very labor-intensive, and high-end computers are needed if a lot of data analysis is performed with the wrapper method. 4.  What is joint sampling and separate sampling? Ans: · Joint sampling is done when there are equal number of events and non-events. Not appropriate for imbalanced data · Separate sampling is done for imbalanced data. For rare event, all observations are kept when target = 1 and only few observations are kept when target = 0.