Data Science & Machine Learning

1. How can you assess a good logistic model? A. An approach to determining the goodness of fit is through the Homer-Lemeshow statistics, which is computed on data after the observations have been segmented into groups based on having similar predicted probabilities. It examines whether the observed proportions of events are similar to the predicted probabilities of occurrence in subgroups of the data set using a Pearson chi-square test. Small values with large p-values indicate a good fit to the data while large values with p-values below 0.05 indicate a poor fit. The null hypothesis holds that the model fits the data and in the below example we would reject H0. 2. What is bias, variance trade off ? A. Bias is the difference between the average prediction of our model and the correct value which we are trying to predict. Variance is the variability of model prediction for a given data point or a value which tells us spread of our data. If our model is too simple and has very few parameters then it may have high bias and low variance. On the other hand if our model has large number of parameters then it’s going to have high variance and low bias. So we need to find the right/good balance without overfitting and underfitting the data. This tradeoff in complexity is why there is a tradeoff between bias and variance. An algorithm can’t be more complex and less complex at the same time. 3. Why is mean square error a bad measure of model performance? A. A disadvantage of the mean-squared error is that it is not very interpretable because MSEs vary depending on the prediction task and thus cannot be compared across different tasks. Assume, for example, that one prediction task is concerned with estimating the weight of trucks and another is concerned with estimating the weight of apples. Then, in the first task, a good model may have an RMSE of 100 kg, while a good model for the second task may have an RMSE of 0.5 kg. Therefore, while RMSE is viable for model selection, it is rarely reported and R2 is used instead. 4. How can the outlier values be treated A. Below are some of the methods of treating the outliers Trimming/removing the outlier: In this technique, we remove the outliers from the dataset. Quantile based flooring and capping : In this technique, the outlier is capped at a certain value above the 90th percentile value or floored at a factor below the 10th percentile value. Mean/Median imputation : As the mean value is highly influenced by the outliers, it is advised to replace the outliers with the median value. 5. What is a confusion matrix? A. A confusion matrix is a method of summarising a classification algorithm's performance. Calculating a confusion matrix can help you understand what your classification model is getting right and where it is going wrong. It gives us: “true positive” for correctly predicted event values, “false positive” for incorrectly predicted event values, “true negative” for correctly predicted no-event values, “false negative” for incorrectly predicted no-event values.

Data science interview questions JP Morgan and chase

1.What is the difference between Deep Learning and Machine Learning? Deep Learning allows machines to make various business-related decisions using artificial neural networks that simulate the human brain, which is one of the reasons why it needs a vast amount of data for training. Machine Learning gives machines the ability to make business decisions without any external help, using the knowledge gained from past data. Machine Learning systems require relatively small amounts of data to train themselves, and most of the features need to be manually coded and understood in advance. 2.What is Cross-validation in Machine Learning? Cross-validation allows a system to increase the performance of the given Machine Learning algorithm. This sampling process is done to break the dataset into smaller parts that have the same number of rows, out of which a random part is selected as a test set and the rest of the parts are kept as train sets. Cross-validation consists of the following techniques: •Holdout method •K-fold cross-validation •Stratified k-fold cross-validation •Leave p-out cross-validation 3.What is Epoch in Machine Learning? Epoch in Machine Learning is used to indicate the count of passes in a given training dataset where the Machine Learning algorithm has done its job. Generally, when there is a large chunk of data, it is grouped into several batches. All these batches go through the given model, and this process is referred to as iteration. Now, if the batch size comprises the complete training dataset, then the count of iterations is the same as that of epochs. 4. What is Dimensionality Reduction? In the real world, Machine Learning models are built on top of features and parameters. These features can be multidimensional and large in number. Sometimes, the features may be irrelevant and it becomes a difficult task to visualize them. This is where dimensionality reduction is used to cut down irrelevant and redundant features with the help of principal variables. These principal variables conserve the features, and are a subgroup, of the parent variables.

1. What are the uses of using RNN in NLP? The RNN is a stateful neural network, which means that it not only retains information from the previous layer but also from the previous pass. Thus, this neuron is said to have connections between passes, and through time. For the RNN the order of the input matters due to being stateful. The same words with different orders will yield different outputs. RNN can be used for unsegmented, connected applications such as handwriting recognition or speech recognition. 2. How to remove values to a python array? Ans: Array elements can be removed using pop() or remove() method. The difference between these two functions is that the former returns the deleted value whereas the latter does not. 3. What are the advantages and disadvantages of views in the database? Answer: Advantages of Views: As there is no physical location where the data in the view is stored, it generates output without wasting resources. Data access is restricted as it does not allow commands like insertion, updation, and deletion. Disadvantages of Views: The view becomes irrelevant if we drop a table related to that view. Much memory space is occupied when the view is created for large tables. 4. Describe the Difference Between Window Functions and Aggregate Functions in SQL. The main difference between window functions and aggregate functions is that aggregate functions group multiple rows into a single result row; all the individual rows in the group are collapsed and their individual data is not shown. On the other hand, window functions produce a result for each individual row. This result is usually shown as a new column value in every row within the window. 5. What is Ribbon in Excel and where does it appear? The Ribbon is basically your key interface with Excel and it appears at the top of the Excel window. It allows users to access many of the most important commands directly. It consists of many tabs such as File, Home, View, Insert, etc. You can also customize the ribbon to suit your preferences. To customize the Ribbon, right-click on it and select the “Customize the Ribbon” option.

Data Science Interview Questions asked in Verizon 1. How many cars are there in Chennai? How do u structurally approach coming up with that number? 2. Multiple Linear Regression? 3. OLS vs MLE? 4. R2 vs Adjusted R2? During Model Development which one do we consider? 5. Lift chart, drift chart 6. Sigmoid Function in Logistic regression 7. ROC what is it? AUC and Differentiation? 8. Linear Regression from Multiple Linear Regression 9. P-Value what is it and its significance? What does P in P-Value stand for? What is Hypothesis Testing? Null hypothesis vs Alternate Hypothesis? 10. Bias Variance Trade off? 11. Over fitting vs Underfitting in Machine learning? 12. Estimation of Multiple Linear Regression 13. Forecasting vs Prediction difference? Regression vs Time Series? 14. p,d,q values in ARIMA models 1. What will happen if d=0 2. What is the meaning of p,d,q values? 15. Is your data for Forecasting Uni or multi-dimensional? 16. How to find the nose to start with in a Decision tree. 17. TYPES of Decision trees - CART vs C4.5 vs ID3 18. Genie index vs entropy 19. Linear vs Logistic Regression 20. Decision Trees vs Random Forests 21. Questions on liner regression, how it works and all 22. Asked to write some SQL queries 23. Asked about past work experience 24. Some questions on inferential statistics (hypothesis testing, sampling techniques) 25. Some questions on table (how to filter, how to add calculated fields etc) 26. Why do u use Licensed Platform when other Open source packages are available? 27. What certification Have u done? 28. What is a Confidence Interval? 29. What are Outliers? How to Detect Outliers? 30. How to Handle Outliers?

1. What is Cross-validation in Machine Learning? Cross-validation is a resampling method that uses different portions of the data to test and train a model on different iterations. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice This sampling process is done to break the dataset into smaller parts that have the same number of rows, out of which a random part is selected as a test set and the rest of the parts are kept as train sets. Cross-validation consists of the following techniques: • Holdout method • K-fold cross-validation • Stratified k-fold cross-validation • Leave p-out cross-validation 2. What is bagging and boosting in Machine Learning? Bagging is a homogeneous weak learners’ model that learns from each other independently in parallel and combines them for determining the model average. Boosting is also a homogeneous weak learners’ model but works differently from Bagging. In this model, learners learn sequentially and adaptively to improve model predictions of a learning algorithm. 3.What is systematic sampling and cluster sampling ? Systematic sampling is a type of probability sampling method. The sample members are selected from a larger population with a random starting point but a fixed periodic interval. This interval is known as the sampling interval. The sampling interval is calculated by dividing the population size by the desired sample size. Cluster sampling involves dividing the sample population into separate groups, called clusters. Then, a simple random sample of clusters is selected from the population. Analysis is conducted on data from the sampled clusters. 4.What is market basket analysis? Market Basket Analysis is a modeling technique based upon the theory that if you buy a certain group of items, you are more (or less) likely to buy another group of items.

1. Explain some cases where k-Means clustering fails to give good results k-means has trouble clustering data where clusters are of various sizes and densities.Outliers will cause the centroids to be dragged, or the outliers might get their own cluster instead of being ignored. Outliers should be clipped or removed before clustering.If the number of dimensions increase, a distance-based similarity measure converges to a constant value between any given examples. Dimensions should be reduced before clustering them. 2. If your Time-Series Dataset is very long, what architecture would you use? If the dataset for time-series is very long, LSTMs are ideal for it because it can not only process single data points, but also entire sequences of data. A time-series being a sequence of data makes LSTM ideal for it.For an even stronger representational capacity, making the LSTM's multi-layered is better.Another method for long time-series dataset is to use CNNs to extract information. 3. How would you define Power BI as an effective solution ? Power BI is a strong business analytical tool that creates useful insights and reports by collating data from unrelated sources. This data can be extracted from any source like Microsoft Excel or hybrid data warehouses. Power BI drives an extreme level of utility and purpose using interactive graphical interface and visualizations. 4. Why is the KNN Algorithm known as Lazy Learner? When the KNN algorithm gets the training data, it does not learn and make a model, it just stores the data. Instead of finding any discriminative function with the help of the training data, it follows instance-based learning and also uses the training data when it actually needs to do some prediction on the unseen datasets. As a result, KNN does not immediately learn a model rather delays the learning thereby being referred to as Lazy Learner. 5. Explain the difference between drop and truncate. In SQL, the DROP command is used to remove the whole database or table indexes, data, and more. Whereas the TRUNCATE command is used to remove all the rows from the table. ————————————————————-

Hey 👋 Here you can access Data Science Interview Preparation Books ❤️‍🔥👇 https://dataanalysts.gumroad.com/l/datascienceinterview/data?a=363448787 ◾How to get it: 1. Click on the link 2. Enter the amount you like [Can be 0 as well :) ] 3. Click the 'I Want This' Button 4. Enter your email and get it delivered! I'd appreciate it if you could give it a 5 star when you download it. Join for more: https://t.me/DataScienceFree Thanks 😊

1. How is the Error calculated in a Linear Regression model? Measuring the distance of the observed y-values from the predicted y-values at each value of x. Squaring each of these distances. Calculating the mean of each of the squared distances. MSE = (1/n) * Σ(actual – forecast)2 The smaller the Mean Squared Error, the closer you are to finding the line of best fit How bad or good is this final value always depends on the context of the problem, but the main goal is that its value is as minimal as possible. 2. Explain the intuition behind the Gradient Descent algorithm. Gradient descent is an optimization algorithm that’s used when training a machine learning model and is based on a convex function and tweaks its parameters iteratively to minimize a given function to its local minimum (that is, slope = 0). For a start, we have to select a random bias and weights, and then iterate over the slope function to get a slope of 0. The way we change update the value of the bias and weights is through a variable called the learning rate. We have to be wise on the learning rate because choosing: A small leaning rate may lead to the model to take some time to learn A large learning rate will make the model converge as our pointer will shoot and we’ll not be able to get to minima. 3. How is a Random Forest related to Decision Trees? Random forest is an ensemble learning method that works by constructing a multitude of decision trees. A random forest can be constructed for both classification and regression tasks. Random forest outperforms decision trees, and it also does not have the habit of overfitting the data as decision trees do. A decision tree trained on a specific dataset will become very deep and cause overfitting. To create a random forest, decision trees can be trained on different subsets of the training dataset, and then the different decision trees can be averaged with the goal of decreasing the variance. 4. What are some disadvantages of using Naive Bayes Algorithm? Some disadvantages of using Naive Bayes Algorithm are: It relies on a very big assumption that the independent variables are not related to each other. It is generally not suitable for datasets with large numbers of numerical attributes. It has been observed that if a rare case is not in the training dataset but is in the testing dataset, then it will most definitely be wrong.

Q.   How can outlier values be treated? A.  An outlier is an observation in a dataset that differs significantly from the rest of the data. This signifies that an outlier is much larger or smaller than the rest of the data. Given are some of the methods of treating the outliers: Trimming or removing the outlier, Quantile based flooring and capping, Mean/Median imputation. Q.   What is root cause analysis? A.  A root cause is a component that contributed to a nonconformance and should be eradicated permanently through process improvement. The root cause is the most fundamental problem—the most fundamental reason—that puts in motion the entire cause-and-effect chain that leads to the problem (s). Root cause analysis (RCA) is a word that refers to a variety of approaches, tools, and procedures used to identify the root causes of problems. Some RCA approaches are more directed toward uncovering actual root causes than others, while others are more general problem-solving procedures, and yet others just provide support for the root cause analysis core activity. Q.  What is bias and variance in Data Science? A.  The model's simplifying assumptions simplify the target function, making it easier to estimate. Bias is the difference between the Predicted Value and the Expected Value in its most basic form. Variance refers to how much the target function's estimate will fluctuate as a result of varied training data. In contrast to bias, variance occurs when the model takes into account the data's fluctuations, or noise. Q.  What is a confusion matrix? A.   A confusion matrix is a method of summarising a classification algorithm's performance. Calculating a confusion matrix can help you understand what your classification model is getting right and where it is going wrong. This gives us the following: "True positive" for event values that were successfully predicted. "False positive" for event values that were mistakenly predicted. For successfully anticipated no-event values, "true negative" is used. "False negative" for no-event values that were mistakenly predicted.

Data scientists spend 80% of their time working on the data. Books spend 80% of their time talking about algorithms. Today, there's a large gap between academia and reality. Between what they say is important, and what really is. Better data is better than better models.

1. How Are Weights Initialized in a Neural network? Ans: There are two methods here: we can either initialize the weights to zero or assign them randomly. Initializing all weights to 0: This makes your model similar to a linear model. All the neurons and every layer perform the same operation, giving the same output and making the deep net useless. Initializing all weights randomly: Here, the weights are assigned randomly by initializing them very close to 0. It gives better accuracy to the model since every neuron performs different computations. This is the most commonly used method. 2. What are the variants of Gradient descent? Ans: Stochastic Gradient Descent: We use only a single training example for calculation of gradient and update parameters. Batch Gradient Descent: We calculate the gradient for the whole dataset and perform the update at each iteration. Mini-batch Gradient Descent: It’s one of the most popular optimization algorithms. It’s a variant of Stochastic Gradient Descent and here instead of single training example, mini-batch of samples is used. 3. What are the feature selection methods used to select the right variables? Ans: There are two main methods for feature selection: Filter Methods This involves: • Linear discrimination analysis • ANOVA • Chi-Square The best analogy for selecting features is "bad data in, bad answer out." When we're limiting or selecting the features, it's all about selecting the useful feature. Wrapper Methods This involves: • Forward Selection: We test one feature at a time and keep adding them until we get a good fit • Backward Selection: We test all the features and start removing them to see what works better • Recursive Feature Elimination: Recursively looks through all the different features and how they pair together. Wrapper methods are very labor-intensive, and high-end computers are needed if a lot of data analysis is performed with the wrapper method. 4.  What is joint sampling and separate sampling? Ans: · Joint sampling is done when there are equal number of events and non-events. Not appropriate for imbalanced data · Separate sampling is done for imbalanced data. For rare event, all observations are kept when target = 1 and only few observations are kept when target = 0.