Lead Data Scientist Interview Questions And Answers

Setwd(“dir_path”)

Different syntax can be asked in R Data science interview questions.

At last, we have to iterate the list of files in the current working directory. Also, we need to put them together to form a data frame. Moreover, when the script encounters the first file in the file_list, then it creates the main data frame to merge everything into. This is done using the !exists conditional:

If their dataset exists, then a temp_dataset called temporary data frame will be created and added to the dataset. Moreover, we have to delete temporary data frame. That is been removed when we’re done with it using the rm(temp_dataset) command.
If dataset doesn’t exist (!exists is true), then we have to create it.

It's likely that you'll be interviewed by an HR rep, an end business user, and an IT pro. Each person will probably ask you to explain what big data is, and how the data analysis discipline works with big data to produce insights.

You can start your answer with something fundamental, such as "big data analysis involves the collection and organization of data, and the ability to discover correlations between the data that provide revelations or insights that are actionable." You must be able to explain this in terms that resonate with each interviewer; the best way to do this is to illustrate the definition with an example.

Bias:
“Bias is error introduced in your model due to over simplification of machine learning algorithm.” It can lead to underfitting. When you train your model at that time model makes simplified assumptions to make the target function easier to understand.

Low bias machine learning algorithms - Decision Trees, k-NN and SVM
Hight bias machine learning algorithms - Liear Regression, Logistic Regression
Variance:
“Variance is error introduced in your model due to complex machine learning algorithm, your model learns noise also from the training dataset and performs bad on test dataset.” It can lead high sensitivity and overfitting.

Random forest is a versatile machine learning method capable of performing both regression and classification tasks. It is also used for dimentionality reduction, treats missing values, outlier values. It is a type of ensemble learning method, where a group of weak models combine to form a powerful model.

In Random Forest, we grow multiple trees as opposed to a single tree. To classify a new object based on attributes, each tree gives a classification. The forest chooses the classification having the most votes(Over all the trees in the forest) and in case of regression, it takes the average of outputs by different trees.

Regularization is the process of adding tunning parameter to a model to induce smoothness in order to prevent overfitting. This is most often done by adding a constant multiple to an existing weight vector. This constant is often the L1(Lasso) or L2(ridge). The model predictions should then minimize the loss function calculated on the regularized training set.

A feature vector is an n-dimensional vector of numerical features that represent some object. In machine learning, feature vectors are used to represent numeric or symbolic characteristics, called features, of an object in a mathematical, easily analyzable way.

It is a model validation technique for evaluating how the outcomes of a statistical analysis will generalize to an independent data set. It is mainly used in backgrounds where the objective is forecast and one wants to estimate how accurately a model will accomplish in practice. The goal of cross-validation is to term a data set to test the model in the training phase (i.e. validation data set) in order to limit problems like overfitting and gain insight on how the model will generalize to an independent data set.

These are extraneous variables in a statistical model that correlate directly or inversely with both the dependent and the independent variable. The estimate fails to account for the confounding factor.

Selection bias, in general, is a problematic situation in which error is introduced due to a non-random population sample.

This question's purpose is to learn the programming languages and applications the candidate knows and has experience using. The answer will show the candidate's need for additional training of basic programming languages and platforms or any transferable skills. This is vital to understand as it can cost more time and money to train if the candidate is not knowledgeable in all of the languages and applications required for the position. Answers to look for include:

☛ Experience in SAS and R programming
☛ Understanding of Python, PHP or Java programming languages
☛ Experience using data visualization tools

Proposed methods for model validation:

☛ If the values predicted by the model are far outside of the response variable range, this would immediately indicate poor estimation or model inaccuracy.
☛ If the values seem to be reasonable, examine the parameters; any of the following would indicate poor estimation or multi-collinearity: opposite signs of expectations, unusually large or small values, or observed inconsistency when the model is fed new data.
☛ Use the model for prediction by feeding it new data, and use the coefficient of determination (R squared) as a model validity measure.
☛ Use data splitting to form a separate dataset for estimating model parameters, and another for validating predictions.
☛ Use jackknife resampling if the dataset contains a small number of instances, and measure validity with R squared and mean squared error (MSE).

rnorm function-

Basically, it generates “n” normal random numbers. That is totally based on the mean and standard deviation arguments passed to the function.

Syntax of rnorm function –

rnorm(n, mean = , sd = )

runif function-

Basically, it generates “n” unform random numbers in the interval. That is of minimum and maximum values passed to the function.

Syntax of runif function –

runif(n, min = , max = )

A loop is a sequence of instructions that is repeated until a certain condition is been reached. for, while and repeat, with the additional clauses break and next are used to construct loops.

For Example-

It is executed a known number of times for a block is been contained within curly braces.

x = c(1,2,3,4,5)
for(i in 1:5){
print(x[i])
}
[1] 1
[1] 2
[1] 3
[1] 4
[1] 5

Many big data analysts come from statistics, engineering, and computer science disciplines; they're brilliant analysts, but their people and communications skills lag. Businesses understand that to obtain results, you need both strong execution and strong communication. You can expect your HR, end business, and IT interviewers to focus on your communications skills, and to try to test them with a hypothetical situation.

Instead of using k-fold cross-validation, you should be aware to the fact that a time series is not randomly distributed data - It is inherently ordered by chronological order.

In case of time series data, you should use techniques like forward chaining – Where you will be model on past data then look at forward-facing data.

fold 1: training[1], test[2]

fold 1: training[1 2], test[3]

fold 1: training[1 2 3], test[4]

fold 1: training[1 2 3 4], test[5]

Resampling is done in any of these cases:

☛ Estimating the accuracy of sample statistics by using subsets of accessible data or drawing randomly with replacement from a set of data points
☛ Substituting labels on data points when performing significance tests
☛ Validating models by using random subsets (bootstrapping, cross validation)

Data scientists must be able to go beyond classroom theoretical applications to real-world applications. Your candidate's answer to this question will show how they allocate their time to finding the best way to detect outliers. This information is important to know because it demonstrates the candidate's analytical skills. Look for answers that include:

☛ Raw data analysis
☛ Models
☛ Approaches

Keywords
Character

Usage
strsplit(x, split, fixed = FALSE, perl = FALSE, useBytes = FALSE)

Arguments

a. x
It is a character vector, each element of which is to be split.

b. split
Basically, it is a character vector containing regular expression(s). That is used for splitting.

c. fixed
Since it is TRUE then it will match split exactly.

d. perl
Should Perl-compatible regexps be used?

e. useBytes
It is TRUE then the matching will do byte-by-byte rather than character-by-character, and inputs with marked encodings are not converted.

To find out if elements of a character vector are non-empty strings or not then nzchar is the fastest way.

Keywords
character

Usage
nchar(x, type = “chars”, allowNA = FALSE, keepNA = NA)
nzchar(x, keepNA = FALSE)

Arguments

a. x
Basically, a character vector or a vector will be restricted to a character vector. Giving a factor is an error.

b. type
character string: partial matching to one of c(“bytes”, “chars”, “width”).

c. allowNA
Should NA will return for invalid multibyte strings or “bytes”-encoded strings

d. keepNA
The default for nchar(), NA, means to use keepNA = TRUE unless type is “width”. Used to be hardcoded to FALSE in R versions ≤ 3.2.0.

Data scientists frequently have to combine large amounts of information from various devices in several formats, such as data from a smartwatch or cellphone. Answers to this question will demonstrate how your candidate's methods for organizing large data. This information is important to know because data scientists need clean data to analyze information accurately to offer recommendations that solve business problems. Possible answers may include:

☛ Automation tools
☛ Value correction methods
☛ Comprehension of data sets

read.spss Function – read.spss
What it does – Reads spss data file
For Example- spss(“myfile”)
read.xport Function – read.xport
What it does – Reads SAS export file
For Example- export(“myfile”)
read.dta Function – read.dta
What it does – Reads stata binary file
For Example – read.dta(“myfile”)

“Exploding gradients are a problem where large error gradients accumulate and result in very large updates to neural network model weights during training.” At an extreme, the values of weights can become so large as to overflow and result in NaN values.

This has the effect of your model being unstable and unable to learn from your training data. Now let’s understand what is the gradient.

Gradient:
Gradient is the direction and magnitude calculated during training of a neural network that is used to update the network weights in the right direction and by the right amount.

Logistic Regression often referred as logit model is a technique to predict the binary outcome from a linear combination of predictor variables. For example, if you want to predict whether a particular political leader will win the election or not. In this case, the outcome of prediction is binary i.e. 0 or 1 (Win/Lose). The predictor variables here would be the amount of money spent for election campaigning of a particular candidate, the amount of time spent in campaigning, etc.

☛ Take the entire data set as input.
☛ Look for a split that maximizes the separation of the classes. A split is any test that divides the data into two sets.
☛ Apply the split to the input data (divide step).
☛ Re-apply steps 1 to 2 to the divided data.
☛ Stop when you meet some stopping criteria.
☛ This step is called pruning. Clean up the tree if you went too far doing splits.