Data science is an interdisciplinary field that applies information from data across various application fields by using scientific methods, procedures, algorithms, and systems to infer knowledge and insights from noisy, structured, and unstructured data.
Most asked questions and answers related to Data Science.
Technical Questions
1. Describe data science.
Statistics, algebra, specialized software, artificial intelligence, machine learning, and other fields are all combined in data science. The use of particular ideas and analytical methods to extract information from data used in strategic planning, decision-making, etc. is known as data science. Data science is the practice of analyzing data to get meaningful insights.
2. What distinguishes supervised from unsupervised learning?
Supervised learning
- Using known and labeled data as input
- There is a feedback mechanism in supervised learning.
- Among supervised learning algorithms, decision trees, logistic regression, and support vector machines are the most often used ones.
Unsupervised learning
- uses the input of unlabeled data.
- Without supervision, there is no feedback system.
- K-means clustering, hierarchical clustering, and algorithm are the three most popular unsupervised learning algorithms.
3. What is the process of logistic regression?
By estimating probability using its underlying logistic function, logistic regression analyses the relationship between the dependent variable (our label for what we want to predict) and one or more independent variables (our features) (sigmoid)
4. Describe how to create a decision tree.
- Use the complete collection of data as your input.
- Determine the entropy of the target variable and the characteristics of the predictors
- Do the math to determine your information gain for all qualities (we gain information on sorting different objects from each other)
- As the root node, pick the property with the greatest information gain.
- until each branch's decision node is reached, carry out the same steps on each branch.
5. How is a random forest model created?
Many different decision trees are used to create a random forest. The random forest puts all the trees together if the data is divided into many packages and a decision tree is created for each package of data.
How to construct a random forest model:
- Choose 'k' features at random from a total of 'm' features where k <<m
- Calculate node D using the best split point among the "k" characteristics.
- Utilize the optimum split to divide the node into daughter nodes.
- In order to complete the leaf nodes, repeat steps two and three.
- Create a forest by repeating steps one through four n times to produce n trees.
6. How do you prevent your model from becoming overfitting?
How do you prevent your model from becoming overfitting?
A model that is overfitted ignores the wider picture and is only tuned for a relatively tiny quantity of data. To prevent overfitting, there are three basic strategies:
- Keep the model straightforward by considering fewer variables, which will help to reduce some of the noise in the training data.
- Utilize cross-validation methods, such as the k-folds method.
- If you want to avoid overfitting, employ regularisation techniques like LASSO that penalize specific model parameters.
7. A data collection containing variables with more than 30% of their values missing is handed to you. How are you going to handle them?
How to We can easily delete the rows with missing data values if the data set is huge. We use the remaining data to forecast the values, making it the quickest method.manage missing data values includes the following:
Using the panda's data frame in Python, we may replace missing values with the mean or average of the remaining data for smaller data sets. There are other methods for doing this, including df.mean() and df.fillna (mean).
8. What are the advantages of dimensionality reduction?
To communicate the same information succinctly, a data collection with several dimensions is reduced in size using a technique known as "dimensionality reduction."
This decrease aids in data compression and storage space reduction. Additionally, because there are fewer dimensions, there is a reduction in computation time. It eliminates features that are unnecessary; for instance, holding a value in two separate units serves no purpose (meters and inches).
9. What are Recommender system?
Based on user preferences, a recommender system predicts how a user will evaluate a certain product. It can be divided into two sections:
Teamwork in Filtering
For instance, Last.FM suggests songs based on what other users who share your interests frequently listen to. Customers may get the following message along with product recommendations after completing a purchase on Amazon: "Users who bought this also bought..."
Filtering based on content
As an illustration, Pandora uses a song's characteristics to suggest songs with related characteristics. Instead than focusing on who else is listening to music, we are here focusing on the substance.
10. How do you choose k for the k-means?
In order to choose k for k-means clustering, we employ the elbow approach. The elbow technique works by applying the k-means clustering algorithm on the data set, where k is the number of clusters.
It is referred to as the sum of the squared distances between each cluster member and its centroid.
11. Age, gender, and blood cholesterol levels are the three risk factors we will use to forecast the likelihood of mortality from heart disease. What algorithm is best suitable in this situation?
Select the appropriate response:
- Rational Regression
- Regular Regression
- Apriori K-means clustering algorithm
- Logistic regression is the most suited
12. Describe the Confusion Matrix.
The summary of the results of a certain problem's predictions is the confusion matrix. It is a table that is used to summarise the model's performance. The Confusion Matrix, a n*n matrix, assesses how well the categorization model performs.
13. What are, respectively, the true-positive rate and the false-positive rate?
The true-positive rate represents the percentage of correct predictions made for the positive class. This statistic also calculates the proportion of genuine positives that are accurately validated.
The false-positive rate indicates the proportion of incorrect predictions made for the positive
14. What distinguishes data science from conventional application programming?
The fundamental distinction between standard application programming and data science is that traditional programming requires the creation of rules to convert input to output. The rules are generated automatically from the data in data science. class. A false positive determines that something is true when it is initially false.
15. What are some of the most widely used libraries in data science?
Popular data science libraries include
- Flow Tensor
- Pandas
- NumPy
- sSciPy
- sScrapy
- sLibrosa
- sMatPlotLib