Data Science
Data science is the practice of using data to answer questions, make predictions, and guide decisions. It blends statistics, programming, domain knowledge, and communication — and the hardest parts are usually problem framing, data quality, and evaluation, not the model itself.
A fancy model on the wrong problem, leaky data, or a metric that doesn't reflect business cost will fail no matter how sophisticated it is. The most valuable data scientists spend their effort framing the question and validating the data before reaching for an algorithm.
TL;DR
- The hard parts are problem framing, data quality, and evaluation — not modeling.
- Start with a simple baseline before anything complex.
- Watch for data leakage, correlation vs causation, and distribution shift.
- Make work reproducible: versioned data, deterministic pipelines, tracked experiments.
Quick Example
A reliable project follows a workflow, not a leap to modeling:
Core Concepts
What data scientists do
Exploratory analysis and visualization, experiment design and causal reasoning, forecasting and predictive modeling, segmentation/clustering, building decision-support metrics, and partnering with product/engineering to operationalize insights.
The ideas that bite
- Correlation vs causation — observational patterns aren't cause; controlled experiments (A/B tests) are the cleanest path to causal claims.
- Data leakage — features that encode future information inflate offline scores and collapse in production.
- Distribution shift — training data may not match production reality, so models degrade over time.
- Reproducibility — version data, make pipelines deterministic, and track experiments.
Best Practices
- Start interpretable — linear/logistic regression or trees before deep models.
- Validate carefully — use cross-validation, and time-aware splits for time-series.
- Calibrate probabilities when decisions depend on confidence thresholds.
- Define the metric first, tied to the real cost of errors, not just accuracy.
Comparison: data science vs related fields
See Data Engineering and MLOps.
Common Mistakes
Data leakage
Reaching for a model before framing the problem
FAQ
What's the difference between data science, ML, and analytics?
Analytics describes what happened (dashboards, reports). Machine learning builds predictive models. Data science is the broader practice of using data — including statistics, experimentation, and communication — to answer questions and drive decisions, often using both analytics and ML.
What is data leakage and why is it dangerous?
Leakage is when information unavailable at prediction time sneaks into training features (e.g. a value computed after the outcome). It produces excellent offline metrics that evaporate in production. Prevent it by splitting data before feature engineering and only using features known at prediction time.
Correlation vs causation — why does it matter?
Most data shows correlation, which doesn't prove one thing causes another (confounders abound). When you need to act on a cause (will this change increase retention?), a controlled experiment like an A/B test is the cleanest evidence; observational methods require careful causal-inference techniques.
How do I make data science reproducible?
Version your data and code, make pipelines deterministic (fixed seeds, pinned dependencies), and track experiments (parameters, metrics, artifacts) with a tool like MLflow. Reproducibility is what lets others — and future you — trust and rebuild a result.
Related Topics
- Python for Data Science — The common toolkit
- MLOps — Shipping and operating models
- Data Engineering — The data that feeds models
- Model Evaluation — Measuring models honestly
- Databases — Where the data lives
- R — Statistics-first alternative to Python