Feature Importance Example - Lightgbm Typing CST Test
Loading…
Feature Importance Example — Lightgbm Code
Display feature importance after training a model.
import lightgbm as lgb
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
data = load_iris()
X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.2, random_state=42)
train_data = lgb.Dataset(X_train, label=y_train)
params = {'objective':'multiclass','num_class':3,'metric':'multi_logloss'}
model = lgb.train(params, train_data, num_boost_round=100)
lgb.plot_importance(model)
plt.show()Lightgbm Language Guide
LightGBM (Light Gradient Boosting Machine) is a fast, distributed, high-performance gradient boosting framework based on decision tree algorithms, used for ranking, classification, and many other machine learning tasks.
Primary Use Cases
- ▸Binary and multiclass classification
- ▸Regression problems
- ▸Ranking tasks (learning-to-rank)
- ▸Feature selection and importance analysis
- ▸Integration in ML pipelines for large-scale structured data
Notable Features
- ▸Faster training with histogram-based decision tree algorithm
- ▸Low memory usage compared to XGBoost
- ▸Supports parallel and GPU learning
- ▸Handles categorical features directly
- ▸Scales efficiently with large datasets
Origin & Creator
LightGBM was developed by Microsoft’s DMTK team and released in 2016 to provide a faster and more memory-efficient gradient boosting framework compared to existing solutions.
Industrial Note
LightGBM is widely used in Kaggle competitions, finance, advertising, recommendation systems, and any scenario requiring high-speed gradient boosting on large datasets.
Quick Explain
- ▸LightGBM enables efficient training of large-scale datasets with lower memory usage.
- ▸It implements gradient-based one-side sampling (GOSS) and exclusive feature bundling (EFB) for speed and accuracy.
- ▸LightGBM integrates seamlessly with Python ML workflows, including scikit-learn, XGBoost, and other pipelines.
Core Features
- ▸Gradient-based One-Side Sampling (GOSS)
- ▸Exclusive Feature Bundling (EFB)
- ▸Leaf-wise tree growth with depth limitation
- ▸Support for custom objective functions
- ▸Integration with Python, R, and CLI interfaces
Learning Path
- ▸Learn Python and scikit-learn basics
- ▸Understand decision trees and gradient boosting
- ▸Practice LightGBM on classification and regression tasks
- ▸Explore hyperparameter tuning and early stopping
- ▸Integrate into ML pipelines and production workflows
Practical Examples
- ▸Train a classifier: clf = lgb.LGBMClassifier(); clf.fit(X_train, y_train)
- ▸Predict: y_pred = clf.predict(X_test)
- ▸Evaluate: accuracy_score(y_test, y_pred)
- ▸Feature importance: clf.feature_importances_
- ▸Custom objective function: define function and pass to lgb.train
Comparisons
- ▸LightGBM vs XGBoost: faster and more memory-efficient
- ▸LightGBM vs CatBoost: better for categorical-heavy data
- ▸LightGBM vs RandomForest: gradient boosting vs bagging
- ▸LightGBM vs scikit-learn GBM: highly optimized for large datasets
- ▸LightGBM vs TensorFlow/PyTorch: tabular ML vs deep learning
Strengths
- ▸High-speed training and low memory usage
- ▸Excellent predictive accuracy
- ▸Handles large datasets efficiently
- ▸Supports parallel, GPU, and distributed learning
- ▸Works well with sparse data and categorical variables
Limitations
- ▸Leaf-wise tree growth can overfit on small datasets
- ▸Less interpretable than simple decision trees
- ▸Parameter tuning is essential for optimal performance
- ▸Not ideal for extremely small datasets
- ▸Python API is feature-rich but some advanced options are less documented
When NOT to Use
- ▸Extremely small datasets (overfitting risk)
- ▸Text, image, or unstructured data without preprocessing
- ▸When interpretability is more important than accuracy
- ▸GPU not available for extremely large datasets
- ▸Highly imbalanced datasets without sampling or weighting
Cheat Sheet
- ▸lgb.LGBMClassifier() = classification model
- ▸lgb.LGBMRegressor() = regression model
- ▸lgb.Dataset() = dataset object for training
- ▸train() = train booster with parameters
- ▸predict() = generate predictions
FAQ
- ▸Is LightGBM free?
- ▸Yes - open-source under MIT license.
- ▸Which languages are supported?
- ▸Python, R, CLI, C++ interface.
- ▸Can LightGBM handle large datasets?
- ▸Yes, optimized for millions of rows and features.
- ▸Does LightGBM support GPU?
- ▸Yes, optional via CUDA-enabled GPU training.
- ▸Is LightGBM suitable for ranking?
- ▸Yes - built-in ranking objective for learning-to-rank tasks.
30-Day Skill Plan
- ▸Week 1: train simple classifier/regressor
- ▸Week 2: hyperparameter tuning and cross-validation
- ▸Week 3: ranking tasks and custom objective functions
- ▸Week 4: GPU training and distributed learning
- ▸Week 5: deployment and integration into pipelines
Final Summary
- ▸LightGBM is a high-performance gradient boosting framework.
- ▸Optimized for speed, memory efficiency, and large datasets.
- ▸Supports classification, regression, and ranking tasks.
- ▸Integrates easily with Python ML workflows.
- ▸Widely used in industry, competitions, and large-scale tabular ML.
Project Structure
- ▸main.py / notebook.ipynb - training and evaluation scripts
- ▸data/ - raw and preprocessed datasets
- ▸models/ - saved LightGBM model files
- ▸utils/ - feature engineering and helper functions
- ▸notebooks/ - experiments and parameter tuning
Monetization
- ▸Financial risk models
- ▸Recommendation engines
- ▸Ad targeting scoring systems
- ▸Kaggle competition solutions
- ▸Enterprise ML consulting
Productivity Tips
- ▸Use LGBMClassifier/LGBMRegressor for fast prototyping
- ▸Enable early stopping to prevent overfitting
- ▸Batch large datasets efficiently
- ▸Use GPU for speed on big datasets
- ▸Tune num_leaves, learning_rate, and max_depth carefully
Basic Concepts
- ▸Dataset: structured tabular data with features and labels
- ▸Booster: core model object (tree-based)
- ▸Leaf-wise tree growth: splits the most important leaf
- ▸Objective function: defines learning goal (e.g., regression, classification)
- ▸Hyperparameters: control learning rate, depth, boosting type, etc.