Python-sklearn-特征工程

Sklearn 特征工程包含sklearn.feature_extraction特征提取、sklearn.feature_selection特征选择和sklearn.calibration概率校准。 文本特征提取1.CountVectorizer— 词频向量化 ⭐fromsklearn.feature_extraction.textimportCountVectorizer vectorizerCountVectorizer(inputcontent,# content,filename,fileencodingutf-8,decode_errorstrict,# strict,ignore,replacestrip_accentsNone,# ascii,unicode,NonelowercaseTrue,# 全部转为小写preprocessorNone,# 自定义预处理函数tokenizerNone,# 自定义分词函数stop_wordsNone,# english, list, Nonetoken_patternr(?u)\b\w\w\b,# 正则表达式ngram_range(1,1),# (min_n, max_n)analyzerword,# word,char,char_wbmax_df1.0,# 文档频率上限过滤高频词min_df1,# 文档频率下限过滤低频词max_featuresNone,# 最大词汇量vocabularyNone,# 预定义词汇表binaryFalse,# True出现/不出现非计数dtypenp.int64)Xvectorizer.fit_transform(documents)# 关键属性print(vectorizer.vocabulary_)# 词→索引映射print(vectorizer.get_feature_names_out())# 特征名print(vectorizer.stop_words_)# 被去除的停用词print(vectorizer.fixed_vocabulary_)# 是否使用预定义词汇表# 查看词频term_freqsX.sum(axis0)# 每个词的总出现次数2.TfidfVectorizer— TF-IDF 向量化 ⭐使用频率最高的文本向量化方法。fromsklearn.feature_extraction.textimportTfidfVectorizer vectorizerTfidfVectorizer(inputcontent,encodingutf-8,lowercaseTrue,stop_wordsenglish,ngram_range(1,2),# uni-grams bi-gramsmax_df0.8,# 过滤出现在 80% 以上文档的词min_df2,# 过滤出现少于 2 次的词max_features5000,norml2,# 归一化: l1,l2,Noneuse_idfTrue,# 是否使用 IDFsmooth_idfTrue,# 平滑 IDF防止除以 0sublinear_tfFalse,# 使用 1log(tf)binaryFalse,vocabularyNone)Xvectorizer.fit_transform(documents)# 关键属性print(vectorizer.idf_)# IDF 向量print(vectorizer.vocabulary_)print(vectorizer.fixed_vocabulary_)3.TfidfTransformer— TF-IDF 变换器将词频矩阵转换为 TF-IDF 矩阵。fromsklearn.feature_extraction.textimportTfidfTransformer transformerTfidfTransformer(norml2,use_idfTrue,smooth_idfTrue,sublinear_tfFalse)X_tfidftransformer.fit_transform(X_counts)print(transformer.idf_)CountVectorizer TfidfTransformer TfidfVectorizer上面更简洁4.HashingVectorizer— 哈希向量化使用特征哈希Hashing Trick无词汇表、内存高效。fromsklearn.feature_extraction.textimportHashingVectorizer vectorizerHashingVectorizer(n_features2**20,# 哈希特征维度inputcontent,encodingutf-8,lowercaseTrue,stop_wordsenglish,ngram_range(1,2),analyzerword,norml2,alternate_signTrue,# 使用符号哈希binaryFalse,dtypenp.float64)Xvectorizer.fit_transform(documents)# 注意: 无 vocabulary_ 属性不可逆️ 图像特征提取fromsklearn.feature_extraction.imageimport(extract_patches_2d,# 提取 2D 图像块reconstruct_from_patches_2d,# 从块重建图像PatchExtractor,# 块提取器grid_to_graph,# 像素网格图img_to_graph,# 图像到图)fromsklearn.feature_extraction.imageimportextract_patches_2dimportnumpyasnp imagenp.arange(16).reshape(4,4)# 提取所有 (2, 2) 的图像块patchesextract_patches_2d(image,patch_size(2,2),max_patchesNone,# None所有, int随机采样random_state42)# patches.shape: (9, 2, 2) 对于 4x4 图像# 从块重建图像reconstructedreconstruct_from_patches_2d(patches,image_size(4,4))️ 特征选择1. 过滤法Filter MethodsVarianceThreshold— 方差阈值fromsklearn.feature_selectionimportVarianceThreshold selectorVarianceThreshold(threshold0.0)# 移除方差为 0 的特征X_selectedselector.fit_transform(X)print(selector.variances_)# 每个特征的方差print(selector.get_support())# 布尔掩码SelectKBest— 选最佳 K 个特征 ⭐fromsklearn.feature_selectionimportSelectKBest,f_classif,chi2,mutual_info_classif# 分类: f_classif(F 检验), chi2(卡方), mutual_info_classif(互信息)selectorSelectKBest(score_funcf_classif,# 评分函数k10# 保留的特征数)X_selectedselector.fit_transform(X,y)print(selector.scores_)# 每个特征的得分print(selector.pvalues_)# 每个特征的 p 值部分函数print(selector.get_support())# 被选中的特征# 回归对应的评分函数fromsklearn.feature_selectionimportf_regression,mutual_info_regression selector_regSelectKBest(score_funcf_regression,k10)常用评分函数:分类回归说明f_classiff_regressionF 检验chi2—卡方检验仅非负值mutual_info_classifmutual_info_regression互信息捕获非线性—r_regressionPearson 相关系数SelectPercentile— 按百分比选择fromsklearn.feature_selectionimportSelectPercentile selectorSelectPercentile(score_funcf_classif,percentile50# 保留前 50% 的特征)X_selectedselector.fit_transform(X,y)SelectFpr/SelectFdr/SelectFwe— 基于假设检验fromsklearn.feature_selectionimportSelectFpr,SelectFdr,SelectFwe# 控制假阳性率selectorSelectFpr(score_funcf_classif,alpha0.05)# 控制错误发现率selectorSelectFdr(score_funcf_classif,alpha0.05)# 按家族错误率选择selectorSelectFwe(score_funcf_classif,alpha0.05)2. 包装法Wrapper MethodsRFE— 递归特征消除 ⭐fromsklearn.feature_selectionimportRFEfromsklearn.linear_modelimportLogisticRegression estimatorLogisticRegression(max_iter1000)selectorRFE(estimatorestimator,n_features_to_select10,# 或 float (0~1) 表示比例step1,# 每次移除的特征数verbose0,importance_getterauto# auto,coef_,feature_importances_)selector.fit(X,y)print(selector.support_)# 被选中特征的掩码print(selector.ranking_)# 特征的排名1最优print(selector.n_features_)# 选中特征数print(selector.estimator_)# 训练好的最终估计器# 变换X_selectedselector.transform(X)RFECV— 带交叉验证的 RFE ⭐fromsklearn.feature_selectionimportRFECVfromsklearn.svmimportSVC estimatorSVC(kernellinear)selectorRFECV(estimatorestimator,step1,min_features_to_select1,cv5,# 或 StratifiedKFold 等scoringaccuracy,verbose0,n_jobs-1,importance_getterauto)selector.fit(X,y)print(selector.support_)print(selector.ranking_)print(selector.n_features_)# 最优特征数print(selector.cv_results_)# 各特征数的交叉验证结果print(selector.grid_scores_)# 已弃用使用 cv_results_# 可视化importmatplotlib.pyplotasplt n_featuresrange(selector.min_features_to_select,len(selector.cv_results_[mean_test_score])1)plt.figure(figsize(10,6))plt.errorbar(n_features,selector.cv_results_[mean_test_score],yerrselector.cv_results_[std_test_score])plt.xlabel(Number of features)plt.ylabel(Cross-validation score)plt.title(RFECV: Optimal Number of Features)plt.axvline(selector.n_features_,colorr,linestyle--,labelfOptimal:{selector.n_features_})plt.legend()plt.show()SequentialFeatureSelector— 顺序特征选择fromsklearn.feature_selectionimportSequentialFeatureSelector selectorSequentialFeatureSelector(estimatorLogisticRegression(max_iter1000),n_features_to_select10,# 或 auto用 tol 判断tolNone,# 分数改善低于 tol 则停止directionforward,# forward(前向) 或 backward(后向)scoringaccuracy,cv5,n_jobs-1)selector.fit(X,y)print(selector.support_)print(selector.get_support())X_selectedselector.transform(X)3. 嵌入法Embedded MethodsSelectFromModel⭐使用任何有coef_或feature_importances_属性的估计器选择特征。fromsklearn.feature_selectionimportSelectFromModelfromsklearn.linear_modelimportLassoCVfromsklearn.ensembleimportRandomForestClassifier# 方式一: L1 正则化LassolassoLassoCV(cv5,random_state42).fit(X,y)selectorSelectFromModel(estimatorlasso,thresholdmedian,# 或 mean, 1.25*mean, floatprefitTrue,# True已拟合, False先 fitnorm_order1,# 系数范数max_featuresNone# 最大特征数)X_selectedselector.transform(X)# 方式二: 树模型特征重要性rfRandomForestClassifier(n_estimators100,random_state42)selectorSelectFromModel(estimatorrf,threshold0.5*mean,# 阈值为平均重要性的 0.5 倍prefitFalse)X_selectedselector.fit_transform(X,y)# 属性print(selector.estimator_)# 训练好的估计器print(selector.threshold_)# 使用的阈值print(selector.get_support())# 选中的特征print(selector.max_features_)# 最大特征数 特征字典提取DictVectorizer将字典列表转换为特征矩阵自动 One-Hot 编码类别值。fromsklearn.feature_extractionimportDictVectorizer data[{city:Beijing,temp:25},{city:Shanghai,temp:28,humidity:70},{city:Beijing,temp:22,humidity:55}]vecDictVectorizer(dtypenp.float64,separator,sparseTrue)Xvec.fit_transform(data)# temp cityBeijing cityShanghai humidity# 0 25.0 1.0 0.0 0.0# 1 28.0 0.0 1.0 70.0# 2 22.0 1.0 0.0 55.0print(vec.feature_names_)print(vec.vocabulary_)# 逆变换data_reconstructedvec.inverse_transform(X) 特征特征Feature CharacterizerFeatureHasher— 特征哈希fromsklearn.feature_extractionimportFeatureHasher hasherFeatureHasher(n_features2**10,# 输出特征维度input_typedict,# dict,pair,stringdtypenp.float64,alternate_signTrue)Xhasher.fit_transform(feature_dicts)# 无 vocabulary_ — 不可逆 概率校准CalibratedClassifierCV— 概率校准 ⭐让模型的概率估计更准确。fromsklearn.calibrationimportCalibratedClassifierCVfromsklearn.svmimportSVC# 方法一: 包裹任意分类器base_modelSVC(probabilityFalse)# 不一定要开启概率calibratedCalibratedClassifierCV(estimatorbase_model,methodsigmoid,# sigmoid(Platt Scaling) 或 isotoniccv5,# prefit 或 int 或 cross-validatorn_jobsNone,ensembleTrue# True每个 fold 一个模型集成False单模型)calibrated.fit(X_train,y_train)y_probcalibrated.predict_proba(X_test)y_predcalibrated.predict(X_test)# 关键属性print(calibrated.calibrated_classifiers_)# 校准后的分类器列表print(calibrated.classes_)Platt Scaling vs Isotonic Regression:method适用场景数据量sigmoid默认更稳定较少数据也可isotonic更灵活非参数需要更多数据1000calibration_curve()— 校准曲线fromsklearn.calibrationimportcalibration_curveimportmatplotlib.pyplotasplt prob_true,prob_predcalibration_curve(y_true,y_prob,n_bins10,strategyuniform# uniform 或 quantile)# 绘制plt.plot([0,1],[0,1],k--,labelPerfectly calibrated)plt.plot(prob_pred,prob_true,s-,labelModel)plt.xlabel(Mean predicted probability)plt.ylabel(Fraction of positives)plt.legend()plt.show() 完整特征工程 Pipeline 模板fromsklearn.pipelineimportPipelinefromsklearn.composeimportColumnTransformerfromsklearn.preprocessingimportStandardScaler,OneHotEncoderfromsklearn.imputeimportSimpleImputerfromsklearn.feature_selectionimportSelectFromModelfromsklearn.ensembleimportRandomForestClassifier# 1. 预处理preprocessorColumnTransformer([(num,StandardScaler(),numerical_cols),(cat,OneHotEncoder(handle_unknownignore),categorical_cols),])# 2. 特征选择feature_selectorSelectFromModel(RandomForestClassifier(n_estimators100,random_state42),thresholdmedian)# 3. 最终模型final_modelRandomForestClassifier(n_estimators200,random_state42)# 完整管道pipelinePipeline([(preprocessor,preprocessor),(feature_selection,feature_selector),(classifier,final_model)])pipeline.fit(X_train,y_train)print(fTest accuracy:{pipeline.score(X_test,y_test):.3f})print(fSelected features:{pipeline.named_steps[feature_selection].get_support().sum()})[[sklearn-总览|← 返回总览]]