K-Nearest Neighbor Classification Algorithm Case

My friend Helen has been using the online dating website to find the right person for her. Although dating websites recommend different people, she doesn't find anyone she likes. After some summary, she found that she has met three types of people: "people who don't like it", "people who are charming" and "people who are very charming". Despite finding these patterns, Helen still can't categorize the matches recommended by dating websites appropriately. She feels that she can date the most attractive people from Monday to Friday, while she prefers to spend the weekend with the most attractive people. Helen wants our categorization software to better help her categorize matches into exact categories. Helen also collects data that dating websites haven't recorded, which she believes will help her categorize matches better.

1. Data preparation

Helen has been collecting dating data for some time, and she keeps it in a text file dating TestSet. In the txt, each sample takes up one row, with a total of 106 Helen samples featuring three main features:'Flight frequent miles per year','Percentage of time spent playing video Games'and'Ice cream liters consumed per week'.

2. Data Preprocessing

Due to inconsistent dimensions of data attributes, normalization is required.

3. Data Modeling

Under the guidance of classification accuracy criterion, the optimal parameter k is determined by ten-fold cross-validation.

4. Model prediction

Now that we've tested the classifier on our data, we can finally use it to classify people for Helen. We'll give Helen a little program where she finds someone on the dating website and enters his information. The program gives her a prediction of how much she likes her partner.

import pandas as pd import numpy as np from sklearn.neighbors import KNeighborsClassifier from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.model_selection import cross_val_score import matplotlib.pyplot as plt df=pd.read_csv("datingTestSet.txt",sep="\t",names=['flight','ice','game','type']) features=df.drop(columns=['type'],axis=0) targets=df['type'] X_train,X_test,y_train,y_test=train_test_split(features,targets,test_size=0.25,random_state=2022) #Standardization scaler=StandardScaler() X_train=scaler.fit_transform(X_train) X_test=scaler.fit_transform(X_test) # Combining K-fold cross-validation with learning curve to obtain optimal K-value scores_cross = [] Ks = [] for k in range(3, 20): knn = KNeighborsClassifier(n_neighbors=k) # Instantiate Model Object score_cross = cross_val_score(knn, X_train, y_train, cv=6).mean() # Cross-validation based on training data and return the score of cross-validation scores_cross.append(score_cross) Ks.append(k) # Convert to Array Type scores_arr = np.array(scores_cross) Ks_arr = np.array(Ks) # Draw learning curve plt.plot(Ks, scores_cross) plt.show() # Get the highest score and the array subscript corresponding to the highest score to get the optimal K value score_best = scores_arr.max() # Find the highest score in the array of stored scores index_best = scores_arr.argmax() # Find the subscript corresponding to the highest score in array Ks_best = Ks_arr[index_best] # Find the K value corresponding to the highest score based on the subscript print(Ks_best) # The optimal K value found by the combination of K-fold cross validation and learning curve is used to instantiate the model object knn=KNeighborsClassifier(n_neighbors=Ks_best) knn.fit(X_train,y_train) score=knn.score(X_test,y_test) print("score:",score) dscore=knn.predict([[40000,10,0.5]]) print("Forecast results:",dscore[0])

Handwriting recognition is a common image recognition task. Computers recognize characters in pictures by handwriting pictures. Unlike typefaces, handwriting styles and sizes vary from person to person, making it difficult for computers to recognize handwriting. Digital handwriting recognition has become a relatively simple handwriting recognition task due to its limited categories (0~9 of 10 numbers).

1. Data preparation

There are 204 txt files in the file "trainingDigits" folder of the compressed file "digits.zip", each of which corresponds to one of the numbers "1-9" (processed as a 0-1 digital matrix of 3232), as shown below. Read this 204 file and convert the data in each file into 1024-dimensional row vectors for training set. Note: Take "0_2.txt" as an example, where 0 in the file name represents the corresponding number 0 for this file, that is, the class label for this data vector is 2. Tip: Get all the files in the current folder using from os import listdir, convert them to vectors, and use the split("") of the string [0] Method gets the class label of each data.

2. Model building

The optimal number of neighbors K is selected by cross-validation.

3. Forecast classification

Reads all the data files in the folder testDigits in "digits.zip", vectorizes them using the above methods, and predicts them using the built model.

4. Model evaluation

What is the prediction classification accuracy of the model?

from os import listdir import pandas as pd import numpy as np from sklearn.neighbors import KNeighborsClassifier from sklearn.model_selection import train_test_split from sklearn.model_selection import cross_val_score import matplotlib.pyplot as plt name1=listdir('digits/trainingDigits') name2=listdir('digits/testDigits') train_data=[] train_label=[] test_data=[] test_label=[] for name in name1: label=int(name.split("_")[0]) train_label.append(label) Vect = np.zeros((1, 1024)) fr = open("./digits/trainingDigits/{}".format(name)) for i in range(32): lineStr = fr.readline() for j in range(32): Vect[0, 32*i+j] = int(lineStr[j]) train_data.append(list(Vect[0])) for name in name2: label=int(name.split("_")[0]) test_label.append(label) Vect = np.zeros((1, 1024)) fr = open("./digits/trainingDigits/{}".format(name)) for i in range(32): lineStr = fr.readline() for j in range(32): Vect[0, 32*i+j] = int(lineStr[j]) test_data.append(list(Vect[0])) train_data=np.array(train_data) test_data=np.array(test_data) # Combining K-fold cross-validation with learning curve to obtain optimal K-value scores_cross = [] Ks = [] for k in range(3, 20): knn = KNeighborsClassifier(n_neighbors=k) # Instantiate Model Object score_cross = cross_val_score(knn, train_data, train_label, cv=6).mean() # Cross-validation based on training data and return the score of cross-validation scores_cross.append(score_cross) Ks.append(k) # Convert to Array Type scores_arr = np.array(scores_cross) Ks_arr = np.array(Ks) # learning curve plt.plot(Ks, scores_cross) plt.show() # Get the highest score and the array subscript corresponding to the highest score to get the optimal K value score_best = scores_arr.max() # Find the highest score in the array of stored scores index_best = scores_arr.argmax() # Find the subscript corresponding to the highest score in array Ks_best = Ks_arr[index_best] # Find the K value corresponding to the highest score based on the subscript print(Ks_best) #model training knn=KNeighborsClassifier(n_neighbors=Ks_best) knn.fit(train_data,train_label) score=knn.score(test_data,test_label) print("score:",score) #Scoring curve import matplotlib.pyplot as plt k_range = range(1, 51) k_scores = [] for k in k_range: knn = KNeighborsClassifier(n_neighbors=k) knn.fit(train_data,train_label) score=knn.score(test_data,test_label) k_scores.append(score) plt.plot(k_range, k_scores) plt.xlabel('Vlaue of K for KNN') plt.ylabel('Cross-Validated Accuracy') plt.show()

Also attached is image processing, because our data is processed txt, the real data is png pictures

from PIL import Image import matplotlib.pylab as plt import numpy as np def picTo01(filename): #Read pictures img=Image.open(filename).convert('RGBA') raw_data=img.load() #Noise reduction based on the RGB value of the picture for y in range(img.size[1]): for x in range(img.size[0]): if raw_data[x,y][0]<90: raw_data[x,y]=(0,0,0,255) for y in range(img.size[1]): for x in range(img.size[0]): if raw_data[x,y][1]<136: raw_data[x,y]=(0,0,0,255) for y in range(img.size[1]): for x in range(img.size[0]): if raw_data[x,y][2]>0: raw_data[x,y]=(255,255,255,255) # Convert pictures to 32x32 img=img.resize((32,32),Image.LANCZOS) #Save the processed pictures for easy comparison img.save('test'+filename.split('.')[0]+'.png') #Make the black part of the picture 1 and the white part 0 according to the formula array=plt.array(img) gray_array=np.zeros((32,32)) for x in range(array.shape[0]): for y in range(array.shape[1]): gray=0.299*array[x][y][0]+0.587 * array[x][y][1] + 0.114 * array[x][y][2] #white if gray==255: gray_array[x][y]=0 #black else: gray_array[x][y]=1 #Name the txt file this number and save it name01=filename.split('.')[0] name01=name01+'.txt' np.savetxt(name01,gray_array,fmt='%d',delimiter='') picTo01('./png/0_03.png') # import os.path # import numpy as np # #Convert 32*32 to 1*1024 # def img32to1024(filename): # returnVect=np.zeros((1,1024)) # fr=open(filename,'r') # for i in range(32): # lineStr=fr.readline() # for j in range(32): # returnVect[0,32*i+j]=int(lineStr[j]) # return returnVect # #Collate train data and labels (file name label) # hwLabels=[] # trainingFileList=os.listdir('png') # m=len(trainingFileList) # trainingMat=np.zeros((m,1024)) # for i in range(m): # fileNameStr=trainingFileList[i] # fileStr=fileNameStr.split('.')[0] # classNumStr=int(fileStr.split('_')[0]) # hwLabels.append(classNumStr) # trainingMat[i,:]=img32to1024('trainingDigits/%s' % fileNameStr) # import os.path # import numpy as np # import train as tr # def classify(inX,k): # dataSetSize=tr.m # diffMat=inX-tr.trainingMat # sqDiffMat=diffMat**2 # sqDistances=sqDiffMat.sum(axis=1) # distances=sqDistances**0.5 # #get the rank # sortedDistances=distances.argsort() # classCount={} # for i in range(k): # voteIlabel=tr.hwLabels[sortedDistances[i]] # classCount[voteIlabel]=classCount.get(voteIlabel,0)+1 # sortedClassCount=sorted(classCount.items(),reverse=True) # return sortedClassCount[0][0]