Classification of Community Complaints Against Public Services on Twitter

The Surakarta Al-Islam Vocational School is a private educational institution that requires all students to pay school tuition fees. Education is an obligation for all Indonesian citizens. The cost of education is one of the most important input components in implementing education. Because cost is the main requirement in achieving educational goals. SPP School is a routine school fee that is carried out every month. Based on last year's School Admin report, many students were late in paying school tuition fees, around 60%. This is a very big problem because the income of school funds comes from school tuition. The purpose of this research is that the researcher will build a prediction system using the best classification method, which is to compare the accuracy level of the Naïve Bayes method with the K-K-Nearest Neighbor method. Because both methods can make class classifications right or late, in paying school fees. processing using dapodic data for 2017/2018 as many as 236 data. In improving accuracy, the researcher also applies feature selection with Information Gain, which is useful for selecting optimal parameters. System testing is carried out using the Confusion Matrix method. The final results of this study indicate that the Naïve Bayes Method + Information Gain Method produces the highest accuracy, namely 95% compared to the Naïve Bayes method alone, namely 85% and the K-NN method, namely 81%. Keywords— Big Data, Twitter, Naïve Bayes, Classification, Complaint.


I. INTRODUCTION
The development of the internet today is very rapid and has a great influence on human life, one of which is the increasingly rapid dissemination of information. At present almost all information that we want is on the internet, both from websites, blogs, social media, and others.One of the most popular media in sharing information is Twitter social media. Twitter is a social network that allows each user to share information [1]. Besides being used as social media, Twitter is also used as a medium to read news. Users who want to get information from an account must first become a follower of the account. The large amount of information available on Twitter can also be used to conduct a study, especially in terms of sentiment analysis by categorizing information based on certain categories.
Text categorization methods that can be used at this time are quite large, including Bayes classification algorithm, K-Nearest Neighbor (KNN), Neural Network (NN), Support Vector Machine (SVM), The Decision Tree, K-Means, etc.
In this study the author will use the Naïve Bayes method. This method was chosen because Naïve Bayes was able to carry out the classification process, which would be able to predict complaints or not on public services based on the results of posts from Twitter of the Indonesian people. The Naïve Bayes method is also included in the Top 10 algorithms in data mining. Based on the paper of (ICDM) International Conference on Data Mining [2].
In a previous study entitled Sentiment Classification of Twitter Belonging Data to Saudi Arabian Telecommunication Companies, using the KNN method to cluster tweets into positive and negative classes based on twitter data, so that 80.1% accuracy is obtained [3].
In a previous study conducted by (Effendy, 2016), the SVM method was used to determine positive and negative opinions on public transportation via twitter, which then the results of the analysis will be used to determine the factors that are the main causes of the inability to use public transportation and factors that make people choose to use this type of transportation [4] The main aim of this study to predicting by classifying the public services that include on complaint category or not based on 3 parameters is Transportation, Hospital, and market. This analysis results can be used to predict is it a complaint or not in the future, so it will help the government to decide the general policy by using this information.

II. RESEARCH METHODS
The method used in classifying complaints and not complaints on public services uses the Naïve Bayes method. This method was chosen because it was able to carry out the classification process.

Method Naïve Bayes
Naïve Bayes algorithm is a classification method with probability and statistical methods. This method was put forward by British scientist Thomas Bayes, namely predicting opportunities in the future based on previous experience so that it is known as the Bayes theorem. The theorem is assumed to have mutually independent attributes [5].
The theoretical basis used in carrying out this classification process is the Bayes theorem shown by the equation At opportunity A as B, it is obtained from opportunity B when A multiplied by opportunity A and divided by opportunity B. The use of Naïve Bayes on a data with more than one feature / attribute causes equation (1) to be more complex as shown in equation (2).
The value of P (B1 ... Bn) is constant for each experiment so that the maximum value of a class is determined by the maximum value between P (A) P (B.. Bn | A). The function of the equation formed into a maximum multiplication for the prior value and the likelihood function, the function is shown in the equation (3).

Classification Of Twitter
The System Experiment conducted aims to create an analysis system in classifying tweets using the Naïve Bayes method. At this stage it is known that the suitability of the output produced by the system and the most suitable parameters to be used as input in the classification process. The analysis in this study is to classify tweets using Indonesian. The following are the stages in making the system as shown in Figure 1.

ClassificationAnalysis
Analysis in this research the author will process the data sourced from the Twitter post. The purpose of this study is to predict complaints or not on public services based on activities in posts on Twitter. System built by applying the Naïve Bayes method [6]. At the complaint classification stage using training data from the results of previous experiments. Data processing in this study will use tweet data of 700 tweets or records.From 700 data records, the researcher will divide into two parts, 75% for training data, 525 records and 25% for data testing, which is 175 records.
Because in determining the good proportion for training data is 75% and testing data is 25%.

Performance Evaluation
System testing is useful to determine the level of accuracy of system performance. In Big Data for the classification of public service complaints there are several parameters that are mutually influential in predicting public service complaints.For this reason, in evaluating the performance of classifiers, the researchers used the Confusion Matrix test with Accuracy, Precision, Recall, F-Measure tests on the system that had been built. The system testing formula with the matrix confusion method is shown in the equation (4).

……………(4)
The results of the Accuracy, Recall, Precision, F-Measure values produce a percentage value from 0% -100% and good results are values above 70% and close to 100% [7] III. RESULT AND DISCUSSION Big Data in the classification of public service complaints is expected to make future predictions on public service complaints. The system was built by taking the twitter post data of the Indonesian community.From the results of twitter posts that have been post by the community, it generates large data collection so that the implementation of machine learning can help a model in classification of exclusion and not complaints on public services [8].

Data Analysis
The data used in this study comes from geolocation posts in the Indonesian region. Retrieving data using R studio tools by taking word categories based on parameters, namely transportation, hospital and market [9]. The results of data retrieval are still in the form of raw data that does not yet have a label, so before conducting the machine learning process the researcher conducts the cleaning data and labelling process, so that it becomes a dataset that can be used in predicting public service complaints.Retrieval of initial data as many as 1000 data, then after carrying out the cleaning process data obtained 700 data.The data is then labelled based on the types of parameters such as: transportation, hospitals and markets. For the results of class classification namely Complaints and Not Complaints.

Training and Testing
The dataset that has been collected is 700 data and has a label based on three parameter attributes and classes. From 700 data then 525 data were taken for Training and 175 Data Testing. Based on the data usage reference that In determining the proportion of training data and testing data is 75% training data and 25% testing data [10]. After the dataset is well formed, it can be used in the process of machine learning using the Naïve Bayes method. Based on the results of data collection, the researcher grouped the data into two parts, namely parameter data related to twitter text, which is shown in Table 1 and parameter data that has nothing to do with twitter text, shown in Table 2. Table 1.

Data Category Including Parameters
Based on the data in table 1, the results of the classification of complaints have been obtained and not the complaints that the researcher made manually by giving a label Yes on each twitter text that has to do with the category of parameters and provides class values (complaints and not complaints). The above data can also be described in Figure 2 below.

Figure 2. Data Category Including Parameters
Grouping of twitter text data that has nothing to do with parameters is shown in table 2. Table 2

. Data Category Excludes Parameters
Based on the data in table 2, the results of the categorization of data from text that has no connection to the three parameters have been obtained. The above data can also be described in Figure 3 beside.

Figure 3. Data Category Excludes Parameters
Based on the results of comparisons in Figures 2 and 3, it can be concluded that the posts that discuss the extensibility of public services are quite small. Because the number of class grades that are not complaints is more than the amount of the complaint. This shows that the Indonesian people are quite satisfied with the public services provided by the government. To measure the system, an accuracy test will be conducted using the matrix confusion method.

Testing Result
In the system testing phase, the researcher conducted two test scenarios, namely testing data sampling as much as 40 data and testing the overall data of 700 data.In testing sampling data with 40 data, namely 30 training data and 10 data testing, we obtained the results of confusion matrix testing with True Positive (   Based on the results of the two test scenarios that have been carried out, it can be concluded that by applying 700 data obtained an accuracy of 86%,better compared to 40 data which obtained an accuracy of 80%. However, the value of precision is 72% lower in the results of the second scenario if compared with the results of the first scenario, 75%. Based on the results of a comparison of testing of sampling data with the overall data, it can be described in Figure 4 below.  Based on the results of the two system tests the results are above 50% on each test parameter both from the value of Accuracy, Precision, Recall and F-Measure. While the best results are carried out using 700 data obtained the value of Accuracy: 86%, Precision: 72%, Recall: 81% and F-Measure: 76%.From all parameter test values the results are above 70%.So with this shows the system performance has run very well in conducting data classification.

IV. CONCLUSION
In this study has produced a data mining model that is able to process the classification of complaints or not complaints based on three parameters, namely transportation, hospitals and markets. System testing is done with 2 test scenarios, namely the first test scenario with 40 data, then produces an accuracy value: 80%, precision: 75%, recall: 75% and f-measure: 75%. While the second test scenario with 700 data, it produces an accuracy value: 86%, precision: 72%, recall: 81% and f-measure: 76%.Comparison of the two test scenarios that have been done, the best test results are obtained using 700 data. Because 86% accuracy is obtained. [