Evaluation Information System Using UTAUT ( Case Study : UMS Vocational School ) 1 stNendy

This admissions study evaluates users and the use of models for the Vocational School information system at the Muhammadiyah University of Surakarta with the UTAUT model by Venkatesh (2003) which has been modified by adding the System Service and Information Quality variables. The respondents are 100 students. From the results of the F and T tests, it is concluded that the variables Performance Expectance, Effort Expectance, Social Influence, System Service and Information Quality have an influence on Behavioral Intention, but only Social Influence and System Service variables have a significant partial effect on Behavioral Intention. In addition, another result is the variable Facilitating Condition and Behaviural Intention simultaneously affect Behavioral Usage. Validity, reliability and assumptions of classical test data show value as a measure of this analysis. Keywords— UTAUT, Evaluation, Vocational School, Information System


I. INTRODUCTION
In the current era, the development of Information Technology (IT) is very fast and has become an inseparable part of everyday human life. (Rais et al.) SalaryThe development of Information Technology is very fast and cannot be separated from everyday human life. One of them is the education sector that takes advantage of the development of information technology to simplify the operational process, especially in tertiary institutions. Information systems can provide services to its users in various fields, for example the fields of education, transportation, politics etc. (Rais)The UMS Vocational School is a faculty under the Muhammadiyah University of Surakarta, founded on the background of the desire of the Muhammadiyah University of Surakarta to provide human resources who are ready to go down in the world of work. Providing education with a weight of 30% theory and 70% practice with a learning motto that is active, innovative, creative, effective, fun, joyful and strong. UMS Vocational School has a mission to be a reformer in producing skilled, character and Islamic graduates who are ready to work and be independent. UMS Vocational School uses information systems in carrying out business processes and academic processes in the UMS Vocational environment. The systems used in Vocational UMS include academic information systems, value information systems, alumni information systems and Field Practice information systems. This information system has been implemented on a website basis to make it easier for students to access. Based on internal data from the UMS Vocational School which was obtained during an interview with the Director of the UMS Vocational School, there were only about 30% of students who had access to the Information System website. In addition, TU officers often receive questions from students about study activities and requests for files that are already in the system. From the results of interviews conducted by researchers, researchers found a problem, namely the lack of interest in students in accessing the information system of the UMS Vocational School. This caused disruption of the academic process at the UMS Vocational School, including the information provided by the campus to students not being conveyed thoroughly, the Administration still had to make and post announcements on the board, the Administration still often received questions from students. This is the basis of this research, so that this study will identify what factors influence the interest in student access to the UMS Vocational Information System. The theory used for this research is Unified Theory of Acceptance and Use of Technology (UTAUT). The reason the researcher chose the UTAUT method is because this method combines the features of the eight previous acceptance theories into one theory. This method is also considered to be more successful than the other eight theories in explaining up to 70% of user variants (Vankatesh et al. 2003). The UTAUT model is shown in the following figure 1: Research using the UTAUT model has been carried out, including research by Hannix Sulistyowati (2017), which discusses the analysis of user acceptance of the application of the e-office system at Airlangga University with the UTAUT model. The research objective was to determine what factors influence the acceptance and use of the e-office system at Airlangga University. In addition, Luzi Dwi Oktaviana, Zanuar Rifa'I and Kurnia Utami (2017) examined the Analysis of the Application of the Online KRS System on Student Satisfaction at STIMIK Amikom Purwokerto using the UTAUT method. The goal is to find out the shortcomings of information systems and improve information systems to make them better. Based on the above research, the researcher conducted research by modifying the UTAUT model which was added with system service variables and information quality that had an effect on user intensity (Use Intention).4. Research conducted by I Gusti Nyoman Sedana and St. Wisnu Wijaya (2018) entitled "Application of the UTAUT Model to Understand the Acceptance and Use of the Learning Management System in Experential E-Learning Of Sanata Dharma University". In this study, it is concluded that the test results with the Spearman correlation show that performance expectancy, effort expectancy, social influence, and facilitating conditions each have a positive and significant correlation (p-value <0.01) on behavioral intention. Likewise, behavioral intention has a positive and significant correlation with use behavior (p-value <0.05). Meanwhile, the facilitating condition does not have a significant correlation with the use behavior.

II.
RESEARCH METHODS The research was conducted on the UMS Vocational School Information System by distributing questionnaires to 100 students in the UMS Vocational School with a nonprobability sampling sample collection method. The questionnaire distributed certainly represents the variables in this study, namely performance expectancy, effort expectancy, social influence, facilitating conditions, system service, information quality and behavioral intention. From the results of the questionnaire collected is 100 for later processing. The results of the questionnaire were converted into quantitative data with a Likert scale which would later be tested and analyzed. The scale used is the five Likert scale with the following conditions : The model used in this research is the modified UTAUT model by adding two variables of System Service and Information Quality. The following is the research framework along with an explanation of each component in Figure 2: individual confidence that using a system will help get performance gains on the job. 2. Effort expectancy (EE) is the level of ease associated with using a system. 3. Social influence (SI) is the level of the individual perceives the interests that are believed by other people who will influence using the new system. 4. Facilitating conditions (FC) is the level of individuals believing that the organizational and technical infrastructure is in place to support the system. 5. System Service (SS) used to measure the quality of service to its users. 6. Information Quality (IQ) is the output of information generated by the information system used. 7. Behavioral Intention) (BI) is the level of desire of users in continuously utilizing existing systems. 8. Use Behavior (UB) is the intensity or frequency of users using information technology. Researchers started this research with literature study from journals, books, research reports, papers and other literature related to the evaluation of information system implementation. Then carried out data and information collection by making observations in the UMS vocational school student information system, interviews with the director of the UMS vocational school and direct surveys to the location to find out conditions and problems which can later be identified as problem formulation and research objectives. In addition, a literature study was also carried out on previous research related to the evaluation of the implementation of information systems to obtain information about the methodology and research results, which will also be used to consider in determining the analysis model and research implementation to be carried out. In the identification and problem analysis stage, the researcher identified the data needs to be used, analyzed the UMS vocational school student information system in order to obtain the problem formulation and research objectives. Then draw conclusions about the formulation of the research model based on the previous stages, the research model that is considered appropriate, namely the UTAUT model by adding variables of system quality and information quality. Next is the stage of compiling and distributing questionnaires to respondents. However, before the questionnaires are distributed, they must first be compiled by taking the questionnaires from previous studies that have been tested and conducting discussions with the experts / research advisers. After the questionnaire is distributed and filled out by the respondents, the questionnaire data is collected and processed. After that the data will be analyzed by validity test, reliability test, and statistical test. This test is carried out with the SEM model to test what variables affect user interest in accessing the UMS Vocational school student information system. The result is a statistic that shows how much influence the independent and dependent variables have on the modified UTAUT model. The final result of this research is in the form of recommendations, conclusions and suggestions which are of course based on the results of data analysis.
Furthermore, feedback will be requested from the Vocational Director whether he agrees or not before producing a final report.The research flow can be seen in Figure 3. Education and training at the UMS Vocational School collaborates with the business world in the industrial world according to their fields. Students are taught theory in class, and workshops for 3 months and continued on the On The Job Training for 6 months in the industrial world. Education and training materials 30% theory and 70% practice, supported through the concept of mastery learning, contextual teaching and learning, active learning and learning is fun, learning by doing.

UMS Vocational School Information System
The UMS Vocational School Information System is an information system built that contains information related to the UMS Vocational School. Information systems are used to help carry out academic processes and those in the campus environment. The information system is implemented based on a website to make it easier for students to access.The main page of the UMS vocational website can be seen in Figure 4.

. Preparation of research questionnaire
The preparation of the questionnaire in this study was taken from the variables in the modified UTAUT model by adding two variables: system quality and information quality. The aim is to determine the effect of performance expectancy, effort expectancy, social influence, system quality, information quality variables on behavioral intention and the effect of facilitating conditions, behavioral intention on use behavior in the UMS Vocational School information system. The scale used is a Likert scale ranging from 1 to 5 in the order of strongly disagree, disagree, neutral, agree and strongly agree.
Questionnaire items compiled as research instruments were compiled based on adaptations of questionnaire items that had been used in previous studies. After determining the original items taken from Jogiyanto (2007), the items were then adapted to the place of research and changed into Indonesian. The questionnaire items were conducted to obtain the validity of the items that make up the research construct. Preparation of research questionnaires based on the adaptation of items which are then adjusted to the research objectives

Test the validity and reliability of the instrument
Validity and Reliability are very important in a data analysis. This is done to test a measuring instrument or research instrument (questionnaire) whether it is valid and reliable or not. The pilot study was conducted on 102 respondents. The results of the validity and reliability tests are as follows:

validity test
The results of testing the validity of the instrument with SPSS version 25 for Windows, obtained the following results: a. The data shows that the r-count value of Spearman rank correlation for all statements of the Performance The data shows that the rcount value of the Spearman rank correlation of all statements of the System Service (SS) variable is greater than the rtable value. Thus, all items of the System Service (SS) variable statement are declared valid and can be used as a data collection tool.
The data shows that the rcount value of the Spearman rank correlation of all statements of the Information Quality (IQ) variable is greater than the rtable value. Thus, all items of the Information Quality (IQ) variable statement are declared valid and can be used as a data collection tool.
The data shows that the rcount value of the Spearman rank correlation for all statements of the Behavioral Intention (BI) variable is greater than the rtable value. Thus, all items of the Behavioral Intention (BI) variable statement are declared valid and can be used as a data collection tool.
The data shows that the rcount value of the Spearman rank correlation of all statements of the Use Behavior (UB) variable is greater than the value of rtable. Thus, all items of the Use Behavior (UB) variable statement are declared valid and can be used as a data collection tool.

Reliability test
To test the reliability of the questionnaire or questions used Cronbach alpha formula. A research questionnaire is declared reliable if the value of the reliability coefficient (ralpha) is greater than the value of rtable.  Based on the table above, it can be seen that all indicators have a p-value of chi square Skewness and Kurtosis greater than 0.05. This shows that univariately, the normality of the data is met. 2. Multivariate normality If the results of the multivariate normality test show that the p-value of chi square Skewness and Kurtosis is greater than 0.05, it means that all variables follow the normal distribution function. The following are the results of the multivariate normality analysis shown in  Based on the table above, the p-value of chi square Skewness and Kurtosis obtained is 0.000. Because the value is lower than 0.05, it can be concluded that multivariate normality is not met. Based on the limit central theom (central limit theorem) ie the larger the sample size, the statistics obtained will approach the normal distribution. The number of units of analysis in this study is n = 102, it is considered to have fulfilled the central limit theorem. Thus the assumption of normality can be ignored.

Evaluation of the UMS Vocational School Information System 1. Validity and Reliability Test
The validity and reliability tests are as expected, namely the data is suitable to be used as a measuring tool for analysis in the UMS Vocational School Information System Evaluation research.

Classic Assumption Test
The results of the Classical Assumption Test, namely Normality, Heteroscedasticity, Autocorrelation and Multicollinearity of the data are in accordance with the requirements as appropriate data to be used as prediction of the dependent variable BI by the predictor variables PE, EE, SI, SS and IQ.

Multiple Regression Test Results
Prediction variables of UB by predictor variables of FC and BI. Has a Variance value of 66.3% for BI by predictors of PE, EE, SI, SS and IQ. Meanwhile, the variance value at 40.0% for UB by the predictors of FC and BI. Research Hypothesis is a temporary answer to the problem formulation that has been made in the research. Partial testing is carried out to test whether each independent variable has a significant effect on the dependent variable, then the hypothesis is tested as follows: Ho: the independent variable does not have a significant positive effect on the dependent variable Ha : the independent variable has a significant positive effect on the dependent variable. (Sugiono, 2009) Test Criteria: Accept Ha if t count 1.64 or -1.64 Accept Ho if t count < 1.64 or >-1.64 The following is an output image of the t-values estimation results shown in Figure 8 below: Figure 8. t-value In Figure 5 it can be seen that the results of the F test conclude that the variables PE, EE, SI, SS and IQ simultaneously affect the BI variable. Besides that, it can be seen that the predictor variables FC and BI have a simultaneous effect on the bound prediction variable of UB. The t test results for the regression coefficient have given conclusions as in Figure 8 below : Figure 9. t test results In Figure 9, the results can be seen partially that the variables PE, EE and IQ are not significant to the BI variable, while the SS and SI variables partially have a significant effect on the BI variable. In addition, the FC variable significantly affects the UB variable. Meanwhile, the BI variable does not partially have a significant effect on the UB variable.
The regression coefficients that can be drawn are as follows :

IV. CONCLUSION
The conclusion that the researchers found from this study is: 1. Acceptance of the UMS Vocational School information system is simultaneously affected by all variables. The acceptance of the UMS Vocational information system is affected by the ability of the UMS Vocational School information system to improve business & performance, besides that the acceptance of the information system is also influenced by the quality of information, system services and social influence where important people have a positive effect on users in the acceptance of the UMS Vocational information system. 2. The level of acceptance of the UMS Vocational School information system is significantly influenced by Social Influence and System Service factors. Social Influence is more influential than System Service. From these results, it is important to use the social influence of UMS Vocational School officials, class leaders, relatives, influential colleagues or students such as the smartest student or highest GPA to influence or convince users that the use of information systems can improve business performance and job performance. In addition, a good level of service quality will also support the acceptance of this information system. 3. Behavioral Intentions to use this information system have no significant effect on (Use Behavior), while what has a significant effect is the Facilitating Conditions that support the use of the system. The suggestions put forward by the author in this study : 1) In further research, the moderator factors are not ignored, namely Age, Gender, Experience and Voluntariness of Use. 2) In further research carried out with other acceptance models besides UTAUT, for example UTAUT 2.
Besides that, you can also combine or modify the UTAUT model with other models.