A Text-Based Predictive Maintenance Approach for Facility Management Requests Utilizing Association Rule Mining and Large Language Models
Abstract
:1. Introduction
2. Theoretical Background
2.1. Predictive Maintenance for Facilities
2.2. Association Rule Mining
2.3. Extensions of Association Rule Mining
3. Methodology
3.1. ARM with Semantic Similarity
3.2. Temporal Extension
3.3. Expert Evaluation
3.4. Technical Case Study
- Create rules based on all maintenance requests that are available at the time of the current iteration;
- Check for all rules, whether they are new rules or whether they have been found in a previous iteration (by applying a similarity measure also to identify very similar rules);
- Optional: check for each rule whether a human domain expert estimates a contextual connection between the antecedent and the consequence of a rule. Only keep rules with a possible connection;
- Calculate the hits of the new rules on the current data (i.e., the hits that the ARM algorithm used to create the new rules);
- Calculate the hits of the new rules on all available data (also future maintenance requests);
- Calculate the future hits by subtracting the hits on the current data from those on all available data;
- Keep the new rules in the backlog for the next iteration.
4. Results
4.1. Semantic Similarity Comparison
4.2. Comparison of Rules without Human Expert Evaluation
4.3. Comparison of Rules with Human Expert Evaluation
5. Discussion
6. Conclusions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Widodo, A.; Yang, B.-S. Support Vector Machine in Machine Condition Monitoring and Fault Diagnosis. Mech. Syst. Signal Process. 2007, 21, 2560–2574. [Google Scholar] [CrossRef]
- Mobley, R.K. An Introduction to Predictive Maintenance; Elsevier: Amsterdam, The Netherlands, 2002; ISBN 978-0-7506-7531-4. [Google Scholar]
- Alestra, S.; Bordry, C.; Brand, C.; Burnaev, E.; Erofeev, P.; Papanov, A.; Silveira-Freixo, C. Rare Event Anticipation and Degradation Trending for Aircraft Predictive Maintenance. In Proceedings of the 11th World Congress on Computational Mechanics, WCCM, Barcelona, Spain, 20–25 July 2014; Volume 5, p. 6571. [Google Scholar]
- Zhu, H.; Gao, J.; Li, D.; Tang, D. A Web-Based Product Service System for Aerospace Maintenance, Repair and Overhaul Services. Comput. Ind. 2012, 63, 338–348. [Google Scholar] [CrossRef]
- Adu-Amankwa, K.; Attia, A.K.; Janardhanan, M.N.; Patel, I. A Predictive Maintenance Cost Model for CNC SMEs in the Era of Industry 4.0. Int. J. Adv. Manuf. Technol. 2019, 104, 3567–3587. [Google Scholar] [CrossRef]
- Hrnjica, B.; Softic, S. Explainable AI in Manufacturing: A Predictive Maintenance Case Study. In Advances in Production Management Systems. Towards Smart and Digital Manufacturing; Lalic, B., Majstorovic, V., Marjanovic, U., von Cieminski, G., Romero, D., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 66–73. [Google Scholar]
- Antomarioni, S.; Pisacane, O.; Potena, D.; Bevilacqua, M.; Ciarapica, F.E.; Diamantini, C. A Predictive Association Rule-Based Maintenance Policy to Minimize the Probability of Breakages: Application to an Oil Refinery. Int. J. Adv. Manuf. Technol. 2019, 105, 3661–3675. [Google Scholar] [CrossRef]
- Nadj, M.; Jegadeesan, H.; Maedche, A.; Hoffmann, D.; Erdmann, P. A Situation Awareness Driven Design for Predictive Maintenance Systems: The Case of Oil and Gas Pipeline Operations. In Proceedings of the 24th European Conference on Information Systems, Istanbul, Turkey, 12–15 June 2016; pp. 1–10. [Google Scholar]
- Ding, Y.; Ma, J.; Luo, X. Applications of Natural Language Processing in Construction. Autom. Constr. 2022, 136, 104169. [Google Scholar] [CrossRef]
- Wu, C.; Li, X.; Guo, Y.; Wang, J.; Ren, Z.; Wang, M.; Yang, Z. Natural Language Processing for Smart Construction: Current Status and Future Directions. Autom. Constr. 2022, 134, 104059. [Google Scholar] [CrossRef]
- Ghofrani, A.; Nazemi, S.D.; Jafari, M.A. HVAC Load Synchronization in Smart Building Communities. Sustain. Cities Soc. 2019, 51, 101741. [Google Scholar] [CrossRef]
- West, S.R.; Guo, Y.; Wang, X.R.; Wall, J. Automated Fault Detection and Diagnosis of HVAC Subsystems Using Statistical Machine Learning. In Proceedings of the Building Simulation, Sydney, Australia, 14–16 November 2011. [Google Scholar]
- Cheng, J.C.; Chen, W.; Tan, Y.; Wang, M. A BIM-Based Decision Support System Framework for Predictive Maintenance Management of Building Facilities. In Proceedings of the 16th International Conference on Computing in Civil and Building Engineering (ICCCBE2016), Osaka, Japan, 6–8 July 2016; pp. 711–718. [Google Scholar]
- Ur-Rahman, N.; Harding, J.A. Textual Data Mining for Industrial Knowledge Management and Text Classification: A Business Oriented Approach. Expert Syst. Appl. 2012, 39, 4729–4739. [Google Scholar] [CrossRef]
- Bortolini, R.; Forcada, N. Analysis of Building Maintenance Requests Using a Text Mining Approach: Building Services Evaluation. Build. Res. Inf. 2020, 48, 207–217. [Google Scholar] [CrossRef]
- Folino, F.; Folino, G.; Guarascio, M.; Pontieri, L. Semi-Supervised Discovery of DNN-Based Outcome Predictors from Scarcely-Labeled Process Logs. Bus. Inf. Syst. Eng. 2022, 64, 729–749. [Google Scholar] [CrossRef]
- Chandrasekaran, D.; Mago, V. Evolution of Semantic Similarity—A Survey. ACM Comput. Surv. 2021, 54, 41:1–41:37. [Google Scholar] [CrossRef]
- Hashemian, H.M. State-of-the-Art Predictive Maintenance Techniques. IEEE Trans. Instrum. Meas. 2010, 60, 226–236. [Google Scholar] [CrossRef]
- Virk, S.M.; Muhammad, A.; Martinez-Enriquez, A. Fault Prediction Using Artificial Neural Network and Fuzzy Logic. In Proceedings of the 2008 Seventh Mexican International Conference on Artificial Intelligence, Atizapan de Zaragoza, Mexico, 27 October 2008; pp. 149–154. [Google Scholar]
- Carvalho, T.P.; Soares, F.A.A.M.N.; Vita, R.; Francisco, R.d.P.; Basto, J.P.; Alcalá, S.G.S. A Systematic Literature Review of Machine Learning Methods Applied to Predictive Maintenance. Comput. Ind. Eng. 2019, 137, 106024. [Google Scholar] [CrossRef]
- Mo, Y.; Zhao, D.; Du, J.; Syal, M.; Aziz, A.; Li, H. Automated Staff Assignment for Building Maintenance Using Natural Language Processing. Autom. Constr. 2020, 113, 103150. [Google Scholar] [CrossRef]
- Akhbardeh, F.; Desell, T.; Zampieri, M. NLP Tools for Predictive Maintenance Records in MaintNet. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: System Demonstrations; Wong, D., Kiela, D., Eds.; Association for Computational Linguistics: Suzhou, China, 2020; pp. 26–32. [Google Scholar]
- Bhardwaj, A.S.; Veeramani, D.; Zhou, S. Identifying Equipment Health Status from Maintenance Records Using Lexicon Based Unsupervised Sentiment Analysis Adjusted for Negation (LUSAA-N). Comput. Ind. Eng. 2023, 186, 109693. [Google Scholar] [CrossRef]
- Devaney, M.; Ram, A.; Qiu, H.; Lee, J. Preventing Failures by Mining Maintenance Logs with Case-Based Reasoning. In Proceedings of the 59th Meeting of the Society for Machinery Failure Prevention Technology (MFPT-59), Virginia Beach, VA, USA, 18–21 April 2005. [Google Scholar]
- Carrasco, J.; López, D.; Aguilera-Martos, I.; García-Gil, D.; Markova, I.; García-Barzana, M.; Arias-Rodil, M.; Luengo, J.; Herrera, F. Anomaly Detection in Predictive Maintenance: A New Evaluation Framework for Temporal Unsupervised Anomaly Detection Algorithms. Neurocomputing 2021, 462, 440–452. [Google Scholar] [CrossRef]
- da Silva Arantes, J.; da Silva Arantes, M.; Fröhlich, H.B.; Siret, L.; Bonnard, R. A Novel Unsupervised Method for Anomaly Detection in Time Series Based on Statistical Features for Industrial Predictive Maintenance. Int. J. Data Sci. Anal. 2021, 12, 383–404. [Google Scholar] [CrossRef]
- Graß, A.; Beecks, C.; Soto, J.A.C. Unsupervised Anomaly Detection in Production Lines. In Machine Learning for Cyber Physical Systems; Beyerer, J., Kühnert, C., Niggemann, O., Eds.; Springer: Berlin/Heidelberg, Germany, 2019; pp. 18–25. [Google Scholar]
- Agrawal, R.; Imieliński, T.; Swami, A. Mining Association Rules Between Sets of Items in Large Databases. In Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, Washington, DC, USA, 26–28 May 1993; pp. 207–216. [Google Scholar]
- Adamov, A.Z. Mining Term Association Rules from Unstructured Text in Azerbaijani Language. In Proceedings of the 2018 IEEE 12th International Conference on Application of Information and Communication Technologies (AICT), Almaty, Kazakhstan, 17–19 October 2018; pp. 1–4. [Google Scholar]
- Diaz-Garcia, J.A.; Ruiz, M.D.; Martin-Bautista, M.J. A Survey on the Use of Association Rules Mining Techniques in Textual Social Media. Artif. Intell. Rev. 2023, 56, 1175–1200. [Google Scholar] [CrossRef]
- Segura-Delgado, A.; Gacto, M.J.; Alcalá, R.; Alcalá-Fdez, J. Temporal Association Rule Mining: An Overview Considering the Time Variable as an Integral or Implied Component. WIREs Data Min. Knowl. Discov. 2020, 10, e1367. [Google Scholar] [CrossRef]
- Agrawal, R.; Srikant, R. Fast Elgorithms for Mining Association Rules. In Proceedings of the 20th International Conference on Very Large Data Bases (VLDB), Santiago, Chile, 12–15 September 1994; Volume 1215, pp. 487–499. [Google Scholar]
- Kumbhare, T.A.; Chobe, S.V. An Overview of Association Rule Mining Algorithms. Int. J. Comput. Sci. Inf. Technol. 2014, 5, 927–930. [Google Scholar]
- Kireev, V.S.; Guseva, A.I.; Bochkaryov, P.V.; Kuznetsov, I.A.; Filippov, S.A. Association Rules Mining for Predictive Analytics in IoT Cloud System. In Biologically Inspired Cognitive Architectures 2018; Samsonovich, A.V., Ed.; Springer International Publishing: Cham, Switzerland, 2019; pp. 107–112. [Google Scholar]
- Jurafsky, D.; Martin, J.H. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition; Prentice Hall: Upper Saddle River, NJ, USA, 2008; ISBN 0-13-095069-6. [Google Scholar]
- Luhn, H.P. A Statistical Approach to Mechanized Encoding and Searching of Literary Information. IBM J. Res. Dev. 1957, 1, 309–317. [Google Scholar] [CrossRef]
- Turian, J.; Ratinov, L.; Bengio, Y. Word Representations: A Simple and General Method for Semi-Supervised Learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, 11–16 July 2010; pp. 384–394. [Google Scholar]
- Hemalatha, B.; Velmurugan, T. Direct-Indirect Association Rule Mining for Online Shopping Customer Data Using Natural Language Processing. Int. J. Recent Technol. Eng. 2019, 8, 2277–3878. [Google Scholar] [CrossRef]
- Ren, S.; Li, Z.; Wang, H.; Li, Y.; Shen, K.; Cheng, S. NEARM: Natural Language Enhanced Association Rules Mining. In Proceedings of the 2018 IEEE International Conference on Data Mining Workshops (ICDMW), Singapore, 17–20 November 2018; pp. 438–445. [Google Scholar]
- Lakshmi, K.S.; Vadivu, G. Extracting Association Rules from Medical Health Records Using Multi-Criteria Decision Analysis. Proc. Comput. Sci. 2017, 115, 290–295. [Google Scholar] [CrossRef]
- Edwards, B.; Zatorsky, M.; Nayak, R. Clustering and Classification of Maintenance Logs Using Text Data Mining. In Proceedings of the 7th Australasian Data Mining Conference, Glenelg, SA, Australia, 27–28 November 2008; Volume 87, pp. 193–199. [Google Scholar]
- Usuga-Cadavid, J.P.; Lamouri, S.; Grabot, B.; Fortin, A. Using Deep Learning to Value Free-Form Text Data for Predictive Maintenance. Int. J. Prod. Res. 2022, 60, 4548–4575. [Google Scholar] [CrossRef]
- Devlin, J.; Chang, M.-W.; Lee, K.; Toutanova, K. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA, 2–7 June 2019; Association for Computational Linguistics: Minneapolis, MN, USA, 2019; pp. 4171–4186. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention Is All You Need. Adv. Neural Inf. Process. Syst. 2017, 30, 5998–6008. [Google Scholar]
- OpenAI GPT-4 Technical Report. arXiv 2023, arXiv:2303.08774.
- Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; Stoyanov, V. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv 2019, arXiv:1907.11692. [Google Scholar]
- Wang, W.; Wei, F.; Dong, L.; Bao, H.; Yang, N.; Zhou, M. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. In Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, Vancouver, BC, Canada, 6–12 December 2020. [Google Scholar]
- Kaluarachchi, A.C.; Varde, A.S.; Bedathur, S.; Weikum, G.; Peng, J.; Feldman, A. Incorporating Terminology Evolution for Query Translation in Text Retrieval with Association Rules. In Proceedings of the 9th ACM International Conference on Information and Knowledge Management, New York, NY, USA, 6–11 November 2010; Association for Computing Machinery: New York, NY, USA, 2010; pp. 1789–1792. [Google Scholar]
- Zeng, A.P.; Liu, D.; Chen, H.M. An Improved Apriori Algorithm Based on Similarity. Adv. Mater. Res. 2012, 532–533, 1825–1829. [Google Scholar]
- Keith Norambuena, B.; Villegas, C. An Extension to Association Rules Using a Similarity-Based Approach in Semantic Cector Spaces. Intell. Data Anal. 2019, 23, 587–607. [Google Scholar] [CrossRef]
- Sammouri, W.; Côme, E.; Oukhellou, L.; Aknin, P. Mining Floating Train Data Sequences for Temporal Association Rules within a Predictive Maintenance Framework. In Advances in Data Mining. Applications and Theoretical Aspects; Perner, P., Ed.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 112–126. [Google Scholar]
- Han, Y.; Yu, D.; Yin, C.; Zhao, Q. Temporal Association Rule Mining and Updating and Their Application to Blast Furnace in the Steel Industry. Comput. Intell. Neurosci. 2020, 2020, 7467213. [Google Scholar] [CrossRef]
- Li, Y.; McLean, D.; Bandar, Z.A.; O’Shea, J.D.; Crockett, K. Sentence Similarity Based on Semantic Nets and Corpus Statistics. IEEE Trans. Knowl. Data Eng. 2006, 18, 1138–1150. [Google Scholar] [CrossRef]
- Reimers, N.; Gurevych, I. Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks. arXiv 2019, arXiv:1908.10084. [Google Scholar]
- Reimers, N.; Beyer, P.; Gurevych, I. Task-Oriented Intrinsic Evaluation of Semantic Textual Similarity. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan, 11–16 December 2016; pp. 87–96. [Google Scholar]
- Spasic, I.; Button, K. Patient Triage by Topic Modeling of Referral Letters: Feasibility Study. JMIR Med. Inform. 2020, 8, e21252. [Google Scholar] [CrossRef] [PubMed]
- Lee, M.D.; Pincombe, B.; Welsh, M. An Empirical Evaluation of Models of Text Document Similarity. In Proceedings of the Annual Meeting of the Cognitive Science Society, Stresa, Italy, 21–23 July 2005; Volume 27. [Google Scholar]
- Agirre, E.; Banea, C.; Cer, D.; Diab, M.; Gonzalez Agirre, A.; Mihalcea, R.; Rigau Claramunt, G.; Wiebe, J. Semeval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation. In Proceedings of the 10th International Workshop on Semantic Evaluation, San Diego, CA, USA, 16–17 June 2016; ACL (Association for Computational Linguistics): San Diego, CA, USA, 2016; pp. 497–511. [Google Scholar]
- Reimers, N.; Freire, P.; Becquin, G.; Espejel, O.; Gante, J. Sentence-Transformers/All-MiniLM-L6-v2 Hugging Face. Available online: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 (accessed on 2 June 2023).
- May, P. T-Systems-Onsite/Cross-En-de-Roberta-Sentence-Transformer Hugging Face. Available online: https://huggingface.co/T-Systems-onsite/cross-en-de-roberta-sentence-transformer (accessed on 2 June 2023).
- May, P. T-Systems-Onsite/German-Roberta-Sentence-Transformer-v2 Hugging Face. Available online: https://huggingface.co/T-Systems-onsite/german-roberta-sentence-transformer-v2 (accessed on 2 June 2023).
- Papineni, K.; Roukos, S.; Ward, T.; Zhu, W.-J. Bleu: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA, 7–12 July 2002; Isabelle, P., Charniak, E., Lin, D., Eds.; Association for Computational Linguistics: Philadelphia, PA, USA, 2002; pp. 311–318. [Google Scholar]
- Lin, C.-Y. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out; Association for Computational Linguistics: Barcelona, Spain, 2004; pp. 74–81. [Google Scholar]
- Banerjee, S.; Lavie, A. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, MI, USA, 25–30 June 2005; Goldstein, J., Lavie, A., Lin, C.-Y., Voss, C., Eds.; Association for Computational Linguistics: Ann Arbor, MI, USA, 2005; pp. 65–72. [Google Scholar]
- Zhang, T.; Kishore, V.; Wu, F.; Weinberger, K.Q.; Artzi, Y. BERTScore: Evaluating Text Generation with BERT. arXiv 2020, arXiv:1904.09675. [Google Scholar]
- Agirre, E.; Cer, D.; Diab, M.; Gonzalez-Agirre, A. Semeval-2012 Task 6: A Pilot on Semantic Textual Similarity, Proceedings of the SEM 2012: The First Joint Conference on Lexical and Computational Semantics—Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), Montréal, QC, Canada, 7–8 June 2012; Association for Computational Linguistics: Kerrville, TX, USA, 2012; pp. 385–393. [Google Scholar]
- O’Shea, J.; Bandar, Z.; Crockett, K.; McLean, D. A Comparative Study of Two Short Text Semantic Similarity Measures. In Lecture Notes in Computer Science; Nguyen, N.T., Jo, G.S., Howlett, R.J., Jain, L.C., Eds.; Springer: Berlin/Heidelberg, Germany, 2008; Volume 4953, pp. 172–181. [Google Scholar]
- Miok, K.; Corcoran, P.; Spasić, I. The Value of Numbers in Clinical Text Classification. Mach. Learn. Knowl. Extr. 2023, 5, 746–762. [Google Scholar] [CrossRef]
- Laureate, C.D.P.; Buntine, W.; Linger, H. A Systematic Review of the Use of Topic Models for Short Text Social Media Analysis. Artif. Intell. Rev. 2023, 56, 14223–14255. [Google Scholar] [CrossRef] [PubMed]
- Qiang, J.; Qian, Z.; Li, Y.; Yuan, Y.; Wu, X. Short Text Topic Modeling Techniques, Applications, and Performance: A Survey. IEEE Trans. Knowl. Data Eng. 2022, 34, 1427–1445. [Google Scholar] [CrossRef]
Model Identifier | Type of Model | Languages | Huggingface Handle | Publisher |
---|---|---|---|---|
German RoBERTa | RoBERTa | German | T-Systems-onsite/german-roberta-sentence-transformer-v2 | T-Systems on site services GmbH, Berlin, Germany |
Cross RoBERTa | RoBERTa | German, English | T-Systems-onsite/cross-en-de- roberta-sentence-transformer | T-Systems on site services GmbH, Berlin, Germany |
English RoBERTa | RoBERTa | English | sentence-transformers/all-roberta-large-v1 | Nils Reimers, Ubiquitous Knowledge Processing (UKP) Lab, Technical University of Darmstadt, Darmstadt, Germany |
English MimiLM | MiniLM | English | sentence-transformers/all-MiniLM-L6-v2 | Nils Reimers, Ubiquitous Knowledge Processing (UKP) Lab, Technical University of Darmstadt, Darmstadt, Germany |
Item 1 | Item 2 | Semantic Similarity |
---|---|---|
Room [room number]—Fire bulkhead defective (defect from test [test number]) | Room [room number]—The fire bulkhead on the ceiling is damaged (defect from test [test number]) | 0.78 |
Meter Reading November 2020 | Meter Reading December 2020 | 0.91 |
[room number]; no electricity in the entire area | No electricity in the [company name] warehouse in the [room number] | 0.76 |
Cable break at plug | Replacement circuit breaker UV outdoor lighting [room number] | 0.19 |
Power failure at pillar [pillar number] | [room number], the blue cover is missing on a socket, socket still OK. | 0.17 |
German RoBERTa | Cross RoBERTa | English RoBERTa | English MiniLM | ||
---|---|---|---|---|---|
Pearson’s Correlation Coefficient | German RoBERTa | 1 | |||
Cross RoBERTa | 0.95 | 1 | |||
English RoBERTa | 0.59 | 0.57 | 1 | ||
English MiniLM | 0.62 | 0.60 | 0.80 | 1 | |
Mean | 0.25 | 0.25 | 0.22 | 0.19 | |
Median | 0.24 | 0.25 | 0.20 | 0.17 | |
Standard Deviation | 0.14 | 0.15 | 0.14 | 0.13 |
Semantic Text Similarity Model | Human Domain Judge | Human Technical Judge | ||
---|---|---|---|---|
Pearson Correlation | F1-Score | Pearson Correlation | F1-Score | |
German RoBERTa | 0.65 | 0.66 | 0.54 | 1.0 |
Cross RoBERTa | 0.64 | 0.66 | 0.56 | 1.0 |
English RoBERTa | 0.63 | 0.5 | 0.69 | 0.67 |
English MiniLM | 0.57 | 0.5 | 0.74 | 0.8 |
BLEU | 0.34 | 0.0 | 0.39 | 0.0 |
ROUGE-L | 0.50 | 0.5 | 0.65 | 0.8 |
METEOR | 0.46 | 0.29 | 0.63 | 0.5 |
BERTScore | 0.49 | 0.5 | 0.65 | 0.08 |
Model | Antecedent | Consequent | |
---|---|---|---|
German RoBERTa | Room X: power socket problem; voltage is only 137 V | → | Room Z: fire bulkhead in ceiling defect |
Room Y: PEN error; voltage is only 138 V | → | Room Z: fire bulkhead in ceiling defect | |
Room Y: PEN error; voltage is only 138 V | → | External system labeling is missing | |
Room X: power socket problem; voltage is only 137 V | → | External system labeling is missing | |
Cross RoBERTa | Room W: no electricity | → | Room V: fire bulkhead in wall defect |
Room W: no electricity | → | Room Z: fire bulkhead in ceiling defect | |
Room Y: PEN error; voltage is only 138 V | → | External system labeling is missing | |
Room X: power socket problem; voltage is only 137 V | → | External system labeling is missing | |
Room W: no electricity | → | Room T: Coffee machine not working | |
Room U: no electricity | → | Room T: Coffee machine not working |
Minimum Similarity Threshold | German RoBERTa | Cross RoBERTa | English RoBERTa | English MiniLM | ||||
---|---|---|---|---|---|---|---|---|
Rules | Hits | Rules | Hits | Rules | Hits | Rules | Hits | |
0.65 | 104 | 46 | 120 | 48 | 228 | 80 | 54 | 23 |
0.7 | 38 | 13 | 22 | 9 | 46 | 21 | 0 | 0 |
0.75 | 14 | 5 | 16 | 4 | 4 | 2 | 0 | 0 |
0.8 | 0 | 0 | 0 | 0 | 4 | 2 | 0 | 0 |
Minimum Similarity Threshold | German RoBERTa | Cross RoBERTa | English RoBERTa | English MiniLM | ||||
---|---|---|---|---|---|---|---|---|
Rules | Hits | Rules | Hits | Rules | Hits | Rules | Hits | |
0.65 | 42 | 31 | 44 | 24 | 113 | 67 | 19 | 17 |
0.7 | 15 | 11 | 9 | 7 | 13 | 12 | 0 | 0 |
0.75 | 4 | 4 | 6 | 4 | 0 | 0 | 0 | 0 |
LLM | Minimum Similarity | German RoBERTa | Cross RoBERTa | English RoBERTa | English MiniLM |
---|---|---|---|---|---|
German RoBERTa | 0.65 | - | 0.77 | 0.05 | 0.52 |
Cross RoBERTa | 0.65 | 0.79 | - | 0.04 | 0.48 |
English RoBERTa | 0.65 | 0.13 | 0.07 | - | 0.56 |
English MiniLM | 0.65 | 0.21 | 0.12 | 0.13 | - |
German RoBERTa | 0.7 | - | 1.0 | 0.04 | N/A |
Cross RoBERTa | 0.7 | 0.68 | - | 0.04 | N/A |
English RoBERTa | 0.7 | 0.16 | 0.09 | - | N/A |
English MiniLM | 0.7 | 0.0 | 0.0 | 0.0 | N/A |
German RoBERTa | 0.75 | - | 0.5 | 0.0 | N/A |
Cross RoBERTa | 0.75 | 0.57 | - | 0.0 | N/A |
English RoBERTa | 0.75 | 0.0 | 0.0 | - | N/A |
English MiniLM | 0.75 | 0.0 | 0.0 | 0.0 | N/A |
Minimum Similarity Threshold | German RoBERTa | Cross RoBERTa | English RoBERTa | English MiniLM | |||||
---|---|---|---|---|---|---|---|---|---|
Temporal Lift Filter | No | Yes | No | Yes | No | Yes | No | Yes | |
0.65 | 0.15 | 0.15 | 0.12 | 0.14 | 0.17 | 0.18 | 0.04 | 0.05 | |
0.7 | 0.26 | 0.20 | 0.27 | 0.33 | 0.26 | 0.38 | |||
0.75 | 0.29 | 0.00 | 0.50 | 0.33 | 0.00 |
German RoBERTa | Cross RoBERTa | English RoBERTa | English MiniLM | ||||||
---|---|---|---|---|---|---|---|---|---|
Temporal Lift Filter | No | Yes | No | Yes | No | Yes | No | Yes | |
Rules | 18 | 7 | 18 | 7 | 40 | 20 | 2 | 1 | |
Hits | 7 | 6 | 10 | 8 | 11 | 9 | 1 | 1 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lowin, M. A Text-Based Predictive Maintenance Approach for Facility Management Requests Utilizing Association Rule Mining and Large Language Models. Mach. Learn. Knowl. Extr. 2024, 6, 233-258. https://doi.org/10.3390/make6010013
Lowin M. A Text-Based Predictive Maintenance Approach for Facility Management Requests Utilizing Association Rule Mining and Large Language Models. Machine Learning and Knowledge Extraction. 2024; 6(1):233-258. https://doi.org/10.3390/make6010013
Chicago/Turabian StyleLowin, Maximilian. 2024. "A Text-Based Predictive Maintenance Approach for Facility Management Requests Utilizing Association Rule Mining and Large Language Models" Machine Learning and Knowledge Extraction 6, no. 1: 233-258. https://doi.org/10.3390/make6010013