Mathematical Proposal for Securing Split Learning Using Homomorphic Encryption and Zero-Knowledge Proofs
Abstract
:1. Introduction
Background and Related Work
2. Materials and Methods
2.1. Datasets
2.2. Experimental Setup
3. Mathematical Approach Using Homomorphic Encryption and Zero-Knowledge Proofs
3.1. Homomorphic Encryption for Data Security
3.2. Encrypting Data with Homomorphic Encryption
3.3. Zero-Knowledge Proofs for Verifying Model Integrity
3.4. Zero-Knowledge Proofs for Training
3.5. Server Verification with ZKP
3.6. Rollback in Case of Suspicious Behavior
3.7. Results
3.7.1. Processing Time for Homomorphic Encryption and Model Accuracy
3.7.2. Timing Measurements for Generating and Verifying ZKP Proofs
3.7.3. SafeSplit Performance Against Backdoor Attacks
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A. Implementation Details and Code Snippets
library(openssl) # Key Generation for Homomorphic Encryption generate_keys <- function() { pk <- rsa_keygen() # Generate public and private key pair sk <- as.list(pk)$key # Extract private key return(list(public_key = pk, secret_key = sk)) } # Homomorphic Encryption encrypt_data <- function(data, pk) { # Encrypt the data using the public key return(base64_encode(encrypt_aes(data, pk))) } decrypt_data <- function(enc_data, sk) { # Decrypt the encrypted data using the secret key return(decrypt_aes(base64_decode(enc_data), sk)) } # Client-Side Model Training up to L_cut client_train <- function(X, theta) { # Placeholder for neural network training up to layer L_cut return(X %*% theta) #matrix multiplication as placeholder for model output } | # Server-Side Encrypted Data Processing server_process <- function(Z_enc, theta_s) { # Placeholder for encrypted operations on the data return(Z_enc * theta_s) } # Zero-Knowledge Proof for Client ZKP_prove <- function(data, pk) { # Generate proof (placeholder) return(“client_proof”) } # Zero-Knowledge Proof Verification for Server ZKP_verify <- function(proof, pk) { # Verify proof (placeholder, always returns TRUE for this simulation) return(TRUE) } # Main Split Learning Process with Homomorphic Encryption and ZKP split_learning <- function() { # Generate encryption keys for client and server client_keys <- generate_keys() server_keys <- generate_keys() # Client Data and Initial Model Parameters X_i <- matrix(runif(100), nrow = 10) |
References
- Rechberger, C.; Walch, R. Privacy-Preserving Machine Learning Using Cryptography. In Security and Artificial Intelligence; Springer: Berlin/Heidelberg, Germany, 2022; Volume 13049. [Google Scholar] [CrossRef]
- Kalra, S.; Wen, J.; Cresswell, J.C.; Volkovs, M.; Tizhoosh, H.R. Decentralized federated learning through proxy model sharing. Nat. Commun. 2023, 14, 2899. [Google Scholar] [CrossRef] [PubMed]
- Dhiman, S.; Mahato, G.K.; Chakraborty, S.K. Homomorphic Encryption Library, Framework, Toolkit and Accelerator: A Review. SN Comput. Sci. 2024, 5, 24. [Google Scholar] [CrossRef]
- Choi, H.; Kim, J.; Kim, S.; Park, S.; Park, J.; Choi, W.; Kim, H. UniHENN: Designing Faster and More Versatile Homomorphic Encryption-Based CNNs Without im2col. IEEE Access 2024, 12, 109323–109341. [Google Scholar] [CrossRef]
- Liu, X.; Xie, L.; Wang, Y.; Zou, J.; Xiong, J.; Ying, Z.; Vasilakos, A.V. Privacy and Security Issues in Deep Learning: A Survey. IEEE Access 2021, 9, 4566–4593. [Google Scholar] [CrossRef]
- Gupta, O.; Raskar, R. Distributed learning of deep neural network over multiple agents. J. Netw. Comput. Appl. 2018, 116, 1–8. [Google Scholar] [CrossRef]
- Kaissis, G.A.; Makowski, M.R.; Rückert, D.; Braren, R.F. Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell. 2020, 2, 305–311. [Google Scholar] [CrossRef]
- Gentry, C. Fully Homomorphic Encryption Using Ideal Lattices. In Proceedings of the STOC ‘09: Symposium on Theory of Computing, Bethesda, MD, USA, 31 May–2 June 2009. [Google Scholar]
- Cheon, J.H.; Kim, A.; Kim, M.; Song, Y. Homomorphic Encryption for Arithmetic of Approximate Numbers. In Advances in Cryptology–ASIACRYPT 2017. ASIACRYPT 2017; Springer: Cham, Switzerland, 2017. [Google Scholar]
- Fan, J.; Vercauteren, F. Somewhat Practical Fully Homomorphic Encryption. Cryptol. Eprint Arch. 2012. [Google Scholar]
- Acar, A.; Aksu, H.; Uluagac, A.S.; Conti, M. A Survey on Homomorphic Encryption Schemes: Theory and Implementation. ACM Comput. Surv. 2018, 51, 1–35. [Google Scholar] [CrossRef]
- Goldwasser, S.; Micali, S. Probabilistic Encryption and Zero-Knowledge Proofs; ACM: New York, NY, USA, 1984. [Google Scholar]
- Ben-Sasson, E.; Chiesa, A.; Tromer, E.; Virza, M. Succinct Non-Interactive Zero-Knowledge for a von Neumann Architecture. In Proceedings of the 23rd USENIX Security Symposium (USENIX Security 14), San Diego, CA, USA, 20–22 August 2014. [Google Scholar]
- Groth, J. Short Non-Interactive Zero-Knowledge Proofs. In Advances in Cryptology-ASIACRYPT 2010: 16th International Conference on the Theory and Application of Cryptology and Information Security, Singapore, 5–9 December 2010. Proceedings 16; Springer: Berlin/Heidelberg, Germany, 2010. [Google Scholar]
- Yang, Q.; Liu, Y.; Chen, T.; Tong, Y. Federated Machine Learning: Concept and Applications. ACM Trans. Intell. Syst. Technol. 2019, 10, 1–19. [Google Scholar] [CrossRef]
- Vepakomma, P.; Gupta, O.; Swedish, T.; Raskar, R. Split Learning for Health: Distributed Deep Learning without Sharing Raw Patient Data. arXiv 2018, arXiv:1812.00564. [Google Scholar]
- Thapa, C.; Arachchige, M.A.P.C.; Camtepe, S.A. Advancements of Federated Learning Towards Privacy Preservation: From Federated Learning to Split Learning. arXiv 2021, arXiv:2011.14818. [Google Scholar]
- Singh, A.; Vepakomma, P.; Gupta, O.; Raskar, R. Detailed Comparison of Communication Efficiency of Split Learning and Federated Learning. arXiv 2019, arXiv:1909.09145. [Google Scholar]
- He, Y.; Shen, Z.; Hua, J.; Dong, Q. Backdoor Attack against Split Learning-Based Vertical Federated Learning. IEEE Trans. Inf. Forensics Secur. 2023, 19, 748–763. [Google Scholar] [CrossRef]
- Yu, F.; Wang, L.; Zeng, B.; Pang, Z.; Wu, T. How to Backdoor Split Learning. Neural Netw. 2023, 168, 326–336. [Google Scholar] [CrossRef]
- Kariyappa, S.; Qureshi, M.K. Exploit: Extracting Private Labels in Split Learning. In Proceedings of the 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), Raleigh, NC, USA, 8–10 February 2023. [Google Scholar]
- Dwork, C.; Roth, A. The Algorithmic Foundations of Differential Privacy. Found. Trends Theor. Comput. Sci. 2014, 9, 211–407. [Google Scholar] [CrossRef]
- Abadi, M.; Chu, A.; Goodfellow, I.; McMahan, H.B.; Mironov, I.; Talwar, K.; Zhang, L. Deep Learning with Differential Privacy; ACM: New York, NY, USA, 2016. [Google Scholar]
- Bonawitz, K.; Ivanov, V.; Kreuter, B.; Marcedone, A.; McMahan, H.B.; Patel, S.; Ramage, D.; Segal, A.; Seth, K. Practical Secure Aggregation for Privacy-Preserving Machine Learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, USA, 30 October–3 November 2017. [Google Scholar]
- Bagdasaryan, E.; Veit, A.; Hua, Y.; Estrin, D.; Shmatikov, V. How to Backdoor Federated Learning. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR, Online, 26–28 August 2020. [Google Scholar]
- Blanchard, P.; Mhamdi, E.M.; Guerraoui, R.; Stainer, J. Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent. Adv. Neural Inf. Process. Syst. 2017, 30. [Google Scholar]
- Roux, C.; Zimmer, M.; Pokutta, S. On the Byzantine-resilience of distillation-based federated learning. arXiv 2024, arXiv:2402.12265. [Google Scholar]
- Cao, X.; Jia, J.; Gong, N.Z. Provably Secure Federated Learning Against Malicious Clients. Proc. AAAI Conf. Artif. Intell. 2021, 35, 6885–6893. [Google Scholar] [CrossRef]
- Bhagoji, A.N.; Chakraborty, S.; Mittal, P.; Calo, S. Analyzing Federated Learning Through an Adversarial Lens. In Proceedings of the 2019 International Conference on Machine Learning, Long Beach, CA, USA, 10–15 June 2019. [Google Scholar]
- Erdogan, E.; Kupcu, A.; Cicek, A.E. SafeSplit: Detecting and Mitigating Training-Hijacking Attacks in Split Learning. In Proceedings of the CCS ’22: Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, Los Angeles, CA, USA, 7–11 November 2022. [Google Scholar]
- Yu, F.; Zeng, B.; Zhao, K.; Pang, Z.; Wang, L. Chronic Poisoning: Backdoor Attack against Split Learning. Proc. AAAI Conf. Artif. Intell. 2024, 38, 16531–16538. [Google Scholar] [CrossRef]
- Gao, X.; Zhang, L. PCAT: Functionality and Data Stealing from Split Learning by Pseudo-Client Attack. In Proceedings of the 2023 USENIX Annual Technical Conference, Boston, MA, USA, 10–12 July 2023. [Google Scholar]
- Ahmed, N.; Natarajan, T.; Rao, K.R. Discrete Cosine Transform. IEEE Trans. Comput. 1974, 100, 90–93. [Google Scholar] [CrossRef]
- Rahaman, N.; Baratin, A.; Arpit, D. On the Spectral Bias of Neural Networks. In Proceedings of the 2019 International Conference on Machine Learning, Long Beach, CA, USA, 10–15 June 2019. [Google Scholar]
- Xu, Z.Q.J.; Zhang, Y.; Xiao, Y. Training Behavior of Deep Neural Network in Frequency Domain. In Proceedings of the 2019 Conference on Neural Information Processing Systems, Vancouver, BC, Canada, 8–14 December 2019. [Google Scholar]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going Deeper with Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015. [Google Scholar]
- Munoz-Gonzalez, L.; Co, K.T.; Lupu, E.C. Byzantine-Robust Federated Machine Learning Through Adaptive Model Averaging. arXiv 2019, arXiv:1909.05125. [Google Scholar]
- Bai, Y.; Chen, Y.; Zhang, H.; Xu, W.; Weng, H.; Goodman, D. VILLAIN: Backdoor Attacks Against Vertical Split Learning. In Proceedings of the 2023 USENIX Annual Technical Conference, Boston, MA, USA, 10–12 July 2023. [Google Scholar]
- Pasquini, D.; Ateniese, G.; Bernaschi, M. Unleashing the Tiger: Inference Attacks on Split Learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, Virtual, 15–19 November 2021. [Google Scholar]
- Shumailov, I.; Shumaylov, Z.; Kazhdan, D.; Zhao, Y.; Papernot, N.; Erdogdu, M.A.; Anderson, R.J. Manipulating SGD with Data Ordering Attacks. In Proceedings of the 2021 Conference on Neural Information Processing Systems, Virtual, 6–19 December 2021. [Google Scholar]
- Nguyen, T.; Xu, D.; Thai, M.T. Attacking Federated Learning Systems by Injecting Invisible Backdoors. In Proceedings of the 2021 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Virtual, 9–12 May 2021; pp. 1–6. [Google Scholar]
- Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P. Language Models are Few-Shot Learners. Adv. Neural Inf. Process. Syst. 2020, 33, 1877–1901. [Google Scholar]
- Sun, Z.; Cao, X.; Yu, W.; Zhang, T. Local Differential Privacy for Federated Learning and Split Learning. J. Cryptol. 2022, 14. [Google Scholar]
- Zhang, Y.; Cisse, M.; Dauphin, Y.N.; Lopez-Paz, D. mixup: Beyond Empirical Risk Minimization. In Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada, 30 April–3 May 2018. [Google Scholar]
- Gentry, C.; Halevi, S.; Vaikuntanathan, V. i-hop Homomorphic Encryption and Its Applications. In Proceedings of the 30th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Tallinn, Estonia, 15–19 May 2011. [Google Scholar]
- Konecny, J.; McMahan, H.B.; Yu, F.X.; Richtarik, P. Federated Learning: Strategies for Improving Communication Efficiency. arXiv 2016, arXiv:1610.05492. [Google Scholar]
- Erlinghagen, S.; Sachdeva, S.; Lauter, K. Privacy-Enhancing Machine Learning in Healthcare: Trends and Implications. IEEE Access 2020, 8, 120295–120310. [Google Scholar]
- Zhang, Z.; Luo, T.; Peng, Z. Privacy-preserving Machine Learning Techniques for Image Processing: A Survey. IEEE Trans. Image Process. 2019, 28, 6109–6121. [Google Scholar]
- Kim, M.; Song, W.; Shim, J. Trustworthy AI for Collaborative Learning. J. Artif. Intell. Res. 2023, 56, 98–111. [Google Scholar]
- Pascal, J. Applications of Homomorphic Encryption in Biometric Systems. IEEE Trans. Inf. Forensics Secur. 2019, 14, 1127–1139. [Google Scholar]
- Shokri, R.; Stronati, M.; Song, C.; Shmatikov, V. Membership Inference Attacks Against Machine Learning Models. In Proceedings of the IEEE Symposium on Security and Privacy 2017, San Jose, CA, USA, 22–26 May 2017; pp. 3–18. [Google Scholar]
- Hardy, S.; Smith, J.; Jones, D. Advances in Privacy-Preserving Federated Learning with Multi-Party Computation. ACM Trans. Priv. Secur. 2023, 25. [Google Scholar]
- Zhao, H.; Gu, J.; Yan, W. Differential Privacy in Machine Learning: Advances and Applications. IEEE Trans. Big Data 2020, 8, 234–248. [Google Scholar]
- Dhillon, M.; Raj, K.; Verma, P. Secure Aggregation Protocols for Federated Learning. In Proceedings of the 15th ACM Symposium on Applied Computing, Virtual, 22–26 March 2021. [Google Scholar]
- Papernot, N.; Abadi, M.; Erlingsson, U.; Goodfellow, I.; Talwar, K. Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data. In Proceedings of the International Conference on Learning Representations, Toulon, France, 24–26 April 2017. [Google Scholar]
- Hsu, T.; Qi, H.; Brown, B. Measuring the Robustness of Split Learning. Proceedings of 2022 Conference on Neural Information Processing Systems, New Orleans, LA, USA, 28 November–9 December 2022. [Google Scholar]
- Goldreich, O.; Micali, S.; Wigderson, A. How to Play any Mental Game or a Completeness Theorem for Protocols with Honest Majority. In Proceedings of the 19th Annual ACM Symposium on Theory of Computing, New York, NY, USA, 25–27 May 1987. [Google Scholar]
- Wu, Z.; Lin, H.; Li, Z. Layer-wise Gradient Manipulation for Robust Distributed Learning. IEEE Trans. Neural Netw. Learn. Syst. 2024. [Google Scholar]
- Baruch, G.; Baruch, B.; Bar-Or, A. Backdoor Attacks on Federated Learning: Analysis and Defenses. IEEE Trans. Neural Netw. Learn. Syst. 2021. [Google Scholar]
- Rivest, R.L.; Shamir, A.; Adleman, L. A Method for Obtaining Digital Signatures and Public-Key Cryptosystems. Commun. ACM 1978, 21, 120–126. [Google Scholar] [CrossRef]
- Wang, X.; Ma, T.; Cui, W. Efficient Zero-Knowledge Proof Protocols for Privacy-Preserving Machine Learning. IEEE Trans. Knowl. Data Eng. 2021, 33, 2975–2987. [Google Scholar]
Feature | Federated Learning | Split Learning | Layered Learning |
---|---|---|---|
Data Sharing | Model updates only | Encrypted intermediate activations | Layer-wise model sharing |
Privacy | Medium | High (encrypted computations) | Medium |
Security Risks | Backdoor attacks | Backdoor attacks, adversarial modifications | Model leakage |
Computational Load | Heavy on clients | Shared between clients and server | Moderate |
Application in HavenSL | Not directly applied | Primary focus | Limited |
Dataset | Model | Encryption Time (s/image) | Processing Time (s/image) | Main Task Accuracy (MA) (%) |
---|---|---|---|---|
CIFAR-10 | ResNet-18 | 5.8 | 13.5 | 87 |
MNIST | Simple CNN | 3.2 | 10.8 | 98 |
Fashion-MNIST | Simple CNN | 4.0 | 11.3 | 95 |
Dataset | Model | ZKP Generation Time (s/epoch) | ZKP Verification Time (s/epoch) (s/image) |
---|---|---|---|
CIFAR-10 | ResNet-18 | 4.8 | 3.5 |
MNIST | Simple CNN | 2.5 | 2.0 |
Fashion-MNIST | Simple CNN | 3.0 | 2.5 |
Dataset | Model | Rollback Count | Backdoor Accuracy (BA) (%) | Detection Efficiency (%) |
---|---|---|---|---|
CIFAR-10 | ResNet-18 | 4 | 9 | 96 |
MNIST | Simple CNN | 2 | 5 | 98 |
Fashion-MNIST | Simple CNN | 3 | 7 | 97 |
Experiment | Training Time (s) | Communication Overhead (%) | Cryptographic Overhead (%) |
---|---|---|---|
Standard Split Learning | 340 | 15 | 0 |
HavenSL (HE + ZKP) | 390 | 18 | 7 |
HavenSL (HE + ZKP + DCT) | 420 | 20 | 10 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kokaj, A.; Mollakuqe, E. Mathematical Proposal for Securing Split Learning Using Homomorphic Encryption and Zero-Knowledge Proofs. Appl. Sci. 2025, 15, 2913. https://doi.org/10.3390/app15062913
Kokaj A, Mollakuqe E. Mathematical Proposal for Securing Split Learning Using Homomorphic Encryption and Zero-Knowledge Proofs. Applied Sciences. 2025; 15(6):2913. https://doi.org/10.3390/app15062913
Chicago/Turabian StyleKokaj, Agon, and Elissa Mollakuqe. 2025. "Mathematical Proposal for Securing Split Learning Using Homomorphic Encryption and Zero-Knowledge Proofs" Applied Sciences 15, no. 6: 2913. https://doi.org/10.3390/app15062913
APA StyleKokaj, A., & Mollakuqe, E. (2025). Mathematical Proposal for Securing Split Learning Using Homomorphic Encryption and Zero-Knowledge Proofs. Applied Sciences, 15(6), 2913. https://doi.org/10.3390/app15062913