Optimization of XOR Cryptographic Keys using a Hybrid Genetic Algorithm and Simulated Annealing

Naufal Muzakki, Nur Rochmah Dyah Puji Astuti

Abstract


The security level of basic encryption algorithms, such as XOR, is highly dependent on the randomness and bit distribution pattern of the applied key. The use of stochastic optimization approaches, such as Genetic Algorithm (GA), in key generation often faces challenges due to premature convergence, a condition in which the search halts at a local optimum before achieving maximal entropy. This study proposes a sequential hybrid algorithm strategy that combines GA with Simulated Annealing (SA) to address fitness stagnation in PDF document encryption. The strategy is implemented through a two-phase mechanism: GA performs global exploration to identify potential solution regions, followed by SA performing local exploitation with a perturbation mechanism guided by the Metropolis probability. The algorithm’s performance is evaluated through a comparative study between conventional GA and the hybrid GA-SA. Experimental results indicate that the hybrid strategy successfully increases the average fitness value by 4.86%, achieves a Shannon entropy of 7.8952, and attains an NIST test P-value of 0.5299. These improvements demonstrate that the integration of SA effectively enhances the final solution quality of GA, producing cryptographic keys with more uniform bit distribution, passing statistical randomness tests, and exhibiting robustness against pattern analysis.

Keywords


genetic algorithm; key optimization; shannon entropy; simulated annealing; XOR cryptography

Full Text:

PDF

References


K. Khairani and M. Z. Siambaton, “Pengamanan Data Teks menggunakan Algoritma Kriptografi Elgamal dan XOR dari Serangan Hacker,” Sudo J. Tek. Inform., Vol. 2, No. 4, pp. 176–187, Dec. 2023, DOI: 10.56211/sudo.v2i4.401.

F. S. Febriyani and A. Arfriandi, “Implementasi Algoritma RC4 pada Sistem Pengamanan Dokumen Digital Soal Ujian,” JISKA (Jurnal Inform. Sunan Kalijaga), Vol. 6, No. 3, pp. 171–177, Sep. 2021, DOI: 10.14421/jiska.2021.6.3.171-177.

B. D. Kurniawan, M. A. Rosid, I. A. Kautsar, and N. E. Pratama, “Rancang Bangun Library Web Token untuk Enkripsi HTTP Data menggunakan Eksklusif-OR (XOR),” Phys. SCI. Life SCI. Eng., Vol. 1, No. 1, p. 14, Jan. 2024, DOI: 10.47134/pslse.v1i1.164.

D. Jonas, I. Sembiring, A. Setiawan, D. Manongga, I. R. Widiasari, and D. Julianingsih, “XOR Encryption based on Index Value Enumeration used in IoT based Healthcare,” in 2023 11th International Conference on Cyber and IT Service Management (CITSM), IEEE, Nov. 2023, pp. 1–5. DOI: 10.1109/CITSM60085.2023.10455527.

P. Simamora, V. Verawijaya, and S. A. Pasaribu, “Kunci Simetris pada Perangkat IoT: Kajian Literatur,” J. Sist. Inf. dan Teknol. Jar., Sep. 2024, DOI: 10.63703/sisfotekjar.v5i2.62.

M. D. Asrofa, S. Bahri, and K. Kasliono, “One-Time Pad Cryptography for Secure Data Transmission in IoT Smart Door using QR Code,” J. Media Inf. Teknol., Vol. 2, No. 2, pp. 133–148, Oct. 2025, DOI: 10.69616/mit.v2i2.248.

M. T. Gaata, Z. O. Ahmed, and R. S. Ali, “Secure Big Data Transmission based on Modified Reverse Encryption and Genetic Algorithm,” Iraqi J. SCI., pp. 439–451, Jan. 2023, DOI: 10.24996/ijs.2023.64.1.39.

R. B. Abduljabbar, O. K. Hamid, and N. J. Alhyani, “Features of Genetic Algorithm for Plain Text Encryption,” Int. J. Electr. Comput. Eng., Vol. 11, No. 1, p. 434, Feb. 2021, DOI: 10.11591/ijece.v11i1.pp434-441.

P. Bagane and D. K. S., “Enriching AES Through the Key Generation from Genetic Algorithm,” Indian J. Comput. SCI. Eng., Vol. 12, No. 4, pp. 955–963, Aug. 2021, DOI: 10.21817/indjcse/2021/v12i4/211204141.

S. S. Sultana, T. Tanabe, T. Fausten, and M. Irie, “Avoiding Premature Convergence to Local Optima with Adaptive Exploration for Genetic Algorithms,” in GECCO 2025 Companion - Proceedings of the 2025 Genetic and Evolutionary Computation Conference Companion, New York, NY, USA: ACM, Jul. 2025, pp. 539–542. DOI: 10.1145/3712255.3726640.

B. A. Kumar, B. Jyothi, A. R. Singh, M. Bajaj, R. S. Rathore, and M. B. Tuka, “Hybrid Genetic Algorithm-Simulated Annealing based Electric Vehicle Charging Station Placement for Optimizing Distribution Network Resilience,” SCI. Rep., Vol. 14, No. 1, p. 7637, Apr. 2024, DOI: 10.1038/s41598-024-58024-8.

M. A. Rahman, M. J. N. Mim, S. Afrin, A. Islam, and R. Ahmed, “Simulated Annealing Optimization Algorithm with Self-Escape Mechanism for Travelling Salesman Problem,” Saudi J. Eng. Technol., Vol. 10, No. 05, pp. 236–242, May 2025, DOI: 10.36348/sjet.2025.v10i05.003.

P. Hidayatullah, I. Irwansyah, Q. A, and B. N. Syechah, “Pipeline Network Optimization using Hybrid Algorithm between Simulated Annealing and Genetic Algorithms,” Eig. Math. J., pp. 30–39, Jan. 2022, DOI: 10.29303/emj.v4i2.100.

W. Yu, Y. Huang, Y. Yang, and A. Garcia-Ortiz, “Hybrid Genetic Algorithm Combng Simulated Annealing for Task Allocation with Data Security,” in 2023 19th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT), IEEE, Jun. 2023, pp. 134–141. doi: 10.1109/DCOSS-IoT58021.2023.00033.

G. Singla and G. Kaur, “New Enhanced Hybrid Cryptographic Algorithms in Cloud Network,” in 2023 1st International Conference on Advances in Electrical, Electronics and Computational Intelligence, ICAEECI 2023, IEEE, Oct. 2023, pp. 1–6. DOI: 10.1109/ICAEECI58247.2023.10370940.

R. Reymond, Johanes Manullang, Jhoische Tamba, Farel Parasian Sitohang, and Eikel Nioisha Ginting, “Cryptography With One-Time Pad (OTP) Algorithm Xor Based,” J. Tek. Indones., Vol. 3, No. 02, pp. 54–60, Nov. 2024, DOI: 10.58471/ju-ti.v3i02.664.

R. A. Q. and B. S. Mahdi, “Image Protection using Genetic Algorithm and Cipher Technique,” Iraqi J. Comput. Commun. Control Syst. Eng., pp. 160–166, Dec. 2022, DOI: 10.33103/uot.ijccce.22.4.13.

A. A. Hussein and N. K. Ayoob, “Key Generation for Vigenere Ciphering based on Genetic Algorithm,” J. Univ. BABYLON Pure Appl. SCI., pp. 200–208, Mar. 2022, DOI: 10.29196/jubpas.v30i1.4134.

P. Mukherjee, H. Garg, C. Pradhan, S. Ghosh, S. Chowdhury, and G. Srivastava, “Best Fit DNA-based Cryptographic Keys: The Genetic Algorithm Approach,” Sensors, Vol. 22, No. 19, p. 7332, Sep. 2022, DOI: 10.3390/s22197332.

S. Juniawan, U. Roysen, F. Alam, Z. Zainuddin, D. Daruki, and H. Taher, “Optimization of Flight Routes Employing the Simulated Annealing Method in the Context of the Indonesian National Airline Industry,” IJIEM - Indones. J. Ind. Eng. Manag., Vol. 5, No. 3, p. 691, Mar. 2025, DOI: 10.22441/ijiem.v5i3.21748.

Y. Pyrih, M. Klymash, M. Kaidan, and B. Strykhalvuk, “Investigating the Efficiency of Tournament Selection Operator in Genetic Algorithm for Solving TSP,” in 5th IEEE International Conference on Advanced Information and Communication Technologies, AICT 2023 - Proceedings, IEEE, Nov. 2023, pp. 170–173. DOI: 10.1109/AICT61584.2023.10452423.

I. W. Supriana, M. A. Raharja, I. M. S. Bimantara, and D. Bramantya, “Implementasi Dua Model Crossover pada Algoritma Genetika untuk Optimasi Penggunaan Ruang Perkuliahan,” J. Resist. (Rekayasa Sist. Komputer), Vol. 4, No. 2, pp. 167–177, Oct. 2021, DOI: 10.31598/jurnalresistor.v4i2.758.

P. Larrañaga, C. M. H. Kuijpers, R. H. Murga, I. Inza, and S. Dizdarevic, “Genetic Algorithms for the Travelling Salesman Problem: A Review of Representations and Operators,” Artif. Intell. Rev., Vol. 13, No. 2, pp. 129–170, Apr. 1999, DOI: 10.1023/A:1006529012972.

M. Mitchell, An Introduction to Genetic Algorithms. The MIT Press, 1996. DOI: 10.7551/mitpress/3927.001.0001.

A. Hemmak, “Optimal Adjusting of Simulated Annealing Parameters,” Vojnoteh. Glas., Vol. 72, No. 1, pp. 80–93, 2024, DOI: 10.5937/vojtehg72-47242.

D. Henderson, S. H. Jacobson, and A. W. Johnson, “The Theory and Practice of Simulated Annealing,” in Handbook of Metaheuristics, Boston: Kluwer Academic Publishers, 2006, pp. 287–319. DOI: 10.1007/0-306-48056-5_10.

J. Jasmani and A. Mahmudi, “Optimalisasi Jalur Terpendek menggunakan Algoritma Genetika,” J-INTECH, Vol. 11, No. 1, pp. 128–139, Jul. 2023, DOI: 10.32664/j-intech.v11i1.809.

P. B. Utomo and W. Setiafindari, “Optimasi Penjadwalan Produksi menggunakan Metode Simulated Annealing di Industri XYZ,” Borobudur Eng. Rev., Vol. 1, No. 1, pp. 49–55, Mar. 2021, DOI: 10.31603/benr.4610.

D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Vol. 27, No. 02. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1989. DOI: 10.5860/choice.27-0936.

C. E. Shannon, “A Mathematical Theory of Communication,” Bell Syst. Tech. J., Vol. 27, No. 3, pp. 379–423, Jul. 1948, DOI: 10.1002/j.1538-7305.1948.tb01338.x.

A. Rukhin, J. Soto, and J. Nechvatal, “A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications,” Gaithersburg, MD, 2010. DOI: 10.6028/NIST.SP.800-22r1a.




DOI: https://doi.org/10.32520/stmsi.v15i3.6170

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.