Integrating Multimodal Generative AI and Blockchain for Enhancing Generative Design in the Early Phase of Architectural Design Process
Abstract
:1. Introduction
2. Materials and Methods
- Research design: The hypothesis posits that combining generative AI with blockchain technology enhances architectural design by ensuring secure data storage and traceability. The research followed a structured sequence, progressing from design conceptualisation to implementing blockchain technology.
- Material selection: Generative AI tools, such as Midjourney, were selected for their ability to generate diverse designs using both text and image inputs. The Ethereum blockchain was chosen for its secure NFT-based storage capabilities. Architectural datasets were sourced from online repositories, focusing on incorporating diverse styles.
- Procedure: In the design phase, initial sketches and text inputs were fed into AI for design generation, with iterative refinement using blending and upscaling features. For blockchain integration, design metadata were created and stored as NFTs on Ethereum, utilising a Java application for uploading and converting metadata on Firebase.
- Result: Documented outcomes with metadata linking each AI-generated design to its NFT, verifying authenticity and traceability and evaluating supported by relevant papers, highlighting the innovative use of AI in design and the critical role of blockchain in protecting intellectual property and ensuring data integrity.
3. Results
3.1. AI Serves as a Creative Catalyst for Multimodal Design Generation
3.2. Blockchain Provides Methods of Verifying and Tracing the Authenticity of AI–Human Generative Design
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
3DI | 3-Dimension Invariant |
AECO | Architecture, Engineering, Construction, and Operation. |
CGAN | Conditional Generative Adversarial Network |
CLIP | Contrastive Language-Image Pre-Training |
dApp | Decentralied Application |
Dde | Data-driven evaluator |
ERC721 | Ethereum Request for Comment 721 |
FER | Facial Expression Recognition |
GAI | Generative Artificial Intelligence |
GAIS | Generative Artificial Intelligence System |
GAN | Generative Adversarial Network |
GenAIVA | Generative Artificial Intelligence for Virtual Avatar |
GPT | Generative Pre-trained Transformer |
ID | Identifier |
JSON | JavaScript Object Notation |
JobID | Job Identifier |
LLM | Large Language Model |
MMAIR | Multimodal Artificial Intelligence Recognition |
MML | Multimodal Machine Learning |
NFT | non-fungible token |
NLP | Natural Language Processing |
U-Net | U-shaped Convolutional Neural Network |
URI | Uniform Resource Identifier |
URL | Uniform Resource Locators |
URN | Uniform Resource Names |
UUID | Universally Unique Identifier |
References
- Caetano, I.; Santos, L.; Leitão, A. Computational design in architecture: Defining parametric, generative, and algorithmic design. Front. Arch. Res. 2020, 9, 287–300. [Google Scholar] [CrossRef]
- Ma, W.; Wang, X.; Wang, J.; Xiang, X.; Sun, J. Generative design in building information modelling (BIM): Approaches and requirements. Sensors 2021, 21, 5439. [Google Scholar] [CrossRef] [PubMed]
- Zheng, H.; Yuan, P.F. A generative architectural and urban design method through artificial neural networks. Build. Environ. 2021, 205, 108178. [Google Scholar] [CrossRef]
- Liao, W.; Lu, X.; Fei, Y.; Gu, Y.; Huang, Y. Generative AI design for building structures. Autom. Constr. 2024, 157, 105187. [Google Scholar] [CrossRef]
- Dreith, B. How AI Software Will Change Architecture and Design; Dezeen Limited. Available online: https://www.dezeen.com/2022/11/16/ai-design-architecture-product/ (accessed on 15 April 2024).
- Yıldırım, E. Comparative Analysis Of Leonardo Ai, Midjourney, And Dall-E: AI’s Perspective On Future Cities. Urbanizm 2023, 28, 82–96. [Google Scholar] [CrossRef]
- Agkathidis, A.; Song, Y.; Symeonidou, I. AI-Assisted Design: Utilising artificial intelligence as a generative form-finding tool in architectural design studio teaching. Presented at the eCAADe 2024, Leukosia, Cyprus, 9–13 September 2024; Available online: https://www.researchgate.net/publication/381917897_AI-Assisted_Design_Utilising_artificial_intelligence_as_a_generative_form-finding_tool_in_architectural_design_studio_teaching (accessed on 15 July 2024).
- Milošević, J.; Đukanović, L.; Zivkovic, M.; Žujović, M.; Gavrilović, M. Automated Compositions: Artificial Intelligence Aided Conceptual Design Explorations in Architecture. Presented at the 9th International Scientific Conference on Geometry and Graphics MONGEOMETRIJA 2023, Novi Sad, Serbia, 7–10 June 2023; pp. 103–115. Available online: https://raf.arh.bg.ac.rs/handle/123456789/1282 (accessed on 15 July 2024).
- Almaz, A.; El-Agouz, E.; Abdelfatah, M.; Raafat, I. The Future Role of Artificial Intelligence (AI) Design’s Integration into Architectural and Interior Design Education is to Improve Efficiency, Sustainability, and Creativity. Civ. Eng. Archit. 2024, 12, 1749–1772. [Google Scholar] [CrossRef]
- Gao, J.; Fischer, M. Framework and Case Studies Comparing Implementations and Impacts of 3D/4D Modeling across Projects, Citeseer. 2008. Available online: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=358d76d9a14294eb5bef59da664c5f1ed6409438 (accessed on 15 July 2024).
- Ploennigs, J.; Berger, M. AI art in architecture. AI Civ. Eng. 2023, 2, 8. [Google Scholar] [CrossRef]
- Graikos, A.; Yellapragada, S.; Le, M.-Q.; Kapse, S.; Prasanna, P.; Saltz, J.; Samaras, D. Learned representation-guided diffusion models for large-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition) 2024, Seattle, WA, USA, 17–21 June 2024; pp. 8532–8542. Available online: https://openaccess.thecvf.com/content/CVPR2024/papers/Graikos_Learned_Representation-Guided_Diffusion_Models_for_Large-Image_Generation_CVPR_2024_paper.pdf (accessed on 15 July 2024).
- Litan, A. Why Trust and Security Are Essential for the Future of Generative AI; Gartner Inc.: Stamford, CT, USA, 2023; Available online: https://www.gartner.com/en/newsroom/press-releases/2023-04-20-why-trust-and-security-are-essential-for-the-future-of-generative-ai (accessed on 15 July 2024).
- Reichwein, F.L. Ethical and Societal Implications of Generative AI-Models. Bachelor’s Thesis, Technische Hochschule Ingolstadt, Ingolstadt, Germany, 2024. Available online: https://opus4.kobv.de/opus4-haw/frontdoor/index/index/year/2024/docId/4694 (accessed on 15 January 2024).
- AI. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile; NIST Trustworthy and Responsible AI: Gaithersburg, MD, USA, 2024. [Google Scholar] [CrossRef]
- Carvalko, J.R. Generative AI, Ingenuity, and Law. IEEE Trans. Technol. Soc. 2024, 5, 169–182. [Google Scholar] [CrossRef]
- Samuelson, P. Generative AI meets copyright. Science 2023, 381, 158–161. [Google Scholar] [CrossRef]
- Hosseinzadeh Kassani, S. Towards Secure and Intelligent Diagnosis: Deep Learning and Blockchain Technology for Computer-Aided Diagnosis Systems. Ph.D. Thesis, University of Saskatchewan, Saskatoon, SK, Canada, 2021. Available online: https://harvest.usask.ca/bitstream/10388/13302/1/HOSSEINZADEHKASSANI-DISSERTATION-2021.pdf (accessed on 15 January 2024).
- Mohsan, S.A.H.; Razzaq, A.; Ghayyur, S.A.K.; Alkahtani, H.K.; Al-Kahtani, N.; Mostafa, S.M. Decentralised patient-centric report and medical image management system based on blockchain technology and the inter-planetary file system. Int. J. Environ. Res. Public Health 2022, 19, 14641. [Google Scholar] [CrossRef]
- Nawari, N.O.; Ravindran, S. Blockchain technology and BIM process: Review and potential applications. J. Inf. Technol. Constr. 2019, 24, 209–238. Available online: https://www.itcon.org/paper/2019/12 (accessed on 15 January 2024).
- Lu, W.; Wu, L. A blockchain-based deployment framework for protecting building design intellectual property rights in collaborative digital environments. Comput. Ind. 2024, 159, 104098. [Google Scholar] [CrossRef]
- Kifokeris, D.; Dounas, T.; Tezel, A.; Moon, S. What is the potential value of tokens and token engineering for the architecture, engineering, and construction industry? A positional paper. In Proceedings of the 2023 European Conference on Computing in Construction (EC3 2023) co-located with the 40th International CIB (International Council for Research and Innovation in Building and Construction) conference (CIB W78 2023), Heraklion, Greece, 10–12 July 2023. [Google Scholar] [CrossRef]
- Dounas, T.; Jabi, W.; Lombardi, D. Non-fungible building components: Using smart contracts for a circular economy in the built environment. Blucher Des. Proc. 2021, 9, 1189–1198. [Google Scholar] [CrossRef]
- Naderi, H.; Ly, R.; Shojaei, A. From Data to Value: Introducing an NFT-Powered Framework for Data Exchange of Digital Twins in the AEC Industry. In Proceedings of the Construction Research Congress 2024, Des Moines, IA, USA, 20–23 March 2024; pp. 299–308. [Google Scholar] [CrossRef]
- Fitriawijaya, A.; Taysheng, J. Multimodal Generative AI and NFT Metadata, Version 1; National Cheng Kung University: Taipei, Taiwan, 2024. [Google Scholar] [CrossRef]
- Golden, A.; Hsia, S.; Sun, F.; Acun, B.; Hosmer, B.; Lee, Y.; Devito, Z.; Johnson, J.; Wei, G.-Y.; Brooks, D. Generative AI beyond LLMs: System Implications of Multimodal Generation. arXiv 2023, arXiv:2312.14385. [Google Scholar] [CrossRef]
- Hariri, W. Unlocking the potential of ChatGPT: A comprehensive exploration of its applications, advantages, limitations, and future directions in natural language processing. arXiv 2023, arXiv:2304.02017. [Google Scholar] [CrossRef]
- Del Castillo, A.P. AI: Discovering the Many Faces of a Faceless Technology; ETUI aisbl: Brussels, Belgium, 2023; Available online: https://www.etui.org/sites/default/files/2023-05/AI-Guide-discovering%20the%20many%20faces%20of%20a%20faceless%20technology-2023.pdf (accessed on 15 January 2024).
- Ma, S.Y. Exploring Ambiguity in Generative AI Images and Its Impact on Collaborative Design Ideation. Master’s Thesis, Industrial Engineering and Innovation Sciences, Eindhoven University of Technology, Eindhoven, The Netherlands, 2024. Available online: https://pure.tue.nl/ws/portalfiles/portal/320754968/MTP_thesis_report_Sherry_Ma.pdf (accessed on 15 January 2024).
- Jaruga-Rozdolska, A. Artificial intelligence as part of future practices in the architect’s work: MidJourney generative tool as part of a process of creating an architectural form. Architectus 2022, 3, 95–104. [Google Scholar] [CrossRef]
- Meeran, A. AI and Architecture: Image-Based Machine Learning for Early-Stage Design Conceptualisation. Doctoral Dissertation, Singapore University of Technology and Design, Singapore, 2021. [Google Scholar]
- Basole, R.C.; Major, T. Generative AI for Visualization: Opportunities and Challenges. IEEE Comput. Graph. Appl. 2024, 44, 55–64. [Google Scholar] [CrossRef]
- Harreis, H.; Koullias, T.; Roberts, R.; Te, K. Generative AI: Unlocking the Future of Fashion; McKinsey Company: New York, NY, USA, 2023; Available online: https://digital-humanai.io/wp-content/uploads/2023/03/Generative-AI-Unlocking-the-future-of-fashion.pdf (accessed on 15 January 2024).
- Zhong, C.; Yi’an Shi, L.H.C.; Wang, L. AI-enhanced performative building design optimisation and exploration. Presented at the 29th International Conference on Computer-Aided Architectural Design Research in Asia, CAADRIA 2024, Singapore, 23–25 April 2024; The Association for Computer-Aided Architectural Design Research in Asia (CAADRIA): Hong Kong, China, 2024; Volume 1. Available online: https://papers.cumincad.org/data/works/att/caadria2024_15.pdf (accessed on 15 January 2024).
- Bstieler, L.; Noble, C.H. The PDMA Handbook of Innovation and New Product Development; John Wiley & Sons: Hoboken, NJ, USA, 2023. [Google Scholar]
- Li, C.; Zhang, T.; Du, X.; Zhang, Y.; Xie, H. Generative AI for Architectural Design: A Literature Review. arXiv 2024, arXiv:2404.01335. [Google Scholar] [CrossRef]
- Zhang, K.; Cai, S.; Yang, W.; Wu, W.; Shen, H. Exploring Optimal Combinations: The Impact of Sequential Multimodal Inspirational Stimuli in Design Concepts on Creativity. In Proceedings of the 2024 ACM Designing Interactive Systems Conference, Copenhagen, Denmark, 1–5 July 2024; IT University of Copenhagen, Denmark Association for Computing Machinery: New York, NY, USA, 2024; pp. 2788–2801. [Google Scholar] [CrossRef]
- Ochoa, K.S. Can Artificial Intelligence Mark the Next Architectural Revolution? Design Exploration in the Realm of Generative Algorithms and Search Engines; Springer: Berlin/Heidelberg, Germany, 2024. [Google Scholar] [CrossRef]
- Paananen, V.; Oppenlaender, J.; Visuri, A. Using text-to-image generation for architectural design ideation. Int. J. Arch. Comput. 2023. [Google Scholar] [CrossRef]
- Albaghajati, Z.M.; Bettaieb, D.M.; Malek, R.B. Exploring text-to-image application in architectural design: Insights and implications, Architecture. Struct. Constr. 2023, 3, 475–497. [Google Scholar] [CrossRef]
- Yuan, C.; Marion, T.; Moghaddam, M. Dde-gan: Integrating a data-driven design evaluator into generative adversarial networks for desirable and diverse concept generation. J. Mech. Des. 2023, 145, 041407. [Google Scholar] [CrossRef]
- Salem, A.A.; Mansour, Y.; Eldaly, H. Generative vs. Non-Generative AI: Analysing the Effects of AI on the Architectural Design Process. Eng. Res. J. 2024, 53, 119–128. [Google Scholar] [CrossRef]
- Rane, N.; Choudhary, S.; Rane, J. Integrating ChatGPT, Bard, and leading-edge generative artificial intelligence in architectural design and engineering: Applications, framework, and challenges. Int. J. Arch. Plan. 2023, 3, 92–124. [Google Scholar] [CrossRef]
- Shen, Y.; Shen, Y.; Cheng, J.; Jiang, C.; Fan, M.; Wang, Z. Neural Canvas: Supporting Scenic Design Prototyping by Integrating 3D Sketching and Generative AI. In Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 11–16 May 2024; Association for Computing Machinery: New York, NY, USA, 2024; p. 1056. [Google Scholar] [CrossRef]
- Makatura, L.; Foshey, M.; Wang, B.; Hähnlein, F.; Ma, P.; Deng, B.; Tjandrasuwita, M.; Spielberg, A.; Owens, C.E.; Chen, P.Y.; et al. Large Language Models for Design and Manufacturing. MIT Explor. Gener. AI 2024. [CrossRef]
- Maqbool, J.; Hassan, S.T.; Cheema, M.I. Application of conditional generative adversarial networks toward time-efficient and high-fidelity imaging via multimode fibers, in AI and Optical Data Sciences V. SPIE 2024, 12903, 69–73. [Google Scholar] [CrossRef]
- Florance, G.; Nirmala, M.; Khan, J.; Saifudeen Hisham, K.M. Envisioning the interactive convergence of Generative AI and Facial Expression Recognition. In Proceedings of the 2024 IEEE 9th International Conference for Convergence in Technology (I2CT), Pune, India, 5–7 April 2024; pp. 1–5. [Google Scholar] [CrossRef]
- Koga, S. Evaluating ChatGPT in pathology: Towards multimodal AI in medical imaging. J. Clin. Pathol. 2024, 77. [Google Scholar] [CrossRef]
- Micheal, A.A.; Prasanth, A.; Aswin, T.; Krisha, B. Advancing Educational Accessibility: The LangChain LLM Chatbot’s Impact on Multimedia Syllabus-Based Learning. 2024. [Google Scholar] [CrossRef]
- Dollar, O. Deep Inverse Design, Discovery, and Optimisation of Molecular Structure through 3D Invariant and Multimodal Machine Learning. Ph.D. Thesis, University of Washington, Seattle, WA, USA, 2023. Available online: http://hdl.handle.net/1773/50264 (accessed on 5 February 2024).
- Zou, H.; Zhao, Q.; Bariah, L.; Tian, Y.; Bennis, M.; Lasaulce, S.; Debbah, M.; Bader, F. GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoning. arXiv 2024, arXiv:2402.16631. [Google Scholar] [CrossRef]
- Zürcher, A. Developing a Chatbot for Internal Documents. Bachelor’s Thesis, Business Information Technology, Haaga-Helia University of Applied Sciences, Helsinki, Finland,, 2024. Available online: https://www.theseus.fi/bitstream/handle/10024/861594/Zurcher_Alexandre.pdf?sequence=2 (accessed on 5 February 2024).
- Parati, I.; Zolotova, M. Using Future Thinking as a steering tool for Generative AI creative output: A case study aiming at rethink lighting in the next future. In Proceedings of the 11th International Conference on Human Interaction & Emerging Technologies: Artificial Intelligence & Future Applications (IHIET-AI 2024), Lausanne, Switzerland, 25–27 April 2024; AHFE International Open Access: New York, NY, USA, 2024. [Google Scholar] [CrossRef]
- Nistler, J.; Pojeta, T. Graphical use of AI. VTEI 2023, 65, 54–56. Available online: https://www.vtei.cz/wp-content/uploads/2023/08/6575-casopis-VTEI-4-23-EN-AI.pdf (accessed on 10 March 2024).
- Bagnato, V.P. Artificial Intelligence for Design: The Artificial Intelligence of Objects. Interdiscip. J. Archit. Built Environ. 2023, 27, 30–35. Available online: https://www.researchgate.net/profile/Valerio-Perna/publication/379573623_FORUM_AP_27_Venturing_into_the_Age_of_AI_Insights_and_Perspectives/links/660fb14db839e05a20bd9cfb/FORUM-A-P-27-Venturing-into-the-Age-of-AI-Insights-and-Perspectives.pdf#page=31 (accessed on 10 March 2024). [CrossRef]
- Schraml, N.T. ‘You’ve Got All the Weapons You Need. Now Fight!’. Database Trends Appl. 2023, 47, 32. [Google Scholar]
- Sharma, S.K.; Dwivedi, Y.K.; Metri, B.; Lal, B.; Elbanna, A. Transfer, Diffusion and Adoption of Next-Generation Digital Technologies. In Proceedings of the IFIP WG 8.6 International Working Conference on Transfer and Diffusion of IT (TDIT 2023), Nagpur, India, 15–16 December 2023; Proceedings, Part I. Springer Nature: Berlin/Heidelberg, Germany, 2023. Available online: https://link.springer.com/book/10.1007/978-3-031-50192-0 (accessed on 10 March 2024).
- Tomaževič, N.; Ravšelj, D.; Aristovnik, A. Artificial Intelligence for Human-Centric Society: The Future Is Here; European Liberal Forum: Brussels, Belgium, 2023; Available online: https://liberalforum.eu/wp-content/uploads/2023/12/Artificial-Intelligence-for-human-centric-society.pdf (accessed on 24 January 2024).
- Mcnamara, T. Artificial intelligence and the emergence of co-creativism in contemporary art. INSAM J. Contemp. Music. Art Technol. 2023, 11, 12–38. Available online: https://www.ceeol.com/search/article-detail?id=1210421 (accessed on 10 June 2024).
- Zhang, B.; Chen, G.; Ooi, B.C.; Shou, M.Z.; Tan, K.L.; Tung, A.K.H.; Xiao, X.; Yip, J.W.L.; Zhang, M. Managing Metaverse Data Tsunami: Actionable Insights. IEEE Trans. Knowl. Data Eng. 2024, 1–20. [Google Scholar] [CrossRef]
- Guo, Y.; Liu, Q.; Chen, J.; Xue, W.; Jensen, H.; Rosas, F.; Shaw, J.; Wu, X.; Zhang, J.; Xu, J. Pathway to Future Symbiotic Creativity. arXiv 2022, arXiv:2209.02388. [Google Scholar] [CrossRef]
- Parra Pennefather, P. Prototyping with Generative AI. In Creative Prototyping with Generative AI: Augmenting Creative Workflows with Generative AI; Springer: Berlin/Heidelberg, Germany, 2023; pp. 109–143. [Google Scholar] [CrossRef]
- Ioannıdıs, S.; Kontıs, A.P. The 4 Epochs of the Metaverse. J. Metaverse 2023, 3, 152–165. [Google Scholar] [CrossRef]
- Kalpokas, I. Work of art in the Age of Its AI Reproduction. Philos. Soc. Crit. 2023. [CrossRef]
- Rudolf, I. Understanding the Influence of Artificial Intelligence Art on Transaction in the Art World. Master’s Thesis, School of Humanities, Social, Science, and Economics, International Hellenic University, Thessaloniki, Greece, 2024. Available online: https://repository.ihu.edu.gr/xmlui/bitstream/handle/11544/30356/Ion%20Rudolf.pdf?sequence=1 (accessed on 5 June 2024).
- Gupta, R.; Pal, S.K. Introduction to Metaverse; Springer Books: Berlin/Heidelberg, Germany, 2023. [Google Scholar] [CrossRef]
- Popescu, A.-D. Non-fungible tokens (nft)–innovation beyond the craze, in 5th International Conference on Innovation in Business. Econ. Mark. Res. 2021, 32, 26–30. Available online: https://www.researchgate.net/publication/353973149_Non-Fungible_Tokens_NFT_-_Innovation_beyond_the_craze#fullTextFileContent (accessed on 5 June 2024).
- Sahu, B.; Chandramohan Jha, A.M. NFT Marketplaces: The Future of Digital Asset Trading, International Journal of Scientific Research in Computer Science. Eng. Inf. Technol. (IJSRCSEIT) 2023, 9, 513–519. [Google Scholar] [CrossRef]
- Han, Y.; Wang, C.; Wang, H. Research on Blockchain Cross-Chain Model Based on “NFT + Cross-Chain Bridge”. IEEE Access 2024, 12, 77065–77078. [Google Scholar] [CrossRef]
- Wang, Q.; Li, R.; Wang, Q.; Chen, S. Non-fungible token (NFT): Overview, evaluation, opportunities and challenges. arXiv 2021, arXiv:2105.07447. [Google Scholar] [CrossRef]
- Moreaux, A. Visual content tracking, IPR management, & blockchain: From process abstraction to functional interoperability. Doctoral Thesis, Institut Polytechnique de Paris, Paris, France, 2023. Available online: https://theses.hal.science/tel-04418984/ (accessed on 5 June 2024).
- Truong, V.T.; Le, L.; Niyato, D. Blockchain meets metaverse and digital asset management: A comprehensive survey. IEEE Access 2023, 11, 26258–26288. [Google Scholar] [CrossRef]
- Park, A.; Kietzmann, J.; Pitt, L.; Dabirian, A. The evolution of non-fungible tokens: Complexity and novelty of NFT use-cases. IT Prof. 2022, 24, 9–14. [Google Scholar] [CrossRef]
- Huang, M.-H.; Rust, R.T. Artificial intelligence in service. J. Serv. Res. 2018, 21, 155–172. [Google Scholar] [CrossRef]
- Morháč, D.; Valaštín, V.; Košťál, K.; Kotuliak, I. Cross-Chain Payments on Blockchain Networks: An Apartment Booking Use-Case. In Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing, Avila, Spain, 8–12 April 2024; Association for Computing Machinery: New York, NY, USA, 2024; pp. 608–611. [Google Scholar] [CrossRef]
- Battah, A.; Madine, M.; Yaqoob, I.; Salah, K.; Hasan, H.R.; Jayaraman, R. Blockchain and NFTs for trusted ownership, trading, and access of AI models. IEEE Access 2022, 10, 112230–112249. [Google Scholar] [CrossRef]
- Bhujel, S.; Rahulamathavan, Y. A survey: Security, transparency, and scalability issues of nft’s and its marketplaces. Sensors 2022, 22, 8833. [Google Scholar] [CrossRef] [PubMed]
- Khalil, U.; Uddin, M.; Malik, O.A.; Hong, O.W. A Novel NFT Solution for Assets Digitization and Authentication in Cyber-Physical Systems: Blueprint and Evaluation. IEEE Open J. Comput. Soc. 2024, 5, 131–143. [Google Scholar] [CrossRef]
- Castro Pena, M.L.; Carballal, A.; Rodríguez-Fernández, N.; Santos, I.; Romero, J. Artificial intelligence applied to conceptual design. A review of its use in architecture. Autom. Constr. 2021, 124, 103550. [Google Scholar] [CrossRef]
- Chalmers, D.; Fisch, C.; Matthews, R.; Quinn, W.; Recker, J. Beyond the bubble: Will NFTs and digital proof of ownership empower creative industry entrepreneurs? J. Bus. Ventur. Insights 2022, 17, e00309. [Google Scholar] [CrossRef]
- Chen, C.; Fu, J.; Lyu, L. A pathway towards responsible ai generated content. arXiv 2023, arXiv:2303.01325. [Google Scholar] [CrossRef]
- Ramdurai, B.; Adhithya, P. The impact, advancements and applications of generative AI. Int. J. Comput. Sci. Eng. 2023, 10, 1–8. [Google Scholar] [CrossRef]
- Griebel, M.; Flath, C.; Friesike, S. Augmented creativity: Leveraging artificial intelligence for idea generation in the creative sphere. In Proceedings of the Twenty-Eighth European Conference on Information Systems (ECIS2020), Marrakech, Morocco, 15–17 June 2020; p. 77. Available online: https://aisel.aisnet.org/ecis2020_rip/77 (accessed on 5 June 2024).
Input | Output | |
---|---|---|
Image Options | Selected Images | |
Design Intention: Micro-library is located in the rice field with a building concept modern house with a pyramid roof Produce a hand sketch depicting the intended shape of the building to ensure alignment with the desired design | Prompt: https://s.mj.run/mzXZOLMb-Xw people walking, sunny day, architectural rendering --s 750 | Job 1 ID: 09effa33-8139-42cc-8704-feeddebf8186 |
Modify the building and add the environment to adjust with design intention | Prompt: Modern stilt house building with long cube shape with wooden material and perforated building façade | Job ID: c4e63e2f-b027-4234-986b-ef2294080713 |
Modify the building element | Prompt: trees and sky—No building—Variations (region) | Job ID: 8084c539-22e0-4cec-8163-84eba9190947 |
Generative AI Image Linked to Metadata in Firebase Storage: | |
Image Linked in Firebase1 Storage: | |
“https://firebasestorage.googleapis.com/v0/b/genainft-ac24b.appspot.com/o/image%2F1.png?alt=media&token=0aab1e85-3c98-4e04-ac79-db175ab7c82e (accessed on 15 April 2024)” | |
Metadata (.json file): | |
{ "attributes": [ { "username": “digicliffnotes”, “user_id”: “1084483399319822387”, “job_id”: “c4e63e2f-b027-4234-986b-ef2294080713”, “creator_name”: “Adam”, “creation_date”: “April 13th, 2024 11:26 pm”, “creation_tool”: “Midjourney”, “prompt”: “modern_stilt_house_building_with_long_cube_shape_with_wooden_material_and_perforated_building_facade”, “image_link”: “https://cdn.discordapp.com/attachments/1087237286707605535/1228727456182177862/digicliffnotes_modern_stilt_house_building_with_long_cube_shape_c4e63e2f-b027-4234-986b-ef2294080713.png?ex=662d189e&is=661aa39e&hm=e6931926dc71ae46f1f1966444fa2b5f344916e0a168a2b74e71688c1f69a478&” } ], “image”:”https://firebasestorage.googleapis.com/v0/b/genainft-ac24b.appspot.com/o/image%2F1.png?alt=media&token=0aab1e85-3c98-4e04-ac79-db175ab7c82e”, “name”: “Design_option #1” } | |
Link of Metadata in Firebase Storage: | |
https://firebasestorage.googleapis.com/v0/b/genainft-ac24b.appspot.com/o/metadata%2F1.json?alt=media&token=c08b907d-b6f4-4653-9ceb-c8966e48e659 (accessed on 15 April 2024) | |
Deploy and Transaction the Metadata using Smart Contract: | |
The contract address: | 0xb27A31f1b0AF2946B7F582768f03239b1eC07c2c |
Token address 1: | 0x5B38Da6a701c568545dCfcB03FcB875f56beddC4 |
Token URI 2: | “https://firebasestorage.googleapis.com/v0/b/genainft-ac24b.appspot.com/o/metadata%2F1.json?alt=media (accessed on 15th April 2024)” |
Key Points | Description |
---|---|
Framework for integration | Combines generative AI and blockchain for architectural design using a hypothesis scenario framework to outline the design process flow and real-life applications. |
Generative AI design process | Involves initial sketches, prompt engineering, and iterative refinement to generate accurate AI outputs. Key elements include building typology, site details, materials, spatial layout, and rendering style. |
Features of generative AI | Utilises variants, upscale, blends, remixes, and prompts to provide multiple design possibilities. AI models are trained on large datasets to identify artistic trends and stylistic components. |
Data ownership and legal aspects | Emphasises the importance of data ownership in AI training datasets, impacting legal, moral, and regulatory implications. It includes rights to the usage, modification, distribution, and monetisation of AI-generated outputs. |
Blockchain for data storage | Uses blockchain to store prompt data and AI-generated images as NFTs, ensuring secure data ownership. The process includes generating images, producing metadata, storing data in Google Firebase, and converting metadata into the NFT format. |
Results and implementation | Showcases the outcomes of the generative design process with metadata linked to final images. Provides examples of metadata, storage links, and smart contract details for NFT deployment, demonstrating the practical application of the method. |
Unimodal | |||
Input | Output (Generated Image by Generative AI) | ||
Image Options | Selected Image | ||
Design objective | Design iteration: Find the reference building with writing the prompt to develop the building shape suitable with the design intention | Job ID: 7c006bd2-07ad-4e8d-b9ef-47f00abc4732 | Job ID: f918bc31-bcff-4a1c-a49a-d98959652c05 |
Image | - | ||
Prompts | Create a prompt as a trigger to draw the environment: [“micro-library, incorporating vernacular and contemporary architecture, combination of perforated metal panel and transparent muted glass wall as facade, mir rendering, perspective view, located in the rice field near the village in Taiwan, natural light”] | ||
Multimodal | |||
Design objective | Design iteration: Combine the building to obtain wider range design options using blend | Job ID: f4e81d0e-b2e2-4057-8ee3-42a59f97551c seed 2798360210 | Job ID: 8ede5825-a45c-452b-ba8e-bb22e6c2d14a seed 2798360210 |
Image | |||
Prompts | No prompt |
Aspect | Resume | Specific Generative AI Applications or Technology—Source |
---|---|---|
Computational efficiency | AI enables rapid design generation, exploration, and iteration. | 1 MidJourney—[28,29,30]; NS 2—[31,32]; Dall-E 3—[33]. |
Designers can quickly produce, evaluate, and refine multiple options, leading to more innovative and optimised solutions. | NLP 4 and MMAIR 5—[34] NS—[35,36,37,38]; Dall-E—[39,40]; Dde 6—GAN 7—[41]; CLIP 8—[42]. | |
Generative AI tools for architecture need high computational power and complex algorithms. | NS—[4,36]; ChatGPT 9—[43]; Bard AI 10—[43]; Neural Canvas 11—[44]. | |
Ensuring efficiency and accessibility for all firms is challenging due to large datasets, diverse inputs, and multiple design constraints. | NS—[4,36]; ChatGPT—[43]; Bard AI—[43]; NS—[34,35,36,37,38]; LLMs 12—[45] | |
These demands can slow down processing and increase resource consumption. | NS—[4,36]; Neural Canvas [44]. | |
Accuracy | Significant improvement in imaging accuracy ensures high-fidelity imaging for precise applications. | CGANs 13—[46]; U-Net Arch 14—[46]. |
Enhances reliability of multimodal communication and AI diagnostic processes. | GenAIVA 15 and FER 16—[47]; ChatGPT—[48]; | |
Improves accuracy and transparency with visual explanations and textual analysis. | ChatGPT—[48] | |
Maintaining high accuracy while optimising resource usage and ensuring adaptability across diverse contexts is challenging. | LangChain LLM—[49]; | |
Ensuring consistent and reliable accuracy, generalisability, and efficient knowledge transfer in resource-limited environments is crucial. | 3DI 17—[50]; MML 18—[50]; GenAINet 19—[51]. | |
Making visual explanations and textual analyses both accurate and comprehensible is challenging. | ChatGPT—[48]. | |
User Experience (UX) | AI tools, like chatbots, improve adaptability, responsiveness, and user interaction by managing tasks and information efficiently. | ChatGPT—[52]; |
Enhanced visualisation and engagement build trust in AI systems | MidJourney—[53,54]; | |
Integrating text, image, and voice modalities into one tool is technically complex. | NS—[55]. | |
Generative AI tools require new skills and workflows, causing potential frustration and reduced productivity. | Dall-E—[53]; OpenAI—[56] | |
Interoperability issues and variable AI output quality may need refinement. | MidJourney—[53]; Dall-E—[53]. | |
Limited customisation can constrain designers’ creativity. | NS—[55]. | |
Building user trust is challenging due to past unreliable performance and data privacy and security concerns. | 20 GAIS (IBM Watson)—[57] |
Aspect | Resume | Source |
---|---|---|
Authenticity, certification and ownership | NFTs provide a robust method for certifying the authenticity of digital assets through blockchain technology. | [28,58,59,60,61,62,63,64] |
Blockchain’s immutable nature ensures that the ownership records of NFTs remain tamper-proof and verifiable, thus guaranteeing the authenticity of AI-generated content. | [64,65,66] | |
Proving ownership and authenticity in a decentralised NFT market can be complex. | [67,68] | |
Ensuring the security of blockchain and NFTs against hacking, fraud, and other malicious activities is a significant concern that can impact the reliability of authenticity and ownership records. | [59,63,64] | |
Integration in the creative process | NFTs facilitate creating, owning, and distributing collaborative AI–human creations. This integration supports a new dimension of creativity where digital assets are co-created by humans and AI. | [59,61,65] |
Interoperability issues between different blockchain platforms can hinder seamless integration and data exchange. | [69,70,71] | |
Complexity integrating NFT-based. | [21,72,73] | |
Scalability and traceability process | Blockchain-based NFTs simplify the registration, verification, and tracing of financial transactions related to digital assets, thus enhancing these transactions’ overall security and transparency. | [60,63,66,74,75] |
NFTs offer a transparent and secure means to track and verify ownership of digital assets. | [76,77] | |
Implementing blockchain solutions on a large scale can be complex and costly, limiting their practicality for verifying and tracking digital assets. | [21,72,78] |
Aspect | Challenges | Addressing Challenges | Proposed Solution | Limitations |
---|---|---|---|---|
Authenticity and traceability | Ensuring the authenticity and traceability of AI-generated images | Developing a blockchain-integrated framework that ensures the authenticity and traceability of the generative AI process. | A blockchain system can be used to store AI-generated images and their metadata as NFTs, ensuring secure and traceable data. | Scalability and performance of integrated systems in large-scale applications |
Integration of technologies | Integrating multimodal generative AI and blockchain technologies seamlessly | Develop a structured framework for integration. | Combine multimodal generative AI and blockchain technology in a streamlined workflow for architectural design. Potential integration with architecture design software. | Interoperability between different generative AI tools and blockchain platforms. |
Data ownership and legal issues | Managing data ownership for AI-generated content | Addressing data ownership and regulatory issues by ensuring proper attribution and legal compliance through blockchain records. | Store AI-generated images and metadata in a blockchain system, ensuring data ownership and legal compliance through NFTs. | Comprehensive studies on legal and regulatory frameworks required to govern the use of AI and blockchain in architectural design. |
User experience and interaction | Improving design efficiency, accuracy, and user interaction | Utilising detailed prompt engineering to ensure accurate and relevant AI-generated images that align with the intended architectural designs. | Use generative AI applications to create and refine architectural designs, ensuring user-friendly interaction and high-quality outputs. | User acceptance and trust in AI-generated designs and blockchain-based data management. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Fitriawijaya, A.; Jeng, T. Integrating Multimodal Generative AI and Blockchain for Enhancing Generative Design in the Early Phase of Architectural Design Process. Buildings 2024, 14, 2533. https://doi.org/10.3390/buildings14082533
Fitriawijaya A, Jeng T. Integrating Multimodal Generative AI and Blockchain for Enhancing Generative Design in the Early Phase of Architectural Design Process. Buildings. 2024; 14(8):2533. https://doi.org/10.3390/buildings14082533
Chicago/Turabian StyleFitriawijaya, Adam, and Taysheng Jeng. 2024. "Integrating Multimodal Generative AI and Blockchain for Enhancing Generative Design in the Early Phase of Architectural Design Process" Buildings 14, no. 8: 2533. https://doi.org/10.3390/buildings14082533