Search Results (147)

This paper reports a systematic study of low-rank adaptation (LoRA)-based fine-tuning applied to Qwen3 language models (4B, 8B, and 14B parameters) for the task of legal question answering within the jurisdiction of the Republic of Kazakhstan. The bilingual dataset comprises 63,114 question–answer pairs (76.2% Russian, 23.8% Kazakh) covering 11 legal domains. Models are evaluated through both automated metrics (BERTScore, citation accuracy, and hallucination rate) and blind expert assessment by a panel of two practising legal experts. Key findings: (1) all fine-tuned models reach BERTScore F1 close to 90% (89.6–90.2%) versus 82.2–83.1% for untuned base models; (2) fine-tuned models outperform GPT-4o (87.2%) and GPT-4o-mini (86.7%) on semantic similarity while exhibiting far lower hallucination rates (27–29% vs. 83–90%); (3) blind expert assessment confirms the advantage of fine-tuned models, with panel mean completeness scores of 4.28/5 versus 1.95/5 for base models (quadratically weighted Cohen’s κ = 0.80–0.95 across rating dimensions, indicating substantial to almost perfect inter-rater agreement); and (4) we identify a practical scaling plateau: paired Wilcoxon tests (n = 500) detect statistically significant but practically small differences across the 4B, 8B, and 14B fine-tuned variants (largest mean gap 0.67 pp on BERTScore F1; Cohen’s |d_z| ≤ 0.34), gains too small to justify the 3.5× parameter increase. These findings show that parameter-efficient adaptation of compact open-source models can match or exceed commercial LLMs for specialised legal QA in a low-resource bilingual context. We note one scope restriction: three domains (administrative, criminal, and housing law) are represented only in Russian, so the model is not validated for Kazakh language queries in these areas. Full article

Side-scan sonar images play an important role in underwater target detection, seabed mapping, and marine environment monitoring. However, the performance of deep learning-based detectors is often limited by the small scale of available sonar datasets, the high cost of data acquisition, and class imbalance among target categories. To address these issues, this paper proposes a diffusion-model-based data augmentation method for side-scan sonar target detection. A FLUX.1 diffusion model is adopted as the base generative framework and is fine-tuned using low-rank adaptation (LoRA) to adapt the pretrained model to the side-scan sonar image domain under limited training data conditions. The generated samples are further filtered and added only to the training set, while the validation and test sets are kept unchanged and contain only real sonar images. To ensure a fair evaluation of the augmentation strategy, all detection experiments are conducted using a fixed YOLOv8n (You Only Look Once version 8 nano) detector under the same training hyperparameters and three random seeds. Compared with training on the original dataset, the proposed FLUX+LoRA augmentation improves mean average precision (mAP)@0.5 from 0.7400 ± 0.0132 to 0.8582 ± 0.0328 and mAP@0.5:0.95 from 0.3994 ± 0.0187 to 0.5115 ± 0.0164. It also outperforms conventional augmentation methods under the same real-only validation/test protocol. In addition, Fréchet Inception Distance (FID)/Kernel Inception Distance (KID)-based image quality evaluation, generated-sample amount ablation, screening-strategy ablation, LoRA-rank sensitivity analysis, and a controlled 600-sample diffusion-backbone comparison are conducted. The results show that the 600-sample manually annotated FLUX+LoRA subset selected from generated samples achieves better image quality and detection performance than FLUX-base and SD1.5+LoRA under the same annotation budget. These findings demonstrate that FLUX+LoRA-generated sonar images can provide useful structural diversity for detector training and improve target detection performance under limited-data conditions. Full article

Accurate identification of drilling operation states is essential for improving drilling efficiency and operational safety. However, existing methods often suffer from limited temporal feature extraction capability, weak cross-well generalization, and insufficient model interpretability. To address these issues, this study proposes a drilling-state recognition framework based on MultiHead-BiLSTM and low-rank adaptation (LoRA) transfer learning. The MultiHead-BiLSTM model combines multi-head attention with bidirectional long short-term memory to capture both critical temporal segments and global sequential dependencies in drilling time series data. To improve cross-well adaptability while reducing training computational cost, a parameter-efficient LoRA fine-tuning strategy is introduced within the transfer learning framework. In addition, SHAP-based feature attribution and attention visualization are employed to enhance model interpretability. Experimental results show that the proposed method achieves an accuracy of 95.11% and an F1-score of 94.00%, outperforming LSTM, GRU, BiLSTM, and Transformer baselines. The LoRA-based transfer strategy reduces the cross-well error rate to 1.91%, compared with 8.79% for direct transfer and 4.48–5.39% for partial-layer freezing methods. Interpretability analysis qualitatively suggests that bit depth, weight on bit, and block position contribute strongly to drilling-state discrimination, while attention visualization qualitatively suggests that the model tends to focus on operational transition periods. The proposed framework provides an effective and computationally efficient solution for intelligent drilling-state recognition and cross-well deployment. Full article

Network intrusion detection systems (NIDS) play a crucial role in modern network environments where diverse and rapidly evolving traffic patterns are observed. Although deep learning-based NIDS have demonstrated strong performance within specific datasets, their effectiveness significantly degrades when applied to unseen network environments due to domain discrepancies. In this paper, we first experimentally demonstrate the performance degradation of time-series-based NIDS under cross-domain conditions using multiple benchmark datasets. Then, we propose a LoRA-based domain adaptation framework for time-series-based NIDS models. Instead of retraining the entire model, the proposed approach freezes the backbone network and applies low-rank updates to selected layers, enabling parameter-efficient adaptation to new domains. Experimental results show that the proposed method consistently improves cross-domain detection performance across multiple dataset combinations, particularly in terms of recall, while requiring only a small number of additional parameters. Full article

The emergence of multimodal large language models (MLLMs) is opening a new avenue for explainable and interactive intelligent diagnosis in agriculture. However, generic MLLMs still face two major obstacles in plant disease recognition—insufficient fine-grained visual perception and misalignment between visual and linguistic features—which jointly limit diagnostic accuracy. To address these issues, we propose a Qwen2.5-VL-based full-chain fine-tuning framework termed dual-side synergistic low-rank adaptation. Unlike the mainstream paradigm that freezes the vision encoder, our method injects trainable LoRA adapters into both the vision encoder and the large language model, while establishing end-to-end gradient backpropagation across the entire multimodal pipeline. By using the supervision signal from autoregressive text generation (text-supervised visual learning), the framework directly drives deep optimization of visual representations, thereby enabling coordinated alignment between pixel-level perception and semantic-level understanding. We trained Qwen over CDDM and conducted in-domain (CDDM) and cross-domain (PlantVillage) experiments. The results show that the proposed 7B-parameter model achieves 98.8 and 96.0% diagnostic accuracy under in-domain and cross-domain scenarios, respectively. The recognition accuracy of Qwen in the case of cross-domain only decreases slightly, which demonstrates that the MLLM trained by our method exhibits excellent cross-domain recognition capability. This indicates that our method can significantly improve the robustness and generalization ability of MLLM in complex agricultural scenarios. Full article

Graphene and carbon nanotube (CNT) membranes are promising for filtration, desalination, and water treatment, yet their performance requires the joint interpretation of their architecture, nanoconfined transport, selectivity, fouling, swelling, defects, stability, and operating conditions. Here, GCMembrane-LLM was developed as an evidence-grounded domain-specific large language model. A curated 582-paper corpus generated 12,208 cleaned membrane-specific question–answer pairs for Low-Rank Adaptation (LoRA)-based supervised fine-tuning of Llama-3.1-8B-Instruct, and retrieval-augmented generation provided article-title and page-level traceability. GCMembraneBench included 100 application-oriented questions on graphene oxide (GO) membranes, CNT membranes, GO/CNT hybrids, and cross-material reasoning. Under direct answering without retrieval context, the anonymized and shuffled automatic evaluation showed that GCMembrane-LLM achieved a mean weighted score of 4.237/5.0, exceeding Llama-3.1-8B-Instruct and Doubao-1.5-lite. A stratified 30-question blinded manual assessment showed the same ranking. The application cases further yielded membrane science conclusions: CNT-assisted GO/CNT transport should be evaluated with dispersion, interfacial compatibility, defects, and stability; GO desalination depends on swelling control, interlayer spacing, and defect suppression; and CNT high flux requires joint examination of pore diameter, entrance chemistry, hydration barriers, ion rejection, and operating conditions. GCMembrane-LLM supports source-traceable evidence organization and preliminary hypothesis formulation before experimental validation. Full article

Despite recent advances, medical question answering systems still struggle with domain-specific reasoning and data efficiency. This paper presents Med-LLaMA3, a family of medical large language models developed by parameter-efficient fine-tuning of the LLaMA-3.1 (8 billion) and LLaMA-3.2 (1 and 3 billion) architectures using quantized low-rank adaptation (QLoRA) and low-rank adaptation (LoRA) with 4-bit quantization. Beyond model training, this work contributes the following: (1) a formalized dataset curation taxonomy (source type × clinical granularity × task format) with a source-category ablation confirming that the multi-source combination drives benchmark gains beyond any single category; (2) a systematic characterization of low-rank-adaptation rank-scaling behavior for the LLaMA-3 family in the medical domain (monotonic improvement up to rank 128, with no observed plateau); and (3) statistically validated comparisons using McNemar’s test and 95% bootstrap confidence intervals. We curated a medical instruction dataset of over 1.5 million samples spanning medical examinations, clinical dialogues, and biomedical literature. Our approach trains only ∼4% of the base model’s parameters and, consistent with prior studies of parameter-efficient methods in the medical domain, achieves performance comparable to full fine-tuning at a fraction of the memory footprint. Evaluated with five in-context examples per prompt, the 8-billion-parameter model attains a mean accuracy of 75.71% across the eight medical-domain subsets of the Massive Multitask Language Understanding benchmark; improvements over the unmodified LLaMA-3.1-8B-Instruct baseline are statistically significant on the medical multiple-choice benchmark MedMCQA and, after Bonferroni correction across the eight subsets, on three subsets (Clinical Knowledge, Medical Genetics, and Nutrition), with two further subsets being significant only before correction. A structured named-entity-recognition evaluation on 100 hospital discharge summaries (macro-averaged F1 0.94; dual-annotator agreement $\kappa =0.87$) provides complementary evidence of clinical-text utility. A safety mitigation pilot shows that context-disambiguation preprocessing reduces the highest-severity abbreviation-ambiguity error rate from 30% to 10% on a 30-case held-out set. These results show that parameter-efficient fine-tuning can deliver high-performance medical large language models while training only ∼4% of the model’s parameters and reducing memory use by roughly 75%, enabling development on low-cost consumer-grade hardware. Full article

Addressing the challenges of dense conceptual content and intricate knowledge relations in computer science textbooks, where traditional pipeline-based information extraction suffers from error propagation and semantic decoupling, this paper proposes a concept-driven joint extraction method termed CDE (Concept-Driven Extraction).First, the model’s ability to focus on domain-specific terminology is enhanced through conceptual priors and attention re-weighting. This is integrated with a predefined schema and structured instruction templates to achieve normalized output for both entities and relations. Second, efficient domain knowledge transfer for computer science textbooks is realized by performing Low-Rank Adaptation (LoRA) fine-tuning on the Qwen3-4B large language model. Finally, the construction of the computer science textbook knowledge graph is accomplished using the Neo4j graph database. On a self-constructed instruction dataset of computer science textbooks, CDE achieves an F1 score of 81.83%, representing an improvement of approximately 2.47 percentage points over the LKD-KGC baseline. This performance significantly surpasses that of traditional pipeline models and existing joint extraction approaches. Experimental results demonstrate that CDE can effectively improve knowledge extraction accuracy in the textbook domain, thereby providing a novel research avenue for the rapid construction of knowledge graphs for computer science educational materials. Full article

To address challenges such as semantic distortion, poor controllability, and the lack of domain-specific structural knowledge in AIGC-driven furniture design, this study proposes a feature-tag-driven semantic control framework based on LoRA (Low-Rank Adaptation) fine-tuning. First, Facet Analysis Theory is introduced to construct a structured representation system, deconstructing furniture into multi-dimensional components (e.g., silhouette, base, backrest, and armrests) and establishing a standardized feature-tag dictionary for deep annotation. Subsequently, domain design knowledge is precisely embedded into the latent space of a diffusion model via LoRA training to reinforce structural consistency. The framework was evaluated through single- and multi-feature comparative tests using a hybrid metric of Feature Hit Rate (FHR) and CLIP-based semantic similarity. Results indicate that the proposed method significantly outperforms general-purpose models in feature-level controllability and structural logic. This research provides a transferable methodology for integrating domain knowledge into generative models, offering significant value for the digital modular design and intelligent manufacturing of upholstered furniture systems. Full article

Intelligent autonomous driving systems must not only predict the appropriate driving manoeuvre but also provide human-interpretable evidence that justifies the decision. However, existing methods typically address these objectives separately, leading to three practical limitations: multi-stage perception-to-language pipelines can propagate upstream perception errors into downstream explanations; post hoc saliency methods often produce pixel-level highlights that are difficult to interpret semantically; and decoupled decision and explanation modules cannot guarantee that the explanation reflects the same scene evidence used for behaviour prediction. In this paper, we propose a unified framework that jointly performs vehicle behaviour prediction and human-centric interpretation from a shared visual backbone. Specifically, a hierarchical Swin Transformer encodes the driving scene into a sequence of spatial tokens, which are processed by two complementary branches. The first branch, termed the Object Selection Module (OSM), learns a compact scene-level semantic representation through query-guided cross-attention, while the second branch extracts a small set of class-agnostic object-centric tokens without requiring bounding-box or segmentation supervision. These two representations are subsequently integrated by a Semantic–Object Fusion (SOF) module based on scaled dot-product attention, residual connections, and a feed-forward network. The behaviour prediction head operates on the fused representation, whereas the interpretation head leverages the semantic representation through a skip connection to preserve decision-relevant context. For surround-view perception, learnable per-camera embeddings are introduced to maintain viewpoint identity with negligible additional parameter cost. Furthermore, a compact language model fine-tuned via Low-Rank Adaptation (LoRA) generates fluent, label-conditioned natural-language justifications. Extensive experiments on two public benchmarks, BDD-OIA and nu-AD, demonstrate that the proposed framework consistently delivers superior performance and provides effective, human-readable interpretations of driving decisions. Full article

Automated visual inspection of photovoltaic (PV) cells is an important component of solar-module quality assurance. However, adapting modern pre-trained vision backbones to PV defect classification remains challenging because full fine-tuning requires substantial memory, naturally imbalanced datasets can reduce sensitivity to rare defect classes, and edge-oriented inspection workflows impose computational constraints. Parameter-efficient fine-tuning (PEFT) methods, such as Low-Rank Adaptation (LoRA) and Quantized LoRA (QLoRA), have been widely studied in natural language processing, but their use for PV defect classification remains underexplored. This study presents a controlled benchmark of LoRA and QLoRA against full fine-tuning for PV cell defect classification. Four adaptation strategies—full fine-tuning, LoRA with rank 8, LoRA with rank 16, and 4-bit QLoRA with rank 16—are evaluated using a ConvNeXt-Tiny backbone on a 17,377-image polycrystalline PV cell electroluminescence dataset referred to as POLY, covering five classes: intact, cracked, broken, surface-diffuse, and surface-point. The natural 6.7× class imbalance is preserved without synthetic resampling, and a group-aware StratifiedGroupKFold protocol based on available cell or panel-image identifiers is used to reduce identifiable leakage across folds. All PEFT variants slightly outperform full fine-tuning in macro-F1 while training 26–52× fewer parameters. QLoRA_r16 achieves the highest macro-F1 score of 79.92 ± 0.75%, compared with 78.26 ± 0.94% for full fine-tuning, while training the same number of parameters as LoRA_r16 (1.060 M; 3.67% of the adapted model). QLoRA_r16 also improves F1 on the intact (+4.75 points) and surface-diffuse (+2.62 points) classes relative to full fine-tuning. This class-wise pattern suggests that quantized low-rank adaptation may influence minority and visually ambiguous categories; however, the present experiments do not isolate the independent effect of NF4 quantization from adapter rank, batch size, or optimization dynamics. Under the training configuration used, QLoRA_r16 records the lowest observed peak training GPU memory, approximately 30% below full fine-tuning (1727 MB versus 2478 MB). Because QLoRA_r16 was trained with batch size 16 whereas the other methods used batch size 32, this reduction should be interpreted as an end-to-end configuration effect rather than as the isolated effect of 4-bit quantization. Overall, the results indicate that PEFT is a promising and resource-efficient alternative to full fine-tuning for PV defect classification, although batch-matched memory experiments, direct embedded-device profiling, and cross-dataset validation remain necessary before making deployment-level claims. Full article

Deep convolutional neural networks have made substantial progress in automated crop disease diagnosis. However, their practical application remains constrained by limited interpretability and insufficient structured reasoning, as these models largely operate as black boxes. Although they are effective in extracting visual features, they often fail to provide semantically grounded explanations, which may reduce their reliability in complex and open agricultural environments. To address these issues, this study constructs a Vision Language Model for Multi-Crop Pest and Disease Diagnosis (VLM-MCPDD). Specifically, the LLaVA-1.5 model is fine-tuned using low-rank adaptation (LoRA) to better align visual symptom representations with domain-specific agricultural knowledge. In addition, a Pests and Diseases Semantic Dataset (PDSD) is constructed to support multimodal learning. Based on PDSD, a chain-of-thought (CoT) mechanism is introduced to simulate the diagnostic workflow of agronomists, covering symptom observation, causal analysis, and final decision-making. The experimental results show that compared with comparative models such as Swin Transformer and ConvNeXt, VLM-MCPDD performs better in overall performance and can provide some reference for disease and pest diagnosis in intelligent agriculture. Full article

Amid China’s rapid urbanization, many historic districts face complex challenges, including fragmented traditional fabrics, disordered spatial morphology, and discontinuous street networks. To tackle these issues, this study proposes a multimodal deep learning framework that combines Generative Adversarial Networks (GANs) and Diffusion Models, establishing an integrated generation-optimization workflow for the renewal of historic districts. The methodology begins by using Pix2PixHD to generate high-precision fabric layouts, followed by fine-tuning a Diffusion Model through Low-Rank Adaptation (LoRA) to achieve diversified morphological expansion. The candidate proposals are quantitatively evaluated using a ten-indicator evaluation matrix that covers both architectural fabric and street network dimensions. Afterwards, these proposals undergo iterative optimization with a multi-objective framework to enhance both urban fabric morphology and network performance. The framework was validated through an empirical study of the Yuehe Historic District in Jiaxing. The results indicate that the generated schemes closely align with the original urban fabric. Compared with the existing expanded area (EA), the weighted comprehensive fitness score of the optimized scheme group improved from 0.66 to 0.89 ± 0.02 (a 34.8% increase), with the standard deviation decreasing from 0.07 to 0.02, indicating significantly enhanced stability. Deep learning balances morphological authenticity, generative diversity, and performance in historic district preservation and renewal. Full article

Large language models (LLMs) have demonstrated strong potential in medical knowledge applications; however, their reliability in knowledge-intensive medical reasoning—remains limited due to hallucination, inadequate domain grounding, and unstable inference behavior. These limitations are particularly pronounced in tasks of professional medical reasoning that require strict logical consistency and authoritative knowledge support. This study proposes a generative AI architecture that integrates RAG (Retrieval-Augmented Generation) with parameter-efficient supervised fine-tuning based on Low-Rank Adaptation (LoRA) to improve reasoning stability and diagnostic accuracy in complex medical domains. The architecture combines internalized domain reasoning learned through LoRA-based fine-tuning with external knowledge grounding enabled by a dynamic RAG mechanism, allowing the model to selectively retrieve domain-specific knowledge only when it is semantically relevant and evidence supported. To validate the proposed architecture, a large-scale real-world dataset comprising 11,476 multiple-choice questions from Taiwan’s national Traditional Chinese Medicine (TCM) licensing examinations (2005–2025) is constructed as a representative case study of knowledge-intensive medical reasoning. The experimental results show that the baseline LLM achieves an accuracy of 61.0%. Incorporating RAG improves accuracy to 89.0%, while combined LoRA-based fine-tuning and RAG architecture further increases accuracy to 90.1%, with reduced variance across repeated evaluations. Statistical analysis using McNemar’s test confirms that the performance improvements introduced by the retrieval mechanism are highly significant. The results demonstrate that integrating parameter-efficient fine-tuning with dynamically controlled retrieval is critical to balancing reasoning stability and knowledge enhancement in generative AI systems. Beyond the specific medical case study examined in this work, the proposed architecture offers a reproducible and extensible framework for developing reliable generative AI systems in other knowledge-intensive professional reasoning and educational domains. Full article

Murals are not merely independent visual artworks. Rather, they are an integral part of architectural heritage, directly attached to buildings’ structural elements, such as brick walls and vaults. However, murals are susceptible to various building-related types of damage, including structural cracks and moisture-induced peeling, due to long-term exposure to environmental factors and geological changes. As the progressive deterioration of these murals hastens the loss of mural value, professional assessment and restoration are urgently required. To tackle the issues of low efficiency in traditional structural damage detection and the absence of predictable repair plans, this paper presents a semi-automatic building-mural protection solution that integrates morphological assessment of mural deterioration with computer vision technology. This study establishes an image prediction system that integrates intelligent damage identification with virtual restoration. First, employing the PaddleSeg deep learning framework and the DeepLabv3 semantic segmentation model, this study used existing mural damage datasets to build a recognition model. The model allows for intelligent identification and labeling of multiple damage types. Subsequently, relying on the ComfyUI platform, Stable Diffusion was used to construct a virtual restoration model. LoRA (low-rank adaptation) technology was introduced to fine-tune the model specifically for the mural style, thus enhancing the directivity and accuracy of virtual restoration. Finally, by applying the results of the recognition model to the virtual restoration model, this study built an integrated system for mural damage diagnosis and virtual restoration. The results show that the damage recognition model achieved a mean intersection over union (mIoU) of 47.8% and a pixel accuracy of 77.97% on the test set, validating the feasibility of using semantic segmentation for mural damage detection. This study presents an integrated workflow framework integrating automatic damage identification and intelligent repair. As an expert-assisted tool, this framework shows application potential for preliminary exploration of mural disease diagnosis and virtual restoration plans, providing technical references for the digital protection of cultural heritage. Full article