Jadwal Sholat

Memuat jadwal sholatโ€ฆ

Teknik & Rekayasa editorial

Open AccessOA2025

Semantic and Lexical Analysis of Pre-Trained Vision Language Artificial Intelligence Models for Automated Image Descriptions in Civil Engineering

This study evaluates pre-trained vision-language models, specifically ChatGPT-4v, for describing civil engineering images without fine-tuning. Results show an average semantic similarity of 76% compared to human descriptions, with better performance on a public dataset.
Pedram Bazrafshan; Kris Melag; A. Ebrahimkhanlouยท AI in Civil Engineeringยท 2025ยท DOI 10.1007/s43503-025-00063-9

The core problem

The paper addresses the application of pre-trained Vision-Language Models (VLMs) for describing images from civil engineering materials and construction sites, focusing on construction components, structural elements, and materials. The novelty lies in investigating VLMs for this specialized domain, which has not been previously addressed. As a case study, the authors evaluate ChatGPT-4v's ability to serve as a descriptor tool by comparing its performance with three human descriptions (a civil engineer and two engineering interns). The contributions include adapting a pre-trained VLM to civil engineering applications without additional fine-tuning and benchmarking its performance using both semantic similarity analysis (SentenceTransformers) and lexical similarity methods.

Innovation

The study utilizes two datasets: one from a publicly available online repository and another manually collected by the authors. The evaluation employs whole-text and sentence pair-wise similarity analyses to assess the model's alignment with human descriptions. Semantic similarity is measured using SentenceTransformers, while lexical similarity methods are also applied. The human descriptions serve as the benchmark for comparison. The VLM (ChatGPT-4v) is used out-of-the-box without any domain-specific fine-tuning. The analysis compares the model's descriptions against those of a civil engineer and two engineering interns.
Introduction
The paper addresses the application of pre-trained Vision-Language Models (VLMs) for describing images from civil engineering materials and construction sites, focusing on construction components, structural elements, and materials. The novelty lies in investigating VLMs for this specialized domain, which has not been previously addressed. As a case study, the authors evaluate ChatGPT-4v's ability to serve as a descriptor tool by comparing its performance with three human descriptions (a civil engineer and two engineering interns). The contributions include adapting a pre-trained VLM to civil engineering applications without additional fine-tuning and benchmarking its performance using both semantic similarity analysis (SentenceTransformers) and lexical similarity methods.

Why it matters

The findings suggest that pre-trained VLMs like ChatGPT-4v can be effectively used for automated image description in civil engineering without additional fine-tuning, achieving a reasonable level of similarity to human descriptions. The better performance on the public dataset implies that dataset characteristics influence model performance. The study highlights the potential of VLMs for automating image documentation in civil engineering, but also notes the limitations and the need for further refinement. The authors discuss the implications for practice and future research directions, including the potential for fine-tuning and expanding the datasets.

Who should read this

Pembaca Research Digest dan praktisi rumpun ini.

Opening member contentโ€ฆ