{"id":15,"date":"2024-07-02T07:51:34","date_gmt":"2024-07-01T22:51:34","guid":{"rendered":"https:\/\/www.ist.hokudai.ac.jp\/labo\/lml\/?page_id=15"},"modified":"2024-08-30T11:15:09","modified_gmt":"2024-08-30T02:15:09","slug":"publications","status":"publish","type":"page","link":"https:\/\/www.ist.hokudai.ac.jp\/labo\/lml\/?page_id=15","title":{"rendered":"Publications \ud83d\udcd7"},"content":{"rendered":"\n

<\/p>\n\n\n\n

<\/a>
<\/div><\/div><\/form>
828 entries<\/span> «<\/a> ‹<\/a> 1 of 21 ›<\/a> »<\/a> <\/div><\/div>
1.<\/div>

Rafal Rzepka; Jin Hosono; Jun Nakamura<\/p>

Beyond Description: Comparing AI and Human Interpretation of Visual Art in Social Communication Contexts Proceedings Article<\/span> <\/p>

In: <\/span> Liberona, Dario; Nguyen, Luong Vuong; Ting, I-Hsien; Wang, Kai (Ed.): Multidisciplinary Social Networks Research, <\/span>pp. 169\u2013183, <\/span>Springer Nature Switzerland, <\/span>Cham, <\/span>2027<\/span>, ISBN: 978-3-032-33384-1<\/span>.<\/p>

Abstract<\/a><\/span> | BibTeX<\/a><\/span><\/p>

@inproceedings{10.1007\/978-3-032-33384-1_13,
\r\ntitle = {Beyond Description: Comparing AI and Human Interpretation of Visual Art in Social Communication Contexts},
\r\nauthor = {Rafal Rzepka and Jin Hosono and Jun Nakamura},
\r\neditor = {Dario Liberona and Luong Vuong Nguyen and I-Hsien Ting and Kai Wang},
\r\nisbn = {978-3-032-33384-1},
\r\nyear  = {2027},
\r\ndate = {2027-01-01},
\r\nbooktitle = {Multidisciplinary Social Networks Research},
\r\npages = {169\u2013183},
\r\npublisher = {Springer Nature Switzerland},
\r\naddress = {Cham},
\r\nabstract = {Visual content plays a central role in online communication, where images are often accompanied by human interpretation and commentary. With the rapid development of Vision\u2013Language Models (VLMs), artificial intelligence systems are increasingly capable of generating such interpretations. This paper examines whether current VLMs can produce art descriptions and interpretations comparable to those written by human students. We conduct a comparative study using a structured evaluation framework in which a Large Language Model (GPT-4o) acts as an automated judge assessing interpretations across seven dimensions, including symbolism, contextual awareness, originality, and coherence. Although the LLM-judge evaluates models higher than humans, experiments with GPT-4o, LLaVA1.5, Qwen2.5-VL, and Gemma3 reveal distinct differences. While humans map abstract concepts onto spatial reality and personal metaphor, advanced VLMs rely on detached, formal art critique terminology. Crucially, our variance analysis debunks the assumption that higher generation temperatures induce creativity; advanced models like GPT-4o exhibited zero or small variance, acting as unyielding, consistent critics. On the other hand, smaller models like LLaVA exhibited erratic, above-human variance due to abstract instability. These results highlight a critical design trajectory for multimodal AI: choosing between the rigid reliability of an automated ``art critic'' and the unpredictable, divergent brainstorming of an ``idea-giver.''},
\r\nkeywords = {},
\r\npubstate = {published},
\r\ntppubtype = {inproceedings}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>
Visual content plays a central role in online communication, where images are often accompanied by human interpretation and commentary. With the rapid development of Vision\u2013Language Models (VLMs), artificial intelligence systems are increasingly capable of generating such interpretations. This paper examines whether current VLMs can produce art descriptions and interpretations comparable to those written by human students. We conduct a comparative study using a structured evaluation framework in which a Large Language Model (GPT-4o) acts as an automated judge assessing interpretations across seven dimensions, including symbolism, contextual awareness, originality, and coherence. Although the LLM-judge evaluates models higher than humans, experiments with GPT-4o, LLaVA1.5, Qwen2.5-VL, and Gemma3 reveal distinct differences. While humans map abstract concepts onto spatial reality and personal metaphor, advanced VLMs rely on detached, formal art critique terminology. Crucially, our variance analysis debunks the assumption that higher generation temperatures induce creativity; advanced models like GPT-4o exhibited zero or small variance, acting as unyielding, consistent critics. On the other hand, smaller models like LLaVA exhibited erratic, above-human variance due to abstract instability. These results highlight a critical design trajectory for multimodal AI: choosing between the rigid reliability of an automated ``art critic'' and the unpredictable, divergent brainstorming of an ``idea-giver.''<\/div>
Close<\/a><\/p><\/div><\/div><\/div>
2.<\/div>
Binghao Wang; Rafal Rzepka<\/p>
SafeSight: Auditing False Alarms in Embodied Hazard Perception<\/a> Proceedings Article<\/span> Forthcoming<\/span><\/p>
In: <\/span>Proceedings of The Pacific Rim International Conference on Artificial Intelligence (PRICAI 2026), <\/span>Forthcoming.<\/p>
Abstract<\/a><\/span> | Links<\/a><\/span> | BibTeX<\/a><\/span><\/p>
@inproceedings{BinghaoWangPRICAI2026,
\r\ntitle = {SafeSight: Auditing False Alarms in Embodied Hazard Perception},
\r\nauthor = {Binghao Wang and Rafal Rzepka},
\r\nurl = {https:\/\/github.com\/takashiwangbh\/SafeSight},
\r\nyear  = {2026},
\r\ndate = {2026-11-17},
\r\nurldate = {2026-11-17},
\r\nbooktitle = {Proceedings of The Pacific Rim International Conference on Artificial Intelligence (PRICAI 2026)},
\r\nabstract = {Embodied agents increasingly rely on large language and vision-language models (LLMs\/VLMs) to perceive household hazards such as running faucets, broken glass, or unattended stoves. Existing
\r\nbenchmarks evaluate this ability primarily through recall on hazardous scenes, which rewards models that trivially predict \u201cdangerous\u201d for every input. This paper presents SafeSight, an AI2-THOR-grounded evaluation framework, and SafeSight-Bench, its frozen benchmark instantiation with 33 executable hazard templates, 891 hazardous scenes, 356 safe controls, and semantic sensor noise. Across 12 modern open-source LLMs and VLMs spanning five architecture families and three input tracks (text-only, image-only, text+image), the audit collects 142,542 raw model-inference JSON results, which are normalized into 137,182 standardized scoring records, and reports precision, recall, F1, False-Alarm Rate (FAR) on safe scenes, Hazard Alignment (HA), and Phantom Contamination
\r\n(PC). The analysis reveals that five of twenty-two configurations combine recall \u22650.96 with FAR between 0.87 and 0.99, exposing conservative over-prediction; that sensor noise raises text-bearing FAR from 0.43 at \u03bb= 0 to 0.77 at \u03bb= 0.7 (all tracks: 0.41 \u21920.67); and that keyword-based Hazard Alignment does not exceed approximately 0.60 even on correctly flagged scenes. A lightweight Embodied Gated Alignment Verification (EGAV) baseline reduces FAR by roughly half for two representative VLMs while improving F1. Automatic metrics are complemented with machine-consensus cross-verification. The code, hazard templates, standardized scoring records, and qualitative visual examples are publicly available in our GitHub repository.},
\r\nkeywords = {},
\r\npubstate = {forthcoming},
\r\ntppubtype = {inproceedings}
\r\n}
\r\n<\/pre><\/div>
Close<\/a><\/p><\/div>
Embodied agents increasingly rely on large language and vision-language models (LLMs\/VLMs) to perceive household hazards such as running faucets, broken glass, or unattended stoves. Existing
\r\nbenchmarks evaluate this ability primarily through recall on hazardous scenes, which rewards models that trivially predict \u201cdangerous\u201d for every input. This paper presents SafeSight, an AI2-THOR-grounded evaluation framework, and SafeSight-Bench, its frozen benchmark instantiation with 33 executable hazard templates, 891 hazardous scenes, 356 safe controls, and semantic sensor noise. Across 12 modern open-source LLMs and VLMs spanning five architecture families and three input tracks (text-only, image-only, text+image), the audit collects 142,542 raw model-inference JSON results, which are normalized into 137,182 standardized scoring records, and reports precision, recall, F1, False-Alarm Rate (FAR) on safe scenes, Hazard Alignment (HA), and Phantom Contamination
\r\n(PC). The analysis reveals that five of twenty-two configurations combine recall \u22650.96 with FAR between 0.87 and 0.99, exposing conservative over-prediction; that sensor noise raises text-bearing FAR from 0.43 at \u03bb= 0 to 0.77 at \u03bb= 0.7 (all tracks: 0.41 \u21920.67); and that keyword-based Hazard Alignment does not exceed approximately 0.60 even on correctly flagged scenes. A lightweight Embodied Gated Alignment Verification (EGAV) baseline reduces FAR by roughly half for two representative VLMs while improving F1. Automatic metrics are complemented with machine-consensus cross-verification. The code, hazard templates, standardized scoring records, and qualitative visual examples are publicly available in our GitHub repository.<\/div>
Close<\/a><\/p><\/div>