The enclosed documents pertain to the proposal and the desired structure of the thesis, which focuses on developing a plugin for the Moodle forum system that utilizes a large language model. This Thesis focuses on comparing newest pretrained Image to Text Models
- UForm (unum-cloud)
- InstructBLIP-Vicuna13B
- CLIP-Interrogator
- Img2Prompt
- LLaMA-13B
- LLaMA-7B
- BLIP-2
- OpenAI GPT-4-Vision-Preview
Some of the Evaluation Metrics could be
- SQA (Semantic Quality Assessment)
- MME (Multimodal Embedding Evaluation)
- MMBench (Multimodal Benchmarking)
- Average Size (of the model)
- Caption Length (generated text length)
- CLIPScore (for measuring the relevance of the generated text to the image)
- RefCLIPScore (reference-based CLIPScore for contextual accuracy)
- VQAv2 (Visual Question Answering version 2 performance)
- Token Speed (generation speed measured in tokens per second)
The Thesis requirses a working Plugin for Moodle Image to Text writen in Php/Javascript to generate Alternative Text in the Field “image_altentry” for Atto and TinyMCE