Objective:
This literature review aims to analyze, compare, and summarize the performance and efficiency of selected pre-trained machine learning models that specialize in generating relevant text descriptions from images. The review will focus on evaluating these models against a set of numerical metrics to determine their capabilities and limitations in real-world applications.
Scope:
The review will cover the following pre-trained models:
- UForm (unum-cloud)
- InstructBLIP-Vicuna13B
- CLIP-Interrogator
- Img2Prompt
- LLaMA-13B
- LLaMA-7B
- BLIP-2
- OpenAI GPT-4-Vision-Preview
Evaluation Metrics:
The models will be evaluated based on the following metrics:
- SQA (Semantic Quality Assessment)
- MME (Multimodal Embedding Evaluation)
- MMBench (Multimodal Benchmarking)
- Average Size (of the model)
- Caption Length (generated text length)
- CLIPScore (for measuring the relevance of the generated text to the image)
- RefCLIPScore (reference-based CLIPScore for contextual accuracy)
- VQAv2 (Visual Question Answering version 2 performance)
- Token Speed (generation speed measured in tokens per second)
you can use different metrics aswell
Conclusion and Recommendations: Summarize the key findings and suggest which models show the most promise based on the evaluation metrics. Offer recommendations for future research or application areas where these models could be utilized effectively.